Video coding and decoding method, device, equipment, system and storage medium

Abstract

The embodiment of the invention provides a video encoding and decoding method, device, equipment and system and a storage medium. A decoding end decodes a code stream and determines a reconstruction block of a current block; decoding the code stream, and determining a first quantization parameter of the current block, the first quantization parameter being determined based on a second quantization parameter predicted by a quantization parameter prediction model; and filtering the reconstruction block of the current block through a filter based on a neural network according to the first quantization parameter to obtain a filtered reconstruction block. According to the embodiment of the invention, the quantization parameter is predicted through the quantization parameter prediction model, the selectable range of the quantization parameter is expanded, the accuracy of the quantization parameter is further improved, and when filtering is carried out based on the accurately determined quantization parameter, the filtering effect can be improved, so that the video compression performance is improved.

Description

Video encoding and decoding method, device, equipment, system and storage medium Technical Field

[0001] The present application relates to the field of video coding and decoding technology, and in particular to a video coding and decoding method, apparatus, device, system and storage medium. Background Art

[0002] Digital video technology can be incorporated into a variety of video devices, such as digital televisions, smartphones, computers, e-readers, or video players. With the development of video technology, the amount of video data included has increased. To facilitate the transmission of video data, video devices implement video compression technology to enable more efficient transmission or storage of video data.

[0003] The video compression process introduces distortion. To reduce this distortion, the reconstructed image is filtered to improve its quality. With the rapid development of neural network technology, it is now possible to filter the reconstructed image using neural network-based filters. Since the quantization parameter setting plays a decisive role in the quality of the encoded image, to further improve the filtering quality of the image, it is necessary to determine the quantization parameter and use it as the input of the neural network-based filter.

[0004] However, the current method for determining the quantization parameter does not provide a high accuracy rate for the determined quantization parameter, which results in poor encoding and decoding effects.

[0005] Summary of the Invention

[0006] The embodiments of the present application provide a video encoding and decoding method, apparatus, device, system and storage medium to improve video compression effect.

[0007] In a first aspect, the present application provides a video decoding method, comprising:

[0008] Decode the code stream and determine the reconstructed block of the current block;

[0009] Decoding the bitstream to determine a first quantization parameter of the current block, where the first quantization parameter is determined based on a second quantization parameter predicted by a quantization parameter prediction model;

[0010] According to the first quantization parameter, the reconstructed block of the current block is filtered by a neural network-based filter to obtain a filtered reconstructed block.

[0011] In a second aspect, an embodiment of the present application provides a video encoding method, including:

[0012] Reconstructing the current block to obtain a reconstructed block of the current block;

[0013] Predicting a second quantization parameter of the current block using a quantization parameter prediction model;

[0014] determining a first quantization parameter of the current block according to the second quantization parameter;

[0015] According to the first quantization parameter, the reconstructed block of the current block is filtered by a neural network-based filter to obtain a filtered reconstructed block.

[0016] In a third aspect, the present application provides a video encoder configured to execute the method of the first aspect or its respective implementations. Specifically, the encoder includes a functional unit configured to execute the method of the first aspect or its respective implementations.

[0017] In a fourth aspect, the present application provides a video decoder for executing the method of the second aspect or its respective implementations. Specifically, the decoder includes a functional unit for executing the method of the second aspect or its respective implementations.

[0018] In a fifth aspect, a video encoder is provided, comprising a processor and a memory, wherein the memory is configured to store a computer program, and the processor is configured to call and execute the computer program stored in the memory to execute the method of the first aspect or its respective implementations.

[0019] In a sixth aspect, a video decoder is provided, comprising a processor and a memory, wherein the memory is configured to store a computer program, and the processor is configured to call and execute the computer program stored in the memory to perform the method of the second aspect or its respective implementations.

[0020] In a seventh aspect, a video encoding and decoding system is provided, comprising a video encoder and a video decoder. The video encoder is configured to execute the method of the first aspect or its respective implementations, and the video decoder is configured to execute the method of the second aspect or its respective implementations.

[0021] In an eighth aspect, a chip is provided for implementing the method described in any one of the first and second aspects above, or their respective implementations. Specifically, the chip includes a processor configured to load and execute a computer program from a memory, causing a device equipped with the chip to perform the method described in any one of the first and second aspects above, or their respective implementations.

[0022] In a ninth aspect, a computer-readable storage medium is provided for storing a computer program, which enables a computer to execute the method of any one of the first to second aspects or their respective implementations.

[0023] In a tenth aspect, a computer program product is provided, comprising computer program instructions, which enable a computer to execute the method of any one of the first to second aspects or their respective implementations.

[0024] In an eleventh aspect, a computer program is provided, which, when executed on a computer, enables the computer to execute the method in any one of the first to second aspects or their respective implementations.

[0025] In the twelfth aspect, a code stream is provided, including the code stream generated by any aspect of the second aspect.

[0026] Based on the above technical solution, the decoding end decodes the bitstream to determine a reconstructed block of the current block; the decoding end then determines a first quantization parameter for the current block, where the first quantization parameter is determined based on a second quantization parameter predicted by a quantization parameter prediction model; and based on the first quantization parameter, the reconstructed block of the current block is filtered using a neural network-based filter to obtain a filtered reconstructed block. Specifically, the embodiment of the present application predicts the quantization parameter using a quantization parameter prediction model, thereby expanding the selectable range of the quantization parameter and thereby improving the accuracy of the quantization parameter. When filtering based on the accurately determined quantization parameter, the filtering effect can be improved, thereby enhancing video compression performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] FIG1 is a schematic block diagram of a video encoding and decoding system according to an embodiment of the present application;

[0028] FIG2 is a schematic block diagram of a video encoder according to an embodiment of the present application;

[0029] FIG3 is a schematic block diagram of a video decoder according to an embodiment of the present application;

[0030] FIG4A is a schematic diagram of a filter based on a neural network according to an embodiment of the present application;

[0031] FIG4B is a schematic diagram of another neural network-based filter according to an embodiment of the present application;

[0032] FIG5 is a flow chart of a video decoding method provided in an embodiment of the present application;

[0033] FIG6 is a schematic diagram of a quantitative parameter prediction model provided in an embodiment of the present application;

[0034] FIG7 is a schematic diagram of image filtering provided by an embodiment of the present application;

[0035] FIG8 is a schematic diagram of a loop filter according to an embodiment of the present application;

[0036] FIG9 is a schematic diagram of a video decoding process according to an embodiment of the present application;

[0037] FIG10 is a schematic diagram of a flow chart of a video encoding method provided in an embodiment of the present application;

[0038] FIG11 is a schematic diagram of a video encoding process according to an embodiment of the present application;

[0039] FIG12 is a schematic block diagram of a video decoding device provided in an embodiment of the present application;

[0040] FIG13 is a schematic block diagram of a video encoding apparatus according to an embodiment of the present application;

[0041] FIG14 is a schematic block diagram of an electronic device provided in an embodiment of the present application;

[0042] FIG15 is a schematic block diagram of a video encoding and decoding system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0043] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0044] The present application can be applied to the field of image coding and decoding, the field of video coding and decoding, the field of hardware video coding and decoding, the field of dedicated circuit video coding and decoding, the field of real-time video coding and decoding, etc. For example, the solution of the present application can be combined with an audio and video coding standard (AVS), such as the H.264 / audio and video coding (AVC) standard, the H.265 / high efficiency video coding (HEVC) standard, and the H.266 / versatile video coding (VVC) standard. Alternatively, the solution of the present application can be combined with other proprietary or industry standards and operated, and the standards include ITU-TH.261, ISO / IEC MPEG-1 Visual, ITU-TH.262 or ISO / IEC MPEG-2 Visual, ITU-TH.263, ISO / IEC MPEG-4 Visual, ITU-TH.264 (also known as ISO / IEC MPEG-4 AVC), including scalable video coding (SVC) and multi-view video coding (MVC) extensions. It should be understood that the technology of this application is not limited to any specific coding standard or technology.

[0045] For ease of understanding, the video encoding and decoding system involved in the embodiment of the present application is first introduced with reference to FIG1 .

[0046] FIG1 is a schematic block diagram of a video encoding and decoding system involved in an embodiment of the present application. It should be noted that FIG1 is only an example, and the video encoding and decoding system of the embodiment of the present application includes but is not limited to that shown in FIG1. ​​As shown in FIG1, the video encoding and decoding system 100 includes an encoding device 110 and a decoding device 120. The encoding device is used to encode (which can be understood as compressing) the video data to generate a code stream, and transmit the code stream to the decoding device. The decoding device decodes the code stream generated by the encoding device to obtain decoded video data.

[0047] The encoding device 110 of the embodiment of the present application can be understood as a device with a video encoding function, and the decoding device 120 can be understood as a device with a video decoding function, that is, the embodiment of the present application includes a wider range of devices for the encoding device 110 and the decoding device 120, such as smartphones, desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, televisions, cameras, display devices, digital media players, video game consoles, car computers, etc.

[0048] In some embodiments, the encoding device 110 may transmit the encoded video data (eg, a code stream) to the decoding device 120 via a channel 130. The channel 130 may include one or more media and / or devices capable of transmitting the encoded video data from the encoding device 110 to the decoding device 120.

[0049] In one example, the channel 130 includes one or more communication media that enable the encoding device 110 to transmit the encoded video data directly to the decoding device 120 in real time. In this example, the encoding device 110 can modulate the encoded video data according to a communication standard and transmit the modulated video data to the decoding device 120. The communication media includes wireless communication media, such as radio frequency spectrum. Optionally, the communication media may also include wired communication media, such as one or more physical transmission lines.

[0050] In another example, channel 130 includes a storage medium that can store the video data encoded by encoding device 110. The storage medium includes various locally accessible data storage media, such as optical disks, DVDs, and flash memories. In this example, decoding device 120 can retrieve the encoded video data from the storage medium.

[0051] In another example, the channel 130 may include a storage server that can store the video data encoded by the encoding device 110. In this example, the decoding device 120 can download the stored encoded video data from the storage server. Alternatively, the storage server can store the encoded video data and transmit the encoded video data to the decoding device 120, such as a web server (e.g., for a website), a file transfer protocol (FTP) server, etc.

[0052] In some embodiments, the encoding device 110 includes a video encoder 112 and an output interface 113. The output interface 113 may include a modulator / demodulator (modem) and / or a transmitter.

[0053] In some embodiments, the encoding device 110 may further include a video source 111 in addition to the video encoder 112 and the input interface 113 .

[0054] The video source 111 may include at least one of a video acquisition device (eg, a video camera), a video archive, a video input interface, and a computer graphics system, wherein the video input interface is used to receive video data from a video content provider, and the computer graphics system is used to generate video data.

[0055] The video encoder 112 encodes the video data from the video source 111 to generate a bitstream. The video data may include one or more pictures or a sequence of pictures. The bitstream contains the coding information of the picture or picture sequence in the form of a bitstream. The coding information may include the coded picture data and associated data. The associated data may include a sequence parameter set (SPS), a picture parameter set (PPS), and other syntax structures. The SPS may contain parameters that apply to one or more sequences. The PPS may contain parameters that apply to one or more pictures. The syntax structure refers to a set of zero or more syntax elements arranged in a specified order in the bitstream.

[0056] The video encoder 112 transmits the encoded video data directly to the decoding device 120 via the output interface 113. The encoded video data may also be stored in a storage medium or a storage server for subsequent reading by the decoding device 120.

[0057] In some embodiments, the decoding device 120 includes an input interface 121 and a video decoder 122 .

[0058] In some embodiments, the decoding device 120 may further include a display device 123 in addition to the input interface 121 and the video decoder 122 .

[0059] The input interface 121 includes a receiver and / or a modem and can receive the encoded video data via the channel 130 .

[0060] The video decoder 122 is configured to decode the encoded video data to obtain decoded video data, and transmit the decoded video data to the display device 123 .

[0061] The decoded video data is displayed on the display device 123. The display device 123 may be integrated with the decoding device 120 or external to the decoding device 120. The display device 123 may include various display devices, such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or other types of display devices.

[0062] In addition, Figure 1 is only an example, and the technical solution of the embodiment of the present application is not limited to Figure 1. For example, the technology of the present application can also be applied to unilateral video encoding or unilateral video decoding.

[0063] The following is an introduction to the video encoder involved in the embodiments of the present application.

[0064] FIG2 is a schematic block diagram of a video encoder according to an embodiment of the present application. It should be understood that the video encoder 200 can be used to perform lossy compression or lossless compression on an image. The lossless compression can be visually lossless or mathematically lossless.

[0065] The video encoder 200 can be applied to image data in a luminance and chrominance (YCbCr, YUV) format. For example, the YUV ratio can be 4:2:0, 4:2:2, or 4:4:4, where Y represents brightness (Luma), Cb (U) represents blue chrominance, Cr (V) represents red chrominance, and U and V represent chrominance (Chroma) for describing color and saturation. For example, in terms of color format, 4:2:0 means that every 4 pixels have 4 luminance components and 2 chrominance components (YYYYCbCr), 4:2:2 means that every 4 pixels have 4 luminance components and 4 chrominance components (YYYYCbCrCbCr), and 4:4:4 represents full pixel display (YYYYCbCrCbCrCbCrCbCr).

[0066] For example, the video encoder 200 reads video data, and for each frame of the video data, divides the frame into a number of coding tree units (CTUs). In some examples, CTUs may be referred to as "tree blocks", "largest coding units" (LCUs) or "coding tree blocks" (CTBs). Each CTU may be associated with a pixel block of equal size within the image. Each pixel may correspond to one luminance (luminance or luma) sample and two chrominance (chroma) samples. Therefore, each CTU may be associated with one luminance sample block and two chrominance sample blocks. The size of a CTU is, for example, 128×128, 64×64, 32×32, etc. A CTU may be further divided into a number of coding units (CUs) for encoding. A CU may be a rectangular block or a square block. A CU can be further divided into prediction units (PUs) and transform units (TUs), allowing for separation of coding, prediction, and transform, and greater flexibility in processing. In one example, a CTU is divided into CUs using a quadtree, and a CU is divided into TUs and PUs using a quadtree.

[0067] The video encoder and video decoder can support various PU sizes. Assuming that the size of a particular CU is 2N×2N, the video encoder and video decoder can support PU sizes of 2N×2N or N×N for intra-frame prediction, and support symmetric PUs of 2N×2N, 2N×N, N×2N, N×N, or similar sizes for inter-frame prediction. The video encoder and video decoder can also support asymmetric PUs of 2N×nU, 2N×nD, nL×2N, and nR×2N for inter-frame prediction.

[0068] In some embodiments, as shown in FIG2 , the video encoder 200 may include a prediction unit 210, a residual unit 220, a transform / quantization unit 230, an inverse transform / quantization unit 240, a reconstruction unit 250, a loop filter unit 260, a decoded image buffer 270, and an entropy coding unit 280. It should be noted that the video encoder 200 may include more, fewer, or different functional components.

[0069] Optionally, in this application, the current block may be referred to as a current coding unit (CU) or a current prediction unit (PU), etc. A prediction block may also be referred to as a predicted image block or an image prediction block, and a reconstructed image block may also be referred to as a reconstructed block or an image reconstructed image block.

[0070] In some embodiments, the prediction unit 210 includes an inter-frame prediction unit 211 and an intra-frame prediction unit 212. Because there is a strong correlation between adjacent pixels in a video frame, intra-frame prediction is used in video coding and decoding technologies to eliminate spatial redundancy between adjacent pixels. Because there is a strong similarity between adjacent frames in a video, inter-frame prediction is used in video coding and decoding technologies to eliminate temporal redundancy between adjacent frames, thereby improving coding efficiency.

[0071] The inter-frame prediction unit 211 can be used for inter-frame prediction. Inter-frame prediction can refer to image information from different frames. Inter-frame prediction uses motion information to find a reference block from a reference frame and generate a prediction block based on the reference block to eliminate temporal redundancy. The frames used for inter-frame prediction can be P frames and / or B frames. P frames refer to forward-predicted frames, and B frames refer to bidirectionally predicted frames. Motion information includes the reference frame list in which the reference frame is located, the reference frame index, and the motion vector. The motion vector can be integer pixel or fractional pixel. If the motion vector is fractional pixel, interpolation filtering is required to generate the required fractional pixel block in the reference frame. Here, the integer pixel or fractional pixel block in the reference frame found based on the motion vector is called a reference block. Some technologies directly use the reference block as the prediction block, while others further process the reference block to generate a prediction block. Reprocessing the reference block to generate a prediction block can also be understood as using the reference block as the prediction block and then processing the prediction block to generate a new prediction block.

[0072] The intra-frame prediction unit 212 only refers to the information of the same frame image to predict the pixel information within the current code image block to eliminate spatial redundancy. The frame used for intra-frame prediction can be an I frame. For example, as shown in Figure 5, the white 4×4 block is the current block, and the gray pixels in the left row and the upper column of the current block are the reference pixels of the current block. Intra-frame prediction uses these reference pixels to predict the current block. These reference pixels may all be available, that is, all have been encoded and decoded. Some may also be unavailable, for example, if the current block is the leftmost part of the entire frame, then the reference pixels on the left of the current block are unavailable. Or when encoding and decoding the current block, the lower left part of the current block has not been encoded and decoded, then the reference pixels on the lower left are also unavailable. For the case where reference pixels are unavailable, available reference pixels or certain values ​​or certain methods can be used for filling, or no filling can be performed.

[0073] In some embodiments, the intra prediction method further includes a multiple reference line (MRL) intra prediction method. MRL can use more reference pixels to improve coding efficiency.

[0074] There are multiple intra-frame prediction modes. H.264 uses nine intra-frame prediction modes for 4×4 blocks. Mode 0 copies the pixels above the current block vertically to the current block as the prediction value; Mode 1 copies the reference pixels to the left horizontally to the current block as the prediction value; Mode 2 (DC) uses the average of the eight points A to D and I to L as the prediction value for all points; Modes 3 to 8 copy the reference pixels to the corresponding positions in the current block at a specific angle. Because some positions in the current block cannot correspond exactly to the reference pixels, it may be necessary to use a weighted average of the reference pixels, or interpolated sub-pixels of the reference pixels.

[0075] HEVC uses planar, DC, and 33 angle modes for intra-frame prediction, for a total of 35 prediction modes. VVC uses planar, DC, and 65 angle modes for intra-frame prediction, for a total of 67 prediction modes. AVS3 uses DC, Plane, Bilinear, and 63 angle modes for intra-frame prediction, for a total of 66 prediction modes.

[0076] It should be noted that with the increase of angle modes, intra-frame prediction will be more accurate and more in line with the needs of the development of high-definition and ultra-high-definition digital videos.

[0077] The residual unit 220 may generate a residual block for the CU based on the pixel blocks of the CU and the prediction blocks of the PUs of the CU. For example, the residual unit 220 may generate the residual block for the CU such that each sample in the residual block has a value equal to the difference between the sample in the pixel blocks of the CU and the corresponding sample in the prediction blocks of the PUs of the CU.

[0078] The transform / quantization unit 230 may quantize the transform coefficients. The transform / quantization unit 230 may quantize the transform coefficients associated with the TUs of a CU based on a quantization parameter (QP) value associated with the CU. The video encoder 200 may adjust the degree of quantization applied to the transform coefficients associated with the CU by adjusting the QP value associated with the CU.

[0079] The inverse transform / quantization unit 240 may apply inverse quantization and inverse transform, respectively, to the quantized transform coefficients to reconstruct a residual block from the quantized transform coefficients.

[0080] Reconstruction unit 250 may add samples of the reconstructed residual block to corresponding samples of one or more prediction blocks generated by prediction unit 210 to generate a reconstructed image block associated with the TU. By reconstructing the sample blocks of each TU of a CU in this manner, video encoder 200 can reconstruct the pixel blocks of the CU.

[0081] Loop filtering unit 260 may perform a deblocking filtering operation to reduce blocking artifacts of pixel blocks associated with a CU.

[0082] In some embodiments, the loop filtering unit 260 includes a deblocking filtering unit, a sample adaptive offset (SAO) unit, and an adaptive loop filtering (ALF) unit.

[0083] The decoded image buffer 270 may store the reconstructed pixel blocks. The inter-prediction unit 211 may use a reference image containing the reconstructed pixel blocks to perform inter-prediction on PUs of other images. In addition, the intra-prediction unit 212 may use the reconstructed pixel blocks in the decoded image buffer 270 to perform intra-prediction on other PUs in the same image as the CU.

[0084] The entropy coding unit 280 may receive the quantized transform coefficients from the transform / quantization unit 230. The entropy coding unit 280 may perform one or more entropy coding operations on the quantized transform coefficients to generate entropy-coded data.

[0085] The basic process of video encoding involved in this application is as follows: At the encoding end, the current image is divided into blocks. For the current block, the prediction unit 210 uses intra-frame prediction or inter-frame prediction to generate a prediction block for the current block. The residual unit 220 calculates a residual block based on the predicted block and the original block of the current block. This residual block is the difference between the predicted block and the original block of the current block. This residual block can also be referred to as residual information. This residual block undergoes transformation and quantization by the transform / quantization unit 230, removing information that is insensitive to the human eye and eliminating visual redundancy. Optionally, the residual block before transformation and quantization by the transform / quantization unit 230 can be referred to as a time-domain residual block, and the time-domain residual block after transformation and quantization by the transform / quantization unit 230 can be referred to as a frequency residual block or a frequency-domain residual block. The entropy coding unit 280 receives the quantized transform coefficients output by the transform and quantization unit 230, performs entropy coding on the quantized transform coefficients, and outputs a bitstream. For example, the entropy coding unit 280 can eliminate character redundancy based on the target context model and probability information of the binary bitstream.

[0086] In addition, the video encoder dequantizes and inverse transforms the quantized transform coefficients output by the transform and quantization unit 230 to obtain a residual block for the current block. This residual block is then added to the prediction block for the current block to obtain a reconstructed block for the current block. As encoding progresses, reconstructed blocks corresponding to other image blocks in the current image are obtained. These reconstructed blocks are then spliced ​​together to obtain a reconstructed image of the current image. Because errors are introduced during the encoding process, the reconstructed image is filtered to reduce them. For example, ALF filtering is used to reduce the difference between the pixel values ​​of the reconstructed image and the original pixel values ​​of the pixels in the current image. The filtered reconstructed image is stored in the decoded image buffer 270 and can be used as a reference frame for inter-frame prediction for subsequent frames.

[0087] It should be noted that the block division information determined by the encoder, as well as mode information or parameter information such as prediction, transform, quantization, entropy coding, and loop filtering, etc., are carried in the bitstream when necessary. The decoder parses the bitstream and analyzes the existing information to determine the same block division information, prediction, transform, quantization, entropy coding, loop filtering, etc. mode information or parameter information as the encoder, thereby ensuring that the decoded image obtained by the encoder and the decoder are identical.

[0088] FIG3 is a schematic block diagram of a video decoder provided in an embodiment of the present application.

[0089] 3 , the video decoder 300 includes an entropy decoding unit 310, a prediction unit 320, an inverse quantization / transformation unit 330, a reconstruction unit 340, a loop filter unit 350, and a decoded picture buffer 360. It should be noted that the video decoder 300 may include more, fewer, or different functional components.

[0090] The video decoder 300 may receive a bitstream. The entropy decoding unit 310 may parse the bitstream to extract syntax elements from the bitstream. As part of parsing the bitstream, the entropy decoding unit 310 may parse the entropy-encoded syntax elements in the bitstream. The prediction unit 320, the inverse quantization / transform unit 330, the reconstruction unit 340, and the loop filter unit 350 may decode the video data based on the syntax elements extracted from the bitstream, thereby generating decoded video data.

[0091] In some embodiments, the prediction unit 320 includes an inter-frame prediction unit 321 and an intra-frame prediction unit 322 .

[0092] The inter-frame prediction unit 321 may perform intra-frame prediction to generate a prediction block for the PU. The inter-frame prediction unit 321 may use an intra-frame prediction mode to generate a prediction block for the PU based on the pixel blocks of spatially neighboring PUs. The inter-frame prediction unit 321 may also determine the intra-frame prediction mode of the PU based on one or more syntax elements parsed from the codestream.

[0093] The intra-frame prediction unit 322 may construct a first reference picture list (List 0) and a second reference picture list (List 1) based on syntax elements parsed from the codestream. In addition, if the PU is encoded using inter-frame prediction, the entropy decoding unit 310 may parse the motion information of the PU. The intra-frame prediction unit 322 may determine one or more reference blocks for the PU based on the motion information of the PU. The intra-frame prediction unit 322 may generate a prediction block for the PU based on the one or more reference blocks of the PU.

[0094] The inverse quantization / transform unit 330 may inversely quantize (ie, dequantize) the transform coefficients associated with the TU. The inverse quantization / transform unit 330 may use the QP value associated with the CU of the TU to determine the degree of quantization.

[0095] After inverse quantizing the transform coefficients, the inverse quantization / transform unit 330 may apply one or more inverse transforms to the inverse quantized transform coefficients in order to generate a residual block associated with the TU.

[0096] The reconstruction unit 340 uses the residual block associated with the TU of the CU and the prediction block of the PU of the CU to reconstruct the pixel block of the CU. For example, the reconstruction unit 340 can add samples of the residual block to corresponding samples of the prediction block to reconstruct the pixel block of the CU to obtain a reconstructed image block.

[0097] The loop filtering unit 350 may perform a deblocking filtering operation to reduce blocking artifacts of pixel blocks associated with a CU.

[0098] In some embodiments, the loop filtering unit 350 includes a deblocking filtering unit, a sample adaptive offset (SAO) unit, and an adaptive loop filtering (ALF) unit.

[0099] The video decoder 300 may store the reconstructed image of the CU in the decoded image buffer 360. The video decoder 300 may use the reconstructed image in the decoded image buffer 360 as a reference image for subsequent prediction, or transmit the reconstructed image to a display device for presentation.

[0100] The basic process of video decoding involved in this application is as follows: the entropy decoding unit 310 can parse the code stream to obtain the prediction information, quantization coefficient matrix, etc. of the current block, and the prediction unit 320 uses intra-frame prediction or inter-frame prediction for the current block based on the prediction information to generate a prediction block of the current block. The inverse quantization / transformation unit 330 uses the quantization coefficient matrix obtained from the code stream to inverse quantize and inverse transform the quantization coefficient matrix to obtain a residual block. The reconstruction unit 340 adds the prediction block and the residual block to obtain a reconstructed block. The reconstructed blocks constitute a reconstructed image, and the loop filtering unit 350 performs loop filtering on the reconstructed image based on the image or block to obtain a decoded image. The decoded image can also be called a reconstructed image. On the one hand, the reconstructed image can be displayed by a display device, and on the other hand, it can be stored in the decoded image cache 360 ​​as a reference frame for inter-frame prediction for subsequent frames.

[0101] The above is the basic process of the video codec under the block-based hybrid coding framework. With the development of technology, some modules or steps of the framework or process may be optimized. This application is applicable to the basic process of the video codec under the block-based hybrid coding framework, but is not limited to the framework and process.

[0102] As can be seen above, in existing video codec standards, such as VVC, traditional loop filtering modules mainly include tools such as the deblocking filter (DBF), sample adaptive offset (SAO), and adaptive correction filter (ALF). In recent years, with the development of deep learning technology, the exploration of neural network-based filters has also gradually begun.

[0103] In the VVC research work, the Neural Network-Based Video Coding Exploration Experiment Group (NNVC, EE1) was established to encourage related exploration and research. For the loop filter module, many high-performance neural network-based filters were proposed.

[0104] In some embodiments, JVET-X0065 shown in FIG4A is proposed. JVET-X0065 is a convolutional neural network filter based on residual learning to perform filtering optimization on the input reconstructed image, so that the final output image is closer to the original image, thereby bringing better coding performance.

[0105] Because the quantization parameter setting plays a crucial role in the quality of the encoded image, JVET-X0065 generated different training sets for different base quantization parameters (BQP), such as {22, 27, 32, 37, 42}, and trained multiple filter network models to process the corresponding reconstructed images. However, this multi-model approach also presents certain challenges, such as requiring more space to store the models, limited generalization capabilities, and longer training times.

[0106] Therefore, in some embodiments, JVET-X0066 further inputs the basic quantization parameter into the network, so that only one filter network model needs to be trained. A candidate basic quantization parameter set, such as {BQP, BQP-5, BQP+5}, is proposed. By traversing the rate-distortion cost at the encoder end, the optimal basic quantization parameter is selected, resulting in better performance for the filter network model.

[0107] In some embodiments, further attempts are made to input more quantization parameter-related information into the filtering network model. For example, JVET-Y0078 shown in Figure 4B adopts a multi-layer residual network structure. In addition to the basic quantization parameter BQP, a slice-level quantization parameter (SQP) is also introduced as the input of the network.

[0108] As can be seen from the above, to improve the filtering performance of neural network-based filters, quantization parameters must also be input into the neural network-based filters. Current methods for determining the quantization parameters for neural network-based filters include directly using a preset basic quantization parameter as input, or selecting one of a preset set of basic quantization parameter candidates as input. However, due to the limited range of selectable quantization parameters, the accuracy of the selected input quantization parameter is low, resulting in poor filtering performance and unsatisfactory compression quality.

[0109] To address the above technical issues, the present embodiment predicts a second quantization parameter using a quantization parameter prediction model, determines a first quantization parameter based on the second quantization parameter, and filters a reconstructed block of the current block using a neural network-based filter based on the first quantization parameter to obtain a filtered reconstructed block. In other words, the present embodiment predicts the quantization parameter using a quantization parameter prediction model, thereby expanding the selectable range of the input quantization parameter and improving the accuracy of the quantization parameter. When filtering based on the accurately determined quantization parameter, the filtering effect can be improved, thereby enhancing video compression performance.

[0110] The technical solutions provided in the embodiments of the present application are described in detail below in conjunction with specific embodiments.

[0111] First, referring to FIG5 , the decoding end is taken as an example for introduction.

[0112] FIG5 is a flow chart of a video decoding method provided by an embodiment of the present application, which is applied to the video decoder shown in FIG1 or FIG3. As shown in FIG5, the method of the embodiment of the present application includes:

[0113] S501: Decode a first bitstream and determine a reconstructed block of a current block.

[0114] In some embodiments, the current block is also referred to as a current CU, a current image block, a current decoding block, a current decoding unit, a current block to be decoded, a current image block to be decoded, etc.

[0115] In some embodiments, the current block of the embodiment of the present application only includes chrominance components and can be understood as a chrominance block.

[0116] In some embodiments, the current block of the embodiment of the present application only includes a luminance component and can be understood as a luminance block.

[0117] In some embodiments, the current block includes both luma and chroma components.

[0118] During video decoding, the decoding end receives and decodes the code stream, determines the block division mode of the current CTU, and divides the current CTU into blocks using the block division mode of the current CTU to obtain at least one CU.

[0119] For a current block in at least one CU, the decoding end may decode the code stream and determine a reconstructed block of the current block.

[0120] The specific manner in which the decoding end determines the reconstructed block of the current block by decoding the bitstream includes the following two cases:

[0121] In case 1, if the encoder transforms the residual values ​​of the current block without quantizing them, and instead directly encodes the transform coefficients of the current block to obtain a bitstream, the decoder decodes the bitstream to obtain the transform coefficients of the current block, and then inversely transforms the transform coefficients to obtain the residual block of the current block. Alternatively, the current block is predicted using an inter-frame or intra-frame prediction method to obtain a predicted block for the current block. Based on the predicted block and the residual block of the current block, a reconstructed block for the current block is determined. For example, the sum of the predicted block and the residual block of the current block is determined as the reconstructed block of the current block.

[0122] In case 2, if the encoder transforms the residual values ​​of the current block and then quantizes them, obtaining the quantization coefficients of the current block, the quantization coefficients of the current block are encoded to obtain a bitstream. In this case, the decoder decodes the bitstream, obtains the quantization coefficients of the current block, dequantizes the quantization coefficients to obtain the transform coefficients of the current block, and de-transforms the transform coefficients to obtain the residual block of the current block. Alternatively, the current block is predicted using an inter-frame or intra-frame prediction method to obtain a prediction block for the current block. Based on the prediction block and the residual block of the current block, a reconstructed block for the current block is determined. For example, the sum of the prediction block and the residual block of the current block is determined as the reconstructed block of the current block.

[0123] S502: Decode the code stream and determine a first quantization parameter of the current block.

[0124] The first quantization parameter is determined based on a second quantization parameter predicted by a quantization parameter prediction model.

[0125] In an embodiment of the present application, in order to improve the accuracy of the quantization parameter, the quantization parameter is predicted using a quantization parameter prediction mode to expand the optional range of the quantization parameter, thereby improving the accuracy of determining the quantization parameter. For example, the quantization parameter prediction mode is used to predict the second quantization parameter, and the distortion corresponding to the first quantization parameter and the preset basic quantization parameter is determined. Based on the distortion, the first quantization parameter is determined from the predicted second quantization parameter and the preset basic quantization parameter, thereby improving the accuracy of determining the first quantization parameter. When filtering based on the accurately determined first quantization parameter through a neural network-based filter, the filtering effect can be improved, thereby improving video compression performance.

[0126] In the above S502, methods for determining the first quantization parameter of the current block include but are not limited to the following:

[0127] Method 1 decodes the bitstream to obtain the first quantization parameter for the current block. Specifically, the encoder calculates and determines the first quantization coefficient for the current block and then writes it into the bitstream. This allows the decoder to obtain the first quantization coefficient for the current block by decoding the bitstream, reducing its workload and improving decoding efficiency.

[0128] In the second method, the decoding end determines the first quantization coefficient of the current block through the following steps S502-A1 and S502-A2:

[0129] S502-A1. Predict a second quantization parameter of the current block using a quantization parameter prediction model.

[0130] The quantization parameter prediction model of the embodiment of the present application can predict a second quantization parameter that meets the feature information of the current block based on the feature information of the current block, so that when subsequent image filtering is performed based on the second quantization parameter, the effect of image filtering can be improved.

[0131] In some embodiments, the quantization parameter prediction model is trained using N preset candidate quantization parameters as labels. Specifically, the quantization parameter prediction model is trained iteratively and its parameters updated using the cross-entropy loss function, using the N candidate quantization parameters as labels and following the principles of classification network training. This trained quantization parameter prediction model can accurately predict the second quantization parameter of the current block based on the image feature information of the current block.

[0132] In some embodiments, the second quantization parameter of the current block predicted by the quantization parameter prediction model of the embodiment of the present application is the candidate quantization parameter with the highest probability among the preset N candidate quantization parameters. In other words, the quantization parameter prediction model of the embodiment of the present application can predict the probability of selection of each candidate quantization parameter among the N candidate quantization parameters, and determine the candidate quantization parameter with the highest probability of selection among the N candidate quantization parameters as the second quantization parameter of the current block.

[0133] In some embodiments, in order to further improve the prediction accuracy of the quantization parameter prediction model, the embodiment of the present application can expand the number of candidate quantization parameters in the N candidate quantization parameters, so that the quantization parameter prediction model can predict the second quantization parameter for the current block from more candidate quantization parameters, thereby improving the prediction accuracy of the second quantization parameter.

[0134] In some embodiments, in the embodiments of the present application, the number of the above-mentioned N candidate quantization parameters is greater than the number of candidate quantization parameters corresponding to the basic quantization parameter.

[0135] For example, the number of candidate quantization parameters corresponding to the basic quantization parameter is usually 3, such as {BQP, BQP-5, BQP+5} or {BQP, BQP-5, BQP-10}, etc. However, in the embodiment of the present application, the number of N candidate quantization parameters is greater than 3.

[0136] In one example, the N candidate quantization parameters in the embodiment of the present application are obtained by expanding the basic quantization parameters, for example, {BQP-floor(N / 2), ..., BQP+floor(N / 2)}. Assuming that N is 11, the N candidate quantization parameters in the embodiment of the present application are {BQP-5, BQP-4, ..., BQP, ..., BQP+4, BQP+5}.

[0137] In an embodiment of the present application, a quantization parameter prediction model can accurately select one candidate quantization parameter from the above-mentioned N candidate quantization parameters as the second quantization parameter of the current block. When image filtering is performed based on the accurate second quantization parameter, the filtering effect can be improved.

[0138] The embodiments of the present application do not limit the specific network structure of the quantization parameter prediction model.

[0139] In some embodiments, the quantization parameter prediction model includes multiple convolutional layers and at least one fully connected layer. For example, the quantization parameter prediction model includes three convolutional layers and two fully connected layers. Exemplarily, the two fully connected layers are located after the three convolutional layers.

[0140] In some embodiments, the quantization parameter prediction model further includes a plurality of pooling layers, which are used to downsample the feature dimensions of the feature information output by the convolution layer.

[0141] Optionally, at least one pooling layer among the multiple pooling layers is a maximum pooling layer.

[0142] Optionally, at least one pooling layer among the multiple pooling layers is an average pooling layer.

[0143] In the embodiment of the present application, there is no limitation on the specific connection method between the above-mentioned multiple convolutional layers and the above-mentioned multiple pooling layers.

[0144] In one possible implementation, each of the multiple convolutional layers is connected to a pooling layer, that is, the number of convolutional layers included in the above-mentioned quantization parameter prediction model is the same as the number of pooling layers, and one convolutional layer is connected to a pooling layer.

[0145] In some embodiments, the last layer of the quantitative parameter prediction model is SoftMax.

[0146] Figure 6 is a schematic diagram of a quantization parameter prediction model provided in an embodiment of the present application. As shown in Figure 6, the quantization parameter prediction model in the embodiment of the present application includes 3 layers of convolution layers Conv, 3 layers of maximum pooling layers MaxPooling, 2 fully connected layers FullConnect, 1 nonlinear activation function, such as a parameterized rectified linear unit (PReLU) and a SoftMax layer. Among them, a maximum pooling layer is connected after a convolution layer, a fully connected layer is connected after the last maximum pooling layer, a nonlinear activation function is connected between the two fully connected layers, and the last layer is a SoftMax layer. It should be noted that Figure 6 is only a quantization parameter prediction model shown in an embodiment of the present application. The network structure of the quantization parameter prediction model in the embodiment of the present application includes but is not limited to that shown in Figure 6. In addition, the number of layers in the quantization parameter prediction model includes but is not limited to the number shown in Figure 6. For example, the quantization parameter prediction model can include 2, 3, 4, 5 or other multiple convolution layers.

[0147] As shown in Figure 6, the decoder inputs the reconstructed block of the current block into the quantization parameter prediction model shown in Figure 6. This quantization parameter prediction model processes the block through three convolutional layers and three layers of maximum pooling to obtain the feature information of the current block. The feature information of the current block then passes through two fully connected layers and one layer of nonlinear activation function PReLU, and finally passes through a softmax layer to output a probability distribution of N candidate quantization parameters. Based on this probability distribution, the second quantization parameter of the current block is obtained.

[0148] It should be noted that the embodiment of the present application does not limit the specific method of predicting the second quantization parameter of the current block through the quantization parameter prediction model in the above S502-A1.

[0149] In some embodiments, the decoding end inputs the reconstructed block of the current block into a quantization parameter prediction model to perform quantization parameter prediction to obtain a second quantization parameter of the current block.

[0150] In some embodiments, in order to improve the prediction accuracy of the quantization parameter prediction model, the decoding end inputs the reconstructed block of the current block into the quantization parameter prediction model, and also inputs at least one preset third quantization parameter into the quantization parameter prediction model for quantization parameter prediction to obtain the second quantization parameter of the current block.

[0151] Specifically, the decoding end fuses the reconstructed block of the current block and at least one third quantization parameter, and inputs the quantization parameter into a quantization parameter prediction model to perform quantization parameter prediction to obtain the second quantization parameter.

[0152] The embodiment of the present application does not limit the specific method of fusing the reconstructed block of the current block and at least one third quantization parameter. For example, after the decoding end splices the reconstructed block of the current block and at least one third quantization parameter, it inputs the quantization parameter prediction model to perform quantization parameter prediction to obtain the second quantization parameter.

[0153] The embodiment of the present application does not limit the specific method of the third quantization parameter.

[0154] In some embodiments, the at least one third quantization parameter includes at least one of a basic quantization parameter, a slice-level quantization parameter, and a reference block quantization parameter of the current block.

[0155] In one example, the at least one third quantization parameter includes a basic quantization parameter (BQP) and a slice-level quantization parameter (SQP). At this point, the decoder inputs the reconstructed block of the current block, the basic quantization parameter (BQP), and the slice-level quantization parameter (SQP) into the quantization parameter prediction model to predict the second quantization parameter for the current block. For example, as shown in FIG7 , the reconstructed block of the current block, the basic quantization parameter (BQP), and the slice-level quantization parameter (SQP) are concatenated and input into the quantization parameter prediction model to predict the second quantization parameter for the current block.

[0156] According to the above method, after the second quantization parameter of the current block is predicted by the quantization parameter prediction model, the following step S502-A2 is performed.

[0157] S502-A2: Determine the first quantization parameter according to the second quantization parameter.

[0158] In the embodiment of the present application, specific implementation methods for determining the first quantization parameter based on the second quantization parameter include but are not limited to the following:

[0159] Method 1: Since the above-mentioned second quantization parameter is obtained by the quantization parameter prediction model based on the reconstructed block prediction of the current block and conforms to the characteristic information of the current block, the decoding end can directly determine the second quantization parameter of the current block obtained by the above prediction as the first quantization parameter of the current block.

[0160] Mode 2: To further improve the accuracy of determining the first quantization parameter, the above S502-A2 includes the following step S502-A2-1:

[0161] S502-A2-1. Determine a first quantization parameter according to a second quantization parameter and a preset basic quantization parameter.

[0162] The preset basic quantization parameter may be understood as a quantization parameter selected from candidate quantization parameters corresponding to the basic quantization parameter.

[0163] Exemplarily, the candidate quantization parameters corresponding to the basic quantization parameter include at least one of 22, 27, 32, 37, 42, etc.

[0164] In this approach 2, the decoding end determines the first quantization parameter of the current block by using the predicted second quantization parameter of the current block and a preset basic quantization parameter, thereby improving the accuracy of determining the first quantization parameter.

[0165] The embodiment of the present application does not limit the specific method of determining the first quantization parameter based on the second quantization parameter and the preset basic quantization parameter in the above S502-A2-1.

[0166] In a possible implementation of S502-A2-1 above, the predicted second quantization parameter of the current block and any one of the preset basic quantization parameters can be determined as the first quantization parameter of the current block. For example, the decoding end determines the minimum quantization parameter of the second quantization parameter of the current block and the basic quantization parameter as the first quantization parameter of the current block. Alternatively, the decoding end determines the cost of the second quantization parameter and the basic quantization parameter of the current block respectively, and determines the quantization parameter with the minimum cost between the second quantization parameter and the basic quantization parameter as the first quantization parameter of the current block.

[0167] In another possible implementation of S502-A2-1, the arithmetic mean or weighted mean of the predicted second quantization parameter of the current block and the preset basic quantization parameter may be determined as the first quantization parameter of the current block.

[0168] In another possible implementation of the above S502-A2-1, the decoder determines the first quantization parameter of the current block from the second quantization parameter and the basic quantization parameter according to the distortion. Specifically, the above S502-A2-1 includes the following steps S502-A2-11 to S502-A2-13:

[0169] S502-A2-11. Determine the distortion corresponding to the second quantization parameter.

[0170] S502-A2-12, determining the distortion corresponding to the basic quantization parameter;

[0171] S502-A2-13. Determine a first quantization parameter from the second quantization parameter and the basic quantization parameter according to the distortion corresponding to the second quantization parameter and the distortion corresponding to the basic quantization parameter.

[0172] In this implementation, the distortion corresponding to the second quantization parameter and the distortion corresponding to the basic quantization parameter are determined respectively, so that the first quantization parameter is determined from the second quantization parameter and the basic quantization parameter according to the distortion.

[0173] The specific process of determining the distortion corresponding to the second quantization parameter and the distortion corresponding to the basic quantization parameter is described below. It should be noted that the method of determining the distortion corresponding to the second quantization parameter and the distortion corresponding to the basic quantization parameter can be the same or different, and the embodiments of the present application do not limit this.

[0174] The embodiment of the present application does not limit the method of determining the distortion corresponding to the second quantization parameter in the above S502-A2-11.

[0175] In one example, the decoding end can filter the prediction block of the current block through a neural network-based filter according to the second quantization parameter to obtain a filtered prediction block, compare the filtered prediction value with the prediction value of the current block, and obtain the distortion corresponding to the second quantization parameter. For example, the difference between the filtered prediction block and the prediction block of the current block is determined as the distortion corresponding to the second quantization parameter of the current block.

[0176] In another example, the decoding end filters the reconstructed block of the current block using a neural network-based filter based on the second quantization parameter to obtain a filtered first reconstructed block. The distortion corresponding to the second quantization parameter is determined based on the filtered first reconstructed block and the reconstructed block of the current block. For example, the second quantization parameter of the current block and the reconstructed block of the current block are concatenated and then input into the neural network-based filter for filtering to obtain the filtered first reconstructed block. The difference between the filtered first reconstructed block and the reconstructed block of the current block is then determined as the distortion corresponding to the second quantization coefficient.

[0177] The embodiment of the present application does not limit the method of determining the distortion corresponding to the basic quantization parameter in the above S502-A2-12.

[0178] In one example, the decoding end can filter the prediction block of the current block through a neural network-based filter according to the basic quantization parameter to obtain a filtered prediction block, compare the filtered prediction value with the prediction value of the current block, and obtain the distortion corresponding to the basic quantization parameter. For example, the difference between the filtered prediction block and the prediction block of the current block is determined as the distortion corresponding to the basic quantization parameter of the current block.

[0179] In another example, the decoding end filters the reconstructed block of the current block using a neural network-based filter based on the basic quantization parameter to obtain a filtered second reconstructed block. The distortion corresponding to the basic quantization parameter is determined based on the filtered second reconstructed block and the reconstructed block of the current block. For example, the basic quantization parameter of the current block and the reconstructed block of the current block are concatenated and then input into the neural network-based filter for filtering to obtain the filtered second reconstructed block. The difference between the filtered second reconstructed block and the reconstructed block of the current block is then determined as the distortion corresponding to the basic quantization coefficient.

[0180] According to the above method, the distortion corresponding to the second quantization coefficient and the distortion corresponding to the basic quantization coefficient are determined, and then, based on the distortion, the first quantization coefficient of the current block is determined from the second quantization coefficient of the current block and the preset basic quantization coefficient. For example, the quantization parameter with the smallest distortion between the second quantization parameter and the basic quantization parameter is determined as the first quantization parameter of the current block, thereby accurately determining the first quantization coefficient. In this way, when performing image filtering on a reconstructed block of the current block based on the accurately determined first quantization coefficient, the filtering effect of the reconstructed block can be improved.

[0181] In the embodiment of the present application, in addition to decoding the code stream through the above-mentioned method 1 to obtain the first quantization parameter of the current block, and predicting the second quantization parameter of the current block through the quantization parameter prediction model through the above-mentioned method 2, the decoding end can also determine the first quantization coefficient of the current block according to the following method 3.

[0182] In a third approach, the decoder determines the first quantization coefficient of the current block according to the following steps S502-B1 to S502-B3:

[0183] S502-B1. Decode the bitstream to obtain a first flag, where the first flag is used to indicate whether the first quantization parameter of the current block is the second quantization parameter predicted by the quantization parameter prediction model;

[0184] S502-B2: If the first flag indicates that the first quantization parameter of the current block is the second quantization parameter predicted by the quantization parameter prediction model, predict the second quantization parameter of the current block using the quantization parameter prediction model;

[0185] S502-B3. Determine the second quantization parameter as the first quantization parameter.

[0186] In this third method, the encoder can use a first flag to indicate whether the first quantization parameter of the current block is the second quantization parameter predicted by the quantization parameter prediction model. For example, when the encoder determines that the first quantization parameter of the current block is the second quantization parameter predicted by the quantization parameter prediction model, the encoder sets the first flag to yes and writes the first flag set to yes into the bitstream. If the encoder determines that the first quantization parameter of the current block is not the second quantization parameter predicted by the quantization parameter prediction model, the encoder sets the first flag to no and writes the first flag set to no into the bitstream. In this way, after receiving the bitstream, the decoder can determine the first quantization coefficient of the current block by decoding the bitstream. For example, the decoder decodes the bitstream and obtains the first flag. If the first flag indicates that the first quantization coefficient of the current block is the second quantization parameter predicted by the quantization parameter prediction model, the decoder predicts the second quantization parameter of the current block through the quantization parameter prediction model and determines the predicted second quantization parameter as the first quantization parameter of the current block.

[0187] In some embodiments, if the decoding end decodes the code stream and determines that the first flag indicates that the first quantization parameter of the current block is not the second quantization parameter predicted by the quantization parameter prediction model, the decoding end can determine the preset basic quantization parameter as the first quantization parameter of the current block.

[0188] As can be seen from the above, in this method three, the first flag at the block level is used to accurately indicate whether the first quantization parameter of the current block is the second quantization parameter predicted by the quantization parameter prediction model, so that the decoding end can accurately determine which method to use to determine the first quantization parameter based on the first flag. For example, when the first flag indicates that the first quantization parameter of the current block is the second quantization parameter predicted by the quantization parameter prediction model, the decoding end predicts the first quantization parameter of the current block through the quantization parameter prediction model. If the first flag indicates that the first quantization parameter of the current block is not the second quantization parameter predicted by the quantization parameter prediction model, the decoding end directly determines the preset basic quantization parameter as the first quantization parameter of the current block. The decoding end does not need to perform an attempt to compare, thereby reducing the amount of calculation required by the decoding end to determine the first quantization parameter of the current block, thereby improving the filtering efficiency of the decoding end.

[0189] In some embodiments, when the value of the first flag is a first numerical value, it indicates that the first quantization parameter of the current block is the second quantization parameter predicted by the quantization parameter prediction model.

[0190] In some embodiments, when the value of the first flag is the second value, it indicates that the first quantization parameter of the current block is not the second quantization parameter predicted by the quantization parameter prediction model.

[0191] The embodiment of the present application does not limit the specific values ​​of the first and second values.

[0192] Optionally, the first value is 1.

[0193] Optionally, the second value is 0.

[0194] In an example, the first flag may be represented by the field patch_nnqp_enable_flag[compIdx][LcuIdx], where compIdx represents the sequence numbers of three color components, and ctuIdx represents the sequence number of the CTU.

[0195] In a specific example, if patch_nnqp_enable_flag[compIdx][LcuIdx] is 1, it indicates that the first quantization parameter of the current block is the second quantization parameter predicted by the quantization parameter prediction model. At this time, the decoding end predicts the second quantization parameter of the current block through the quantization parameter prediction model.

[0196] For example, when patch_nnqp_enable_flag[compIdx][LcuIdx]=1, the decoding end inputs the reconstructed block of the current block into the quantization parameter prediction model to perform quantization parameter prediction to obtain the second quantization parameter of the current block.

[0197] For another example, when patch_nnqp_enable_flag[compIdx][LcuIdx] = 1, the decoder inputs the reconstructed block of the current block and at least one preset third quantization parameter into the quantization parameter prediction model to perform quantization parameter prediction, thereby obtaining a second quantization parameter for the current block. In one possible implementation, the decoder concatenates the reconstructed block of the current block and the at least one third quantization parameter, inputs the concatenated block into the quantization parameter prediction model to perform quantization parameter prediction, obtains a second quantization parameter, and determines the second quantization parameter as the first quantization parameter for the current block.

[0198] This embodiment of the present application does not limit the specifics of the at least one third quantization parameter.

[0199] Optionally, the at least one third quantization parameter includes at least one of a basic quantization parameter, a slice-level quantization parameter, and a reference block quantization parameter of the current block. Exemplarily, the at least one third quantization parameter includes a basic quantization parameter and a slice-based quantization parameter. In this case, the decoder may concatenate the reconstructed block of the current block, the basic quantization parameter, and the slice-based quantization parameter, and input the concatenated data into a quantization parameter prediction model to perform quantization parameter prediction, thereby obtaining the second quantization parameter.

[0200] In another specific example, if patch_nnqp_enable_flag[compIdx][LcuIdx]=0, it indicates that the first quantization parameter of the current block is not the second quantization parameter predicted by the quantization parameter prediction model. At this time, the decoding end determines the preset basic quantization parameter as the first quantization parameter of the current block.

[0201] In the third approach, the decoding end determines the first quantization parameter of the current block through a first flag at the block level.

[0202] In some embodiments, the code stream further includes a second flag at the frame level, and the second flag is used to indicate whether the first quantization parameter of at least one block in the current image is the second quantization parameter predicted by the quantization parameter prediction model. That is, when the encoder determines the first quantization parameter of each coding block in the current image, it not only determines the first flag, but also determines the second flag. Specifically, when the encoder determines the first quantization parameter of each coding block in the current image, for each coding block, if the encoder determines that the first quantization parameter of the coding block is the second quantization parameter preset by the quantization parameter prediction model, the first flag corresponding to the coding block is set to yes; if the encoder determines that the first quantization parameter of the coding block is not the second quantization parameter preset by the quantization parameter prediction model, the first flag corresponding to the coding block is set to no. According to this method, the encoder can determine the first flag of each coding block in the current image. Furthermore, the encoder determines the second flag of the current image based on the first flag of each coding block in the current block. Specifically, if the first flags of all coding blocks in the current image are all negative, for example, all 0, that is, the first quantization parameters of all coding blocks in the current image are not the second quantization parameters predicted by the quantization parameter prediction model, then the encoder sets the second flag of the current image to negative, for example, to 0, and writes the negative second flag into the bitstream. If the first flag of at least one coding block in the current image is positive, for example, the first flag of at least one coding block in the current image is 1, that is, the first quantization parameter of at least one coding block in the current image is the second quantization parameter predicted by the quantization parameter prediction model, then the encoder sets the second flag of the current image to positive, for example, to 1, and writes the positive second flag into the bitstream. In this way, before decoding the first flag from the bitstream, the decoding end first obtains the second flag from the decoded bitstream. If the second flag indicates that the first quantization parameter of at least one block in the current image is the second quantization parameter predicted by the quantization parameter prediction model, the decoding end continues to decode the bitstream to obtain the first flag, and determines the first quantization parameter of the current block based on the first flag.

[0203] In some embodiments, if the decoded second flag indicates that the first quantization parameter of the current image is not the second quantization parameter predicted by the quantization parameter prediction model, that is, if the second flag indicates that the first quantization parameters of all blocks in the current image are not the second quantization parameters predicted by the quantization parameter prediction model, the decoding end no longer decodes the first flag from the bitstream, but determines the preset basic quantization parameter as the first quantization parameter of the current block.

[0204] In some embodiments, when the value of the second flag is a third value, it indicates that the first quantization parameter of at least one block in the current image is the second quantization parameter predicted by the quantization parameter prediction model.

[0205] In some embodiments, when the value of the first flag is the fourth value, it indicates that the first quantization parameter of the current image is not the second quantization parameter predicted by the quantization parameter prediction model.

[0206] The embodiment of the present application does not limit the specific values ​​of the third and fourth values.

[0207] Optionally, the third value is 1.

[0208] Optionally, the fourth value is 0.

[0209] In an example, the second flag may be represented by a field picture_nnqp_enable_flag[compIdx], where compIdx represents the ordinal numbers of the three color components.

[0210] In a specific example, if picture_nnqp_enable_flag[compIdx] is 1, it indicates that the first quantization parameter of at least one block in the current image is the second quantization parameter predicted by the quantization parameter prediction model. At this time, the decoding end continues to decode the bit stream, obtains the first flag, and determines the first quantization parameter of the current block based on the first flag.

[0211] In another specific example, if picture_nnqp_enable_flag[compIdx] is 0, it indicates that the first quantization parameter of the current image is not the second quantization parameter predicted by the quantization parameter prediction model. At this time, the decoding end no longer decodes the bit stream to obtain the first flag, but directly determines the preset basic quantization parameter as the first quantization parameter of the current block.

[0212] In some embodiments, the code stream also includes a third flag at the frame level, which is used to indicate whether the current sequence is allowed to use the quantization parameter prediction model to predict the quantization parameter. That is, in an embodiment of the present application, if the encoding end determines that the current sequence allows the use of the quantization parameter prediction model to predict the quantization parameter, the third flag is set to yes, and the third flag set to yes is written into the code stream. If the encoding end determines that the current sequence does not allow the use of the quantization parameter prediction model to predict the quantization parameter, the third flag is set to no, and the third flag set to no is written into the code stream. Correspondingly, before decoding the code stream and obtaining the second flag, the decoding end first decodes the code stream to obtain the third flag, and based on the third flag, determines whether the current sequence allows the use of the quantization parameter prediction model to predict the quantization parameter. If the third flag indicates that the current sequence is allowed to use the quantization parameter prediction model to predict the quantization parameter, the decoding end continues to obtain the second flag.

[0213] In some embodiments, if the third flag indicates that the current sequence is not allowed to use the quantization parameter prediction model to predict the quantization parameter, the decoding end does not use the quantization parameter prediction model to predict the quantization parameter, that is, the decoding end no longer continues to decode the second flag or the first flag, but uses other methods to determine the first quantization parameter of the current block, for example, determining the preset quantization parameter as the first quantization parameter of the current block.

[0214] In some embodiments, when the value of the third flag is the fifth value, it indicates that the current sequence is allowed to use the quantization parameter prediction model to predict the quantization parameter.

[0215] In some embodiments, when the value of the third flag is the sixth value, it indicates that the current sequence is not allowed to use the quantization parameter prediction model to predict the quantization parameter.

[0216] The embodiment of the present application does not limit the specific values ​​of the fifth and sixth values.

[0217] Optionally, the fifth value is 1.

[0218] Optionally, the sixth value is 0.

[0219] In one example, the third flag may be represented by the field nnqp_enable_flag.

[0220] In a specific example, if nnqp_enable_flag is 1, it indicates that the current sequence is allowed to use the quantization parameter prediction model to predict the quantization parameter. At this time, the decoding end continues to decode the bit stream to obtain the second flag and / or the first flag.

[0221] In another specific example, if nnqp_enable_flag is 0, it indicates that the current sequence is allowed to use the quantization parameter prediction model to predict the quantization parameter. At this time, the decoding end no longer continues to decode the code stream to obtain the second flag and / or the first flag, but uses other methods to determine the first quantization coefficient of the current block, such as directly determining the preset basic quantization parameter as the first quantization parameter of the current block.

[0222] The embodiment of the present application does not limit the specific position of the third flag in the code stream.

[0223] In one example, the third flag is located in the sequence header. In this case, the sequence header including the third flag is defined as shown in Table 1:

[0224] Table 1

[0225]

[0226] Among them, nnqp_enable_flag is the third flag.

[0227] The embodiment of the present application does not limit the specific position of the second flag in the code stream.

[0228] In one example, the second flag is located in the image header. In this case, the image header including the second flag is defined as shown in Table 2:

[0229] Table 2

[0230]

[0231]

[0232] When the third flag nnqp_enable_flag is 1, the second flag picture_nnqp_enable_flag[compIdx] is defined.

[0233] The embodiment of the present application does not limit the specific position of the first flag in the code stream.

[0234] In one example, the first mark is located in the image block. In this case, the image block including the first mark is defined as shown in Table 3:

[0235] Table 3

[0236]

[0237] When the third flag nnqp_enable_flag is 1 and the second flag picture_nnqp_enable_flag[compIdx] is 1, the first flag patch_nnqp_enable_flag[compIdx][LcuIdx] is defined.

[0238] In the embodiment of the present application, after the decoding end determines the first quantization coefficient of the current block according to the above step S502, it performs the following step S503.

[0239] S503 : Filter the reconstructed block of the current block using a neural network-based filter according to the first quantization parameter to obtain a filtered reconstructed block.

[0240] From the above, it can be seen that the first quantization parameter of the embodiment of the present application is determined based on the second quantization parameter predicted by the quantization parameter prediction model. Compared with the trial method to determine the first quantization parameter from multiple candidate quantization parameters, the embodiment of the present application predicts the quantization parameter through the quantization parameter prediction model, which not only reduces the complexity of determining the quantization parameter, but also improves the speed of determining the quantization parameter, thereby improving the efficiency of encoding and decoding, and expands the optional range of the quantization parameter, and improves the accuracy of the quantization parameter. When filtering is performed based on the accurately determined quantization parameter, the filtering effect can be improved, thereby improving the video compression performance.

[0241] After the decoding end determines the first quantization parameter of the current block according to the above method, the reconstructed block of the current block is filtered using a neural network-based filter to obtain a filtered reconstructed block. Because the first quantization parameter of the current block determined in the embodiment of the present application is predicted based on the feature information of the reconstructed block, when the reconstructed block is filtered based on the first quantization parameter, the filtering effect of the reconstructed block can be improved.

[0242] The embodiment of the present application does not limit the specific implementation of the above S503.

[0243] In some embodiments, the decoding end inputs the reconstructed value of the current block and the first quantization parameter of the current block determined above into a neural network-based filter for filtering to obtain a filtered reconstructed block. For example, the decoding end concatenates the reconstructed value of the current block and the first quantization parameter of the current block, and then inputs the concatenated ...

[0244] In some embodiments, to further improve the filtering effect of the reconstructed block, in addition to inputting the first quantization parameter of the current block and the reconstructed block into the neural network-based filter, the decoder may also input other quantization parameters related to the current block. Exemplarily, the decoder may also input the slice-level quantization parameter SQP of the current image slice in which the current block is located and / or the basic quantization parameter of the reference block of the current block, along with the reconstructed value of the current block and the determined first quantization parameter of the current block, into the neural network-based filter for filtering, thereby obtaining a filtered reconstructed block.

[0245] In some embodiments, the reconstructed value of the current block is a reconstructed block that has not been filtered. For example, the reconstructed block of the current block is an image block obtained by adding a residual block of the current block to a prediction block.

[0246] In some embodiments, the decoding end may first use other filtering methods to filter the reconstructed value of the current block before filtering the reconstructed value using a neural network-based filter. In this case, the above-mentioned S503 includes: loop filtering the reconstructed block of the current block to obtain a reconstructed block after loop filtering; filtering the reconstructed block after loop filtering using a neural network-based filter according to the first quantization parameter to obtain a filtered reconstructed block. For example, DB and / or SAO are used to loop filter the reconstructed block of the current block to obtain a reconstructed block after loop filtering, and then the first quantization parameter and the reconstructed block after loop filtering are spliced, and then the filter based on the neural network is input to filter the reconstructed block after loop filtering to obtain a filtered reconstructed block.

[0247] In one possible implementation, as shown in FIG8 , the quantization parameter prediction model proposed in an embodiment of the present application can be located between the SAO and a neural network-based filter, such as a neural network-based loop filter (NNLN). Specifically, as shown in FIG8 , the reconstructed block of the current block is filtered by DB and SAO to obtain a loop-filtered reconstructed block. Next, the loop-filtered reconstructed block, the basic quantization coefficient BQP, and the slice-level quantization coefficient SQP are input into the quantization parameter prediction model, such as the neural network-based quantization parameter (NNQP), and the NNQP predicts the second quantization parameter of the current block. According to the above method, the first quantization parameter of the current block is determined from the second quantization parameter and the preset basic quantization parameter. Then, according to the determined first quantization parameter, the loop-filtered reconstructed block is filtered by the neural network-based filter NNLF to obtain a filtered reconstructed block. Optionally, the filtered reconstructed block can also be filtered again by the ALF.

[0248] The video decoding method provided in the embodiment of the present application comprises: a decoding end decoding a bitstream to determine a reconstructed block of a current block; decoding the bitstream to determine a first quantization parameter of the current block, where the first quantization parameter is determined based on a second quantization parameter predicted by a quantization parameter prediction model; and filtering the reconstructed block of the current block using a neural network-based filter based on the first quantization parameter to obtain a filtered reconstructed block. Specifically, the embodiment of the present application predicts the quantization parameter using a quantization parameter prediction model, thereby expanding the optional range of the quantization parameter and improving the accuracy of the quantization parameter. When filtering based on the accurately determined quantization parameter, the filtering effect can be improved, thereby enhancing video compression performance.

[0249] FIG9 is a schematic diagram of a video decoding process provided by an embodiment of the present application. The video decoding method shown in FIG9 can be understood as a specific and feasible embodiment of the video decoding method shown in FIG5 above. As shown in FIG9, the video decoding method provided by the embodiment of the present application includes:

[0250] S601: Decode the code stream to obtain a third flag.

[0251] The third flag is used to indicate whether the current sequence is allowed to use the quantization parameter prediction model to predict the quantization parameter.

[0252] S602: Determine whether the third flag is equal to 1.

[0253] In some embodiments, if the third flag = 1, indicating that the current sequence is allowed to use the quantization parameter prediction model to predict the quantization parameter, the following S603 is executed.

[0254] In some embodiments, if the third flag = 0, indicating that the current sequence is not allowed to use the quantization parameter prediction model to predict the quantization parameter, the following S609 is executed.

[0255] S603: Decode the code stream to obtain a second flag.

[0256] That is, in the embodiment of the present application, if the third flag indicates that the current sequence is allowed to use the quantization parameter prediction model to predict the quantization parameter, the decoding end continues to decode the code stream to obtain the second flag.

[0257] The second flag is used to indicate whether the first quantization parameter of at least one block in the current image is the second quantization parameter predicted by the quantization parameter prediction model.

[0258] S604: Determine whether the second flag is equal to 1.

[0259] In some embodiments, if the second flag = 1, indicating that the first quantization parameter of at least one block in the current image is the second quantization parameter predicted by the quantization parameter prediction model, the following S605 is executed.

[0260] In some embodiments, if the second flag = 0, indicating that the first quantization parameter of the current image is the second quantization parameter predicted by the quantization parameter prediction model, the following S609 is executed.

[0261] S605: Decode the code stream to obtain a first flag.

[0262] That is, in the embodiment of the present application, if the second flag indicates that the first quantization parameter of at least one block in the current image is the second quantization parameter predicted by the quantization parameter prediction model, the code stream is decoded to obtain the first flag.

[0263] The first flag is used to indicate whether the first quantization parameter of the current block is the second quantization parameter predicted by the quantization parameter prediction model.

[0264] S606: Determine whether the first flag is equal to 1.

[0265] In some embodiments, if the first flag = 1, it indicates that the first quantization parameter of the current block is the second quantization parameter predicted by the quantization parameter prediction model, and the following S607 is executed.

[0266] In some embodiments, if the first flag = 0, it indicates that the first quantization parameter of the current block is not the second quantization parameter predicted by the quantization parameter prediction model, and the following S609 is executed.

[0267] S607 : Predict a second quantization parameter of the current block using a quantization parameter prediction model.

[0268] For example, the reconstructed block of the current block and at least one preset third quantization parameter are input into a quantization parameter prediction model to perform quantization parameter prediction to obtain a second quantization parameter.

[0269] In an example, after the reconstructed block of the current block and at least one third quantization parameter are spliced, the concatenation is input into a quantization parameter prediction model to perform quantization parameter prediction to obtain the second quantization parameter.

[0270] Optionally, the at least one third quantization parameter includes at least one of a basic quantization parameter, a slice-level quantization parameter, and a reference block quantization parameter of the current block.

[0271] S608: Determine the second quantization parameter as the first quantization parameter.

[0272] After executing the above S608, execute the following step S611.

[0273] S609: Determine a preset basic quantization parameter as a first quantization parameter of the current block.

[0274] After executing the above S609, execute the following step S610.

[0275] S610 : Filter a reconstructed block of the current block using a neural network-based filter according to a first quantization parameter to obtain a filtered reconstructed block.

[0276] In some embodiments, loop filtering is performed on a reconstructed block of the current block to obtain a loop-filtered reconstructed block; and the loop-filtered reconstructed block is filtered using a neural network-based filter according to a first quantization parameter to obtain a filtered reconstructed block.

[0277] In an embodiment of the present application, the decoding end can accurately determine which method to use to determine the first quantization parameter by decoding the first flag, the second flag, and the third flag. For example, if it is determined that the first quantization parameter of the current block is the second quantization parameter predicted by the quantization parameter prediction model, the decoding end predicts the first quantization parameter of the current block through the quantization parameter prediction model. If it is determined that the first quantization parameter of the current block is not the second quantization parameter predicted by the quantization parameter prediction model, the decoding end can directly determine the preset basic quantization parameter as the first quantization parameter of the current block, thereby reducing the accuracy and computational complexity of the decoding end in determining the first quantization parameter of the current block, thereby improving the filtering efficiency of the decoding end.

[0278] The above describes the video decoding method involved in the embodiment of the present application. On this basis, the following describes the video encoding method involved in the present application for the encoding end.

[0279] Figure 10 is a flow chart of a video encoding method according to an embodiment of the present application. The execution subject of the embodiment of the present application may be the encoder shown in Figure 1 or Figure 2 above.

[0280] As shown in FIG10 , the method of the embodiment of the present application includes:

[0281] S701: Reconstruct the current block to obtain a reconstructed block of the current block.

[0282] In some embodiments, the current block is also referred to as a current CU, a current image block, a current decoding block, a current decoding unit, a current block to be decoded, a current image block to be decoded, etc.

[0283] In some embodiments, the current block of the embodiment of the present application only includes chrominance components and can be understood as a chrominance block.

[0284] In some embodiments, the current block of the embodiment of the present application only includes a luminance component and can be understood as a luminance block.

[0285] In some embodiments, the current block includes both luma and chroma components.

[0286] During video encoding, the current image is divided into blocks to obtain multiple coding blocks. A prediction method, such as inter-frame or intra-frame prediction, is used to predict the current block, resulting in a prediction block for the current block. Based on the prediction block and the current block, a residual block for the current block is determined. For example, the difference between the current block and the prediction block is used as the residual block for the current block. The residual block for the current block is transformed to obtain transform coefficients for the current block.

[0287] In some embodiments, the encoding end may not quantize the transform coefficients, but directly encode the transform coefficients to obtain a bit stream.

[0288] In some embodiments, the encoding end quantizes the transform coefficients to obtain quantized coefficients of the current block, and encodes the quantized coefficients to obtain a bitstream.

[0289] In addition to generating a bitstream, the encoder also includes a decoding part. In some embodiments, the encoder performs an inverse transform on the transform coefficients of the current block to obtain a residual block of the current block, and adds the residual block to the prediction block to obtain a reconstructed block of the current block.

[0290] In some embodiments, the encoding end inversely quantizes the quantization coefficients of the current block to obtain the transformation coefficients of the current block, then inversely transforms the transformation coefficients of the current block to obtain the residual block of the current block, and adds the residual block to the prediction block to obtain the reconstructed block of the current block.

[0291] S702 : Predict a second quantization parameter of the current block using a quantization parameter prediction model.

[0292] In the embodiment of the present application, in order to improve the accuracy of the quantization parameter, the quantization parameter is predicted through a quantization parameter prediction mode, so as to expand the optional range of the quantization parameter and thereby improve the accuracy of determining the quantization parameter.

[0293] The quantization parameter prediction model of the embodiment of the present application can predict a second quantization parameter that meets the feature information of the current block based on the feature information of the current block, so that when subsequent image filtering is performed based on the second quantization parameter, the effect of image filtering can be improved.

[0294] In some embodiments, the quantization parameter prediction model is trained using N preset candidate quantization parameters as labels. Specifically, the quantization parameter prediction model is trained iteratively and its parameters updated using the cross-entropy loss function, using the N candidate quantization parameters as labels and following the principles of classification network training. This trained quantization parameter prediction model can accurately predict the second quantization parameter of the current block based on the image feature information of the current block.

[0295] In some embodiments, the second quantization parameter of the current block predicted by the quantization parameter prediction model of the embodiment of the present application is the candidate quantization parameter with the highest probability among the preset N candidate quantization parameters. In other words, the quantization parameter prediction model of the embodiment of the present application can predict the probability of selection of each candidate quantization parameter among the N candidate quantization parameters, and determine the candidate quantization parameter with the highest probability of selection among the N candidate quantization parameters as the second quantization parameter of the current block.

[0296] In some embodiments, in order to further improve the prediction accuracy of the quantization parameter prediction model, the embodiment of the present application can expand the number of candidate quantization parameters in the N candidate quantization parameters, so that the quantization parameter prediction model can predict the second quantization parameter for the current block from more candidate quantization parameters, thereby improving the prediction accuracy of the second quantization parameter.

[0297] In some embodiments, in the embodiments of the present application, the number of the above-mentioned N candidate quantization parameters is greater than the number of candidate quantization parameters corresponding to the basic quantization parameter.

[0298] For example, the number of candidate quantization parameters corresponding to the basic quantization parameter is usually 3, such as {BQP, BQP-5, BQP+5} or {BQP, BQP-5, BQP-10}, etc. However, in the embodiment of the present application, the number of N candidate quantization parameters is greater than 3.

[0299] In one example, the N candidate quantization parameters in the embodiment of the present application are obtained by expanding the basic quantization parameters, for example, {BQP-floor(N / 2), ..., BQP+floor(N / 2)}. Assuming that N is 11, the N candidate quantization parameters in the embodiment of the present application are {BQP-5, BQP-4, ..., BQP, ..., BQP+4, BQP+5}.

[0300] In an embodiment of the present application, a quantization parameter prediction model can accurately select one candidate quantization parameter from the above-mentioned N candidate quantization parameters as the second quantization parameter of the current block. When image filtering is performed based on the accurate second quantization parameter, the filtering effect can be improved.

[0301] The embodiments of the present application do not limit the specific network structure of the quantization parameter prediction model.

[0302] In some embodiments, the quantization parameter prediction model includes multiple convolutional layers and at least one fully connected layer. For example, the quantization parameter prediction model includes three convolutional layers and two fully connected layers. Exemplarily, the two fully connected layers are located after the three convolutional layers.

[0303] In some embodiments, the quantization parameter prediction model further includes a plurality of pooling layers, which are used to downsample the feature dimensions of the feature information output by the convolution layer.

[0304] Optionally, at least one pooling layer among the multiple pooling layers is a maximum pooling layer.

[0305] Optionally, at least one pooling layer among the multiple pooling layers is an average pooling layer.

[0306] In the embodiment of the present application, there is no limitation on the specific connection method between the above-mentioned multiple convolutional layers and the above-mentioned multiple pooling layers.

[0307] In one possible implementation, each of the multiple convolutional layers is connected to a pooling layer, that is, the number of convolutional layers included in the above-mentioned quantization parameter prediction model is the same as the number of pooling layers, and one convolutional layer is connected to a pooling layer.

[0308] In some embodiments, the last layer of the quantitative parameter prediction model is SoftMax.

[0309] Figure 6 is a schematic diagram of a quantization parameter prediction model provided by an embodiment of the present application. As shown in Figure 6, the quantization parameter prediction model of the embodiment of the present application includes 3 layers of convolution layers Conv, 3 layers of maximum pooling layers MaxPooling, 2 fully connected layers FullConnect, 1 nonlinear activation function PReLU and a SoftMax layer. Among them, a maximum pooling layer is connected after a convolution layer, a fully connected layer is connected after the last maximum pooling layer, a nonlinear activation function PReLU is connected between the two fully connected layers, and the last layer is a SoftMax layer. It should be noted that Figure 6 is only a quantization parameter prediction model shown in the embodiment of the present application. The network structure of the quantization parameter prediction model of the embodiment of the present application includes but is not limited to that shown in Figure 6. In addition, the number of layers in the quantization parameter prediction model includes but is not limited to the number shown in Figure 6. For example, the quantization parameter prediction model can include 2, 3, 4, 5 and other layers of convolution layers.

[0310] As shown in Figure 6, the encoder inputs the reconstructed block of the current block into the quantization parameter prediction model shown in Figure 6. This quantization parameter prediction model processes the block through three convolutional layers and three layers of maximum pooling to obtain the feature information of the current block. The feature information of the current block then passes through two fully connected layers and one layer of nonlinear activation function PReLU, and finally passes through a softmax layer, outputting a probability distribution of N candidate quantization parameters. Based on this probability distribution, the second quantization parameter of the current block is obtained.

[0311] It should be noted that the embodiment of the present application does not limit the specific method of predicting the second quantization parameter of the current block through the quantization parameter prediction model in the above S702.

[0312] In some embodiments, the encoder inputs the reconstructed block of the current block into a quantization parameter prediction model to perform quantization parameter prediction to obtain a second quantization parameter of the current block.

[0313] In some embodiments, in order to improve the prediction accuracy of the quantization parameter prediction model, the encoding end inputs the reconstructed block of the current block into the quantization parameter prediction model, and also inputs at least one preset third quantization parameter into the quantization parameter prediction model for quantization parameter prediction to obtain the second quantization parameter of the current block.

[0314] Specifically, the encoder fuses the reconstructed block of the current block and at least one third quantization parameter, and inputs the quantization parameter into a quantization parameter prediction model to perform quantization parameter prediction to obtain the second quantization parameter.

[0315] The embodiment of the present application does not limit the specific method of fusing the reconstructed block of the current block and at least one third quantization parameter. For example, the encoding end splices the reconstructed block of the current block and at least one third quantization parameter, and inputs them into the quantization parameter prediction model to perform quantization parameter prediction to obtain the second quantization parameter.

[0316] The embodiment of the present application does not limit the specific method of the third quantization parameter.

[0317] In some embodiments, the at least one third quantization parameter includes at least one of a basic quantization parameter, a slice-level quantization parameter, and a reference block quantization parameter of the current block.

[0318] In one example, the at least one third quantization parameter includes a basic quantization parameter (BQP) and a slice-level quantization parameter (SQP). In this case, the encoder inputs the reconstructed block of the current block, the basic quantization parameter (BQP), and the slice-level quantization parameter (SQP) into the quantization parameter prediction model to predict the second quantization parameter for the current block. For example, as shown in Figure 7, the reconstructed block of the current block, the basic quantization parameter (BQP), and the slice-level quantization parameter (SQP) are concatenated and input into the quantization parameter prediction model to predict the second quantization parameter for the current block.

[0319] According to the above method, after the second quantization parameter of the current block is predicted by the quantization parameter prediction model, the following step S703 is performed.

[0320] S703: Determine the first quantization parameter according to the second quantization parameter.

[0321] In the embodiment of the present application, specific implementation methods for determining the first quantization parameter based on the second quantization parameter include but are not limited to the following:

[0322] Method 1: Since the above-mentioned second quantization parameter is obtained by the quantization parameter prediction model based on the reconstructed block prediction of the current block and conforms to the characteristic information of the current block, the encoding end can directly determine the second quantization parameter of the current block obtained by the above prediction as the first quantization parameter of the current block.

[0323] Mode 2: To further improve the accuracy of determining the first quantization parameter, the above S703 includes the following step S703-A:

[0324] S703-A: Determine a first quantization parameter according to the second quantization parameter and a preset basic quantization parameter.

[0325] The preset basic quantization parameter may be understood as a quantization parameter selected from candidate quantization parameters corresponding to the basic quantization parameter.

[0326] Exemplarily, the candidate quantization parameters corresponding to the basic quantization parameter include at least one of 22, 27, 32, 37, 42, etc.

[0327] In this approach 2, the encoder determines the first quantization parameter of the current block by using the predicted second quantization parameter of the current block and a preset basic quantization parameter, thereby improving the accuracy of determining the first quantization parameter.

[0328] The embodiment of the present application does not limit the specific method of determining the first quantization parameter according to the second quantization parameter and the preset basic quantization parameter in the above S703-A.

[0329] In one possible implementation of S703-A, the predicted second quantization parameter of the current block and any one of the preset basic quantization parameters may be determined as the first quantization parameter of the current block. For example, the encoder determines the minimum quantization parameter of the second quantization parameter of the current block and the basic quantization parameter as the first quantization parameter of the current block. Alternatively, the encoder determines the cost of the second quantization parameter and the basic quantization parameter of the current block, respectively, and determines the quantization parameter with the minimum cost between the second quantization parameter and the basic quantization parameter as the first quantization parameter of the current block.

[0330] In another possible implementation of S703-A, the arithmetic mean or weighted mean of the predicted second quantization parameter of the current block and the preset basic quantization parameter may be determined as the first quantization parameter of the current block.

[0331] In another possible implementation of the above S703-A, the encoder determines the first quantization parameter of the current block from the second quantization parameter and the basic quantization parameter according to the distortion. Specifically, the above S703-A includes the following steps S703-A1 to S703-A3:

[0332] S703-A1. Determine distortion corresponding to the second quantization parameter;

[0333] S703-A2, determining the distortion corresponding to the basic quantization parameter;

[0334] S703-A3: Determine a first quantization parameter from the second quantization parameter and the basic quantization parameter according to the distortion corresponding to the second quantization parameter and the distortion corresponding to the basic quantization parameter.

[0335] In this implementation, the distortion corresponding to the second quantization parameter and the distortion corresponding to the basic quantization parameter are determined respectively, so that the first quantization parameter is determined from the second quantization parameter and the basic quantization parameter according to the distortion.

[0336] The specific process of determining the distortion corresponding to the second quantization parameter and the distortion corresponding to the basic quantization parameter is described below. It should be noted that the method of determining the distortion corresponding to the second quantization parameter and the distortion corresponding to the basic quantization parameter can be the same or different, and the embodiments of the present application do not limit this.

[0337] This embodiment of the present application does not limit the method for determining the distortion corresponding to the second quantization parameter in the above S703-A1.

[0338] In one example, the encoding end may filter the prediction block of the current block through a neural network-based filter according to the second quantization parameter to obtain a filtered prediction block, compare the filtered prediction value with the prediction value of the current block to obtain the distortion corresponding to the second quantization parameter, for example, determine the difference between the filtered prediction block and the prediction block of the current block as the distortion corresponding to the second quantization parameter of the current block.

[0339] In another example, the encoder filters the reconstructed block of the current block using a neural network-based filter based on the second quantization parameter to obtain a filtered first reconstructed block. The encoder then determines the distortion corresponding to the second quantization parameter based on the filtered first reconstructed block and the reconstructed block of the current block. For example, the second quantization parameter of the current block and the reconstructed block of the current block are concatenated and then input into the neural network-based filter for filtering to obtain the filtered first reconstructed block. The difference between the filtered first reconstructed block and the reconstructed block of the current block is then determined as the distortion corresponding to the second quantization coefficient.

[0340] The embodiment of the present application does not limit the method of determining the distortion corresponding to the basic quantization parameter in the above S703-A2.

[0341] In one example, the encoder may filter the prediction block of the current block using a neural network-based filter based on the basic quantization parameter to obtain a filtered prediction block, and compare the filtered prediction value with the prediction value of the current block to obtain the distortion corresponding to the basic quantization parameter. For example, the difference between the filtered prediction block and the prediction block of the current block is determined as the distortion corresponding to the basic quantization parameter of the current block.

[0342] In another example, the encoder filters the reconstructed block of the current block using a neural network-based filter based on the basic quantization parameter to obtain a filtered second reconstructed block. The distortion corresponding to the basic quantization parameter is determined based on the filtered second reconstructed block and the reconstructed block of the current block. For example, the basic quantization parameter of the current block and the reconstructed block of the current block are concatenated and then input into the neural network-based filter for filtering to obtain the filtered second reconstructed block. The difference between the filtered second reconstructed block and the reconstructed block of the current block is then determined as the distortion corresponding to the basic quantization coefficient.

[0343] According to the above method, the distortion corresponding to the second quantization coefficient and the distortion corresponding to the basic quantization coefficient are determined, and then, based on the distortion, the first quantization coefficient of the current block is determined from the second quantization coefficient of the current block and the preset basic quantization coefficient. For example, the quantization parameter with the smallest distortion between the second quantization parameter and the basic quantization parameter is determined as the first quantization parameter of the current block, thereby accurately determining the first quantization coefficient. In this way, when performing image filtering on a reconstructed block of the current block based on the accurately determined first quantization coefficient, the filtering effect of the reconstructed block can be improved.

[0344] In the embodiment of the present application, the encoding end determines the first quantization coefficient of the current block through the second quantization parameter predicted by the quantization parameter prediction model, thereby achieving rapid and accurate determination of the first quantization parameter.

[0345] In some embodiments, after the encoder determines the first quantization parameter of the current block according to the above method, the embodiment of the present application further performs the following step 1:

[0346] Step 1: Determine a first flag according to a first quantization parameter, where the first flag is used to indicate whether the first quantization parameter of the current block is a second quantization parameter predicted by a quantization parameter prediction model.

[0347] In some embodiments, when the encoder determines that the first quantization parameter of the current block is the second quantization parameter predicted by the quantization parameter prediction model, the encoder sets the first flag to yes and writes the set first flag into the bitstream.

[0348] In some embodiments, if the encoder determines that the first quantization parameter of the current block is a preset basic quantization parameter, the encoder sets the first flag to false and writes the false first flag into the bitstream.

[0349] In this way, after receiving the bitstream, the decoder can determine the first quantization coefficient of the current block by decoding the bitstream. For example, the decoder decodes the bitstream and obtains a first flag. If the first flag indicates that the first quantization coefficient of the current block is the second quantization parameter predicted by the quantization parameter prediction model, the decoder predicts the second quantization parameter of the current block using the quantization parameter prediction model and determines the predicted second quantization parameter as the first quantization parameter of the current block. If the decoder decodes the bitstream and determines that the first flag indicates that the first quantization parameter of the current block is not the second quantization parameter predicted by the quantization parameter prediction model, the decoder may determine the preset basic quantization parameter as the first quantization parameter of the current block.

[0350] As can be seen from the above, in the embodiment of the present application, the encoding end accurately indicates whether the first quantization parameter of the current block is the second quantization parameter predicted by the quantization parameter prediction model through the first flag at the block level, so that the decoding end can accurately determine which method to use to determine the first quantization parameter based on the first flag. For example, when the first flag indicates that the first quantization parameter of the current block is the second quantization parameter predicted by the quantization parameter prediction model, the decoding end predicts the first quantization parameter of the current block through the quantization parameter prediction model. If the first flag indicates that the first quantization parameter of the current block is not the second quantization parameter predicted by the quantization parameter prediction model, the decoding end directly determines the preset basic quantization parameter as the first quantization parameter of the current block. The decoding end does not need to perform an attempt to compare, thereby reducing the amount of calculation for the decoding end to determine the first quantization parameter of the current block, thereby improving the filtering efficiency of the decoding end.

[0351] In some embodiments, when the value of the first flag is a first numerical value, it indicates that the first quantization parameter of the current block is the second quantization parameter predicted by the quantization parameter prediction model.

[0352] In some embodiments, when the value of the first flag is the second value, it indicates that the first quantization parameter of the current block is not the second quantization parameter predicted by the quantization parameter prediction model.

[0353] The embodiment of the present application does not limit the specific values ​​of the first and second values.

[0354] Optionally, the first value is 1.

[0355] Optionally, the second value is 0.

[0356] In an example, the first flag may be represented by the field patch_nnqp_enable_flag[compIdx][LcuIdx], where compIdx represents the sequence numbers of three color components, and ctuIdx represents the sequence number of the CTU.

[0357] In a specific example, if patch_nnqp_enable_flag[compIdx][LcuIdx] is 1, it indicates that the first quantization parameter of the current block is the second quantization parameter predicted by the quantization parameter prediction model. At this time, the decoding end predicts the second quantization parameter of the current block through the quantization parameter prediction model.

[0358] For example, when patch_nnqp_enable_flag[compIdx][LcuIdx]=1, the decoding end inputs the reconstructed block of the current block into the quantization parameter prediction model to perform quantization parameter prediction to obtain the second quantization parameter of the current block.

[0359] In some embodiments, the present application embodiment further includes the following step 2:

[0360] Step 2: Determine a second flag according to the first flag of the current block, where the second flag is used to indicate whether the first quantization parameter of at least one block in the current image is the second quantization parameter predicted by the quantization parameter prediction model.

[0361] In this embodiment, the encoding end further indicates, through a second flag at the frame level, whether the first quantization parameter of at least one block in the current image is the second quantization parameter predicted by the quantization parameter prediction model.

[0362] That is, when determining the first quantization parameter of each coding block in the current image, the encoder not only determines the first flag but also determines the second flag. Specifically, when determining the first quantization parameter of each coding block in the current image, for each coding block, if the encoder determines that the first quantization parameter of the coding block is the second quantization parameter preset by the quantization parameter prediction model, the first flag corresponding to the coding block is set to yes; if the encoder determines that the first quantization parameter of the coding block is not the second quantization parameter preset by the quantization parameter prediction model, the first flag corresponding to the coding block is set to no. Referring to this method, the encoder can determine the first flag of each coding block in the current image.

[0363] Furthermore, the encoding end determines the second flag of the current image according to the first flag of each coding block in the current block.

[0364] In one example, if the value of the first flag is a first numerical value, the value of the second flag is set to a third numerical value, and the third numerical value is used to indicate that the first quantization parameter of at least one block in the current image is the second quantization parameter predicted by the quantization parameter prediction model.

[0365] In one example, if the value of the first flag is the second value, and the values ​​of the first flags of other coding blocks in the current image except the current block are all second values, the value of the second flag is set to a fourth value, and the fourth value is used to indicate that the first quantization parameter of the current image is not the second quantization parameter predicted by the quantization parameter prediction model.

[0366] Specifically, if the first flags of all coding blocks in the current image are all false, for example, all 0, meaning that the first quantization parameters of all coding blocks in the current image are not the second quantization parameters predicted by the quantization parameter prediction model, the encoder sets the second flag of the current image to false, for example, 0, and writes the false second flag into the bitstream. If the first flag of at least one coding block in the current image is true, for example, the first flag of at least one coding block in the current image is 1, meaning that the first quantization parameter of at least one coding block in the current image is the second quantization parameter predicted by the quantization parameter prediction model, the encoder sets the second flag of the current image to true, for example, 1, and writes the true second flag into the bitstream. Thus, before decoding the first flag from the bitstream, the decoder first obtains the second flag from the decoded bitstream. If the second flag indicates that the first quantization parameter of at least one block in the current image is the second quantization parameter predicted by the quantization parameter prediction model, the decoder continues decoding the bitstream, obtains the first flag, and determines the first quantization parameter of the current block based on the first flag.

[0367] The embodiment of the present application does not limit the specific values ​​of the third and fourth values.

[0368] Optionally, the third value is 1.

[0369] Optionally, the fourth value is 0.

[0370] In an example, the second flag may be represented by a field picture_nnqp_enable_flag[compIdx], where compIdx represents the ordinal numbers of the three color components.

[0371] In a specific example, if picture_nnqp_enable_flag[compIdx] is 1, it indicates that the first quantization parameter of at least one block in the current picture is the second quantization parameter predicted by the quantization parameter prediction model.

[0372] In another specific example, if picture_nnqp_enable_flag[compIdx] is 0, it indicates that the first quantization parameter of the current image is not the second quantization parameter predicted by the quantization parameter prediction model.

[0373] In some embodiments, before executing S702, the encoder first obtains a third flag indicating whether the quantization parameter prediction model is allowed to predict the quantization parameter of the current sequence. If the third flag indicates that the quantization parameter prediction model is allowed to predict the quantization parameter of the current sequence, the encoder executes S702 and predicts the second quantization parameter of the current block using the quantization parameter prediction model.

[0374] In some embodiments, the encoder writes a third flag into the bitstream. Specifically, if the encoder determines that the current sequence allows the use of a quantization parameter prediction model to predict quantization parameters, the encoder sets the third flag to yes and writes the third flag set to yes into the bitstream. If the encoder determines that the current sequence does not allow the use of a quantization parameter prediction model to predict quantization parameters, the encoder sets the third flag to no and writes the third flag set to no into the bitstream. Correspondingly, before decoding the bitstream and obtaining the second flag, the decoder first decodes the bitstream to obtain the third flag, and based on the third flag, determines whether the current sequence allows the use of a quantization parameter prediction model to predict quantization parameters. If the third flag indicates that the current sequence allows the use of a quantization parameter prediction model to predict quantization parameters, the decoder continues to decode the bitstream to obtain the second flag.

[0375] In some embodiments, when the value of the third flag is the fifth value, it indicates that the current sequence is allowed to use the quantization parameter prediction model to predict the quantization parameter.

[0376] In some embodiments, when the value of the third flag is the sixth value, it indicates that the current sequence is not allowed to use the quantization parameter prediction model to predict the quantization parameter.

[0377] The embodiment of the present application does not limit the specific values ​​of the fifth and sixth values.

[0378] Optionally, the fifth value is 1.

[0379] Optionally, the sixth value is 0.

[0380] In one example, the third flag may be represented by the field nnqp_enable_flag.

[0381] In a specific example, if nnqp_enable_flag is 1, it indicates that the current sequence is allowed to use the quantization parameter prediction model to predict the quantization parameter. At this time, the decoding end continues to decode the bit stream to obtain the second flag and / or the first flag.

[0382] In another specific example, if nnqp_enable_flag is 0, it indicates that the current sequence is allowed to use the quantization parameter prediction model to predict the quantization parameter. At this time, the decoding end no longer continues to decode the code stream to obtain the second flag and / or the first flag, but uses other methods to determine the first quantization coefficient of the current block, such as directly determining the preset basic quantization parameter as the first quantization parameter of the current block.

[0383] The embodiment of the present application does not limit the specific position of the third flag in the code stream.

[0384] In an example, the third flag is located in the sequence header. In this case, the sequence header including the third flag is defined as shown in Table 1.

[0385] The embodiment of the present application does not limit the specific position of the second flag in the code stream.

[0386] In one example, the second flag is located in the image header. In this case, the image header including the second flag is defined as shown in Table 2.

[0387] The embodiment of the present application does not limit the specific position of the first flag in the code stream.

[0388] In an example, the first mark is located in the image block. In this case, the image block including the first mark is defined as shown in Table 3.

[0389] In the embodiment of the present application, after the decoding end determines the first quantization coefficient of the current block according to the above step S703, it performs the following step S703.

[0390] S704 : Filter the reconstructed block of the current block using a neural network-based filter according to the first quantization parameter to obtain a filtered reconstructed block.

[0391] From the above, it can be seen that the first quantization parameter of the embodiment of the present application is determined based on the second quantization parameter predicted by the quantization parameter prediction model. Compared with the trial method to determine the first quantization parameter from multiple candidate quantization parameters, the embodiment of the present application predicts the quantization parameter through the quantization parameter prediction model, which not only reduces the complexity of determining the quantization parameter, but also increases the speed of determining the quantization parameter, thereby improving the efficiency of encoding and decoding, and expands the optional range of the quantization parameter, thereby improving the accuracy of the quantization parameter. When filtering is performed based on the accurately determined quantization parameter, the filtering effect can be improved, thereby improving the video compression performance.

[0392] After the encoder determines the first quantization parameter of the current block according to the above method, it filters the reconstructed block of the current block using a neural network-based filter to obtain a filtered reconstructed block. Because the first quantization parameter of the current block determined in the embodiment of the present application is predicted based on the feature information of the reconstructed block, filtering the reconstructed block based on the first quantization parameter can improve the filtering effect of the reconstructed block.

[0393] The embodiment of the present application does not limit the specific implementation of the above S704.

[0394] In some embodiments, the encoder inputs the reconstructed value of the current block and the first quantization parameter of the current block determined above into a neural network-based filter for filtering to obtain a filtered reconstructed block. For example, the encoder concatenates the reconstructed value of the current block and the first quantization parameter of the current block, and then inputs the concatenated ...

[0395] In some embodiments, to further improve the filtering effect of the reconstructed block, the encoder may input other quantization parameters related to the current block in addition to inputting the first quantization parameter of the current block and the reconstructed block into the neural network-based filter. Exemplarily, the encoder may also input the slice-level quantization parameter SQP of the current image slice in which the current block is located and / or the basic quantization parameter of the reference block of the current block, along with the reconstructed value of the current block and the determined first quantization parameter of the current block, into the neural network-based filter for filtering, thereby obtaining a filtered reconstructed block.

[0396] In some embodiments, the reconstructed value of the current block is a reconstructed block that has not been filtered. For example, the reconstructed block of the current block is an image block obtained by adding a residual block of the current block to a prediction block.

[0397] In some embodiments, the encoding end may first use other filtering methods to filter the reconstructed value of the current block before using the neural network-based filter to filter the reconstructed value. In this case, the above S704 includes: loop filtering the reconstructed block of the current block to obtain a reconstructed block after loop filtering; filtering the reconstructed block after loop filtering through a neural network-based filter according to the first quantization parameter to obtain a filtered reconstructed block. For example, DB and / or SAO are used to loop filter the reconstructed block of the current block to obtain a reconstructed block after loop filtering, and then the first quantization parameter and the reconstructed block after loop filtering are spliced, and then the filter based on the neural network is input to filter the reconstructed block after loop filtering to obtain a filtered reconstructed block.

[0398] In one possible implementation, as shown in FIG8 , the quantization parameter prediction model proposed in the embodiment of the present application can be located between SAO and the neural network-based filter NNLN. Specifically, as shown in FIG8 , the reconstructed block of the current block is filtered by DB and SAO to obtain a reconstructed block after loop filtering. Then, the reconstructed block after loop filtering and the basic quantization coefficient BQP and the slice-level quantization coefficient SQP are input into the quantization parameter prediction model NNQP, and NNQP predicts the second quantization parameter of the current block. According to the above method, the first quantization parameter of the current block is determined from the second quantization parameter and the preset basic quantization parameter. Then, according to the determined first quantization parameter, the reconstructed block after loop filtering is filtered by the neural network-based filter NNLF to obtain a filtered reconstructed block. Optionally, the filtered reconstructed block can also be filtered again by ALF.

[0399] In the video encoding method provided in the embodiments of the present application, the encoding end reconstructs the current block to obtain a reconstructed block of the current block; predicts a second quantization parameter of the current block using a quantization parameter prediction model; determines a first quantization parameter of the current block based on the second quantization parameter; and filters the reconstructed block of the current block using a neural network-based filter based on the first quantization parameter to obtain a filtered reconstructed block. Specifically, the embodiments of the present application predict the quantization parameter using the quantization parameter prediction model, thereby expanding the optional range of the quantization parameter and improving the accuracy of the quantization parameter. When filtering based on the accurately determined quantization parameter, the filtering effect can be improved, thereby enhancing video compression performance.

[0400] FIG11 is a schematic diagram of a video encoding process according to an embodiment of the present application. The video encoding method shown in FIG11 can be understood as a specific and feasible embodiment of the video encoding method shown in FIG10 above. As shown in FIG11 , the video encoding method provided in the embodiment of the present application includes:

[0401] S801: Reconstruct the current block to obtain a reconstructed block of the current block.

[0402] In some embodiments, the encoding end performs an inverse transform on the transform coefficients of the current block to obtain a residual block of the current block, and adds the residual block to the prediction block to obtain a reconstructed block of the current block.

[0403] In some embodiments, the encoding end inversely quantizes the quantization coefficients of the current block to obtain the transformation coefficients of the current block, then inversely transforms the transformation coefficients of the current block to obtain the residual block of the current block, and adds the residual block to the prediction block to obtain the reconstructed block of the current block.

[0404] S802: Decode the code stream to obtain a third flag.

[0405] The third flag is used to indicate whether the current sequence is allowed to use the quantization parameter prediction model to predict the quantization parameter.

[0406] S803: Determine whether the third flag is equal to 1.

[0407] In some embodiments, if the third flag = 1, it indicates that the current sequence is allowed to use the quantization parameter prediction model to predict the quantization parameter, and the following S804 is executed.

[0408] In some embodiments, if the third flag = 0, indicating that the current sequence is not allowed to use the quantization parameter prediction model to predict the quantization parameter, the following S810 is executed.

[0409] S803 : Predict a second quantization parameter of the current block using a quantization parameter prediction model.

[0410] For example, the reconstructed block of the current block and at least one preset third quantization parameter are input into a quantization parameter prediction model to perform quantization parameter prediction to obtain a second quantization parameter.

[0411] In an example, after the reconstructed block of the current block and at least one third quantization parameter are spliced, the concatenation is input into a quantization parameter prediction model to perform quantization parameter prediction to obtain the second quantization parameter.

[0412] Optionally, the at least one third quantization parameter includes at least one of a basic quantization parameter, a slice-level quantization parameter, and a reference block quantization parameter of the current block.

[0413] The specific implementation process of the above S803 refers to the description of the above S702 and will not be repeated here.

[0414] S804: Determine the first quantization parameter according to the second quantization parameter.

[0415] For example, the first quantization parameter is determined according to the second quantization parameter and a preset basic quantization parameter.

[0416] For example, the distortion corresponding to the second quantization parameter is determined; the distortion corresponding to the basic quantization parameter is determined; and the first quantization parameter is determined from the second quantization parameter and the basic quantization parameter according to the distortion corresponding to the second quantization parameter and the distortion corresponding to the basic quantization parameter.

[0417] For example, the quantization parameter with the least distortion between the second quantization parameter and the basic quantization parameter is determined as the first quantization parameter of the current block, thereby accurately determining the first quantization coefficient. In this way, when the reconstructed block of the current block is image filtered based on the accurately determined first quantization coefficient, the filtering effect of the reconstructed block can be improved.

[0418] The specific implementation process of the above S804 refers to the description of the above S703 and will not be repeated here.

[0419] S805: Determine a first quantization parameter of the current block using the preset basic quantization parameter.

[0420] S806 : Filter the reconstructed block of the current block using a neural network-based filter according to the first quantization parameter to obtain a filtered reconstructed block.

[0421] In some embodiments, loop filtering is performed on a reconstructed block of the current block to obtain a loop-filtered reconstructed block; and the loop-filtered reconstructed block is filtered using a neural network-based filter according to a first quantization parameter to obtain a filtered reconstructed block.

[0422] S807 : Determine a first flag of the current block according to the first quantization parameter of the current block, and write the first flag into the bitstream.

[0423] The first flag is used to indicate whether the first quantization parameter of the current block is the second quantization parameter predicted by the quantization parameter prediction model.

[0424] In some embodiments, when the encoder determines that the first quantization parameter of the current block is the second quantization parameter predicted by the quantization parameter prediction model, the encoder sets the first flag to yes and writes the set first flag into the bitstream.

[0425] In some embodiments, if the encoder determines that the first quantization parameter of the current block is a preset basic quantization parameter, the encoder sets the first flag to false and writes the false first flag into the bitstream.

[0426] It should be noted that there is no strict order in which the above-mentioned S807 and the above-mentioned S806 are executed. For example, the above-mentioned S807 can be executed before the above-mentioned S806, or after the above-mentioned S806, or can be executed synchronously with the above-mentioned S806. The embodiment of the present application does not limit this.

[0427] S808: Determine the second flag of the current image according to the first flag of the current block, and write the second flag into the code stream.

[0428] The second flag is used to indicate whether the first quantization parameter of at least one block in the current image is the second quantization parameter predicted by the quantization parameter prediction model.

[0429] In one example, if the value of the first flag is a first numerical value, the value of the second flag is set to a third numerical value, and the third numerical value is used to indicate that the first quantization parameter of at least one block in the current image is the second quantization parameter predicted by the quantization parameter prediction model.

[0430] In one example, if the value of the first flag is the second value, and the values ​​of the first flags of other coding blocks in the current image except the current block are all second values, the value of the second flag is set to a fourth value, and the fourth value is used to indicate that the first quantization parameter of the current image is not the second quantization parameter predicted by the quantization parameter prediction model.

[0431] Furthermore, in some embodiments, the technical solutions provided in the embodiments of the present application are preliminarily trained and tested based on the VTM (VVC reference software testing platform, VVC Test Model) 11.0-nnvc reference platform.

[0432] We first trained the quantization parameter prediction model, NNQP. We then selected an existing NNLF solution, such as JVET-Y0078, and tested its performance based on the proposed NNQP combined with NNLF. We tested the JVET-specified common sequences under the general test conditions of Random Access. The test results for some of these sequences are shown in Tables 4 and 5 below.

[0433] Table 4 RA performance of NNQP+NNLF vs. VTM11.0-nnvc

[0434]

[0435] Table 5 RA performance of NNQP+NNLF vs. NNLF

[0436]

[0437] As shown in Tables 4 and 5 above, a comparison of the performance data in the two tables shows that by introducing the learning-based quantization parameter prediction model of this application, it is possible to achieve certain coding performance improvements based on the neural network-based loop filter, especially in the UV component. At the same time, due to the increased frequency of using this application's solution, the performance of higher-resolution sequences is also relatively good. Further performance improvement can be achieved through subsequent optimization training.

[0438] In an embodiment of the present application, the encoding end determines a first flag and a second flag, and writes the first flag and the second flag into the bitstream, so that the decoding end can obtain the first flag and the second flag by decoding the bitstream, and then accurately determine which method to use to determine the first quantization parameter based on the first flag and the second flag. For example, if it is determined that the first quantization parameter of the current block is the second quantization parameter predicted by the quantization parameter prediction model, the decoding end predicts the first quantization parameter of the current block through the quantization parameter prediction model. If it is determined that the first quantization parameter of the current block is not the second quantization parameter predicted by the quantization parameter prediction model, the decoding end can directly determine the preset basic quantization parameter as the first quantization parameter of the current block, or determine the first quantization parameter through other methods, thereby reducing the accuracy and computational complexity of the decoding end in determining the first quantization parameter of the current block, thereby improving the filtering efficiency of the decoding end.

[0439] It should be understood that Figures 5 to 11 are merely examples of the present application and should not be construed as limiting the present application.

[0440] The preferred embodiments of the present application are described in detail above in conjunction with the accompanying drawings. However, the present application is not limited to the specific details in the above embodiments. Within the technical concept of the present application, a variety of simple modifications can be made to the technical solution of the present application, and these simple modifications all fall within the scope of protection of the present application. For example, the various specific technical features described in the above specific embodiments can be combined in any suitable manner unless there is any contradiction. In order to avoid unnecessary repetition, the present application will not further explain various possible combinations. For another example, the various different embodiments of the present application can also be arbitrarily combined, and as long as they do not violate the ideas of the present application, they should also be regarded as the contents disclosed in the present application.

[0441] It should also be understood that in the various method embodiments of the present application, the size of the sequence numbers of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. In addition, in the embodiments of the present application, the term "and / or" is merely a description of the association relationship of associated objects, indicating that three types of relationships can exist. Specifically, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the related objects before and after are in an "or" relationship.

[0442] The above describes in detail the method embodiment of the present application in conjunction with Figures 5 to 11, and the following describes in detail the device embodiment of the present application in conjunction with Figures 12 to 15.

[0443] FIG12 is a schematic block diagram of a video decoding device according to an embodiment of the present application.

[0444] As shown in FIG12 , the video decoding apparatus 10 includes:

[0445] The reconstruction unit 11 is configured to decode the code stream and determine a reconstructed block of the current block;

[0446] A determination unit 12 is configured to decode the bitstream and determine a first quantization parameter of the current block, where the first quantization parameter is determined based on a second quantization parameter predicted by a quantization parameter prediction model;

[0447] The filtering unit 13 is configured to filter the reconstructed block of the current block using a neural network-based filter according to the first quantization parameter to obtain a filtered reconstructed block.

[0448] In some embodiments, the determination unit 12 is specifically configured to decode the bitstream to obtain the first quantization parameter of the current block.

[0449] In some embodiments, the determining unit 12 is specifically configured to predict the second quantization parameter of the current block using the quantization parameter prediction model;

[0450] The first quantization parameter is determined according to the second quantization parameter.

[0451] In some embodiments, the determining unit 12 is specifically configured to determine the first quantization parameter according to the second quantization parameter and a preset basic quantization parameter.

[0452] In some embodiments, the determination unit 12 is specifically used to determine the distortion corresponding to the second quantization parameter; determine the distortion corresponding to the basic quantization parameter; and determine the first quantization parameter from the second quantization parameter and the basic quantization parameter based on the distortion corresponding to the second quantization parameter and the distortion corresponding to the basic quantization parameter.

[0453] In some embodiments, the determining unit 12 is specifically configured to determine the quantization parameter with the smallest distortion between the second quantization parameter and the basic quantization parameter as the first quantization parameter.

[0454] In some embodiments, the determination unit 12 is specifically used to filter the reconstructed block of the current block through the neural network-based filter according to the second quantization parameter to obtain a filtered first reconstructed block; and determine the distortion corresponding to the second quantization parameter according to the filtered first reconstructed block and the reconstruction value of the current block.

[0455] In some embodiments, the determination unit 12 is specifically configured to concatenate the reconstructed value of the current block and the second quantization parameter, and then input the concatenated concatenated values ​​into the neural network-based filter for filtering to obtain a filtered first reconstructed block.

[0456] In some embodiments, the determination unit 12 is specifically configured to filter the reconstructed block of the current block through the neural network-based filter according to the basic quantization parameter to obtain a filtered second reconstructed block; and determine the distortion corresponding to the basic quantization parameter according to the filtered second reconstructed block and the reconstruction value of the current block.

[0457] In some embodiments, the determination unit 12 is specifically configured to concatenate the reconstructed value of the current block and the basic quantization parameter, and then input the concatenated concatenated values ​​into the neural network-based filter for filtering to obtain a filtered second reconstructed block.

[0458] In some embodiments, the determination unit 12 is specifically used to decode the code stream to obtain a first flag, where the first flag is used to indicate whether the first quantization parameter of the current block is the second quantization parameter predicted by the quantization parameter prediction model; if the first flag indicates that the first quantization parameter of the current block is the second quantization parameter predicted by the quantization parameter prediction model, then the second quantization parameter of the current block is predicted by the quantization parameter prediction model; and the second quantization parameter is determined as the first quantization parameter.

[0459] In some embodiments, the determination unit 12 is further configured to determine a preset basic quantization parameter as the first quantization parameter of the current block if the first flag indicates that the first quantization parameter of the current block is not the second quantization parameter predicted by the quantization parameter prediction model.

[0460] In some embodiments, the determination unit 12 is specifically used to decode the code stream to obtain a second flag, where the second flag is used to indicate whether the first quantization parameter of at least one block in the current image is the second quantization parameter predicted by the quantization parameter prediction model; if the second flag indicates that the first quantization parameter of at least one block in the current image is the second quantization parameter predicted by the quantization parameter prediction model, then the code stream is decoded to obtain the first flag.

[0461] In some embodiments, the determination unit 12 is further configured to determine a preset basic quantization parameter as the first quantization parameter of the current block if the second flag indicates that the first quantization parameter of the current image is not the second quantization parameter predicted by the quantization parameter prediction model.

[0462] In some embodiments, the determination unit 12 is specifically used to decode the code stream to obtain a third flag, where the third flag is used to indicate whether the current sequence is allowed to use the quantization parameter prediction model to predict the quantization parameter; if the third flag indicates that the current sequence is allowed to use the quantization parameter prediction model to predict the quantization parameter, the code stream is decoded to obtain the second flag.

[0463] In some embodiments, the determination unit 12 is specifically configured to input the reconstructed block of the current block and at least one preset third quantization parameter into the quantization parameter prediction model to perform quantization parameter prediction to obtain the second quantization parameter.

[0464] In some embodiments, the determination unit 12 is specifically configured to splice the reconstructed block of the current block and the at least one third quantization parameter, and then input the spliced ​​blocks into the quantization parameter prediction model to perform quantization parameter prediction to obtain the second quantization parameter.

[0465] Optionally, the at least one third quantization parameter includes at least one of a basic quantization parameter, a slice-level quantization parameter, and a reference block quantization parameter of the current block.

[0466] In some embodiments, the quantization parameter prediction model includes multiple convolutional layers and at least one fully connected layer.

[0467] Optionally, the quantization parameter prediction model also includes multiple pooling layers.

[0468] Optionally, a pooling layer is connected after each convolution layer in the multiple convolution layers.

[0469] In some embodiments, the second quantization parameter predicted by the quantization parameter prediction model is a candidate quantization parameter with the highest probability among N preset candidate quantization parameters, where N is a positive integer greater than 1.

[0470] In some embodiments, the number of the N candidate quantization parameters is greater than the number of candidate quantization parameters corresponding to the basic quantization parameter.

[0471] In some embodiments, the filtering unit 13 is specifically used to perform loop filtering on the reconstructed block of the current block to obtain a reconstructed block after loop filtering; according to the first quantization parameter, the reconstructed block after loop filtering is filtered by a neural network-based filter to obtain the filtered reconstructed block.

[0472] It should be understood that the device embodiment and the method embodiment may correspond to each other, and similar descriptions can refer to the method embodiment. To avoid repetition, no further description is given here. Specifically, the video decoding device 10 shown in Figure 12 can correspond to the corresponding subject in the method of the embodiment of the present application, and the aforementioned and other operations and / or functions of the various units in the video decoding device 10 are respectively for implementing the corresponding processes in each method such as the method, and for the sake of brevity, no further description is given here.

[0473] FIG13 is a schematic block diagram of a video encoding device according to an embodiment of the present application.

[0474] As shown in FIG13 , the video encoding apparatus 20 includes:

[0475] A reconstruction unit 21, configured to reconstruct a current block to obtain a reconstructed block of the current block;

[0476] A prediction unit 22 is configured to predict a second quantization parameter of the current block using a quantization parameter prediction model;

[0477] a determining unit 23, configured to determine a first quantization parameter of the current block according to the second quantization parameter;

[0478] The filtering unit 24 is configured to filter the reconstructed block of the current block using a neural network-based filter according to the first quantization parameter to obtain a filtered reconstructed block.

[0479] In some embodiments, the prediction unit 22 is specifically configured to input the reconstructed block of the current block and at least one preset third quantization parameter into the quantization parameter prediction model to perform quantization parameter prediction to obtain the second quantization parameter.

[0480] Optionally, the at least one third quantization parameter includes at least one of a basic quantization parameter, a slice-level quantization parameter, and a reference block quantization parameter of the current block.

[0481] In some embodiments, the determining unit 23 is specifically configured to determine the first quantization parameter according to the second quantization parameter and a preset basic quantization parameter.

[0482] In some embodiments, the determination unit 23 is specifically used to determine the distortion corresponding to the second quantization parameter; determine the distortion corresponding to the basic quantization parameter based on the distortion corresponding to the second quantization parameter and the distortion corresponding to the basic quantization parameter, and determine the first quantization parameter from the second quantization parameter and the basic quantization parameter.

[0483] In some embodiments, the determination unit 23 is specifically used to filter the reconstructed block of the current block through the neural network-based filter according to the second quantization parameter to obtain a filtered first reconstructed block; and determine the distortion corresponding to the second quantization parameter according to the filtered first reconstructed block and the reconstruction value of the current block.

[0484] In some embodiments, the determination unit 23 is specifically configured to concatenate the reconstructed value of the current block and the second quantization parameter, and then input the concatenated concatenated values ​​into the neural network-based filter for filtering to obtain a filtered first reconstructed block.

[0485] In some embodiments, the determination unit 23 is specifically used to filter the reconstructed block of the current block through the neural network-based filter according to the basic quantization parameter to obtain a filtered second reconstructed block; and determine the distortion corresponding to the basic quantization parameter according to the filtered second reconstructed block and the reconstruction value of the current block.

[0486] In some embodiments, the determination unit 23 is specifically configured to concatenate the reconstructed value of the current block and the basic quantization parameter, and then input the concatenated concatenated values ​​into the neural network-based filter for filtering to obtain a filtered second reconstructed block.

[0487] In some embodiments, the determining unit 23 is specifically configured to determine the quantization parameter with the smallest distortion between the second quantization parameter and the basic quantization parameter as the first quantization parameter.

[0488] In some embodiments, the determination unit 23 is further used to determine a first flag based on the first quantization parameter, where the first flag is used to indicate whether the first quantization parameter of the current block is the second quantization parameter predicted by the quantization parameter prediction model.

[0489] In some embodiments, the determination unit 23 is specifically used to set the value of the first flag to a first numerical value if the first quantization parameter is the second quantization parameter, and the first numerical value is used to indicate that the first quantization parameter of the current block is the second quantization parameter predicted by the quantization parameter prediction model.

[0490] In some embodiments, the determination unit 23 is specifically used to set the value of the first flag to a second value if the first quantization parameter is the basic quantization parameter, and the second value is used to indicate that the first quantization parameter of the current block is not the second quantization parameter predicted by the quantization parameter prediction model.

[0491] In some embodiments, the determination unit 23 is further used to determine a second flag based on the first flag, where the second flag is used to indicate whether the first quantization parameter of at least one block in the current image is the second quantization parameter predicted by the quantization parameter prediction model.

[0492] In some embodiments, the determination unit 23 is specifically used to set the value of the second flag to a third value if the value of the first flag is the first numerical value, and the third numerical value is used to indicate that the first quantization parameter of at least one block in the current image is the second quantization parameter predicted by the quantization parameter prediction model.

[0493] In some embodiments, the determination unit 23 is specifically used to set the value of the second flag to a fourth value if the value of the first flag is the second value, and the values ​​of the first flags of other coding blocks in the current image except the current block are all the second value. The fourth value is used to indicate that the first quantization parameter of the current image is not the second quantization parameter predicted by the quantization parameter prediction model.

[0494] In some embodiments, the prediction unit 22 is specifically used to obtain a third flag, where the third flag is used to indicate whether the current sequence is allowed to predict the quantization parameter through the quantization parameter prediction model; if the third flag indicates that the current sequence is allowed to predict the quantization parameter through the quantization parameter prediction model, the second quantization parameter is predicted through the quantization parameter prediction model.

[0495] In some embodiments, the prediction unit 22 is specifically configured to splice the reconstructed block of the current block and the at least one third quantization parameter, and then input the spliced ​​blocks into the quantization parameter prediction model to perform quantization parameter prediction to obtain the second quantization parameter.

[0496] In some embodiments, the quantization parameter prediction model includes multiple convolutional layers and at least one fully connected layer.

[0497] Optionally, the quantization parameter prediction model also includes multiple pooling layers.

[0498] Optionally, a pooling layer is connected after each convolution layer in the multiple convolution layers.

[0499] In some embodiments, the second quantization parameter predicted by the quantization parameter prediction model is a candidate quantization parameter with the highest probability among N preset candidate quantization parameters, where N is a positive integer greater than 1.

[0500] In some embodiments, the number of the N candidate quantization parameters is greater than the number of candidate quantization parameters corresponding to the basic quantization parameter.

[0501] In some embodiments, the filtering unit 24 is specifically used to perform loop filtering on the reconstructed block of the current block to obtain a reconstructed block after loop filtering; according to the first quantization parameter, the reconstructed block after loop filtering is filtered by a neural network-based filter to obtain the filtered reconstructed block.

[0502] It should be understood that the device embodiment and the method embodiment may correspond to each other, and similar descriptions can refer to the method embodiment. To avoid repetition, no further description is given here. Specifically, the video encoding device 20 shown in Figure 13 can correspond to the corresponding subject in the method of the embodiment of the present application, and the aforementioned and other operations and / or functions of each unit in the video encoding device 20 are respectively for implementing the corresponding processes in each method such as the method. For the sake of brevity, no further description is given here.

[0503] The above describes the apparatus and system of the embodiment of the present application from the perspective of functional units in conjunction with the accompanying drawings. It should be understood that the functional unit can be implemented in the form of hardware, can be implemented by instructions in the form of software, or can be implemented by a combination of hardware and software units. Specifically, the steps of the method embodiment in the embodiment of the present application can be completed by the hardware integrated logic circuit and / or software instructions in the processor, and the steps of the method disclosed in the embodiment of the present application can be directly embodied as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software units in the decoding processor. Optionally, the software unit can be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps in the above method embodiment in conjunction with its hardware.

[0504] FIG14 is a schematic block diagram of an electronic device provided in an embodiment of the present application.

[0505] As shown in FIG14 , the electronic device 30 may be a video encoder or a video decoder as described in an embodiment of the present application. The electronic device 30 may include:

[0506] The memory 33 and the processor 32 are configured to store a computer program 34 and transmit the program code 34 to the processor 32. In other words, the processor 32 can call and run the computer program 34 from the memory 33 to implement the method in the embodiment of the present application.

[0507] For example, the processor 32 may be configured to execute the steps of the above method according to the instructions in the computer program 34 .

[0508] In some embodiments of the present application, the processor 32 may include but is not limited to:

[0509] General-purpose processor, Digital Signal Processor (DSP), Application Specific Integrated Circuit (ASIC), Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc.

[0510] In some embodiments of the present application, the memory 33 includes but is not limited to:

[0511] Volatile memory and / or non-volatile memory. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus random access memory (DR RAM).

[0512] In some embodiments of the present application, the computer program 34 may be divided into one or more units, which are stored in the memory 33 and executed by the processor 32 to implement the method provided by the present application. The one or more units may be a series of computer program instruction segments capable of implementing specific functions, and the instruction segments are used to describe the execution process of the computer program 34 in the electronic device 30.

[0513] As shown in FIG14 , the electronic device 30 may further include:

[0514] The transceiver 33 may be connected to the processor 32 or the memory 33 .

[0515] The processor 32 can control the transceiver 33 to communicate with other devices. Specifically, it can send information or data to other devices or receive information or data sent by other devices. The transceiver 33 may include a transmitter and a receiver. The transceiver 33 may further include an antenna, and the number of antennas may be one or more.

[0516] It should be understood that the various components in the electronic device 30 are connected via a bus system, wherein the bus system includes not only a data bus but also a power bus, a control bus and a status signal bus.

[0517] FIG15 is a schematic block diagram of a video encoding and decoding system provided in an embodiment of the present application.

[0518] As shown in Figure 15, the video encoding and decoding system 40 may include: a video encoder 41 and a video decoder 42, wherein the video encoder 41 is used to execute the video encoding method involved in the embodiment of the present application, and the video decoder 42 is used to execute the video decoding method involved in the embodiment of the present application.

[0519] In some embodiments, the present application also provides a code stream, which is obtained by the above encoding method.

[0520] The present application also provides a computer storage medium having a computer program stored thereon, which, when executed by a computer, enables the computer to perform the method of the above-mentioned method embodiment. In other words, the present application also provides a computer program product containing instructions, which, when executed by a computer, enables the computer to perform the method of the above-mentioned method embodiment.

[0521] When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a digital video disc (DVD)), or a semiconductor medium (e.g., a solid state drive (SSD)).

[0522] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0523] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the unit is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0524] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment. For example, the functional units in the various embodiments of the present application may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0525] The above content is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A video decoding method, characterized in that: include: Decode the code stream and determine the reconstructed block of the current block; Decoding the bitstream to determine a first quantization parameter of the current block, where the first quantization parameter is determined based on a second quantization parameter predicted by a quantization parameter prediction model; According to the first quantization parameter, the reconstructed block of the current block is filtered by a neural network-based filter to obtain a filtered reconstructed block.

2. The method according to claim 1, characterized in that The determining the first quantization parameter of the current block includes: The code stream is decoded to obtain a first quantization parameter of the current block.

3. The method according to claim 1, characterized in that The determining the first quantization parameter of the current block includes: Predicting a second quantization parameter of the current block using the quantization parameter prediction model; The first quantization parameter is determined according to the second quantization parameter.

4. The method according to claim 3, characterized in that The determining the first quantization parameter according to the second quantization parameter includes: The first quantization parameter is determined according to the second quantization parameter and a preset basic quantization parameter.

5. The method according to claim 4, characterized in that The determining the first quantization parameter according to the second quantization parameter and a preset basic quantization parameter includes: determining a distortion corresponding to the second quantization parameter; Determining the distortion corresponding to the basic quantization parameter; The first quantization parameter is determined from the second quantization parameter and the basic quantization parameter according to the distortion corresponding to the second quantization parameter and the distortion corresponding to the basic quantization parameter.

6. The method according to claim 5, characterized in that The determining, based on the distortion corresponding to the second quantization parameter and the distortion corresponding to the basic quantization parameter, the first quantization parameter from the second quantization parameter and the basic quantization parameter, includes: The quantization parameter with the smallest distortion between the second quantization parameter and the basic quantization parameter is determined as the first quantization parameter.

7. The method according to claim 5, characterized in that The determining the distortion corresponding to the second quantization parameter includes: filtering the reconstructed block of the current block using the neural network-based filter according to the second quantization parameter to obtain a filtered first reconstructed block; Distortion corresponding to the second quantization parameter is determined according to the filtered first reconstructed block and a reconstructed value of the current block.

8. The method according to claim 7, characterized in that The filtering, according to the second quantization parameter, of the reconstructed block of the current block by the neural network-based filter to obtain a filtered first reconstructed block includes: After the reconstruction value of the current block and the second quantization parameter are spliced, the concatenation is input into the neural network-based filter for filtering to obtain a filtered first reconstructed block.

9. The method according to claim 5, characterized in that The determining of the distortion corresponding to the basic quantization parameter includes: filtering the reconstructed block of the current block using the neural network-based filter according to the basic quantization parameter to obtain a filtered second reconstructed block; The distortion corresponding to the basic quantization parameter is determined according to the filtered second reconstructed block and the reconstructed value of the current block.

10. The method according to claim 9, characterized in that The filtering, according to the basic quantization parameter, the reconstructed block of the current block by the neural network-based filter to obtain a filtered second reconstructed block includes: After the reconstruction value of the current block and the basic quantization parameter are spliced, the concatenation is input into the neural network-based filter for filtering to obtain a filtered second reconstructed block.

11. The method according to claim 1, wherein The determining the first quantization parameter of the current block includes: Decoding the bitstream to obtain a first flag, where the first flag is used to indicate whether the first quantization parameter of the current block is the second quantization parameter predicted by the quantization parameter prediction model; If the first flag indicates that the first quantization parameter of the current block is the second quantization parameter predicted by the quantization parameter prediction model, predicting the second quantization parameter of the current block by using the quantization parameter prediction model; The second quantization parameter is determined as the first quantization parameter.

12. The method according to claim 11, characterized in that The method further comprises: If the first flag indicates that the first quantization parameter of the current block is not the second quantization parameter predicted by the quantization parameter prediction model, a preset basic quantization parameter is determined as the first quantization parameter of the current block.

13. The method according to claim 11, characterized in that The decoding of the code stream to obtain a first flag includes: Decoding the bitstream to obtain a second flag, where the second flag is used to indicate whether a first quantization parameter of at least one block in a current image is a second quantization parameter predicted by the quantization parameter prediction model; If the second flag indicates that the first quantization parameter of at least one block in the current image is the second quantization parameter predicted by the quantization parameter prediction model, the code stream is decoded to obtain the first flag.

14. The method according to claim 13, characterized in that The method further comprises: If the second flag indicates that the first quantization parameter of the current image is not the second quantization parameter predicted by the quantization parameter prediction model, a preset basic quantization parameter is determined as the first quantization parameter of the current block.

15. The method according to claim 13, characterized in that The decoding of the code stream to obtain a second flag includes: Decoding the bitstream to obtain a third flag, where the third flag is used to indicate whether the current sequence is allowed to use the quantization parameter prediction model to predict the quantization parameter; If the third flag indicates that the current sequence is allowed to use the quantization parameter prediction model to predict the quantization parameter, the code stream is decoded to obtain the second flag.

16. The method according to claim 3 or 11, characterized in that The step of predicting the second quantization parameter of the current block by using the quantization parameter prediction model includes: The reconstructed block of the current block and at least one preset third quantization parameter are input into the quantization parameter prediction model to perform quantization parameter prediction to obtain the second quantization parameter.

17. The method according to claim 16, characterized in that The step of inputting the reconstructed block of the current block and at least one preset third quantization parameter into the quantization parameter prediction model to perform quantization parameter prediction to obtain the second quantization parameter includes: After the reconstructed block of the current block and the at least one third quantization parameter are spliced, the concatenation is input into the quantization parameter prediction model to perform quantization parameter prediction, thereby obtaining the second quantization parameter.

18. The method according to claim 16, characterized in that The at least one third quantization parameter includes at least one of a basic quantization parameter, a slice-level quantization parameter, and a reference block quantization parameter of the current block.

19. The method according to any one of claims 1 to 12, characterized in that: The quantization parameter prediction model includes multiple convolutional layers and at least one fully connected layer.

20. The method according to claim 19, characterized in that The quantization parameter prediction model also includes multiple pooling layers.

21. The method according to claim 20, characterized in that A pooling layer is connected after each convolution layer in the multiple convolution layers.

22. The method according to claim 20, characterized in that The second quantization parameter predicted by the quantization parameter prediction model is a candidate quantization parameter with the highest probability among N preset candidate quantization parameters, where N is a positive integer greater than 1.

23. The method according to claim 22, characterized in that The number of the N candidate quantization parameters is greater than the number of candidate quantization parameters corresponding to the basic quantization parameter.

24. The method according to any one of claims 1 to 12, characterized in that The filtering, according to the first quantization parameter, the reconstructed block of the current block by using a filter based on a neural network to obtain a filtered reconstructed block includes: Performing loop filtering on a reconstructed block of the current block to obtain a reconstructed block after loop filtering; The reconstructed block after the loop filtering is filtered using a neural network-based filter according to the first quantization parameter to obtain the filtered reconstructed block.

25. A video encoding method, characterized in that: include: Reconstructing the current block to obtain a reconstructed block of the current block; Predicting a second quantization parameter of the current block using a quantization parameter prediction model; determining a first quantization parameter of the current block according to the second quantization parameter; According to the first quantization parameter, the reconstructed block of the current block is filtered by a neural network-based filter to obtain a filtered reconstructed block.

26. The method according to claim 25, characterized in that The step of predicting the second quantization parameter of the current block by using the quantization parameter prediction model includes: The reconstructed block of the current block and at least one preset third quantization parameter are input into the quantization parameter prediction model to perform quantization parameter prediction to obtain the second quantization parameter.

27. The method according to claim 26, characterized in that The at least one third quantization parameter includes at least one of a basic quantization parameter, a slice-level quantization parameter, and a reference block quantization parameter of the current block.

28. The method according to claim 25, characterized in that The determining the first quantization parameter according to the second quantization parameter includes: The first quantization parameter is determined according to the second quantization parameter and a preset basic quantization parameter.

29. The method according to claim 28, characterized in that The determining the first quantization parameter according to the second quantization parameter and a preset basic quantization parameter includes: determining a distortion corresponding to the second quantization parameter; Determining the distortion corresponding to the basic quantization parameter; The first quantization parameter is determined from the second quantization parameter and the basic quantization parameter according to the distortion corresponding to the second quantization parameter and the distortion corresponding to the basic quantization parameter.

30. The method according to claim 29, wherein The determining the distortion corresponding to the second quantization parameter includes: filtering the reconstructed block of the current block using the neural network-based filter according to the second quantization parameter to obtain a filtered first reconstructed block; Distortion corresponding to the second quantization parameter is determined according to the filtered first reconstructed block and a reconstructed value of the current block.

31. The method according to claim 30, wherein The filtering, according to the second quantization parameter, of the reconstructed block of the current block by the neural network-based filter to obtain a filtered first reconstructed block includes: After the reconstruction value of the current block and the second quantization parameter are spliced, the concatenation is input into the neural network-based filter for filtering to obtain a filtered first reconstructed block.

32. The method according to claim 29, wherein The determining of the distortion corresponding to the basic quantization parameter includes: filtering the reconstructed block of the current block using the neural network-based filter according to the basic quantization parameter to obtain a filtered second reconstructed block; The distortion corresponding to the basic quantization parameter is determined according to the filtered second reconstructed block and the reconstructed value of the current block.

33. The method according to claim 32, characterized in that The filtering, according to the basic quantization parameter, the reconstructed block of the current block by the neural network-based filter to obtain a filtered second reconstructed block includes: After the reconstruction value of the current block and the basic quantization parameter are spliced, the concatenation is input into the neural network-based filter for filtering to obtain a filtered second reconstructed block.

34. The method according to claim 29, wherein The determining, based on the distortion corresponding to the second quantization parameter and the distortion corresponding to the basic quantization parameter, the first quantization parameter from the second quantization parameter and the basic quantization parameter, includes: The quantization parameter with the smallest distortion between the second quantization parameter and the basic quantization parameter is determined as the first quantization parameter.

35. The method according to claim 29, wherein The method further comprises: A first flag is determined according to the first quantization parameter, where the first flag is used to indicate whether the first quantization parameter of the current block is the second quantization parameter predicted by the quantization parameter prediction model.

36. The method according to claim 35, characterized in that The determining of a first flag according to the first quantization parameter includes: If the first quantization parameter is the second quantization parameter, the value of the first flag is set to a first value, where the first value is used to indicate that the first quantization parameter of the current block is the second quantization parameter predicted by the quantization parameter prediction model.

37. The method according to claim 36, characterized in that The determining of a first flag according to the first quantization parameter includes: If the first quantization parameter is the basic quantization parameter, the value of the first flag is set to a second value, where the second value is used to indicate that the first quantization parameter of the current block is not the second quantization parameter predicted by the quantization parameter prediction model.

38. The method according to claim 37, wherein The method further comprises: A second flag is determined according to the first flag, where the second flag is used to indicate whether a first quantization parameter of at least one block in a current image is a second quantization parameter predicted by the quantization parameter prediction model.

39. The method according to claim 38, characterized in that Determining the second flag according to the first flag includes: If the value of the first flag is the first numerical value, the value of the second flag is set to a third numerical value, and the third numerical value is used to indicate that the first quantization parameter of at least one block in the current image is the second quantization parameter predicted by the quantization parameter prediction model.

40. The method according to claim 39, wherein Determining the second flag according to the first flag includes: If the value of the first flag is the second numerical value, and the values ​​of the first flags of other coding blocks in the current image except the current block are all the second numerical value, then the value of the second flag is set to a fourth numerical value, and the fourth numerical value is used to indicate that the first quantization parameter of the current image is not the second quantization parameter predicted by the quantization parameter prediction model.

41. The method according to any one of claims 25 to 40, characterized in that The step of predicting the second quantization parameter of the current block by using a quantization parameter prediction model includes: Obtaining a third flag, where the third flag is used to indicate whether to allow the current sequence to predict the quantization parameter through the quantization parameter prediction model; If the third flag indicates that the current sequence is allowed to predict the quantization parameter through the quantization parameter prediction model, the second quantization parameter is predicted by the quantization parameter prediction model.

42. The method according to claim 26, wherein The step of inputting the reconstructed block of the current block and at least one preset third quantization parameter into the quantization parameter prediction model to perform quantization parameter prediction to obtain the second quantization parameter includes: After the reconstructed block of the current block and the at least one third quantization parameter are spliced, the concatenation is input into the quantization parameter prediction model to perform quantization parameter prediction, thereby obtaining the second quantization parameter.

43. The method according to any one of claims 25 to 40, characterized in that The quantization parameter prediction model includes multiple convolutional layers and at least one fully connected layer.

44. The method according to claim 43, wherein The quantization parameter prediction model also includes multiple pooling layers.

45. The method according to claim 44, wherein A pooling layer is connected after each convolution layer in the multiple convolution layers.

46. ​​The method according to claim 44, wherein The second quantization parameter predicted by the quantization parameter prediction model is a candidate quantization parameter with the highest probability among N preset candidate quantization parameters, where N is a positive integer greater than 1.

47. The method according to claim 46, wherein The number of the N candidate quantization parameters is greater than the number of candidate quantization parameters corresponding to the basic quantization parameter.

48. The method according to any one of claims 25 to 40, characterized in that The filtering, according to the first quantization parameter, the reconstructed block of the current block by using a filter based on a neural network to obtain a filtered reconstructed block includes: Performing loop filtering on a reconstructed block of the current block to obtain a reconstructed block after loop filtering; The reconstructed block after the loop filtering is filtered using a neural network-based filter according to the first quantization parameter to obtain the filtered reconstructed block.

49. A video decoding device, characterized in that include: A reconstruction unit, configured to decode the bitstream and determine a reconstructed block of the current block; a determining unit, configured to decode the bitstream and determine a first quantization parameter of the current block, where the first quantization parameter is determined based on a second quantization parameter predicted by a quantization parameter prediction model; A filtering unit is configured to filter the reconstructed block of the current block using a neural network-based filter according to the first quantization parameter to obtain a filtered reconstructed block.

50. A video encoding device, characterized in that include: a reconstruction unit, configured to reconstruct a current block to obtain a reconstructed block of the current block; A prediction unit, configured to predict a second quantization parameter of the current block using a quantization parameter prediction model; a determining unit, configured to determine a first quantization parameter of the current block according to the second quantization parameter; A filtering unit is configured to filter the reconstructed block of the current block using a neural network-based filter according to the first quantization parameter to obtain a filtered reconstructed block.

51. A video encoding and decoding system, characterized in that: Includes video encoder and video decoder; The video decoder is configured to execute the video decoding method according to any one of claims 1 to 24; The video encoder is used to execute the video encoding method according to any one of claims 25-48.

52. An electronic device, characterized in that: include: Memory, processor; The memory is used to store computer programs; The processor is configured to execute the computer program to implement the method according to any one of claims 1 to 24 or 25 to 48.

53. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 24 or 25 to 48 when executed by a processor.

54. A code stream, characterized in that The method comprises a code stream obtained by the method according to any one of claims 25 to 48.

Images