Filtering method, encoder, decoder, code stream and storage medium

Abstract

The embodiment of the invention discloses a filtering method, an encoder, a decoder, a code stream and a storage medium, and the method comprises the steps: analyzing the code stream, and obtaining a frame-level switch identification bit based on a network filtering model; the frame-level switch identification bit is used for judging whether each block in the current frame is filtered or not; when the frame-level switch identification bit represents that the video component is not turned on and the obtained frame-level use identification bit represents that any video component of the current block is used, obtaining a block-level use identification bit; when the block-level use identifier represents that any video component of the current block is used, obtaining a quantization parameter; wherein the quantization parameter comprises an adjusted block-level quantization parameter or a frame-level optimal quantization parameter; and filtering the current block of the current frame based on the quantization parameter and the network filtering model to obtain first residual information of the current block.

Description

Filtering method, encoder, decoder, code stream and storage medium Technical Field

[0001] The embodiments of the present application relate to the field of image processing technology, and in particular to a filtering method, an encoder, a decoder, a bit stream, and a storage medium. Background Art

[0002] In video codec systems, most video codecs use a block-based hybrid coding framework. Each video frame is divided into several Coding Tree Units (CTUs), which can be further divided into several rectangular Coding Units (CUs), which can be rectangular or square blocks. Because adjacent CUs use different coding parameters, such as different transform processes, different quantization parameters (QPs), different prediction methods, and different reference image frames, and because the error size and distribution characteristics introduced by each CU are independent of each other, the discontinuity of adjacent CU boundaries causes blocking artifacts, which affects the subjective and objective quality of the reconstructed image and even the prediction accuracy of subsequent codecs.

[0003] In this way, during the encoding and decoding process, loop filters are used to improve the subjective and objective quality of the reconstructed image. Neural network-based loop filtering methods offer the most outstanding coding performance. In related technologies, on the one hand, neural network filter models are switched at the coding tree unit level. Different neural network filter models are trained based on different sequence-level quantization parameter values ​​(BaseQP). The encoder tries these different neural network filter models and selects the neural network filter model with the lowest rate-distortion cost as the optimal network model for the current coding tree unit. Using the usage flag and network model index information at the coding tree unit level, the decoder can use the same network model as the encoder for filtering. On the other hand, for different test conditions and quantization parameters, a simplified, low-complexity neural network filter model can be used for loop filtering. When using the low-complexity neural network filter model for filtering, quantization parameter information is added as an additional input. This quantization parameter information is used as the network input to improve the generalization capability of the neural network filter model, thereby achieving good coding performance without switching neural network filter models.

[0004] However, when switching neural network filter models at the coding tree unit level for filtering, the hardware implementation is complex and expensive, as each coding tree unit corresponds to a neural network filter model. When using low-complexity neural network filter models for filtering, the filtering options are not flexible enough due to the influence of quantization parameters, and the encoding and decoding options are still limited, which cannot achieve good encoding and decoding results.

[0005] Summary of the Invention

[0006] The embodiments of the present application provide a filtering method, an encoder, a decoder, a code stream, and a storage medium, which can make the selection of input information for filtering more flexible without increasing the complexity, thereby improving the encoding and decoding efficiency.

[0007] The technical solution of the embodiment of the present application can be implemented as follows:

[0008] In a first aspect, an embodiment of the present application provides a filtering method, applied to a decoder, the method comprising:

[0009] Parse the bitstream to obtain a frame-level switch flag based on a network filtering model; the frame-level switch flag is used to determine whether all blocks in the current frame are filtered;

[0010] When the frame-level switch flag indicates that it is not turned on, and the obtained frame-level usage flag indicates that any video component of the current block is used, obtaining a block-level usage flag;

[0011] When the block-level usage flag indicates that any video component of the current block is in use, a quantization parameter is obtained; wherein the quantization parameter includes: an adjusted block-level quantization parameter or a frame-level optimal quantization parameter;

[0012] Based on the quantization parameter and the network filtering model, a current block of the current frame is filtered to obtain first residual information of the current block.

[0013] In a second aspect, an embodiment of the present application provides a filtering method, applied to an encoder, the method comprising:

[0014] When the obtained sequence level allows the use of a flag indicating permission, obtaining an original value of a current block in a current frame, a reconstructed value of the current block, and a first quantization parameter; the first quantization parameter includes: an original quantization parameter;

[0015] Performing at least two filter estimations on the current block based on a network filter model, a reconstructed value of the current block, at least one frame-level quantization offset parameter, and the original quantization parameter to determine at least two first sub-rate-distortion costs of the current block, and traversing the current frame to determine at least two first rate-distortion costs of the current frame;

[0016] determining a first optimal rate-distortion cost based on at least two of the first rate-distortion costs;

[0017] Determine a block-level usage flag and a first sub-optimal rate-distortion cost of a current block based on at least two of the first sub-rate-distortion costs, and traverse the current frame to determine a second optimal rate-distortion cost of the current frame;

[0018] A frame-level switch flag is determined based on the first best rate-distortion cost and the second best rate-distortion cost.

[0019] In a third aspect, an embodiment of the present application provides a decoder, including:

[0020] The parsing part is configured to parse the code stream and obtain a frame-level switch flag based on the network filtering model; the frame-level switch flag is used to determine whether each block in the current frame is filtered;

[0021] The first determining part is configured to obtain a block-level usage flag when the frame-level switch flag indicates that the switch is not turned on and the obtained frame-level usage flag indicates that any video component of the current block is used; and

[0022] When the block-level usage flag indicates that any video component of the current block is in use, a quantization parameter is obtained; wherein the quantization parameter includes: an adjusted block-level quantization parameter or a frame-level optimal quantization parameter;

[0023] The first filtering part is configured to filter the current block of the current frame based on the quantization parameter and the network filtering model to obtain first residual information of the current block.

[0024] In a fourth aspect, an embodiment of the present application provides a decoder, including:

[0025] a first memory configured to store a computer program executable on the first processor;

[0026] The first processor is configured to execute the method described in the first aspect when running the computer program.

[0027] In a fifth aspect, an embodiment of the present application provides an encoder, including:

[0028] The second determining part is configured to obtain an original value of a current block in a current frame, a reconstructed value of the current block, and a first quantization parameter when the obtained sequence level allows the use of a flag to represent the permission; the first quantization parameter includes: an original quantization parameter;

[0029] a second filtering part, configured to perform at least two filter estimates on the current block based on the network filtering model, the reconstructed value of the current block, at least one frame-level quantization offset parameter, and the original quantization parameter, determine at least two first sub-rate-distortion costs of the current block, and traverse the current frame to determine the at least two first rate-distortion costs of the current frame;

[0030] The second determination part is further configured to determine a first optimal rate-distortion cost based on at least two of the first rate-distortion costs; determine a block-level usage flag and a first sub-optimal rate-distortion cost of a current block based on at least two of the first sub-rate-distortion costs, and traverse the current frame to determine a second optimal rate-distortion cost of the current frame; and determine a frame-level switch flag based on the first optimal rate-distortion cost and the second optimal rate-distortion cost.

[0031] In a sixth aspect, an embodiment of the present application provides an encoder, including:

[0032] a second memory configured to store a computer program executable on the second processor;

[0033] The second processor is configured to execute the method described in the second aspect when running the computer program.

[0034] In a seventh aspect, an embodiment of the present application provides a computer storage medium storing a computer program, which implements the method described in the first aspect when executed by a first processor, or implements the method described in the second aspect when executed by a second processor.

[0035] In an eighth aspect, an embodiment of the present application provides a bitstream, which is generated by bit encoding based on information to be encoded; wherein the information to be encoded includes at least one of the following: first syntax element identification information of the current video sequence, second syntax element identification information of the current frame, third syntax element identification information of the current frame, fourth syntax element identification information of the current block, fifth syntax element identification information of the current block, sixth syntax element identification information of the current frame, a residual scaling factor, and an initial residual value of at least one partition block included in the current frame; wherein the current frame includes at least one partition block, and the current block is any one of the at least one partition block.

[0036] The embodiments of the present application provide a filtering method, an encoder, a decoder, a bitstream, and a storage medium. The method comprises parsing a bitstream to obtain a frame-level switch flag based on a network filtering model. The frame-level switch flag is used to determine whether each block in the current frame is filtered. When the frame-level switch flag indicates that any video component of the current block is in use, the block-level use flag is obtained. When the block-level use flag indicates that any video component of the current block is in use, a quantization parameter is obtained. The quantization parameter includes an adjusted block-level quantization parameter or a frame-level optimal quantization parameter. Based on the quantization parameter and the network filtering model, the current block of the current frame is filtered to obtain first residual information of the current block. The encoder and decoder can determine the quantization parameter input to the neural network filtering model based on the adjusted frame-level quantization parameter or the frame-level optimal quantization parameter. The frame-level quantization parameter or the frame-level optimal quantization parameter are both adjusted parameters, different from the original quantization parameter. This allows for flexible selection and diverse change processing of the frame-level quantization parameter without increasing complexity, and the adjustment amplitude of each block can be different, thereby improving coding efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] 1A-1C are exemplary component distribution diagrams in different color formats provided by an embodiment of the present application;

[0038] FIG2 is a schematic diagram of an exemplary division of coding units provided in an embodiment of the present application;

[0039] FIG3A is a first schematic diagram of a network architecture of an exemplary neural network model provided in an embodiment of the present application;

[0040] FIG3B is a composition structure of an exemplary residual block provided in an embodiment of the present application;

[0041] FIG4 is a second schematic diagram of a network architecture of an exemplary neural network model provided in an embodiment of the present application;

[0042] FIG5 is a structural diagram of an exemplary video encoding system provided in an embodiment of the present application;

[0043] FIG6 is a structural diagram of an exemplary video decoding system provided in an embodiment of the present application;

[0044] FIG7 is a flow chart of a filtering method provided in an embodiment of the present application;

[0045] FIG8 is a flow chart of another filtering method provided in an embodiment of the present application;

[0046] FIG9 is a flow chart of another filtering method provided in an embodiment of the present application;

[0047] FIG10 is a schematic diagram of the structure of a decoder provided in an embodiment of the present application;

[0048] FIG11 is a schematic diagram of the hardware structure of a decoder provided in an embodiment of the present application;

[0049] FIG12 is a schematic diagram of the structure of an encoder provided in an embodiment of the present application;

[0050] FIG13 is a schematic diagram of the hardware structure of an encoder provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] In the embodiments of the present application, digital video compression technology primarily compresses large amounts of digital video data for easier transmission and storage. With the surge in Internet video usage and increasing demand for higher video clarity, while existing digital video compression standards can save significant amounts of video data, there is still a need for better digital video compression technologies to reduce bandwidth and traffic pressures associated with digital video transmission.

[0052] During digital video encoding, the encoder reads unequal pixels from raw video sequences in different color formats, including luminance and chrominance components. This means the encoder reads a black-and-white or color image. The image is then divided into blocks and encoded by the encoder, which typically uses a hybrid frame coding scheme, typically involving intra-frame and inter-frame prediction, transform and quantization, inverse transform and inverse quantization, loop filtering, and entropy coding. Intra-frame prediction refers only to information from the same frame, predicting the pixels within the current block to eliminate spatial redundancy. Inter-frame prediction can reference information from different frames, using motion estimation to search for the motion vector that best matches the current block, eliminating temporal redundancy. Transform and quantization convert the predicted image blocks to the frequency domain, redistributing the energy. Combined with quantization, this process removes information that is insensitive to the human eye, eliminating visual redundancy. Entropy coding eliminates character redundancy based on the current context model and the probabilistic information of the binary bitstream. Loop filtering primarily processes the inverse-transformed and inverse-quantized pixels to compensate for distortion and provide a better reference for subsequent pixel encoding.

[0053] Currently, the scenario in which filtering processing can be performed can be the reference software test platform HPM based on AVS or the VVC reference software test platform (VVC TEST MODEL, VTM) based on Versatile Video Coding (VVC), which is not limited in the embodiments of the present application.

[0054] In a video image, a first video component, a second video component, and a third video component are generally used to represent a current block (Coding Block, CB); wherein, the three image components are a luminance component, a blue chrominance component, and a red chrominance component, respectively. The luminance component is usually represented by the symbol Y, the blue chrominance component is usually represented by the symbol Cb or U, and the red chrominance component is usually represented by the symbol Cr or V; in this way, the video image can be represented in YCbCr format, or in YUV format, or even in RGB format, which is not limited in the embodiments of the present application.

[0055] Video compression technology primarily compresses large amounts of digital video data for easier transmission and storage. With the surge in internet video usage and increasing demand for higher-quality video, while existing digital video compression standards can save significant amounts of video data, there is a continued need for better digital video compression technologies to reduce the bandwidth and traffic pressure of digital video transmission. During video encoding, the encoder reads unequal pixels from the original video sequence in different color formats, including the luminance and chrominance color components. In other words, the encoder reads a black-and-white or color image. It then divides the image into blocks, and the block data is passed to the encoder for encoding.

[0056] Typically, digital video compression technology operates on image data encoded in the YCbCr (YUV) color encoding format. The YUV ratio is typically 4:2:0, 4:2:2, or 4:4:4. Y represents luminance (Luma), Cb (U) represents blue chrominance, and Cr (V) represents red chrominance. U and V represent chroma, used to describe color and saturation. Figures 1A through 1C illustrate the distribution of components in different color formats, with white representing the Y component and dark gray representing the UV components. As shown in Figure 1A, a 4:2:0 color format indicates four luminance components and two chrominance components for every four pixels (YYYYCbCr). As shown in Figure 1B, a 4:2:2 format indicates four luminance components and four chrominance components for every four pixels (YYYYCbCrCbCr). As shown in Figure 1C, a 4:4:4 format indicates a full pixel display (YYYYCbCrCbCrCbCrCbCr).

[0057] Currently, common video codec standards all use a block-based hybrid coding framework. Each video frame is divided into square Largest Coding Units (LCUs) of the same size (e.g., 128×128, 64×64, etc.). Each LCU can be further divided into rectangular Coding Units (CUs) according to a rule. Coding Units may also be divided into smaller Prediction Units (PUs). Specifically, the hybrid coding framework may include modules such as prediction, transform, quantization, entropy coding, and in-loop filtering. The prediction module may include intra-frame prediction and inter-frame prediction, and inter-frame prediction may include motion estimation and motion compensation. Since there is a strong correlation between adjacent pixels within a video frame, the use of intra-frame prediction in video codec technology can eliminate spatial redundancy between adjacent pixels. Inter-frame prediction can refer to image information from different frames and use motion estimation to search for the motion vector information that best matches the current partition block to eliminate temporal redundancy; the transformation converts the predicted image block to the frequency domain, redistributes the energy, and combines it with quantization to remove information that the human eye is not sensitive to, thereby eliminating visual redundancy; entropy coding can eliminate character redundancy based on the current context model and the probability information of the binary code stream.

[0058] It should be noted that during the video encoding process, the encoder first reads the image information and divides the image into several coding tree units (CTUs). A coding tree unit can be further divided into several coding units (CUs). These coding units can be rectangular blocks or square blocks. The specific relationship can be shown in Figure 2.

[0059] During intra-frame prediction, the current coding unit cannot reference information from different frames and can only use adjacent coding units from the same frame as reference information for prediction. This means that, based on the current left-to-right, top-to-bottom coding order, the current coding unit can reference the upper-left coding unit, the upper coding unit, and the left coding unit as reference information to predict the current coding unit. The current coding unit then serves as reference information for the next coding unit, thus predicting the entire image. If the input digital video is in color format, the current mainstream digital video encoder input source is in YUV 4:2:0 format, meaning that every four pixels in the image are composed of four Y components and two UV components. The encoder encodes the Y and UV components separately, using slightly different encoding tools and techniques. The decoder also decodes the video according to the different formats.

[0060] Intra-frame prediction in digital video encoding and decoding primarily uses information from adjacent blocks in the current frame to predict the current block. The residual information is then calculated between the predicted block and the original image block. This residual information is then transformed and quantized before being transmitted to the decoder. After receiving and parsing the bitstream, the decoder performs an inverse transform and quantization to obtain the residual information. This residual information is then superimposed on the predicted image block obtained by the decoder to create the reconstructed image block.

[0061] Currently common video codec standards (such as H.266 / VVC) all adopt a block-based hybrid coding framework. Each video frame is divided into square largest coding units (LCUs) of the same size (e.g., 128x128, 64x64, etc.). Each LCU can be divided into rectangular coding units (CUs) according to a rule. Coding units may also be divided into prediction units (PUs) and transform units (TUs). The hybrid coding framework includes modules such as prediction, transform, quantization, entropy coding, and in-loop filtering. The prediction module includes intra-frame prediction and inter-frame prediction. Inter-frame prediction includes motion estimation and motion compensation. Because adjacent pixels in a video frame are strongly correlated, intra-frame prediction is used in video codecs to eliminate spatial redundancy between adjacent pixels. Because adjacent frames in a video have strong similarities, inter-frame prediction is used in video codecs to eliminate temporal redundancy between adjacent frames, thereby improving coding efficiency.

[0062] The basic process of a video codec is as follows. On the encoder side, a frame is divided into blocks. Intra-frame prediction or inter-frame prediction is used on the current block to generate a prediction block for the current block. The prediction block is subtracted from the original image block to obtain a residual block. This residual block is transformed and quantized to obtain a quantization coefficient matrix. This quantization coefficient matrix is ​​entropy-encoded and output to the bitstream. On the decoder side, intra-frame prediction or inter-frame prediction is used on the current block to generate a prediction block for the current block. The bitstream is then parsed to obtain a quantization coefficient matrix. This quantization coefficient matrix is ​​inversely quantized and inversely transformed to obtain a residual block. The prediction block and residual block are added together to obtain a reconstructed block. The reconstructed blocks form a reconstructed image, which is then subjected to image-based or block-based loop filtering to obtain a decoded image. The encoder side also performs similar operations to the decoder side to obtain a decoded image. The decoded image can serve as a reference frame for inter-frame prediction in subsequent frames. The block division information determined by the encoder, as well as information about the prediction, transform, quantization, entropy coding, loop filtering, and other modes or parameters, are output to the bitstream if necessary. The decoding end determines the same block division information as the encoding end by parsing and analyzing the existing information, as well as the mode information or parameter information such as prediction, transformation, quantization, entropy coding, and loop filtering, thereby ensuring that the decoded image obtained by the encoding end is the same as the decoded image obtained by the decoding end. The decoded image obtained by the encoding end is also usually called a reconstructed image. During prediction, the current block can be divided into prediction units, and during transformation, the current block can be divided into transformation units. The division of prediction units and transformation units can be different. The above is the basic process of the video codec under the block-based hybrid coding framework. With the development of technology, some modules or steps of this framework or process may be optimized. The current block can be the current coding unit (CU) or the current prediction unit (PU), etc.

[0063] JVET, the international video coding standards development organization, has established two exploratory experimental groups, one based on neural network coding and the other beyond VVC, and has also set up several corresponding expert discussion groups.

[0064] The above-mentioned exploratory experimental group that surpasses VVC aims to explore higher coding efficiency based on the latest codec standard H.266 / VVC with strict performance and complexity requirements. The coding method studied by this group is closer to VVC and can be called a traditional coding method. At present, the performance of the algorithm reference model of this exploratory experiment has surpassed the coding performance of the latest VVC reference model VTM by about 15%.

[0065] The method studied by the first exploratory group is an intelligent coding method based on neural networks. Deep learning and neural networks are currently hot topics across various industries, especially in computer vision, where deep learning-based methods often have overwhelming advantages. Experts from the JVET standards organization have introduced neural networks to the field of video codecs. Leveraging the powerful learning capabilities of neural networks, neural network-based coding tools often achieve very high coding efficiency. In the early stages of VVC standard development, many companies focused on coding tools based on deep learning, proposing methods including neural network-based intra-frame prediction, neural network-based inter-frame prediction, and neural network-based loop filtering. Among them, the neural network-based loop filtering method has the most outstanding coding performance. After multiple research and exploration meetings, the coding performance has reached over 8%. The neural network-based loop filtering scheme studied by the first exploratory group has currently achieved coding performance as high as 12%, almost contributing to half a generation of coding performance.

[0066] The present embodiment of the present application improves upon the current exploratory experiments conducted at the JVET conference and proposes a neural network-based loop filtering enhancement scheme. The following will first briefly introduce the neural network-based loop filtering scheme currently used at the JVET conference, followed by a detailed description of the improved method of the present embodiment of the present application.

[0067] In the related art, exploration of neural network-based loop filtering solutions has primarily focused on two approaches: the first employing a multi-model intra-frame switchable scheme, and the second employing a non-intra-frame switchable model. Regardless of the approach, the neural network architecture remains largely unchanged, and the tool is incorporated into the in-loop filtering of traditional hybrid coding frameworks. Therefore, the fundamental processing unit for both approaches is the coding tree unit, which is the maximum coding unit size.

[0068] The biggest difference between the first scheme in which multiple models can be switched within a frame and the second scheme in which models cannot be switched within a frame is that the first scheme can switch the neural network model at will when encoding and decoding the current frame, while the second scheme cannot switch the neural network model. Taking the first scheme as an example, when encoding a frame of an image, each coding tree unit has multiple candidate neural network models to choose from. The encoder selects which neural network model to use for the current coding tree unit with the best filtering effect, and then writes the neural network model index into the bitstream. That is, in this scheme, if the coding tree unit needs to be filtered, it is necessary to first transmit a coding tree unit-level usage flag, and then transmit the neural network model index. If filtering is not required, it is only necessary to transmit a coding tree unit-level usage flag; after parsing the index value, the decoder loads the neural network model corresponding to the index into the current coding tree unit to filter the current coding tree unit.

[0069] Taking the second scheme as an example, when encoding a frame of image, the available neural network model for each coding tree unit in the current frame is fixed, and each coding tree unit uses the same neural network model, that is, the second scheme does not have a model selection process at the encoding end; the decoding end parses and obtains the usage flag of whether the current coding tree unit uses the neural network-based loop filter. If the usage flag is true, the pre-set model (the same as the encoding end) is used to filter the coding tree unit. If the usage flag is false, no additional operation is performed.

[0070] For the first multi-model intra-frame switchable solution, it has strong flexibility at the coding tree unit level and can adjust the model according to local details, that is, local optimization to achieve a better global effect. Usually, this solution has more neural network models, and different neural network models are trained under different quantization parameters for the JVET general test conditions. At the same time, different coding frame types may also require different neural network models to achieve better results. Taking a filter in the related technology as an example, the filter uses up to 22 neural network models to cover different coding frame types and different quantization parameters, and the model switching is performed at the coding tree unit level. This filter can provide up to 10% more coding performance based on VVC.

[0071] For the second solution in which the model cannot be switched within a frame, although the solution has two neural network models overall, the model is not switched within the frame. The solution makes a judgment at the encoding end. If the current encoding frame type is an I frame, the neural network model corresponding to the I frame is imported, and only the neural network model corresponding to the I frame is used in the current frame; if the current encoding frame type is a B frame, the neural network model corresponding to the B frame is imported, and similarly only the neural network model corresponding to the B frame is used in the current frame. This solution can provide 8.65% encoding performance based on VVC. Although it is slightly lower than the first solution, the overall performance is a coding efficiency that is almost impossible to achieve compared to traditional encoding tools.

[0072] While the first approach offers greater flexibility and higher encoding performance, it suffers from a significant hardware implementation drawback. Hardware experts are concerned about the code for intra-frame model switching. Switching models at the coding tree unit level means, in the worst case, that the decoder must reload the neural network model for each coding tree unit processed. This, in addition to the hardware implementation complexity, creates an additional burden on current high-performance graphics processing units (GPUs). Furthermore, the presence of multiple models requires a large number of parameters to be stored, which is a significant hardware implementation overhead. However, the second approach, a neural network loop filter, further explores the powerful generalization capabilities of deep learning. It uses a variety of information as input, rather than simply reconstructed samples. This increased information provides more information for neural network learning, enhancing model generalization and eliminating many unnecessary redundant parameters. Continuously updated approaches have now emerged that can adapt to different test conditions and quantization parameters using a single, simplified, low-complexity neural network model. Compared to the first approach, this eliminates the overhead of constantly reloading the model and the need for larger storage space to accommodate a large number of parameters.

[0073] The following will introduce the neural network architecture of these two solutions.

[0074] Refer to Figure 3A, which shows a schematic diagram of the network architecture of a neural network model. As shown in Figure 3A, the main structure of the network architecture can be composed of multiple residual blocks (ResBlocks). The composition structure of the residual block is shown in Figure 3B. In Figure 3B, a single residual block is composed of multiple convolutional layers (Conv) connected to a convolutional attention mechanism module (Convolutional Blocks Attention Module, CBAM) layer. As an attention mechanism module, CBAM is mainly responsible for further extraction of detail features. In addition, the residual block also has a direct skip connection (Skip Connection) structure between the input and output. Here, the multiple convolutional layers in Figure 3B include the first convolutional layer, the second convolutional layer and the third convolutional layer, and the first convolutional layer is also connected to an activation layer. For example, the size of the first convolutional layer is 1×1×k×n, the size of the second convolutional layer is 1×1×n×k, and the size of the third convolutional layer is 3×3×k×k, where k and n are positive integers. The activation layer can include a rectified linear unit (ReLU) function, also known as a linear rectifier function, which is a commonly used activation function in current neural network models. ReLU is actually a ramp function that is simple and converges quickly.

[0075] As for Figure 3A, there is also a skip connection structure in the network architecture, which connects the input reconstructed YUV information to the output of the pixel reorganization (Pixel Shuffle) module. The main function of Pixel Shuffle is to obtain a high-resolution feature map through convolution and multi-channel reorganization of low-resolution feature maps; as an upsampling method, it can effectively amplify the reduced feature map. In addition, the input of the network architecture mainly includes reconstructed YUV information (rec_yuv), predicted YUV information (pred_yuv), and YUV information with partitioning information (par_yuv). All inputs undergo simple convolution and activation operations and are then spliced ​​(Cat) and sent to the main structure. It is worth noting that the processing of YUV information with partitioning information may be different in I frames and B frames. I frames require input of YUV information with partitioning information, while B frames do not.

[0076] In summary, for every JVET-required parameter point in each I-frame and B-frame, the first solution has a corresponding neural network model. Furthermore, because the three YUV color components are primarily composed of two channels, luminance and chrominance, the color components differ.

[0077] Refer to Figure 4, which shows a schematic diagram of the network architecture of another neural network model. As shown in Figure 4, the network architecture of the first scheme is basically the same as the second scheme in the main structure. The difference is that the input of the second scheme adds quantization parameter information as an additional input compared to the first scheme. The first scheme mentioned above loads different neural network models according to different quantization parameter information to achieve more flexible processing and more efficient encoding effects, while the second scheme uses quantization parameter information as the input of the network to improve the generalization ability of the neural network, so that the model can adapt to different quantization parameter conditions and provide good filtering performance.

[0078] As can be seen from Figure 4, there are two quantization parameters that enter the neural network model as input, one is BaseQP and the other is SliceQP. BaseQP here indicates the sequence-level quantization parameter set by the encoder when encoding the video sequence, that is, the quantization parameter point required by the JVET pass test, and is also the parameter used to determine the neural network model in the first solution. SliceQP is the quantization parameter of the current frame. The quantization parameter of the current frame can be different from the sequence-level quantization parameter. This is because in the video encoding process, the quantization conditions of the B frame are different from those of the I frame, and the quantization parameters are different at different time domain levels. Therefore, SliceQP is generally different from BaseQP in the B frame. Therefore, in the relevant technology, the input of the neural network model of the I frame only requires SliceQP, while the neural network model of the B frame requires both BaseQP and SliceQP as input.

[0079] In addition, the second solution differs from the first solution in one respect. The output of the model in the first solution generally does not require additional processing. Specifically, if the model output is residual information, it is superimposed with the reconstructed samples of the current coding tree unit and used as the output of the neural network-based loop filter tool. If the model output is a complete reconstructed sample, the model output is the output of the neural network-based loop filter tool. The output of the second solution generally requires scaling. For example, the model outputs residual information of the current coding tree unit. This residual information is scaled and then superimposed with the reconstructed sample information of the current coding tree unit. This scaling factor is obtained by the encoder and written into the code stream for transmission to the decoder.

[0080] In summary, it is precisely because the quantization parameters serve as additional information that the reduction in the number of models is possible, making it the most popular solution at the current JVET conference. Furthermore, general neural network-based loop filtering solutions may not be identical to the two solutions described above. While the specific details may differ, the underlying concepts remain largely the same. For example, the differences in solution 2 can be reflected in the design of the neural network architecture, such as the convolution size of ResBlocks, the number of convolution layers, and whether an attention module is included. They can also be reflected in the inputs to the neural network, which can even include more additional information, such as the boundary strength value of the deblocking filter.

[0081] The first solution switches neural network models at the coding tree unit level. These different neural network models are trained using different BaseQPs. The encoder tries these different neural network models, and the network model with the lowest rate-distortion cost is the optimal network model for the current coding tree unit. By using the usage flag and network model index information at the coding tree unit level, the decoder can use the same network model as the encoder for filtering. The second solution uses input quantization parameters to achieve good coding performance without switching models, initially resolving concerns about hardware implementation. However, the overall performance of the second solution is still inferior to the first solution. The main drawback is the switching of BaseQP. The second solution lacks flexibility, and the encoder has limited selectivity, resulting in suboptimal performance.

[0082] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0083] The embodiment of the present application provides a video coding system. FIG5 is a schematic diagram of the composition structure of the video coding system of the embodiment of the present application. The video coding system 10 includes: a transform and quantization unit 101, an intra-frame estimation unit 102, an intra-frame prediction unit 103, a motion compensation unit 104, a motion estimation unit 105, an inverse transform and inverse quantization unit 106, a filter control analysis unit 107, a filtering unit 108, a coding unit 109 and a decoded image cache unit 110, etc., wherein the filtering unit 108 can implement DBF filtering / SAO filtering / ALF filtering, and the coding unit 109 can implement header information encoding and context-based adaptive binary arithmetic coding (Context-based Adaptive Binary Arithmetic Coding, CABAC). For the input original video signal, the coding tree unit (Coding Tree A video coding block can be obtained by dividing the video coding block into a plurality of frames (CTUs) and then the residual pixel information obtained after intra-frame or inter-frame prediction is transformed by the transform and quantization unit 101, including transforming the residual information from the pixel domain to the transform domain and quantizing the obtained transform coefficients to further reduce the bit rate; the intra-frame estimation unit 102 and the intra-frame prediction unit 103 are used to perform intra-frame prediction on the video coding block; specifically, the intra-frame estimation unit 102 and the intra-frame prediction unit 103 are used to determine the intra-frame prediction mode to be used to encode the video coding block; the motion compensation unit 104 and the motion estimation unit 105 are used to perform inter-frame prediction coding of the received video coding block relative to one or more blocks in one or more reference frames to provide temporal prediction information; the motion estimation performed by the motion estimation unit 105 is a process of generating a motion vector, which can estimate the motion of the video coding block, and then the motion compensation unit 104 calculates the motion vector based on the motion vector determined by the motion estimation unit 105. After determining the intra-frame prediction mode, the intra-frame prediction unit 103 is further configured to provide the selected intra-frame prediction data to the encoding unit 109, and the motion estimation unit 105 also sends the calculated motion vector data to the encoding unit 109. In addition, the inverse transform and inverse quantization unit 106 is configured to reconstruct the video coding block and reconstruct a residual block in the pixel domain. The reconstructed residual block is subjected to the filter control analysis unit 107 and the filtering unit 108 to remove the block effect artifacts. The reconstructed residual block is then added to a predictive block in the frame of the decoded image buffer unit 110 to generate a reconstructed video coding block. The encoding unit 109 is configured to encode various coding parameters and quantized transform coefficients. In the CABAC-based coding algorithm, the context content can be based on adjacent coding blocks and can be used to encode information indicating the determined intra-frame prediction mode, and output the code stream of the video signal. The decoded image buffer unit 110 is configured to store the reconstructed video coding block for prediction reference.As the video image encoding proceeds, new reconstructed video encoding blocks are continuously generated, and these reconstructed video encoding blocks are stored in the decoded image buffer unit 110 .

[0084] An embodiment of the present application provides a video decoding system. FIG6 is a schematic diagram of the composition structure of the video decoding system of the embodiment of the present application. The video decoding system 20 includes: a decoding unit 201, an inverse transform and inverse quantization unit 202, an intra-frame prediction unit 203, a motion compensation unit 204, a filtering unit 205 and a decoded image cache unit 206, etc., wherein the decoding unit 201 can implement header information decoding and CABAC decoding, and the filtering unit 205 can implement DBF filtering / SAO filtering / ALF filtering. After the input video signal is encoded as shown in FIG3A , a code stream of the video signal is output; the code stream is input to the video decoding system 20 and first passes through the decoding unit 201 to obtain decoded transform coefficients; the transform coefficients are processed by the inverse transform and inverse quantization unit 202 to generate residual blocks in the pixel domain; the intra-frame prediction unit 203 can be used to generate prediction data for the current video decoding block based on the determined intra-frame prediction mode and data from the previously decoded blocks of the current frame or picture; the motion compensation unit 204 determines the prediction information for the video decoding block by analyzing the motion vector and other associated syntax elements, and uses The prediction information is used to generate a predictive block for the video decoding block being decoded; a decoded video block is formed by summing the residual block from the inverse transform and inverse quantization unit 202 with the corresponding predictive block generated by the intra-frame prediction unit 203 or the motion compensation unit 204; the decoded video signal passes through the filtering unit 205 to remove blocking artifacts, thereby improving video quality; the decoded video block is then stored in the decoded image buffer unit 206, which stores reference images used for subsequent intra-frame prediction or motion compensation, and is also used for outputting the video signal, thereby obtaining the restored original video signal.

[0085] It should be noted that the filtering method provided in the embodiment of the present application can be applied to the filtering unit 108 shown in FIG5 (indicated by a black bold box), and can also be applied to the filtering unit 205 shown in FIG6 (indicated by a black bold box). In other words, the filtering method in the embodiment of the present application can be applied to both a video encoding system (referred to as "encoder") and a video decoding system (referred to as "decoder"), and can even be applied to both a video encoding system and a video decoding system, but this is not limited here.

[0086] The embodiment of the present application can provide the encoder with more selectivity and possibilities by adjusting the input parameters on the basis of the solution of not switching the model within the frame. For example, the input parameters of the network filtering model include quantization parameters, and the quantization parameters include sequence-level quantization parameter values ​​(BaseQP) or frame-level quantization parameter values ​​(SliceQP). The BaseQP and SliceQP used as inputs are adjusted to provide the encoding end with multiple options, and the adjustment range is determined by using the coding tree unit level or the coding unit level as the basic unit, so that the codec end has more options to try, thereby improving the encoding and decoding efficiency.

[0087] An embodiment of the present application provides a filtering method, which is applied to a decoder. As shown in FIG7 , the method may include:

[0088] S101, parsing the bitstream to obtain a frame-level switch flag based on a network filtering model; the frame-level switch flag is used to determine whether all blocks in the current frame are filtered;

[0089] In an embodiment of the present application, it can be applied to a loop filtering method based on a neural network model. Specifically, it can be a loop filtering method obtained based on a neural network model with multiple quantization parameter inputs.

[0090] On the decoding end, the decoder uses intra-frame prediction or inter-frame prediction to generate a prediction block for the current block. Simultaneously, the decoder parses the bitstream to obtain a quantization coefficient matrix, dequantizes and inversely transforms the quantization coefficient matrix to obtain a residual block. The prediction block and residual block are added together to obtain a reconstructed block, which is then used to form the reconstructed image. The decoder then performs loop filtering on the reconstructed image, either on an image-based or block-based basis, to produce the decoded image.

[0091] It should be noted that since the original image can be divided into CTUs (coding tree units), or CTUs can be divided into CUs; ​​therefore, the filtering method of the embodiment of the present application can not only be applied to CU-level loop filtering (the block division information at this time is CU division information), but also to CTU-level loop filtering (the block division information at this time is CTU division information), and the embodiment of the present application does not make specific limitations.

[0092] That is to say, in the embodiment of the present application, a block may refer to a CTU or a CU, and the embodiment of the present application does not limit this.

[0093] In an embodiment of the present application, when the decoder performs loop filtering on the reconstructed image of the current frame, the decoder can first parse the bitstream to obtain the sequence-level enable flag (sps_nnlf_enable_flag), i.e., the first syntax element identification information. The sequence-level enable flag is a switch that determines whether the filtering function is enabled for the entire video sequence to be processed. Here, the first syntax element identification information can determine whether the filtering function is enabled for the entire video sequence to be processed.

[0094] It should be noted that the first syntax element identification information (sps_nnlf_enable_flag) can be determined by decoding the value of the identification information. In some embodiments, the decoder can parse the bitstream to determine the first syntax element identification information of the video sequence to be processed. Specifically, the decoder can obtain the value of the first syntax element identification information.

[0095] In some embodiments of the present application, if the value of the first syntax element identification information is a first value, it is determined that the first syntax element identification information indicates that the filtering function is turned on for the video sequence to be processed; if the value of the first syntax element identification information is a second value, it is determined that the first syntax element identification information indicates that the filtering function is not turned on for the video sequence to be processed.

[0096] In the embodiment of the present application, the first value and the second value are different, and the first value and the second value can be in parameter form or in digital form. Specifically, the first syntax element identification information can be a parameter written in the profile or a flag value, which is not specifically limited here.

[0097] For example, taking flag as an example, there are two ways to set the flag: an enable flag (enable_flag) and a disable flag (disable_flag). Assuming that the value of the enable flag is a first value and the value of the disable flag is a second value; then for the first value and the second value, the first value can be set to 1 and the second value can be set to 0; or, the first value can also be set to true (true) and the second value can also be set to false (false); however, this embodiment of the application is not specifically limited to this.

[0098] In some embodiments of the present application, when the sequence level allows the use of an identification bit representation, the decoder parses the syntax elements of the current frame and obtains a frame-level switch identification bit (i.e., the second syntax element identification information) and a frame-level usage identification bit (i.e., the third syntax element identification information) based on a network filtering model.

[0099] The frame-level usage flag indicates whether filtering is used in the current frame. When the frame-level usage flag indicates "used," it indicates that some or all blocks in the current frame require filtering. When the frame-level usage flag indicates "not used," it indicates that all blocks in the current frame do not require filtering. The decoder can then continue to perform other filtering methods to output a complete reconstructed image.

[0100] In some embodiments of the present application, if the value of the second syntax element identification information is a first value, it is determined that the second syntax element identification information indicates that filtering is used in the current frame; if the value of the second syntax element identification information is a second value, it is determined that the second syntax element identification information indicates that filtering is not used in the current frame.

[0101] It should be noted that the default related syntax element identification information is an initial value or is set to a negative state, that is, the second value.

[0102] In some embodiments of the present application, the frame-level usage flag for the current frame may be embodied by one or more flags. When multiple flags are embodied, different video components of the current frame may each correspond to a respective frame-level usage flag. When the video component here is a color component, different color components of the current frame may each correspond to a respective frame-level usage flag, that is, a frame-level usage flag of the color component. The frame-level usage flag of a color component indicates whether filtering is required in the block of the current frame under the color component.

[0103] It should be noted that the decoder traverses the frame level of each color component of the current frame and uses the identification bit to determine whether to perform filtering processing on the blocks under each color component.

[0104] In some embodiments of the present application, a frame-level switch flag is used to determine whether all blocks in the current frame are filtered. In some embodiments of the present application, if the value of the third syntax element identification information is a first value, it is determined that the third syntax element identification information indicates that all blocks in the current frame are filtered; if the value of the third syntax element identification information is a second value, it is determined that the third syntax element identification information indicates that some blocks in the current frame use a network-based loop filtering technology, while some blocks do not use a neural network-based loop filtering technology, and further analysis of the block-level usage flags of all blocks in the current frame is required.

[0105] The frame-level switch flag may correspond to each video component. When the video component is a color component, the frame-level switch flag may also indicate whether all blocks under the current color component are filtered using a neural network-based loop filtering technique.

[0106] In an embodiment of the present application, if the frame-level switch flag is on, it indicates that all blocks under the current color component are filtered using the neural network-based loop filtering technology, that is, the block-level usage flag of all blocks in the current frame under the color component is automatically set to use; if the frame-level switch flag is not on, it indicates that some blocks in the current color component use the neural network-based loop filtering technology, while some blocks do not use the neural network-based loop filtering technology. If the frame-level switch flag is not on, it is necessary to further analyze the block-level usage flag of all blocks in the current frame under the color component.

[0107] It should be noted that the color component may include at least one of the following: a first color component, a second color component, and a third color component. The first color component may be a luminance color component, and the second and third color components may be chrominance color components (for example, the second color component is a blue chrominance color component, and the third color component is a red chrominance color component; or the second color component is a red chrominance color component, and the third color component is a blue chrominance color component).

[0108] For example, if the color component is a luma color component, the second syntax element identification information may be ph_nnlf_luma_enable_flag, and the third syntax element identification information may be ph_nnlf_luma_ctrl_flag; if the color component is a chroma color component, the second syntax element identification information may be ph_nnlf_chroma_enable_flag, and the third syntax element identification information may be ph_nnlf_chroma_ctrl_flag. In other words, different second syntax element identification information and third syntax element identification information are set for different color components in the current frame.

[0109] S102: When the frame-level switch flag indicates that it is not turned on, and the obtained frame-level usage flag indicates that any video component of the current block is in use, obtain a block-level usage flag;

[0110] In an embodiment of the present application, when the decoder frame-level switch flag indicates that it is not turned on, and the obtained frame-level usage flag indicates that any video component of the current block is in use, the block-level usage flag can also be parsed from the bitstream.

[0111] It should be noted that the block-level usage flag indicates whether filtering is used for the block to be processed in the current frame. When the block-level usage flag indicates "used," filtering is required for the current block; and when the block-level usage flag indicates "not used," filtering is not required for the current block.

[0112] In an embodiment of the present application, the decoder can parse the code stream to determine the fourth syntax element identification information of the current block. Specifically, the decoder can obtain the value of the fourth syntax element identification information.

[0113] In some embodiments of the present application, if the value of the fourth syntax element identification information is the first value, it is determined that the fourth syntax element identification information indicates that the current block is filtered; if the value of the fourth syntax element identification information is the second value, it is determined that the fourth syntax element identification information indicates that the current block is not filtered.

[0114] In the embodiment of the present application, the frame-level usage flag indicates that any video component of the current block is in use, which means that as long as the frame-level usage flag corresponding to any video component, such as any color component, is in use, the block-level usage flag is obtained.

[0115] It should be noted that the frame-level switch flag that is not enabled is the frame-level switch flag corresponding to the video component corresponding to the frame-level usage flag being used.

[0116] In some embodiments of the present application, the block-level usage flag of the current block includes the block-level usage flag corresponding to each color component (video component). When the video component is a color component, the block-level usage flag can also indicate whether the current block under the current color component is filtered using a network-based loop filtering technology.

[0117] In an embodiment of the present application, if the block-level usage flag is used, it means that the current block under the current color component is filtered using a network-based loop filtering technology; if the block-level usage flag is not used, it means that the current block under the current color component is not filtered using a network-based loop filtering technology.

[0118] It should be noted that the block-level usage flag is the fourth syntax element identification information, and for different color components, it can be represented by ctb_nnlf_luma_flag (corresponding to the luminance component) and ctb_nnlf_chroma_flag (corresponding to the chrominance component).

[0119] In some embodiments of the present application, if the value of the fourth syntax element identification information is a first value, it is determined that the fourth syntax element identification information indicates that the current block uses network filtering; if the value of the fourth syntax element identification information is a second value, it is determined that the fourth syntax element identification information indicates that the current block does not use network filtering.

[0120] It should be noted that when the frame-level switch flag is not enabled, the frame-level usage flag can be parsed from the bitstream.

[0121] For example, for the luma component, if ph_nnlf_luma_ctrl_flag is not enabled, parse the bitstream for ph_nnlf_luma_enable_flag. If (ph_nnlf_luma_enable_flag) is enabled, perform for (traverse all ctbs) and parse ctb_nnlf_luma_flag. For the chroma component, if ph_nnlf_chroma_ctrl_flag is not enabled, parse the bitstream for ph_nnlf_chroma_enable_flag. If (ph_nnlf_chroma_enable_flag) is enabled, perform for (traverse all ctbs) and parse ctb_nnlf_chroma_flag.

[0122] S103: When the block-level usage flag indicates that any video component of the current block is in use, obtaining a quantization parameter; wherein the quantization parameter includes: an adjusted block-level quantization parameter or a frame-level optimal quantization parameter;

[0123] In an embodiment of the present application, when a block-level usage flag indicates that any video component of the current block is in use, a quantization parameter is obtained; wherein the quantization parameter includes: an adjusted block-level quantization parameter or a frame-level optimal quantization parameter.

[0124] It should be noted that in the embodiment of the present application, when the decoder performs decoding, it also performs decoding operations on each color component of the current frame. When the frame-level usage flag corresponding to a color component is in use, but the frame-level start flag corresponding to the color component is not turned on, the decoder parses the block-level usage flag corresponding to the color component. If the block-level usage flag is characterized as in use, the quantization parameter is obtained; wherein the quantization parameter includes: an adjusted block-level quantization parameter or a frame-level optimal quantization parameter.

[0125] In the embodiment of the present application, for the current block, as long as any color component corresponding to the block-level usage flag is used, the decoder needs to perform filtering on the current block to obtain the residual information corresponding to each color component. Therefore, for the current block, as long as any color component corresponding to the block-level usage flag is used, it is necessary to obtain the quantization parameter for use in filtering.

[0126] In some embodiments of the present application, a decoder may obtain a quantization parameter. Specifically, the decoder obtains a block-level quantization parameter offset index from a bitstream and obtains an adjusted block-level quantization parameter based on the block-level quantization parameter offset index.

[0127] In some embodiments of the present application, the decoder obtains a block-level quantization parameter offset index based on the bitstream, where the block-level quantization parameter offset index can directly correspond to the block-level input parameter.

[0128] It should be noted that the block-level quantization offset index parsed by the decoder from the bitstream corresponds to the adjusted block-level quantization parameter, or may correspond to the unadjusted block-level quantization parameter, subject to the actual parsing result, and is not limited in this embodiment of the present application. This is determined by whether the block-level quantization parameter was adjusted during encoding.

[0129] In some embodiments of the present application, the decoder may further determine a block-level quantization offset parameter based on a block-level quantization parameter offset index obtained from the bitstream; and determine an adjusted block-level quantization parameter based on an original quantization parameter (e.g., BaseQP or SliceQP) obtained from the bitstream and the block-level quantization offset parameter. The block-level quantization parameter offset index is the fifth syntax element identification information, such as ctb_nnlf_qp_index.

[0130] It should be noted that in this embodiment of the present application, if the frame-level switch flag is the second value (i.e., not turned on) and the frame-level usage flag is in use, then for each block of the current frame, the decoder needs to parse the block-level usage flag in the bitstream, i.e., the fourth syntax element identification information. When the fourth syntax element identification information indicates that any video component of the current block is in use, the fifth syntax element identification information in the bitstream is parsed to perform the subsequent filtering process.

[0131] It should be noted that when the block-level usage flag indicates that any video component of the current block is enabled (i.e., if (ph_nnlf_luma_enable_flag||ph_nnlf_chroma_enable_flag)), the decoder can parse the fifth syntax element identifier (e.g., ctb_nnlf_qp_index) from the bitstream for the current block of the current frame. The decoder's processing principles for other blocks are similar and are not further detailed here.

[0132] In some embodiments of the present application, the adjustment amplitudes of all blocks in the current frame may not be the same, or may be different, and the embodiments of the present application do not limit this.

[0133] It should be noted that when encoding, the encoder will try to adjust the original block-level quantization parameters to obtain different candidate quantization parameters to determine if the quantization parameters need to be adjusted. If adjustment is required, the serial number corresponding to the adjusted block-level quantization bias parameter will be transmitted to the bitstream as the block-level quantization adjustment index. The decoder stores the correspondence between the serial number and the quantization bias parameter. In this way, the decoder can determine the block-level quantization bias parameter based on the block-level quantization adjustment index. The decoder uses the block-level quantization bias parameter and the original quantization parameter to adjust the block-level quantization parameter to obtain the adjusted block-level quantization parameter. Among them, the original quantization parameter can be obtained from the bitstream.

[0134] It should be noted that the decoder obtains the adjusted block-level quantization parameter, which can be the block-level quantization parameter index corresponding to the block-level quantization offset parameter parsed from the bitstream. The block-level quantization offset parameters corresponding to different blocks are further superimposed based on the quantization parameter to obtain the adjusted block-level quantization parameter corresponding to the current block. The current block of the current frame is then filtered based on the adjusted block-level quantization parameter and the network filter model to obtain the first residual information of the current block.

[0135] In the embodiment of the present application, the adjustments between different blocks may be different, that is, the quantization parameter inputs of different blocks may be different.

[0136] For example, if the block-level quantization parameter of the current frame is adjusted, the decoder can adjust the original quantization parameter according to the block-level quantization parameter index. For example, if the block-level quantization parameter index points to offset1, BaseQP is superimposed on the offset parameter offset1 to obtain BasseQPFinal, which replaces BaseQP as the adjusted quantization parameter for the current block of the current frame and is input into the network filter model.

[0137] It should be noted that the decoder can also parse and obtain the unadjusted block-level quantization parameter, which is determined by whether the block-level quantization parameter is adjusted during encoding, subject to the actual parsing result.

[0138] In some embodiments of the present application, the manner in which the decoder obtains the quantization parameter may further include: the decoder obtains the optimal quantization parameter at the frame level.

[0139] It should be noted that the frame-level optimal quantization parameter is the optimal quantization parameter of the current frame calculated by the encoder. Based on the frame-level optimal quantization parameter as input, a network filtering model is used for filtering. When decoding, the decoder can directly parse the bitstream to obtain the frame-level optimal quantization parameter, or it can obtain the quantization parameter difference and obtain the frame-level optimal quantization parameter based on the quantization parameter difference and the original quantization parameter. This is not limited in the embodiments of the present application.

[0140] In some embodiments of the present application, the encoder may transmit the frame-level optimal quantization parameter or quantization parameter difference directly to the decoder through the bitstream for use by the decoder during decoding.

[0141] It should be noted that for the current block, as long as any color component is represented by a block-level usage flag, the decoder needs to filter the current block to obtain the residual information corresponding to each color component. Therefore, for the current block, as long as any color component is represented by a block-level usage flag, the quantization parameter needs to be obtained for use during filtering.

[0142] S104 : Filter the current block of the current frame based on the quantization parameter and the network filtering model to obtain first residual information of the current block.

[0143] In an embodiment of the present application, after the decoder obtains the level quantization parameter, since the frame-level switch flag indicates that it is not turned on, and the obtained frame-level usage flag indicates that any video component of the current block is in use, the decoder needs to obtain the block-level usage flag, and then determine whether the current block needs to be filtered. When it is determined that filtering is required, the decoder can filter the current block of the current frame according to the obtained quantization parameter and the network filtering model to obtain the first residual information of the current block.

[0144] The first residual information includes the residual information corresponding to each color component. The decoder determines the reconstruction value of the color component of the current block based on the block-level usage flag corresponding to each color component. If the block-level usage flag corresponding to the color component is "used", the target reconstruction value corresponding to the color component is the sum of the reconstruction value of the color component of the current block and the residual information of the filtering output under the color component. If the block-level usage flag corresponding to the color component is "used", the target reconstruction value corresponding to the color component is the reconstruction value of the color component of the current block.

[0145] In an embodiment of the present application, the decoder obtains a reconstructed value of the current block before filtering the current block of the current frame based on the quantization parameter and the network filter model to obtain the first residual information of the current block. The decoder uses the network filter model and combines the quantization parameter to filter the reconstructed value of the current block to obtain the first residual information of the current block to complete the filtering of the current block.

[0146] It can be understood that the decoder can determine the quantization parameters of the input neural network filter model based on the adjusted block-level quantization parameters or frame-level optimal quantization parameters, and the block-level quantization parameters or frame-level optimal quantization parameters are adjusted parameters, which are different from the original quantization parameters. This can achieve flexible selection and diverse change processing of block-level quantization parameters, and the adjustment amplitude of each block can be different, thereby improving decoding efficiency.

[0147] In some embodiments of the present application, the decoder filters the current block of the current frame based on the quantization parameter and the network filtering model, and before obtaining the first residual information of the current block, it can also obtain the predicted value of the current block, block partition information and at least one of the deblocking filter boundary strength, and the reconstructed value of the current block.

[0148] In some embodiments of the present application, the decoder uses a network filtering model, combined with the prediction value of the current block, block partition information and at least one of the deblocking filter boundary strength, and the quantization parameter, to filter the reconstructed value of the current block to obtain the first residual information of the current block to complete the filtering of the current block.

[0149] That is, the decoder obtains at least one of the predicted value, block partition information and deblocking filter boundary strength of the current block, and the reconstructed value of the current block; when the frame-level switch flag indicates that it is not turned on, and the obtained frame-level usage flag indicates that any video component of the current block is in use, the block-level usage flag is obtained; when the block-level usage flag indicates that any video component of the current block is in use, the adjusted block-level input parameters are obtained; wherein the adjusted block-level input parameters include: parameters after any one of the parameters among the predicted value, block partition information and deblocking filter boundary strength is adjusted; finally, based on the adjusted block-level input parameters, the obtained original quantization parameter and the network filtering model, the current block of the current frame is filtered to obtain third residual information of the current block.

[0150] It should be noted that during the filtering process, the input parameters input to the network filtering model may include: the predicted value of the current block, block partition information, deblocking filter boundary strength, the reconstructed value of the current block, and the quantization parameter in this application. This application does not limit the type of information of the input parameters. However, the predicted value of the current block, block partition information, and deblocking filter boundary strength are not necessarily required every time and need to be determined based on actual conditions.

[0151] In some embodiments of the present application, the decoder filters the current block of the current frame based on the quantization parameter and the network filter model to obtain first residual information of the current block, and then obtains a second residual scaling factor in the code stream; scales the first residual information of the current block based on the second residual scaling factor to obtain first target residual information; and when the block-level usage flag is used, determines the first target reconstruction value of the current block based on the first target residual information and the reconstruction value of the current block. When the block-level usage flag is not used, determines the reconstruction value of the current block as the first target reconstruction value.

[0152] It should be noted that when the encoder obtains the first residual information, it can use the second residual scaling factor to scale the first residual information to obtain the first residual information. Therefore, the decoder needs to scale the first residual information of the current block based on the second residual scaling factor to obtain the first target residual information. When the block level uses the identification bit representation, the first target reconstruction value of the current block is determined based on the first target residual information and the reconstruction value of the current block. However, if the encoder does not use the residual factor during encoding, but also needs to input the quantization parameter during filtering, the filtering method provided in the embodiment of the present application is also applicable, except that the first residual information does not need to be scaled using the residual factor.

[0153] It should be noted that each color component has corresponding residual information and residual factors.

[0154] In some embodiments of the present application, during the filtering process of encoding and decoding, part of the data in the input parameters of the network filtering model can be adjusted using the aforementioned principles and then filtered, which is not limited in the embodiments of the present application.

[0155] In an embodiment of the present application, at least one of the quantization parameter, the prediction value of the current block, the block partitioning information, and the deblocking filter boundary strength in the input parameters can be adjusted, and the embodiment of the present application does not limit this.

[0156] Exemplarily, if the block-level usage flags for all color components of the current block are not all unused, a neural network-based loop filtering technique is used to filter the current coding tree unit. The current block's reconstructed YUV samples, the current block's predicted YUV samples, the current block's partitioning information (YUV), and the quantization parameter information are used as input to obtain residual information for the current block. The quantization parameter information is adjusted based on a block-level quantization parameter adjustment index or is the optimal frame-level quantization parameter calculated by the encoder. The decoder can also scale the residual information using a residual scaling factor obtained by parsing the bitstream. The scaled residual is then superimposed on the reconstructed samples to obtain reconstructed YUV samples after the neural network-based loop filtering. Based on the block usage flags for each color component of the current block, a reconstructed sample is selected as the neural network-based loop filtering output. If the block-level usage flag for the corresponding color component is used, the reconstructed sample after the neural network-based loop filtering for that color component is used as the output; otherwise, the reconstructed sample that has not undergone the neural network-based loop filtering is used as the output for that color component. After traversing all blocks of the current frame, the neural network-based loop filtering module terminates. It should be noted that the decoding end can also continue to traverse other loop filtering methods and output a complete reconstructed image after completion.

[0157] It can be understood that the decoder can determine the quantization parameters of the input neural network filter model based on the adjusted block-level quantization parameters or frame-level optimal quantization parameters, and the block-level quantization parameters or frame-level optimal quantization parameters are adjusted parameters, which are different from the original quantization parameters. This can achieve flexible selection and diverse change processing of block-level quantization parameters, and the adjustment amplitude of each block can be different, thereby improving decoding efficiency.

[0158] In some embodiments of the present application, a filtering method provided by an embodiment of the present application may further include:

[0159] S101, parsing the code stream to obtain a frame-level switch flag based on a network filtering model; the frame-level switch flag is used to determine whether all blocks in the current frame are filtered.

[0160] It should be noted that S101 has been described above and will not be repeated here.

[0161] S105 : When the frame-level switch flag indicates that the signal is on, obtain a frame-level quantization parameter; the frame-level quantization parameter includes an adjusted frame-level quantization parameter or an optimal frame-level quantization parameter.

[0162] In an embodiment of the present application, when the frame-level quantization parameter is an adjusted frame-level quantization parameter, the decoder obtains the frame-level quantization parameter in the following manner: determining the frame-level quantization bias parameter based on the frame-level quantization parameter bias index obtained in the bitstream; and determining the adjusted frame-level quantization parameter based on the obtained original quantization parameter and the frame-level quantization bias parameter.

[0163] It should be noted that when the frame-level switch flag is turned on, the decoder obtains the frame-level quantization parameter. The decoder can obtain the frame-level quantization parameter offset index from the bitstream and determine the adjusted frame-level quantization parameter based on the frame-level quantization parameter offset index.

[0164] Here, the adjustment amplitudes of all blocks in the current frame are the same, that is, the inputs of the adjusted frame-level quantization parameters of all blocks are the same.

[0165] It should be noted that when encoding, if the encoder determines that the frame-level quantization parameter needs to be adjusted, it will transmit the serial number corresponding to the frame-level quantization bias parameter as the frame-level quantization parameter bias index to the bitstream. The decoder stores the correspondence between the serial number and the quantization bias parameter. In this way, the decoder can determine the frame-level quantization bias parameter based on the frame-level quantization parameter bias index. The decoder uses the frame-level quantization bias parameter to adjust the original quantization parameter to obtain the adjusted frame-level quantization parameter. The original quantization parameter can be obtained from the bitstream.

[0166] For example, if the frame-level quantization parameter of the current frame is adjusted, the original quantization parameter is adjusted according to the frame-level quantization parameter offset index. For example, if the frame-level quantization parameter offset index points to offset1, BaseQP is superimposed on the offset parameter offset1 to obtain BasseQPFinal, which replaces BaseQP as the frame-level quantization parameter for all coding tree units of the current frame and is input into the network model.

[0167] In some embodiments of the present application, when the frame-level switch flag is turned on, the frame-level quantization parameter can also be the frame-level optimal quantization parameter, which is the optimal quantization parameter of the current frame calculated by the encoder. Based on the frame-level optimal quantization parameter as input, a network filtering model is used for filtering. When decoding, the decoder can directly parse the code stream to obtain the frame-level optimal quantization parameter, or it can obtain the quantization parameter difference, and obtain the frame-level optimal quantization parameter based on the combination of the quantization parameter difference and the original quantization parameter. This is not limited to the embodiments of the present application.

[0168] In some embodiments of the present application, the encoder may transmit the frame-level optimal quantization parameter or quantization parameter difference directly to the decoder through the bitstream for use by the decoder during decoding.

[0169] It should be noted that, in the code stream, the frame-level quantization parameter offset index (eg, ph_nnlf_qp_index), the frame-level optimal quantization parameter, or the quantization parameter difference is embodied in the sixth syntax element identification information.

[0170] It should be noted that in the embodiment of the present application, if the frame-level switch flag is not all the second value (i.e., not enabled), the value of the frame-level enable / disable flag can be obtained by parsing or directly assigning a value based on the condition of the frame-level switch flag. If the frame-level switch flag is enabled, the frame-level usage flag is enabled.

[0171] For example, if (ph_nnlf_luma_ctrl_flag||ph_nnlf_chroma_ctrl_flag), ph_nnlf_luma_enable_flag=ph_nnlf_luma_ctrl_flag; ph_nnlf_luma_chroma_flag=ph_nnlf_chroma_ctrl_flag.

[0172] It should be noted that when the decoder is in the frame level switch flag representation turned on, the frame level usage flag is used. At this time, the sixth syntax element identification information can be parsed from the bitstream to obtain the frame level quantization parameter.

[0173] S106 : Filter the current block of the current frame based on the frame-level quantization parameter and the network filtering model to obtain second residual information of the current block.

[0174] In an embodiment of the present application, after the decoder obtains the frame-level quantization parameter, since the frame-level switch flag indicates that it is turned on, the decoder can filter all blocks of the current frame. For filtering of a block, it is necessary to traverse the filtering processing of each color component before decoding the next block.

[0175] It should be noted that when the decoder parses and finds that the frame-level switch flag is turned on, the frame-level usage flag is used, so all blocks of the current frame will be filtered, that is, the block-level usage flags are all used, and the block-level quantization parameters at this time are all frame-level quantization parameters.

[0176] For example, for (traverse all ctbs), ctb_nnlf_luma_flag=ph_nnlf_luma_ctrl_flag, ctb_nnlf_chroma_flag=ph_nnlf_chroma_ctrl_flag, ctb_nnlf_qp_index=ph_nnlf_qp_index.

[0177] In an embodiment of the present application, the decoder uses a network filtering model and combines the adjusted frame-level quantization parameter to filter the reconstructed value of the current block to obtain the second residual information of the current block to complete the filtering of the current block.

[0178] In an embodiment of the present application, the decoder obtains a reconstructed value of the current block before filtering the current block of the current frame based on the frame-level quantization parameter and the network filtering model to obtain the second residual information of the current block. The network filtering model is used to filter the reconstructed value of the current block and the frame-level quantization parameter to obtain the second residual information of the current block, thereby completing the filtering of the current block.

[0179] In some embodiments of the present application, the decoder may further obtain at least one of a prediction value of the current block, block partition information, and a deblocking filter boundary strength, as well as a reconstructed value of the current block.

[0180] In some embodiments of the present application, the decoder can also use a network filtering model to combine the prediction value of the current block, block partition information and at least one of the deblocking filter boundary strength, and frame-level quantization parameters to filter the reconstructed value of the current block to obtain the second residual information of the current block to complete the filtering of the current block.

[0181] In some embodiments of the present application, after the decoder filters the current block of the current frame based on the frame-level quantization parameter and the network filtering model to obtain the second residual information of the current block, the decoder can also obtain a second residual scaling factor in the code stream; scale the second residual information of the current block based on the second residual scaling factor to obtain second target residual information; and determine the second target reconstruction value of the current block based on the second target residual information and the reconstruction value of the current block.

[0182] It should be noted that when the encoder obtains the second residual information, it can use the second residual scaling factor to scale the second residual information to obtain the second residual information. Therefore, the decoder needs to scale the second residual information of the current block based on the second residual scaling factor to obtain the second target residual information, and determine the second target reconstruction value of the current block based on the second target residual information and the reconstruction value of the current block. However, if the encoder does not use the residual factor during encoding, but also needs to input the quantization parameter during filtering, the filtering method provided in the embodiment of the present application is also applicable, except that the second residual information does not need to be scaled using the residual factor.

[0183] It should be noted that the decoding end continues to traverse other loop filtering methods and outputs a complete reconstructed image after completion.

[0184] It can be understood that the decoder can determine the quantization parameters of the input neural network filter model based on the adjusted frame-level quantization parameters or the frame-level optimal quantization parameters, and the frame-level quantization parameters or the frame-level optimal quantization parameters are adjusted parameters, which are different from the original quantization parameters. This can achieve flexible selection and diverse change processing of frame-level quantization parameters, and the adjustment amplitude of each block can be different, thereby improving decoding efficiency.

[0185] An embodiment of the present application provides a filtering method, which is applied to an encoder. As shown in FIG8 , the method may include:

[0186] S201 : When the obtained sequence level allows the use of a flag to represent the permission, obtain an original value of a current block in a current frame, a reconstructed value of the current block, and a first quantization parameter; the first quantization parameter includes: an original quantization parameter.

[0187] In an embodiment of the present application, the encoder traverses intra-frame or inter-frame prediction to obtain the prediction block of each coding unit. The residual of the coding unit can be obtained by subtracting the original image block from the prediction block. The residual obtains the frequency domain residual coefficient by various transformation modes, which is then quantized and inversely quantized. After inverse transformation, the distortion residual information is obtained. The distortion residual information is superimposed on the prediction block to obtain the reconstructed block of the current block. After the image is encoded, the loop filter module filters the image with the coding tree unit level as the basic unit. The embodiment of the present application can describe the coding tree unit as a block, but the block is not limited to CTU, but can also be CU, which is not limited in the embodiment of the present application. The encoder obtains the sequence-level permission flag based on the network filtering model, i.e., sps_nnlf_enable_flag. If the sequence-level permission flag is allowed, the neural network-based loop filtering technology is allowed; if the sequence-level permission flag is not allowed, the neural network-based loop filtering technology is not allowed. The sequence-level permission flag needs to be written into the bitstream when encoding the video sequence.

[0188] In an embodiment of the present application, when the sequence level allows the use of an identification bit to indicate permission, the encoding end attempts a loop filtering technology based on a network filtering model, and the encoder obtains the original value of the current block in the current frame, the reconstructed value of the current block, and the first quantization parameter; if the sequence level based on the network filtering model allows the use of an identification bit to indicate that it is not allowed, the encoding end does not attempt the network-based loop filtering technology; it continues to try other loop filtering tools, such as LF filtering, and outputs a complete reconstructed image after completion.

[0189] S202. Perform at least two filter estimates on the current block based on the network filtering model, the reconstructed value of the current block, at least one frame-level quantization offset parameter, and the original quantization parameter to determine at least two first sub-rate-distortion costs of the current block, and traverse the current frame to determine at least two first rate-distortion costs of the current frame.

[0190] S203: Determine a first optimal rate-distortion cost based on at least two first rate-distortion costs.

[0191] In an embodiment of the present application, when encoding the current block of the current frame, the encoder may use at least one frame-level quantization bias parameter to adjust the original quantization parameter to obtain at least one candidate quantization parameter. Then, based on the at least one candidate quantization parameter and the original quantization parameter, the current block is filtered and estimated at least twice in combination with the network filtering model and the reconstruction value of the current block to obtain at least two corresponding first sub-rate distortion costs. The number of filtering times is consistent with the total number of at least one candidate quantization parameter and the original quantization parameter. By performing the above filtering process on each block of the current frame, at least two first sub-rate distortion costs corresponding to each block can be obtained. The first sub-rate distortion costs corresponding to each quantization parameter are superimposed to obtain a first rate distortion cost for the current frame, that is, the current frame is traversed to determine at least two first rate distortion costs of the current frame.

[0192] In some embodiments of the present application, the encoder performs filter estimation on the current block based on the network filtering model, the reconstructed value of the current block and the original quantization parameter to determine a first reconstruction value; performs rate-distortion cost estimation on the first reconstructed value and the original value of the current block to obtain a first sub-rate-distortion cost of the current block, and traverses the current frame to determine a first rate-distortion cost of the current frame; based on the network filtering model, at least one frame-level quantization bias parameter, the original quantization parameter and the reconstructed value of the current block in the current frame, performs filter estimation on the current block at least once to determine at least one of the first sub-rate-distortion costs of the current block, and traverses the current frame to determine at least one first rate-distortion cost (costFrame2 and costFrame3) of the current frame.

[0193] In an embodiment of the present application, for a current frame, an encoder performs filter estimation on the current block based on a network filter model, a reconstructed value of the current block, and an original quantization parameter to determine first estimated residual information; determines a first residual scaling factor; scales the first estimated residual value using the first residual scaling factor to obtain first scaled residual information; and combines the first scaled residual information with the reconstructed value of the current block to determine a first reconstructed value. Alternatively, the first estimated residual information may be combined with the reconstructed value of the current block without scaling to determine the first reconstructed value.

[0194] In some embodiments of the present application, before determining the first residual scaling factor, for the current frame, at least one of a predicted value, block partition information, and a deblocking filter boundary strength of the current block, and a reconstructed value of the current block are obtained; using a network filtering model, combined with the predicted value, block partition information, and at least one of the deblocking filter boundary strength of the current block, and a frame-level quantization parameter, a filtered estimation is performed on the reconstructed value of the current block to obtain the first residual information of the current block, and the first residual scaling factor is written into the bitstream.

[0195] It should be noted that the encoder first performs a rate-distortion cost estimation based on the first reconstructed value obtained by filtering the original quantization parameter without adjustment and the original value of the current block to obtain a first sub-rate-distortion cost (costCTU1) of the current block; traverses all blocks in the current frame to determine a first rate-distortion cost (i.e., costFrame1) of the current frame; finally, the encoder attempts to perform at least one filtering estimation by changing the input parameters (e.g., quantization parameters) input to the network filter model at least once to obtain at least one first rate-distortion cost of the current frame. Specifically, the encoder adjusts the original quantization parameter based on at least one frame-level quantization offset parameter to obtain at least one candidate quantization parameter. For each candidate quantization parameter, the reconstructed value of the current block is filtered once to determine another first sub-rate-distortion cost (costCTU2 or costCTU3) of the current block, and traverses all blocks in the current frame to determine a first rate-distortion cost (costFrame2 or costFrame3) of the current frame.

[0196] In an embodiment of the present application, after obtaining the first reconstructed value of the current block, the encoder performs rate-distortion cost estimation on the first reconstructed value and the original value of the current block to obtain a first sub-rate-distortion cost of the current block, and then continues encoding the next block until the first sub-rate-distortion costs of all blocks in the current frame are obtained. The first sub-rate-distortion costs of all blocks are then added together to obtain a first rate-distortion cost for the current frame.

[0197] In some embodiments of the present application, the encoder performs at least one filter estimation on the current block based on the network filter model, at least one frame-level quantization offset parameter, the original quantization parameter, and the reconstructed value of the current block in the current frame, and determines at least one first sub-rate-distortion cost of the current block, including:

[0198] Obtaining an i-th frame-level quantization bias parameter, and adjusting the original quantization parameter based on the i-th frame-level quantization bias parameter to obtain an i-th adjusted frame-level quantization parameter; i is a positive integer greater than or equal to 1;

[0199] Performing filtering estimation on the current block based on the network filtering model, the reconstruction value of the current block and the i-th adjusted frame-level quantization parameter to obtain the i-th third reconstruction value;

[0200] The i-th third reconstruction value and the original value of the current block are used to estimate the rate-distortion cost. After traversing all blocks of the current frame, the i-th first sub-rate-distortion cost is obtained. The i+1-th filter estimation is continued based on the i+1-th frame-level quantization offset parameter until at least one third reconstruction value is obtained, thereby determining at least one first sub-rate-distortion cost (costCTU2 and costCTU3) of the current block.

[0201] In an embodiment of the present application, the encoder performs rate-distortion cost estimation on the i-th third reconstruction value and the original value of the current block, traverses all blocks of the current frame, adds the rate-distortion costs of all blocks to obtain the i-th first sub-rate-distortion cost, and continues to perform the i+1-th filtering estimation based on the i+1-th frame-level quantization bias parameter until the filtering estimation of all blocks is completed and the first rate-distortion cost of the current frame is obtained, until at least one round of filtering is completed and at least one first sub-rate-distortion cost of the current block is obtained.

[0202] In an embodiment of the present application, the encoder performs a filtering estimation on the current block based on the network filtering model, the reconstruction value of the current block and the i-th adjusted frame-level quantization parameter to obtain the i-th second estimated residual information; determines the i-th second residual scaling factor corresponding to the i-th adjusted frame-level quantization parameter; uses the i-th second residual scaling factor to scale the i-th second estimated residual information to obtain the i-th second scaled residual information; combines the i-th second scaled residual information with the reconstruction value of the current block to determine the i-th third reconstruction value.

[0203] In some embodiments of the present application, the encoder may further obtain the predicted value of the current block, at least one of the block partitioning information and the deblocking filter boundary strength, and the reconstructed value of the current block; and utilize the network filtering model, in combination with the predicted value of the current block, at least one of the block partitioning information and the deblocking filter boundary strength, and the i-th adjusted frame-level quantization parameter, to perform filtering estimation on the reconstructed value of the current block to obtain the i-th second estimated residual information of the current block.

[0204] In some embodiments of the present application, an encoder may select whether to adjust the original quantization parameter of the current frame based on the encoding frame type. For frames of the first type, the original quantization parameter needs to be adjusted, while for frames of the second type, the original quantization parameter is not adjusted. The second type of frame is a frame type other than the first type. During encoding, the encoder may adjust the original quantization parameter to perform filter estimation when the current frame is of the first type.

[0205] In some embodiments of the present application, when the current frame is a first type frame, based on a network filtering model, at least one frame-level quantization bias parameter, and an original quantization parameter, at least one filtering estimation is performed on the reconstructed value of the current block in the current frame to determine at least one first rate-distortion cost of the current frame.

[0206] It should be noted that, in the embodiment of the present application, the first type of frame may be a B frame or a P frame, and the embodiment of the present application does not limit this.

[0207] For example, the encoder may adjust the BaseQP and SliceQP used as input, so that the encoding end has more options to try, thereby improving encoding efficiency.

[0208] The above adjustments to BaseQP and SliceQP include both unified adjustments for all coding tree units within a frame and individual adjustments to coding tree units or coding units. For unified adjustments to all coding tree units within a frame, adjustments are made to both I and B frames, and the magnitude of the adjustments is the same for all coding tree units within the frame, meaning the quantization parameter inputs for all coding tree units are the same. For individual adjustments to coding tree units, adjustments are made to both I and B frames, and the magnitude of the adjustments for all coding tree units within the frame can be selected based on rate-distortion optimization at the encoder end based on the current coding tree unit. Adjustments can vary between different coding tree units, meaning the quantization parameter inputs for different coding tree units can be different.

[0209] It can be understood that the encoder can determine whether the block-level quantization parameters of the input neural network filter model need to be adjusted based on the block-level quantization parameter adjustment flag, thereby realizing flexible selection and diverse change processing of block-level quantization parameters, thereby improving coding efficiency.

[0210] In some embodiments of the present application, after obtaining a first rate-distortion cost determined based on filtering of the original quantization parameter and at least one first rate-distortion cost determined based on filtering after adjusting the original quantization parameter, the encoder can determine a first optimal rate-distortion cost (i.e., costFrameBest) with the smallest rate-distortion cost.

[0211] In some embodiments of the present application, the encoder obtains a first optimal rate-distortion cost with the smallest rate-distortion cost among at least two first rate-distortion costs; the encoder can obtain a frame-level quantization parameter bias index corresponding to the first optimal rate-distortion cost; the frame-level quantization parameter bias index is determined based on at least one frame-level quantization bias parameter and an original quantization parameter.

[0212] It should be noted that the first rate-distortion cost is the rate-distortion cost corresponding to the filtering operation. In the embodiment of the present application, there are two cases: one is filtering based on the original quantization parameter, and the other is filtering after adjusting the original quantization parameter. The encoder can determine whether it is necessary to adjust the frame-level quantization parameter during filtering based on the first rate-distortion cost (corresponding to the original quantization parameter) and at least one first rate-distortion cost (at least one candidate quantization parameter, i.e., the quantization parameter after the original quantization parameter is adjusted).

[0213] In the embodiment of the present application, if the first rate-distortion cost corresponding to the original quantization parameter is the smallest, the original quantization parameter is written into the bitstream. If a first rate-distortion cost corresponding to the adjusted original quantization parameter is the smallest, the index of the frame-level quantization offset parameter corresponding to the adjusted original quantization parameter is used as the frame-level quantization parameter offset index corresponding to the first optimal rate-distortion cost.

[0214] S204: Determine a block-level usage flag and a first sub-optimal rate-distortion cost of the current block based on at least two first sub-rate-distortion costs, and traverse the current frame to determine a second optimal rate-distortion cost of the current frame.

[0215] S205 : Determine a frame-level switch flag based on the first best rate-distortion cost and the second best rate-distortion cost.

[0216] After determining at least two first sub-rate-distortion costs of the current block, the encoder may determine a block-level usage flag and a first sub-best rate-distortion cost (costCTUBest) of the current block based on the at least two first sub-rate-distortion costs, traverse all blocks of the current frame, and add up the first sub-best rate-distortion costs corresponding to each block of the current frame to obtain a second best rate-distortion cost (i.e., costBest) of the current frame.

[0217] In some embodiments of the present application, the encoder determines a first sub-best rate-distortion cost with the smallest rate-distortion cost from the third sub-rate-distortion cost and at least two first sub-rate-distortion costs of the current block; if the first sub-best rate-distortion cost is the third sub-rate-distortion cost, the block-level usage flag is determined to be unused; if the first sub-best rate-distortion cost is any one of the at least two first sub-rate-distortion costs, the block-level usage flag is determined to be used.

[0218] It should be noted that if the first sub-optimal rate-distortion cost is any one of at least two first sub-rate-distortion costs, the encoder needs to obtain the block-level quantization parameter offset index of the current block for subsequent use in the decoder.

[0219] In some embodiments of the present application, if the first sub-optimal rate-distortion cost among the at least two first sub-rate-distortion costs is a first sub-rate-distortion cost corresponding to a candidate quantization parameter adjusted from an original quantization parameter, then a block-level quantization parameter offset index corresponding to the first sub-optimal rate-distortion cost is obtained; wherein the block-level quantization parameter offset index is determined based on at least one frame-level quantization offset parameter and the original quantization parameter. That is, the block-level quantization parameter offset index is a quantization parameter offset index corresponding to a candidate quantization parameter.

[0220] It should be noted that, in the embodiment of the present application, the encoder encodes different video components during the process of encoding the current frame, such as color components, which may include luminance components and chrominance components.

[0221] In an embodiment of the present application, for different color components, the encoder can obtain the first sub-best rate-distortion cost and the third sub-rate-distortion cost (costCTUorg) of the current block under different color components based on the aforementioned principles. For the luminance component, if the third sub-rate-distortion cost is the smallest among the first sub-best rate-distortion cost and the third sub-rate-distortion cost, the block-level usage flag (ctb_nnlf_luma_flag) corresponding to the current block under the luminance component is the second value. If the first sub-best rate-distortion cost is the smallest among the first sub-best rate-distortion cost and the third sub-rate-distortion cost, the block-level usage flag (ctb_nnlf_luma_flag) corresponding to the current block under the luminance component is the first value, and the block-level quantization parameter offset index corresponding to the first sub-best rate-distortion cost is recorded.

[0222] In an embodiment of the present application, for the chroma component, if the block-level usage flag corresponding to the current block under the luma component is the first value, and if, under the candidate quantization parameter corresponding to the first sub-best rate-distortion cost of the luma component, the first sub-rate-distortion cost of the chroma component is less than the third sub-rate-distortion cost corresponding to the chroma component, then the block-level usage flag (ctb_nnlf_chroma_flag) corresponding to the current block under the chroma component is the first value. Otherwise, if, under the candidate quantization parameter corresponding to the first sub-best rate-distortion cost of the luma component, the first sub-rate-distortion cost of the chroma component is greater than or equal to the third sub-rate-distortion cost corresponding to the chroma component, then the block-level usage flag (ctb_nnlf_chroma_flag) corresponding to the current block under the chroma component is the second value.

[0223] In the embodiment of the present application, the encoder determines a frame-level switch flag based on the first best rate-distortion cost and the second best rate-distortion cost, and writes the frame-level switch flag into the bitstream.

[0224] In some embodiments of the present application, the encoder may further perform a rate-distortion cost estimation based on the original value of the current block and the reconstructed value of the current block in the current frame, determine a third sub-rate-distortion cost (costCTUorg) of the current block, and continue encoding the next block until the third sub-rate-distortion costs of all blocks in the current frame are obtained. The third sub-rate-distortion costs of all blocks are then added together to obtain a fourth rate-distortion cost (costOrg).

[0225] In the embodiment of the present application, after obtaining the first best rate-distortion cost, the second best rate-distortion cost, and the fourth rate-distortion cost, the encoder determines a first minimum rate-distortion cost with the minimum rate-distortion cost from the first best rate-distortion cost, the second best rate-distortion cost, and the fourth rate-distortion cost;

[0226] If the first minimum rate-distortion cost is the fourth rate-distortion cost, determining that the frame-level switch flag is not enabled;

[0227] If the first minimum rate-distortion cost is the first optimal rate-distortion cost, the frame-level switch flag is determined to be on; the encoder writes the frame-level switch flag and the frame-level quantization parameter offset index (corresponding to the frame-level quantization offset parameter) into the bitstream.

[0228] If the first minimum rate-distortion cost is the second best rate-distortion cost, it is determined that the frame-level switch flag is not enabled.

[0229] In some embodiments of the present application, if the first minimum rate-distortion cost is the second optimal rate-distortion cost, the frame-level switch flag is determined to be not turned on and the frame-level usage flag is determined to be used; the encoder writes the frame-level usage flag, the frame-level switch flag, the block-level quantization parameter bias index, and the block usage flag into the bitstream.

[0230] In some embodiments of the present application, the encoder may further write the frame-level quantization offset parameter corresponding to the first optimal rate-distortion cost into the bitstream from at least one frame-level quantization offset parameter, or write the frame-level quantization parameter offset index (offset sequence number) of the frame-level quantization offset parameter corresponding to the first optimal rate-distortion cost into the bitstream. This is not limited in the embodiments of the present application.

[0231] In some embodiments of the present application, after the encoder determines the frame-level switch flag based on the first best rate-distortion cost and the second best rate-distortion cost, if the first minimum rate-distortion cost is the first best rate-distortion cost, a second residual scaling factor corresponding to the first minimum rate-distortion cost is written into the bitstream.

[0232] It should be noted that in the embodiment of the present application, each color component uses the above encoding estimation to obtain a different frame-level rate-distortion cost. In other words, a first optimal rate-distortion cost, a second optimal rate-distortion cost, and a fourth rate-distortion cost can be obtained for each color component.

[0233] In this embodiment of the present application, each color component is traversed. If the value of the fourth rate-distortion cost costOrg is the smallest, the frame-level switch flag corresponding to the color component is set to the second value and written into the bitstream. If the value of the first best rate-distortion cost costFrameBest is the smallest, the frame-level switch flag (ph_nnlf_luma_ctrl_flag / ph_nnlf_chroma_ctrl_flag) corresponding to the color component is set to the first value, and the determined frame-level quantization parameter offset index is also written into the bitstream. If the value of the second best rate-distortion cost costBest is the smallest, the frame-level usage flag corresponding to the color component is set to the first value, the frame-level switch flag is set to the second value, and the block-level usage flag and block-level quantization parameter offset index (i.e., BaseQP index) of each block are also written into the bitstream.

[0234] It can be understood that the encoder can determine the frame-level switch flag, the frame-level usage flag, the block-level usage flag, and the block-level quantization parameter bias index based on the fourth rate-distortion cost without filtering, the first best rate-distortion cost based on filtering (based on the original quantization parameter and the candidate quantization parameter), and the second best rate-distortion cost obtained by superimposing the minimum rate-distortion cost at the block level, and determine whether the quantization parameter of the input network filter model needs to be adjusted, thereby realizing flexible selection of the quantization parameter and diverse change processing, thereby improving coding efficiency.

[0235] An embodiment of the present application provides a filtering method, which is applied to an encoder. As shown in FIG9 , the method may include:

[0236] S201. When the obtained sequence level allows the use of an identification bit to represent the permission, obtain the original value of the current block in the current frame, the reconstructed value of the current block, and a first quantization parameter; the first quantization parameter includes: a frame-level optimal quantization parameter; the frame-level optimal quantization parameter is an optimal quantization parameter directly calculated by the encoder.

[0237] In the embodiment of the present application, except for the step of obtaining the first quantization parameter which is different from the aforementioned S201 , the other steps of the encoder are the same.

[0238] It should be noted that when the acquired sequence level allows the use of a flag to represent the permission, the encoder can obtain the optimal frame-level quantization parameter. This optimal frame-level quantization parameter is the optimal quantization parameter directly calculated by the encoder. Specifically, when the acquired sequence level allows the use of a flag to represent the permission, the encoder obtains the original value and the reconstructed value of the current block in the current frame; and directly calculates the optimal frame-level quantization parameter for the current frame.

[0239] S206 , performing filter estimation on the current block based on the network filter model, the reconstruction value of the current block and the frame-level optimal quantization parameter to determine a second reconstruction value.

[0240] S207 , performing rate-distortion cost estimation on the second reconstructed value and the original value of the current block to obtain a second sub-rate-distortion cost of the current block, and traversing the current frame to determine a third rate-distortion cost of the current frame (second round costFrame).

[0241] S208 , performing rate-distortion cost estimation based on the original value of the current block and the reconstructed value of the current block in the current frame, determining a third sub-rate-distortion cost of the current block, and traversing the current frame to determine a fourth rate-distortion cost.

[0242] S209: Based on the second sub-rate-distortion cost and the third sub-rate-distortion cost of the current block, determine the block-level usage flag and the second sub-best rate-distortion cost of the current block, and traverse the current frame to determine the third best rate-distortion cost of the current frame (third round costBest).

[0243] In the embodiment of the present application, the encoder also determines the best filtering method based on comparison of several situations including no filtering, filtering with optimal frame-level quantization parameters, and block-level rate-distortion cost.

[0244] In an embodiment of the present application, the encoder may perform a filter estimation on the current block using the network filter model, the reconstructed value of the current block, and the optimal frame-level quantization parameter to determine a second reconstructed value. The encoder inputs the optimal frame-level quantization parameter into the network filter model and filters the reconstructed value of the current block to obtain the second reconstructed value.

[0245] In the embodiment of the present application, in the case of filtering: the encoder performs a rate-distortion cost estimation on the second reconstructed value and the original value of the current block to obtain the second sub-rate-distortion cost (costCTUnn) of the current block, traverses each block of the current frame, and determines the third rate-distortion cost (costFrame) of the current frame. In the embodiment of the present application, after obtaining the second reconstructed value of the current block, the encoder performs a rate-distortion cost estimation on the second reconstructed value and the original value of the current block to obtain the rate-distortion cost of the current block, and continues encoding the next block until the rate-distortion costs of all blocks of the current frame are obtained, and then adds up the rate-distortion costs of all blocks to obtain the third rate-distortion cost of the current frame.

[0246] Exemplarily, the encoder attempts a neural network-based loop filtering technique, using the reconstructed sample YUV, predicted sample YUV, YUV with partitioning information, and frame-level optimal quantization parameters (BaseQP and SliceQP) of the current block as input into the network filtering model for inference. The network filtering model outputs the estimated residual information after filtering of the current block, and scales the estimated residual information. The scaling factor in the scaling operation is calculated based on the original image samples of the current frame, the reconstructed samples that have not undergone neural network loop filtering, and the reconstructed samples that have undergone neural network loop filtering. The scaling factors for different color components are different and, when necessary, are all written into the bitstream for transmission to the decoder. The encoder superimposes the scaled residual information on the reconstructed samples that have not undergone neural network loop filtering and outputs it. The encoder calculates the rate-distortion cost value based on the current block after neural network loop filtering and the original image samples of the current block, traverses each block, and obtains the third rate-distortion cost, costFrame, recorded as the current frame.

[0247] In the embodiment of the present application, in the case of no filtering, the encoder performs a rate-distortion cost estimate based on the original value of the current block and the reconstructed value of the current block in the current frame, determines the third sub-rate-distortion cost (costCTUorg) of the current block, and traverses each block of the current frame to determine the fourth rate-distortion cost (costOrg). The acquisition of the fourth rate-distortion cost is consistent with the previous embodiment and will not be repeated in this embodiment.

[0248] It should be noted that the acquisition of the rate-distortion cost when the encoder performs filtering and when the encoder does not perform filtering is consistent with the principle of the aforementioned embodiment and will not be repeated here.

[0249] In some embodiments of the present application, the encoder also needs to determine the block-level usage flag and the second sub-best rate-distortion cost (costCTUBest) of the current block based on the second sub-rate-distortion cost and the third sub-rate-distortion cost of the current block, and traverse each block of the current frame to determine the third best rate-distortion cost (costBest) of the current frame.

[0250] In some embodiments of the present application, the encoder determines the block-level usage flag and the second sub-best rate-distortion cost of the current block based on the second sub-rate-distortion cost and the third sub-rate-distortion cost of the current block as follows: if the third sub-rate-distortion cost is less than the second sub-rate-distortion cost, then the block-level usage flag is determined to be unused, and the second sub-best rate-distortion cost is the third sub-rate-distortion cost; if the third sub-rate-distortion cost is greater than or equal to the second sub-rate-distortion cost, then the block-level usage flag is determined to be used, and the second sub-best rate-distortion cost is the second sub-rate-distortion cost.

[0251] In the embodiment of the present application, different video components, such as color components, can correspond to respective frame-level switch flags, frame-level usage flags, block-level usage flags, and the like. Each color component uses the above encoding to estimate a different frame-level rate-distortion cost. In other words, for each color component, a second sub-rate-distortion cost and a third sub-rate-distortion cost for the current block can be obtained.

[0252] In an embodiment of the present application, for different color components, the encoder can obtain the second sub-rate distortion cost and the third sub-rate distortion cost of the current block under different color components based on the aforementioned principles. For the luma component, if the third sub-rate distortion cost is the smallest among the third sub-rate distortion cost and the second sub-rate distortion cost, the block-level usage flag (ctb_nnlf_luma_flag) corresponding to the current block under the luma component is set to the second value. If the second sub-rate distortion cost is the smallest among the second sub-rate distortion cost and the third sub-rate distortion cost, the block-level usage flag (ctb_nnlf_luma_flag) corresponding to the current block under the luma component is set to the first value.

[0253] In this embodiment of the present application, for the chroma component, if the second sub-rate-distortion cost of the chroma component is less than the third sub-rate-distortion cost corresponding to the chroma component, the block-level usage flag (ctb_nnlf_chroma_flag) corresponding to the current block under the chroma component is set to the first value. Otherwise, if the second sub-rate-distortion cost of the chroma component is greater than or equal to the third sub-rate-distortion cost corresponding to the chroma component, the block-level usage flag (ctb_nnlf_chroma_flag) corresponding to the current block under the chroma component is set to the second value.

[0254] It should be noted that, under each color component, the second sub-optimal rate-distortion cost of each block is superimposed to obtain the third optimal rate-distortion cost of the current frame under each color component.

[0255] S210: Determine a frame-level switch flag based on the third rate-distortion cost, the fourth rate-distortion cost, and the third best rate-distortion cost.

[0256] In the embodiment of the present application, the encoder determines the minimum rate-distortion cost based on the third rate-distortion cost, the fourth rate-distortion cost, and the third best rate-distortion cost, and then determines the frame-level switch flag.

[0257] In some embodiments of the present application, the encoder determines the implementation of the frame-level switch flag based on the third rate-distortion cost, the fourth rate-distortion cost, and the third best rate-distortion cost as follows:

[0258] determining a third minimum rate-distortion cost from the third rate-distortion cost, the fourth rate-distortion cost, and the third best rate-distortion cost;

[0259] If the third minimum rate-distortion cost is the fourth rate-distortion cost, determining that the frame-level switch flag is not enabled;

[0260] If the third minimum rate-distortion cost is the third rate-distortion cost, determining that the frame-level switch flag is on; writing the frame-level switch flag and the frame-level optimal quantization parameter into the bitstream;

[0261] If the third minimum rate-distortion cost is the third best rate-distortion cost, it is determined that the frame-level switch flag is not enabled.

[0262] In some embodiments of the present application, if the third minimum rate-distortion cost is the third optimal rate-distortion cost, the frame-level switch flag is determined to be disabled and the frame-level usage flag is determined to be enabled; the frame-level usage flag, the frame-level switch flag, the frame-level optimal quantization parameter, and the block usage flag are written into the bitstream.

[0263] In the embodiment of the present application, different video components, such as color components, can correspond to their own frame-level switch flags, frame-level usage flags, block-level usage flags, etc. Each color component uses the above encoding estimation to obtain a different frame-level rate-distortion cost. In other words, for each color component, a third rate-distortion cost, a fourth rate-distortion cost, and a third-best rate-distortion cost can be obtained.

[0264] Traverse each color component. If the fourth rate-distortion cost, costOrg, is the smallest, set the frame-level switch flag corresponding to that color component to the second value and write it into the bitstream. If the third rate-distortion cost, costFrame, is the smallest, set the frame-level switch flag (ph_nnlf_luma_ctrl_flag / ph_nnlf_chroma_ctrl_flag) corresponding to that color component to the first value, and write the determined frame-level quantization parameter offset index into the bitstream. If the third rate-distortion cost, costBest, is the smallest, set the frame-level usage flag corresponding to that color component to the first value and the frame-level switch flag to the second value. Also, write the block-level usage flag and frame-level quantization parameter offset index (BaseQP index) of each block into the bitstream.

[0265] It can be understood that the encoder can determine the quantization parameters of the input neural network filter model based on the frame-level optimal quantization parameters, and the frame-level optimal quantization parameters are all adjusted parameters, which are different from the original quantization parameters. This can achieve flexible selection and diverse change processing of frame-level quantization parameters, thereby improving decoding efficiency.

[0266] In some embodiments of the present application, the input parameters input to the network filter model can be at least one of the predicted value of the current block, block partition information, and deblocking filter boundary strength, the reconstructed value of the current block, and the original level quantization parameter, and can also include other information, which is not limited in the embodiments of the present application. The encoder can adjust any one of the original quantization parameter, the predicted value of the current block, the block partition information, and the deblocking filter boundary strength to perform filter estimation, which is not limited in the embodiments of the present application.

[0267] In some embodiments of the present application, when the sequence level allows the use of an identification bit to represent permission, the encoder obtains a prediction value of the current block, at least one of the block partition information and the deblocking filter boundary strength, the reconstruction value of the current block, and a first quantization parameter; based on the prediction value of the current block, at least one of the block partition information and the deblocking filter boundary strength, the network filtering model, the reconstruction value of the current block, and the first quantization parameter, the current block is filtered and estimated to determine a fourth reconstruction value; the fourth reconstruction value and the original value of the current block are subjected to rate-distortion cost estimation to obtain a fourth sub-rate-distortion cost of the current block, and the current frame is traversed to determine a fifth rate-distortion cost of the current frame; based on the prediction value of the current block, at least one of the block partition information and the deblocking filter boundary strength, the network filtering model, at least one frame-level input bias parameter, and the reconstruction value of the current block in the current frame, the current frame is filtered and estimated at least once to determine at least one sixth rate-distortion cost of the current frame; based on the fifth rate-distortion cost and the at least one sixth rate-distortion cost, a frame-level switch identification bit is determined.

[0268] It should be noted that the input parameters input into the network filtering model can be determined according to actual conditions, and the embodiments of the present application do not limit this.

[0269] It should be noted that the encoder transmits the frame-level start flag, frame-level usage flag, block-level usage flag, and sequence-level allowed usage flag, as well as the frame-level quantization parameter offset index and block-level quantization parameter offset index, in the bitstream as syntax element identification information. The specific representation of syntax element identification information has been explained in the decoder embodiment and will not be repeated here.

[0270] It is understandable that the encoder can estimate whether the input parameters of the input network filter model need to be adjusted, determine the adjustment method with the best encoding effect, and realize flexible selection and diverse change processing of input parameters, thereby improving encoding efficiency.

[0271] This embodiment of the application provides a code stream,

[0272] In another embodiment of the present application, based on the decoding method and encoding method described in the aforementioned embodiments, the embodiment of the present application proposes an adjustment of the BaseQP as input, so that the encoder has more options to try, thereby improving encoding efficiency. The main idea is to provide the encoder with multiple options and use the coding tree unit level or the coding unit level as the basic unit to determine the adjustment range.

[0273] The encoder provides multiple BaseQP candidates or multiple offset value candidates superimposed on the BaseQP. The coding tree unit or coding unit inputs each candidate into the neural network, infers and calculates the reconstructed sample block after filtering, and obtains the corresponding rate-distortion cost. The reconstructed sample block corresponding to the candidate with the lowest cost is selected as the output sample of the current filtering technology, and the BaseQP or offset value corresponding to the minimum cost candidate is recorded and written into the bitstream through indexing or direct binarization to be transmitted to the decoder. The decoder parses the neural network-based loop filtering usage flag of the current coding tree unit or coding unit and parses and calculates the aforementioned BaseQP or offset value. If a neural network-based loop filtering technology is used, the final BaseQP is determined as the BaseQP of the current coding tree unit or coding unit and input into the neural network model. The reconstructed sample output from the neural network model is obtained as the output sample of the current filtering technology.

[0274] In a specific embodiment, for the encoding end, the specific process is as follows:

[0275] The encoder traverses the intra-frame or inter-frame prediction to obtain the prediction block of each coding unit. The residual of the coding unit can be obtained by subtracting the original image block from the prediction block. The residual obtains the frequency domain residual coefficient through various transformation modes, which is then quantized and inversely quantized. After inverse transformation, the distortion residual information is obtained. The distortion residual information is superimposed on the prediction block to obtain the reconstructed block. After the image is encoded, the loop filtering module filters the image with the coding tree unit level as the basic unit. The technical solution of the embodiment of the present application is applied here. Obtain the flag that allows the use of loop filtering based on the neural network model, that is, sps_nnlf_enable_flag. If the flag is true, the loop filtering technology based on the neural network model is allowed to be used; if the flag is false, the loop filtering technology based on the neural network model is not allowed to be used. The flag that allows the use of the sequence level can be written into the bitstream when encoding the video sequence.

[0276] Step 1: If the flag bit indicating the use of loop filtering based on the neural network model is true, the encoder attempts the loop filtering technology based on the neural network model, i.e., executes step 2. If the flag bit indicating the use of loop filtering based on the neural network model is false, the encoder does not attempt the loop filtering technology based on the neural network model, i.e., skips step 2 and directly executes step 3.

[0277] Step 2: Initialize the neural network-based loop filtering technology and load the neural network model applicable to the current frame.

[0278] First round:

[0279] The encoder calculates the cost information of not using the loop filtering technology based on the neural network model. That is, it uses the reconstructed samples of the coding tree unit to be used as the input of the neural network model and the original image samples of the coding tree unit to calculate the rate-distortion cost value, which is recorded as costOrg;

[0280] Second round:

[0281] The encoder tries the neural network-based loop filtering technology, uses the reconstructed sample YUV and quantization parameter BaseQP of the current coding tree unit, and inputs them into the loaded network model for inference. The neural network loop filtering model outputs the reconstructed sample block of the current coding tree unit. The rate-distortion cost value is calculated based on the coding tree unit samples after neural network loop filtering and the original image samples of the coding tree unit, which is recorded as costFrame1. The encoder continues to try other BaseQP candidates or use offset value compensation as the input BaseQP. In this embodiment, taking two candidates as an example, the rate-distortion cost values ​​of the two BaseQPs are calculated and recorded as costFrame2 and costFrame3 respectively. The BaseQP with the minimum cost is taken as the optimal output of this round and its index information is recorded, which is recorded as costFrameBest.

[0282] Round 3:

[0283] The encoder attempts to optimize the selection at the coding tree unit level. In the second round, the encoder attempts to use the neural network model loop filter by default for all coding tree units in the current frame. The luminance component and chrominance component are each controlled by a frame-level switch flag, while the coding tree unit level does not need to transmit the flag. This round attempts to use a combination of flags at the coding tree unit level, and each color component can be controlled separately. The encoder traverses the coding tree units and calculates the rate-distortion cost of the reconstructed samples without the neural network model loop filter and the original samples of the current coding tree unit, which is recorded as costCTUorg; the rate-distortion cost of the reconstructed samples based on the neural network loop filter and the original samples of the current coding tree unit are calculated for each BaseQP, which are recorded as costCTU1, costCTU2 and costCTU3 respectively.

[0284] For the luminance component, if the costCTUorg of the current luminance component is smaller than the costCTU of any luminance component, the usage flag (ctb_nnlf_luma_flag) of the neural network model loop filter at the coding tree unit level of the luminance component is set to false; otherwise, the ctb_nnlf_luma_flag is set to true, and the BaseQP parameter offset index corresponding to the minimum cost of costCTU1, costCTU2 or costCTU3 on the luminance component is recorded.

[0285] For chroma components, if ctb_nnlf_luma_flag is true, determine whether the chroma component costCTU corresponding to the current luma component's minimum cost BaseQP is less than costCTUorg. If so, set the chroma component's coding tree unit's neural network loop filter-based flag (ctb_nnlf_chroma_flag) to true; if greater than or equal to costCTUorg, set ctb_nnlf_chroma_flag to false. If ctb_nnlf_luma_flag is false, the execution process is the same as for luma components.

[0286] If all coding tree units in the current frame have been traversed, the rate-distortion cost value of the reconstructed sample of the current frame and the original image sample is calculated, costBest.

[0287] Traverse each color component. If the value of costOrg (from the first round) is the smallest, set the frame-level neural network loop filter switch flag corresponding to that color component to false and write it into the bitstream. If the value of costFrameBest (from the second round) is the smallest, set the frame-level neural network loop filter switch flag (ph_nnlf_luma_ctrl_flag / ph_nnlf_chroma_ctrl_flag) corresponding to that color component to true, and write the frame-level quantization parameter adjustment index determined in the second round into the bitstream. If the value of costBest (from the third round) is the smallest, set the frame-level neural network loop filter usage flag corresponding to that color component to true, set the frame-level switch flag to false, and write the coding tree unit usage flag and BaseQP index determined in the third round into the bitstream.

[0288] Step 3: The encoder continues to try other loop filtering tools and outputs a complete reconstructed image after completion. The specific process is not related to the technical solution of the embodiment of the present application and is therefore not described in detail here.

[0289] In another specific embodiment, for the decoding end, the specific process is as follows:

[0290] The decoding end parses the sequence-level flag. If sps_nnlf_enable_flag is true, it indicates that the current bitstream allows the use of loop filtering technology based on the neural network model, and the subsequent decoding process needs to parse the relevant syntax elements. Otherwise, it indicates that the current bitstream does not allow the use of loop filtering technology based on the neural network model, and the subsequent decoding process does not need to parse the relevant syntax elements. The default relevant syntax elements are initial values ​​or to the false state.

[0291] Step 1. The decoder parses the syntax elements of the current frame to obtain the frame-level switch flag and the frame-level usage flag based on the neural network model. If the frame-level switch flag is not all negative, execute step 2; otherwise, skip step 2 and execute step 3.

[0292] Step 2: parse or directly assign ph_nnlf_luma_enable_flag / ph_nnlf_chroma_enable_flag based on whether the parsed ph_nnlf_luma_ctrl_flag / ph_nnlf_chroma_ctrl_flag is true or false.

[0293] If the frame-level switch flag (ph_nnlf_luma_ctrl_flag / ph_nnlf_chroma_ctrl_flag) is true, it means that all coding tree units under the current color component are filtered using the neural network-based loop filtering technology, that is, the coding tree unit level use flag of all coding tree units in the current frame under this color component is automatically set to true;

[0294] Otherwise, it means that some CTUs use the neural network-based loop filtering technology, while some do not. Therefore, if the frame-level switch flag is false and the frame-level use flag is true, it is necessary to further analyze the CTU-level use flag (ctb_nnlf_luma_flag / ctb_nnlf_chroma_flag) of all CTUs in the current frame under this color component.

[0295] If ph_nnlf_ctrl_flag is true, the use flag of all coding tree units is set to true by default, and the BaseQP adjustment index (ph_nnlf_qp_index) of the current frame is parsed, the final BaseQPFinal as input is calculated and applied to all coding tree units of the current frame;

[0296] Otherwise, the frame-level usage flag is parsed. If the frame-level usage flag is true, the usage flag of each coding tree unit and the BaseQP adjustment index (ctb_nnlf_qp_index) are parsed and applied to the corresponding coding tree unit to calculate the final BaseQPFinal as input.

[0297] The module is initialized before performing neural network-based loop filtering on the current frame, and the network model is loaded for subsequent inference.

[0298] If the coding tree unit-level usage flags for all color components of the current coding tree unit are not all set to "no," the current coding tree unit is filtered using a neural network-based loop filtering technique. The reconstructed YUV samples and quantization parameter information of the current coding tree unit are used as input. The network model performs inference and obtains the reconstructed YUV samples of the current coding tree unit after the neural network-based loop filtering.

[0299] According to the coding tree unit usage identification bit of each color component of the current coding tree unit, the reconstructed sample is selected as the output of the neural network-based loop filtering technology. If the coding tree unit usage identification bit of the corresponding color component is true, the reconstructed sample after the neural network-based loop filtering of the above-mentioned corresponding color component is used as the output; otherwise, the reconstructed sample that has not undergone the neural network-based loop filtering is used as the output of the color component.

[0300] After traversing all coding tree units of the current frame, the neural network-based loop filtering module ends.

[0301] Step 3: The decoding end continues to traverse other loop filtering tools and outputs a complete reconstructed image after completion. The specific process is not related to the technical solution of the embodiment of the present application and is therefore not described in detail here.

[0302] In another specific embodiment, a brief description of the parsing process at the decoding end is shown in Table 1, wherein bold fonts indicate grammatical elements that need to be parsed.

[0303] Table 1

[0304]

[0305]

[0306] In the above syntax table, the chrominance component is represented by only one chroma flag, but in some embodiments, it can also be represented by Cb and Cr respectively, which will not be repeated here.

[0307] In another embodiment of the present application, the encoding end calculates the optimal BaseQP value of the current entire frame, intra-frame coding tree unit or intra-frame coding unit through a non-standard technical method, and the coding tree unit or coding unit uses the optimal BaseQP value as the input of the neural network loop filtering model based on the current basic unit. The reconstructed sample block after filtering is calculated using the model inference, and the rate-distortion cost value is obtained. If the cost value is less than the rate-distortion cost value obtained by the reconstructed sample value without using the neural network loop filtering, the neural network-based loop filtering technology is not used, and the usage flag position of the current frame or coding tree unit level is set to "no"; otherwise, the neural network-based loop filtering technology is used, and the usage flag position of the current frame or coding tree unit level is set to "true". The flag bit is written into the code stream, and the optimal BaseQP value is written into the code stream at the frame level or the basic unit level, and transmitted to the decoding end. The transmission method is not repeated here. The decoding end parses the neural network loop filtering usage flag of the current frame or coding tree unit. If the flag is true, it further parses and calculates the corresponding optimal BaseQP value and inputs it into the neural network model, obtaining the reconstructed sample output from the neural network model as the output sample of the current filtering technology.

[0308] In a specific embodiment, for the encoding end, the specific process is as follows:

[0309] The encoder performs intra-frame or inter-frame prediction to obtain a prediction block for each coding unit. The residual for each coding unit is obtained by subtracting the original image block from the prediction block. This residual is transformed using various transform modes to obtain frequency-domain residual coefficients. This residual is then quantized and inverse-quantized, and then inverse-transformed to obtain the distortion residual information. This distortion residual information is then superimposed with the prediction block to produce the reconstructed block. After the image is encoded, the loop filter module filters the image at the coding tree unit level. This is where the technology proposed in this article is applied. A flag (sps_nnlf_enable_flag) is obtained to enable the use of neural network-based loop filtering. If this flag is true, the neural network-based loop filtering technique is enabled; if it is false, the neural network-based loop filtering technique is disabled. The sequence-level enable flag must be written into the bitstream when encoding a video sequence.

[0310] Step 1: If the flag bit indicating the use of loop filtering based on the neural network model is true, the encoder attempts the loop filtering technology based on the neural network model, i.e., executes step 2. If the flag bit indicating the use of loop filtering based on the neural network model is false, the encoder does not attempt the loop filtering technology based on the neural network model, i.e., skips step 2 and directly executes step 3.

[0311] Step 2: Initialize the neural network-based loop filtering technology and load the neural network model applicable to the current frame.

[0312] First round:

[0313] The encoder calculates the cost information of not using the loop filtering technology based on the neural network model. That is, it uses the reconstructed samples of the coding tree unit to be used as the input of the neural network model and the original image samples of the coding tree unit to calculate the rate-distortion cost value, which is recorded as costOrg;

[0314] Second round:

[0315] The encoder attempts to use neural network-based loop filtering to calculate the optimal BaseQP value for the current coding frame. The reconstructed YUV samples of the current coding tree unit and the quantization parameter BaseQP are input into the loaded network model for inference. The neural network loop filtering model outputs the reconstructed sample block of the current coding tree unit. The rate-distortion cost is calculated by combining the coding tree unit samples after the neural network loop filtering with the original image samples of the coding tree unit, which is recorded as costFrame.

[0316] Round 3:

[0317] The encoder attempts to optimize the selection at the coding tree unit level. In the second round, the encoder attempts to use the neural network model loop filter by default for all coding tree units in the current frame. The luminance component and chrominance component are each controlled by a frame-level switch flag, while the coding tree unit level does not need to transmit the flag. This round attempts to use a combination of flags at the coding tree unit level, and each color component can be controlled separately. The encoder traverses the coding tree units and calculates the rate-distortion cost of the reconstructed samples without the neural network model loop filter and the original samples of the current coding tree unit, which is recorded as costCTUorg; the rate-distortion cost of the reconstructed samples based on the neural network loop filter and the original samples of the current coding tree unit are calculated, which are recorded as costCTUnn respectively.

[0318] For the luminance component, if the costCTUorg of the current luminance component is smaller than the costCTUnn, the use flag (ctb_nnlf_luma_flag) of the neural network loop filter at the coding tree unit level of the luminance component is set to false; otherwise, the ctb_nnlf_luma_flag is set to true.

[0319] For the chroma component, if the costCTUorg of the current chroma component is smaller than the costCTUnn, the use flag (ctb_nnlf_chroma_flag) of the neural network loop filter at the coding tree unit level of the chroma component is set to false; otherwise, the ctb_nnlf_chroma_flag is set to true.

[0320] If all coding tree units in the current frame have been traversed, the rate-distortion cost of the reconstructed sample of the current frame and the original image sample is calculated, which is recorded as costBest;

[0321] Traverse each color component. If the value of costOrg is the smallest, set the frame-level neural network loop filter switch flag corresponding to that color component to false and write it into the bitstream. If the value of costFrame is the smallest, set the frame-level neural network loop filter switch flag (ph_nnlf_luma_ctrl_flag / ph_nnlf_chroma_ctrl_flag) corresponding to that color component to true, and write the calculated BaseQP value into the bitstream. If the value of costCTUBest is the smallest, set the frame-level neural network loop filter usage flag corresponding to that color component to true and the frame-level switch flag to false. At the same time, write the BaseQP value calculated in the second round and the coding tree unit-level usage flag determined in the third round into the bitstream.

[0322] Step 3: The encoder continues to try other loop filtering tools and outputs a complete reconstructed image after completion. The specific process is not related to the technical solution of the embodiment of the present application and is therefore not described in detail here.

[0323] In another specific embodiment, for the decoding end, the specific process is as follows:

[0324] The decoding end parses the sequence-level flag. If sps_nnlf_enable_flag is true, it indicates that the current bitstream allows the use of loop filtering technology based on the neural network model, and the subsequent decoding process needs to parse the relevant syntax elements. Otherwise, it indicates that the current bitstream does not allow the use of loop filtering technology based on the neural network model, and the subsequent decoding process does not need to parse the relevant syntax elements. The default relevant syntax elements are initial values ​​or to the false state.

[0325] Step 1. The decoder parses the syntax elements of the current frame to obtain the frame-level switch flag and the frame-level usage flag based on the neural network model. If the frame-level switch flag is not all negative, execute step 2; otherwise, skip step 2 and execute step 3.

[0326] Step 2: If the frame-level switch flag is true, it indicates that all coding tree units under the current color component are filtered using the neural network-based loop filtering technology, that is, the coding tree unit-level usage flag of all coding tree units in the current frame under the color component is automatically set to true; otherwise, it indicates that some coding tree units under the current color component use the neural network-based loop filtering technology, while some coding tree units do not use the neural network-based loop filtering technology. Therefore, if the frame-level switch flag is false, it is necessary to further analyze the coding tree unit-level usage flag (ctb_nnlf_luma_flag / ctb_nnlf_chroma_flag) of all coding tree units in the current frame under the color component.

[0327] If ph_nnlf_luma_ctrl_flag or ph_nnlf_chroma_ctrl_flag is true, the BaseQP value (ph_nnlf_baseqp) of the current frame is parsed and applied as the input BaseQP to all coding tree units in the current frame. In addition, ph_nnlf_luma_enable_flag or ph_nnlf_chroma_enable_flag, as well as the ctb_nnlf_luma_flag or ctb_nnlf_chroma_flag flags used by all coding tree units in the current frame must be set to true;

[0328] Otherwise, parse ph_nnlf_luma_enable_flag / ph_nnlf_chroma_enable_flag.

[0329] If both ph_nnlf_luma_enable_flag and ph_nnlf_chroma_enable_flag are set to false, all coding tree units in the current frame will use the flag position false;

[0330] Otherwise, if both ph_nnlf_luma_enable_flag and ph_nnlf_chroma_enable_flag are not false, the BaseQP value (ph_nnlf_baseqp) of the current frame and the use flags of all coding tree units of the corresponding color components are parsed.

[0331] If the coding tree unit (CTU) usage flags for all color components of the current CTU are not all set to "no," the current CTU is filtered using a neural network-based loop filtering technique. The reconstructed YUV samples and quantization parameter information (BaseQP) of the current CTU are used as input. The network model performs inference and obtains the reconstructed YUV samples of the current CTU after the neural network-based loop filtering.

[0332] According to the coding tree unit usage identification bit of each color component of the current coding tree unit, the reconstructed sample is selected as the output of the neural network-based loop filtering technology. If the coding tree unit usage identification bit of the corresponding color component is true, the reconstructed sample after the neural network-based loop filtering of the above-mentioned corresponding color component is used as the output; otherwise, the reconstructed sample that has not undergone the neural network-based loop filtering is used as the output of the color component.

[0333] After traversing all coding tree units of the current frame, the neural network-based loop filtering module ends.

[0334] Step 3: The decoding end continues to traverse other loop filtering tools and outputs a complete reconstructed image after completion. The specific process is not related to the technical solution of the embodiment of the present application and is therefore not described in detail here.

[0335] While the residual scaling portion is not described in detail in all of the above embodiments, this does not mean that the present embodiments cannot employ residual scaling techniques. Residual scaling techniques are employed on the output of a neural network model, specifically by scaling the residuals obtained by subtracting the reconstructed samples output by the neural network from the original reconstructed samples. This will not be elaborated upon here.

[0336] It should be noted that the neural network-based loop filter enhancement method can make finer-grained adjustments to the BaseQP input plane. For example, if the BaseQP is fixed to 22, the plane is a 128x128 block with all values ​​22. However, the BaseQP plane can be fine-tuned based on texture complexity or residual errors. This can be done by calculating the standard deviation of the current coding tree unit and scaling the BaseQP proportionally to the standard deviation of each coding unit.

[0337] In yet another embodiment of the present application, the embodiment of the present application provides a code stream, which is generated by bit encoding based on information to be encoded; wherein the information to be encoded includes at least one of the following: first syntax element identification information of a current video sequence, second syntax element identification information of a current frame, third syntax element identification information of the current frame, fourth syntax element identification information of a current block, fifth syntax element identification information of the current block, sixth syntax element identification information of the current frame, a residual scaling factor, and an initial residual value of at least one partition block included in the current frame; wherein the current frame includes at least one partition block, and the current block is any one of the at least one partition block.

[0338] In yet another embodiment of the present application, based on the same inventive concept as the above embodiment, referring to FIG10 , a schematic diagram of the structure of a decoder 1 provided in an embodiment of the present application is shown. As shown in FIG10 , the decoder 1 includes:

[0339] The parsing part 10 is configured to parse the code stream and obtain a frame-level switch flag based on the network filtering model; the frame-level switch flag is used to determine whether all blocks in the current frame are filtered;

[0340] The first determining portion 11 is configured to, when the frame-level switch flag indicates that the switch is not on and the acquired frame-level usage flag indicates that any video component of the current block is in use, acquire a block-level usage flag; and when the block-level usage flag indicates that any video component of the current block is in use, acquire a quantization parameter; wherein the quantization parameter includes: an adjusted block-level quantization parameter or a frame-level optimal quantization parameter;

[0341] The first filtering part 12 is configured to filter the current block of the current frame based on the quantization parameter and the network filtering model to obtain first residual information of the current block.

[0342] In some embodiments of the present application, the first determining part 11 is further configured to obtain a block-level quantization parameter offset index;

[0343] The adjusted block-level quantization parameter is obtained based on the block-level quantization parameter offset index.

[0344] In some embodiments of the present application, the first determining part 11 is further configured to determine the block-level quantization offset parameter based on the block-level quantization parameter offset index obtained from the bitstream;

[0345] The adjusted block-level quantization parameter is determined according to the original quantization parameter obtained from the bitstream and the block-level quantization offset parameter.

[0346] In some embodiments of the present application, the parsing part 10 is further configured to directly parse the bit stream to obtain the frame-level optimal quantization parameter.

[0347] In some embodiments of the present application, the first determining part 11 is further configured to, after obtaining the frame-level switch flag based on the network filtering model, obtain a frame-level quantization parameter when the frame-level switch flag indicates that the frame-level switch flag is turned on; the frame-level quantization parameter includes: an adjusted frame-level quantization parameter or a frame-level optimal quantization parameter;

[0348] The first filtering part 12 is further configured to filter the current block of the current frame based on the frame-level quantization parameter and the network filtering model to obtain second residual information of the current block.

[0349] In some embodiments of the present application, the first determination part 11 is further configured to determine the frame-level quantization bias parameter based on the frame-level quantization parameter bias index obtained in the code stream; and determine the adjusted frame-level quantization parameter based on the obtained original quantization parameter and the frame-level quantization bias parameter.

[0350] In some embodiments of the present application, the first determination part 11 is further configured to filter the current block of the current frame based on the quantization parameter and the network filtering model to obtain the first residual information of the current block; or, to filter the current block of the current frame based on the frame-level quantization parameter and the network filtering model to obtain the reconstructed value of the current block before obtaining the second residual information of the current block.

[0351] In some embodiments of the present application, the first filtering part 12 is further configured to use the network filtering model and combine the quantization parameter to filter the reconstructed value of the current block to obtain the first residual information of the current block to complete the filtering of the current block.

[0352] In some embodiments of the present application, the first filtering part 12 is further configured to utilize the network filtering model and, in combination with the adjusted frame-level quantization parameter, filter the reconstructed value of the current block to obtain the second residual information of the current block to complete the filtering of the current block.

[0353] In some embodiments of the present application, the first determination part 11 is further configured to filter the current block of the current frame based on the quantization parameter and the network filtering model to obtain the first residual information of the current block; or, before filtering the current block of the current frame based on the frame-level quantization parameter and the network filtering model to obtain the second residual information of the current block, obtain at least one of the predicted value, block partitioning information and deblocking filter boundary strength of the current block, and the reconstructed value of the current block.

[0354] The first filtering part 12 is also configured to use the network filtering model, combined with the predicted value of the current block, the block partitioning information and at least one of the deblocking filter boundary strength, and the quantization parameter, to filter the reconstructed value of the current block to obtain the first residual information of the current block to complete the filtering of the current block.

[0355] The first filtering part 12 is also configured to use the network filtering model, combined with the predicted value of the current block, the block partitioning information and at least one of the deblocking filter boundary strength, and the frame-level quantization parameter, to filter the reconstructed value of the current block to obtain the second residual information of the current block to complete the filtering of the current block.

[0356] In some embodiments of the present application, the parsing part 10 is further configured to filter the current block of the current frame based on the quantization parameter and the network filtering model to obtain first residual information of the current block; or, after filtering the current block of the current frame based on the frame-level quantization parameter and the network filtering model to obtain second residual information of the current block, obtain a second residual scaling factor in the bitstream;

[0357] The first determination part 11 is further configured to scale the first residual information or the second residual information of the current block based on the second residual scaling factor to obtain first target residual information or second target residual information; determine the second target reconstruction value of the current block based on the second target residual information and the reconstruction value of the current block; or, when the block level is represented by an identification bit, determine the first target reconstruction value of the current block based on the first target residual information and the reconstruction value of the current block.

[0358] In some embodiments of the present application, the first determining part 11 is further configured to determine the reconstruction value of the current block as the first target reconstruction value when the block-level usage flag characterization is not used.

[0359] In some embodiments of the present application, the first determining part 11 is further configured to, after obtaining the frame-level switch flag based on the network filtering model, obtain at least one of the predicted value, block partition information, and deblocking filter boundary strength of the current block, and the reconstructed value of the current block; when the frame-level switch flag indicates that it is not turned on and the obtained frame-level usage flag indicates that any video component of the current block is used, obtain a block-level usage flag; when the block-level usage flag indicates that any video component of the current block is used, obtain an adjusted block-level input parameter; the adjusted block-level input parameter includes: a parameter after any one of the predicted value, block partition information, and deblocking filter boundary strength is adjusted;

[0360] The first filtering part 12 is further configured to filter the current block of the current frame based on the adjusted block-level input parameters, the obtained original quantization parameters and the network filtering model to obtain third residual information of the current block.

[0361] In some embodiments of the present application, the parsing part 10 is further configured to parse out the sequence-level permission flag before obtaining the frame-level switch flag based on the network filtering model; when the sequence-level permission flag indicates permission, parse the frame-level switch flag based on the network filtering model.

[0362] The embodiment of the present application further provides a decoder 1, as shown in FIG11 , the decoder 1 may include:

[0363] A first memory 13 configured to store a computer program that can be executed on the first processor 14;

[0364] The first processor 14 is configured to execute the filtering method described by the decoder when running the computer program.

[0365] It can be understood that the decoder can determine the quantization parameters of the input neural network filter model based on the adjusted frame-level quantization parameters or the frame-level optimal quantization parameters, and the frame-level quantization parameters or the frame-level optimal quantization parameters are adjusted parameters, which are different from the original quantization parameters. This can achieve flexible selection and diverse change processing of frame-level quantization parameters without increasing complexity, and the adjustment amplitude of each block can be different, thereby improving decoding efficiency.

[0366] Among them, the first processor 14 can be implemented by software, hardware, firmware or a combination thereof, and can use circuits, single or multiple application specific integrated circuits (ASICs), single or multiple general integrated circuits, single or multiple microprocessors, single or multiple programmable logic devices, or a combination of the aforementioned circuits or devices, or other suitable circuits or devices, so that the first processor 14 can execute the corresponding steps of the filtering method on the decoder side in the aforementioned embodiment.

[0367] The embodiment of the present application provides an encoder 2, as shown in FIG12 , the encoder 2 may include:

[0368] The second determining part 20 is configured to obtain the original value of the current block in the current frame, the reconstructed value of the current block and the first quantization parameter when the obtained sequence level allows the use of the flag bit to represent the permission; the first quantization parameter includes: the original quantization parameter;

[0369] The second filtering part 21 is configured to perform at least two filter estimations on the current block based on the network filtering model, the reconstructed value of the current block, at least one frame-level quantization offset parameter and the original quantization parameter, determine at least two first sub-rate-distortion costs of the current block, and traverse the current frame to determine the at least two first rate-distortion costs of the current frame;

[0370] The second determination part 20 is further configured to determine a first optimal rate-distortion cost based on at least two of the first rate-distortion costs; determine a block-level usage flag and a first sub-optimal rate-distortion cost of a current block based on at least two of the first sub-rate-distortion costs, and traverse the current frame to determine a second optimal rate-distortion cost of the current frame; and determine a frame-level switch flag based on the first optimal rate-distortion cost and the second optimal rate-distortion cost.

[0371] In some embodiments of the present application, the second filtering part 21 is further configured to perform filtering estimation on the current block based on the network filtering model, the reconstructed value of the current block and the original quantization parameter to determine a first reconstructed value;

[0372] The second determining part 20 is further configured to perform rate-distortion cost estimation on the first reconstructed value and the original value of the current block to obtain a first sub-rate-distortion cost of the current block, and traverse the current frame to determine a first rate-distortion cost of the current frame;

[0373] The second filtering part 21 is also configured to perform at least one filtering estimation on the current block based on the network filtering model, the at least one frame-level quantization bias parameter, the original quantization parameter and the reconstructed value of the current block in the current frame, determine at least one of the first sub-rate-distortion costs of the current block, and traverse the current frame to determine at least one of the first rate-distortion costs of the current frame.

[0374] In some embodiments of the present application, the first quantization parameter includes: a frame-level optimal quantization parameter; the frame-level optimal quantization parameter is an optimal quantization parameter directly calculated by the encoder;

[0375] The second filtering part 21 is further configured to, after obtaining the original value of the current block in the current frame, the reconstructed value of the current block and the first quantization parameter, perform filtering estimation on the current block based on the network filtering model, the reconstructed value of the current block and the frame-level optimal quantization parameter to determine a second reconstructed value;

[0376] The second determination part 20 is further configured to perform a rate-distortion cost estimation on the second reconstructed value and the original value of the current block to obtain a second sub-rate-distortion cost of the current block, and traverse the current frame to determine a third rate-distortion cost of the current frame; perform a rate-distortion cost estimation based on the original value of the current block and the reconstructed value of the current block in the current frame to determine a third sub-rate-distortion cost of the current block, and traverse the current frame to determine a fourth rate-distortion cost; determine a block-level usage flag and a second sub-optimal rate-distortion cost of the current block based on the second sub-rate-distortion cost and the third sub-rate-distortion cost of the current block, and traverse the current frame to determine a third optimal rate-distortion cost of the current frame; and determine a frame-level switch flag based on the third rate-distortion cost, the fourth rate-distortion cost, and the third optimal rate-distortion cost.

[0377] In some embodiments of the present application, the second determination part 20 is further configured to, before determining the first optimal rate-distortion cost based on the at least two first rate-distortion costs, perform rate-distortion cost estimation based on the original value of the current block and the reconstructed value of the current block in the current frame, determine a third sub-rate-distortion cost of the current block, and traverse the current frame to determine a fourth rate-distortion cost.

[0378] In some embodiments of the present application, the second determining part 20 is further configured to determine the first sub-optimal rate-distortion cost with the smallest rate-distortion cost from the third sub-rate-distortion cost of the current block and at least two of the first sub-rate-distortion costs; if the first sub-optimal rate-distortion cost is the third sub-rate-distortion cost, determine that the block-level usage flag is unused; if the first sub-optimal rate-distortion cost is any one of the at least two first sub-rate-distortion costs, determine that the block-level usage flag is used.

[0379] In some embodiments of the present application, the second determination part 20 is further configured to, if the first sub-best rate-distortion cost is any one of at least two first sub-rate-distortion costs, determine that the block-level usage flag is used, and then obtain a block-level quantization parameter offset index corresponding to the first sub-best rate-distortion cost; the block-level quantization parameter offset index is determined based on the at least one frame-level quantization offset parameter and the original quantization parameter.

[0380] In some embodiments of the present application, the second determining part 20 is further configured to determine a first minimum rate-distortion cost having the minimum rate-distortion cost from the first optimal rate-distortion cost, the second optimal rate-distortion cost, and the fourth rate-distortion cost;

[0381] If the first minimum rate-distortion cost is the fourth rate-distortion cost, determining that the frame-level switch flag is not enabled;

[0382] If the first minimum rate-distortion cost is the first optimal rate-distortion cost, determining that the frame-level switch flag is on;

[0383] If the first minimum rate-distortion cost is the second best rate-distortion cost, it is determined that the frame-level switch flag is not enabled.

[0384] In some embodiments of the present application, the encoder 2 further includes: a writing portion 22;

[0385] The second determining part 20 is further configured to determine that the frame level switch flag is not turned on and the frame level use flag is used if the first minimum rate distortion cost is the second best rate distortion cost;

[0386] The writing part 22 is configured to write the frame level usage flag, the frame level switch flag, the block level quantization parameter offset index, and the block usage flag into the bitstream.

[0387] In some embodiments of the present application, the encoder 2 further includes: a writing part 22; the writing part 22 is configured to write the frame-level switch flag and the frame-level quantization parameter offset index into the bitstream after determining that the frame-level switch flag is on if the first minimum rate-distortion cost is the first optimal rate-distortion cost.

[0388] In some embodiments of the present application, the second determination part 20 is further configured to obtain a frame-level quantization parameter bias index corresponding to the first optimal rate-distortion cost after determining the first optimal rate-distortion cost based on at least two of the first rate-distortion costs; the frame-level quantization parameter bias index is determined based on the at least one frame-level quantization bias parameter and the original quantization parameter.

[0389] In some embodiments of the present application, the second filtering part 21 is further configured to perform filtering estimation on the current block based on the network filtering model, the reconstruction value of the current block and the original quantization parameter for the current frame, and determine first estimated residual information; determine a first residual scaling factor; scale the first estimated residual value using the first residual scaling factor to obtain first scaled residual information; and combine the first scaled residual information with the reconstruction value of the current block to determine the first reconstructed value.

[0390] In some embodiments of the present application, the second determining part 20 is further configured to, before determining the first residual scaling factor,

[0391] For a current frame, obtaining at least one of a prediction value, block partition information, and a deblocking filter boundary strength of a current block, and a reconstructed value of the current block;

[0392] The second filtering part 21 is also configured to use the network filtering model, combined with the predicted value of the current block, the block partitioning information and at least one of the deblocking filter boundary strength, and the frame-level quantization parameter, to filter and estimate the reconstructed value of the current block to obtain the first residual information of the current block.

[0393] In some embodiments of the present application, the encoder 2 further includes: a writing part 22; the writing part 22 is configured to write the first residual scaling factor into the bitstream after determining the first residual scaling factor.

[0394] In some embodiments of the present application, the second filtering part 21 is further configured to obtain the i-th frame-level quantization bias parameter, adjust the original quantization parameter based on the i-th frame-level quantization bias parameter, and obtain the i-th adjusted frame-level quantization parameter; i is a positive integer greater than or equal to 1; filter the current block based on the network filtering model, the reconstruction value of the current block and the i-th adjusted frame-level quantization parameter to obtain the i-th third reconstruction value; perform rate-distortion cost estimation on the i-th third reconstruction value and the original value of the current block, traverse all blocks of the current frame, obtain the i-th first sub-rate-distortion cost, continue to perform i+1th filtering estimation based on the i+1th frame-level quantization bias parameter, until at least one is completed, and at least one third reconstruction value is obtained, thereby determining at least one of the first sub-rate-distortion costs of the current block.

[0395] In some embodiments of the present application, the second filtering part 21 is further configured to perform a filtering estimation on the current block based on the network filtering model, the reconstruction value of the current block and the i-th adjusted frame-level quantization parameter to obtain the i-th second estimated residual information; determine the i-th second residual scaling factor corresponding to the i-th adjusted frame-level quantization parameter; use the i-th second residual scaling factor to scale the i-th second estimated residual information to obtain the i-th second scaled residual information; combine the i-th second scaled residual information with the reconstruction value of the current block to determine the i-th third reconstruction value.

[0396] In some embodiments of the present application, the encoder 2 further includes: a writing part 22; the writing part 22 is configured to, after determining the frame-level switch flag based on the first optimal rate-distortion cost and the second optimal rate-distortion cost, write a second residual scaling factor corresponding to the first minimum rate-distortion cost into the bitstream if the first minimum rate-distortion cost is the first optimal rate-distortion cost.

[0397] In some embodiments of the present application, the second determining part 20 is further configured to obtain the original value of the current block and the reconstructed value of the current block in the current frame; and calculate the frame-level optimal quantization parameter of the current frame.

[0398] In some embodiments of the present application, the second determining part 20 is further configured to, if the third sub-rate-distortion cost is less than the second sub-rate-distortion cost, determine that the block-level usage flag is unused and the second sub-optimal rate-distortion cost is the third sub-rate-distortion cost; if the third sub-rate-distortion cost is greater than or equal to the second sub-rate-distortion cost, determine that the block-level usage flag is used and the second sub-optimal rate-distortion cost is the second sub-rate-distortion cost.

[0399] In some embodiments of the present application, the second determining part 20 is further configured to determine a third minimum rate-distortion cost from the third rate-distortion cost, the fourth rate-distortion cost and the third best rate-distortion cost;

[0400] If the third minimum rate-distortion cost is the fourth rate-distortion cost, determining that the frame-level switch flag is not enabled;

[0401] If the third minimum rate-distortion cost is the third rate-distortion cost, determining that the frame-level switch flag is on;

[0402] If the third minimum rate-distortion cost is the third optimal rate-distortion cost, it is determined that the frame-level switch flag is not enabled.

[0403] In some embodiments of the present application, the encoder 2 further includes: a writing part 22; the second determining part 20 is further configured to, if the third minimum rate-distortion cost is the third optimal rate-distortion cost, determine that the frame-level switch flag is not turned on and determine that the frame-level use flag is used;

[0404] The writing part 22 is configured to write the frame level usage flag, the frame level switch flag, the frame level optimal quantization parameter, and the block usage flag into the bitstream.

[0405] In some embodiments of the present application, the encoder 2 further includes: a writing part 22; the writing part 22 is configured to, if the third minimum rate-distortion cost is the third rate-distortion cost, determine that the frame-level switch flag is on, and then write the frame-level switch flag and the frame-level optimal quantization parameter into the bitstream.

[0406] In some embodiments of the present application, the second determining part 20 is further configured to, when the sequence level allows the use of a flag to represent permission, obtain at least one of a prediction value of the current block, block partition information, and a deblocking filter boundary strength, a reconstructed value of the current block, and a first quantization parameter;

[0407] The second filtering part 21 is further configured to perform filtering estimation on the current block based on the prediction value of the current block, at least one of the block partition information and the deblocking filter boundary strength, the network filtering model, the reconstructed value of the current block and the first quantization parameter, to determine a fourth reconstructed value;

[0408] The second determining part 20 is further configured to perform rate-distortion cost estimation on the fourth reconstructed value and the original value of the current block to obtain a fourth sub-rate-distortion cost of the current block, and traverse the current frame to determine a fifth rate-distortion cost of the current frame;

[0409] The second filtering part 21 is further configured to perform at least one filter estimation on the current frame based on at least one of the predicted value of the current block, the block partition information and the deblocking filter boundary strength, the network filtering model, at least one frame-level input bias parameter, and the reconstructed value of the current block in the current frame, to determine at least one sixth rate-distortion cost of the current frame;

[0410] The second determining part 20 is further configured to determine a frame-level switch flag based on the fifth rate-distortion cost and the at least one sixth rate-distortion cost.

[0411] In some embodiments of the present application, the encoder 2 further includes: a writing part 22; the writing part 22 is configured to write the frame-level switch flag into the bitstream.

[0412] The embodiment of the present application provides an encoder 2, as shown in FIG13 , the encoder 2 may include:

[0413] a second memory 23 configured to store a computer program capable of running on the second processor 24;

[0414] The second processor 24 is configured to execute the method described by the encoder when running the computer program.

[0415] It can be understood that the encoder can determine the quantization parameters of the input neural network filter model based on the adjusted frame-level quantization parameters or the frame-level optimal quantization parameters, and the frame-level quantization parameters or the frame-level optimal quantization parameters are adjusted parameters, which are different from the original quantization parameters. This can achieve flexible selection and diverse change processing of frame-level quantization parameters without increasing complexity, and the adjustment amplitude of each block can be different, thereby improving coding efficiency.

[0416] An embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, which implements the method described in the decoder when executed by a first processor, or implements the method described in the encoder when executed by a second processor.

[0417] The various components in the embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or in the form of software functional modules.

[0418] If the integrated unit is implemented in the form of a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the method described in this embodiment. The aforementioned computer-readable storage medium includes: ferromagnetic random access memory (FRAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, magnetic surface storage, optical disk, or compact disc read-only memory (CD-ROM), etc. Various media that can store program codes are not limited in the embodiments of the present disclosure.

[0419] The above is merely an embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims. Industrial Applicability

[0420] The embodiment of the present application provides a filtering method, an encoder, a decoder, a bitstream, and a storage medium. By parsing the bitstream, a frame-level usage flag based on a neural network filtering model is obtained; when the frame-level usage flag indicates usage, a frame-level switch flag and a frame-level quantization parameter adjustment flag are obtained; the frame-level switch flag is used to determine whether each block in the current frame is filtered; when the frame-level switch flag indicates on and the frame-level quantization parameter adjustment flag indicates usage, an adjusted frame-level quantization parameter is obtained; based on the adjusted frame-level quantization parameter and the neural network filtering model, a current block of the current frame is filtered to obtain first residual information of the current block. In this way, the quantization parameter input to the neural network filtering model can be determined based on the adjusted frame-level quantization parameter or the frame-level optimal quantization parameter. The frame-level quantization parameter or the frame-level optimal quantization parameter are both adjusted parameters, which are different from the original quantization parameter. This can achieve flexible selection and diverse change processing of the frame-level quantization parameter without increasing complexity, and the adjustment amplitude of each block can be different, thereby improving coding efficiency.

Claims

1. A filtering method, applied to a decoder, comprising: Parse the bitstream and obtain the frame-level switch flag based on the network filtering model; The frame-level switch flag is used to determine whether all blocks in the current frame are filtered; When the frame-level switch flag indicates that it is not turned on, and the obtained frame-level usage flag indicates that any video component of the current block is used, obtaining a block-level usage flag; When the block-level usage flag indicates that any video component of the current block is in use, a quantization parameter is obtained; wherein the quantization parameter includes: an adjusted block-level quantization parameter or a frame-level optimal quantization parameter; Based on the quantization parameter and the network filtering model, a current block of the current frame is filtered to obtain first residual information of the current block.

2. The method according to claim 1, wherein The obtaining of the quantization parameter includes: Get the block-level quantization parameter offset index; The adjusted block-level quantization parameter is obtained based on the block-level quantization parameter offset index.

3. The method according to claim 1, wherein The obtaining of the quantization parameter includes: Determining a block-level quantization bias parameter based on a block-level quantization parameter bias index obtained from a bitstream; The adjusted block-level quantization parameter is determined according to the original quantization parameter obtained from the bitstream and the block-level quantization offset parameter.

4. The method according to claim 1, wherein The obtaining of the quantization parameter includes: The optimal frame-level quantization parameter is directly obtained by parsing the bitstream.

5. The method according to claim 1, wherein After obtaining the frame-level switch flag based on the network filtering model, the method further includes: When the frame-level switch flag indicates that the switch is on, a frame-level quantization parameter is obtained; the frame-level quantization parameter includes: an adjusted frame-level quantization parameter or an optimal frame-level quantization parameter; Based on the frame-level quantization parameter and the network filtering model, filtering is performed on the current block of the current frame to obtain second residual information of the current block.

6. The method according to claim 5, wherein: The obtaining of the frame-level quantization parameter includes: Determining a frame-level quantization bias parameter based on a frame-level quantization parameter bias index obtained from a bitstream; The adjusted frame-level quantization parameter is determined according to the obtained original quantization parameter and the frame-level quantization offset parameter.

7. The method according to any one of claims 1 to 6, wherein: Before filtering the current block of the current frame based on the quantization parameter and the network filtering model to obtain first residual information of the current block; or filtering the current block of the current frame based on the frame-level quantization parameter and the network filtering model to obtain second residual information of the current block, the method further includes: Get the reconstruction value of the current block.

8. The method according to claim 7, wherein: The filtering of the current block of the current frame based on the quantization parameter and the network filtering model to obtain first residual information of the current block includes: The network filtering model is used in combination with the quantization parameter to filter the reconstructed value of the current block to obtain the first residual information of the current block, thereby completing the filtering of the current block.

9. The method according to claim 7, wherein: The filtering of the current block of the current frame based on the frame-level quantization parameter and the network filtering model to obtain second residual information of the current block includes: The network filtering model is used in combination with the adjusted frame-level quantization parameter to filter the reconstructed value of the current block to obtain the second residual information of the current block, thereby completing the filtering of the current block.

10. The method according to any one of claims 1 to 6, wherein: Before filtering the current block of the current frame based on the quantization parameter and the network filtering model to obtain first residual information of the current block; or filtering the current block of the current frame based on the frame-level quantization parameter and the network filtering model to obtain second residual information of the current block, the method further includes: Obtaining at least one of a prediction value, block partition information, and a deblocking filter boundary strength of a current block, and a reconstructed value of the current block; The filtering of the current block of the current frame based on the quantization parameter and the network filtering model to obtain first residual information of the current block includes: Using the network filtering model, in combination with the predicted value of the current block, the block partitioning information, at least one of the deblocking filter boundary strength, and the quantization parameter, filtering the reconstructed value of the current block to obtain the first residual information of the current block, thereby completing filtering of the current block; The filtering of the current block of the current frame based on the frame-level quantization parameter and the network filtering model to obtain second residual information of the current block includes: Utilizing the network filtering model, in combination with the predicted value of the current block, the block partitioning information, at least one of the deblocking filter boundary strength, and the frame-level quantization parameter, the reconstructed value of the current block is filtered to obtain the second residual information of the current block, thereby completing the filtering of the current block.

11. The method according to any one of claims 7 to 10, wherein: After filtering the current block of the current frame based on the quantization parameter and the network filtering model to obtain first residual information of the current block; or filtering the current block of the current frame based on the frame-level quantization parameter and the network filtering model to obtain second residual information of the current block, the method further includes: Obtaining a second residual scaling factor in the bitstream; Scaling the first residual information or the second residual information of the current block based on the second residual scaling factor to obtain first target residual information or second target residual information; determining a second target reconstruction value of the current block based on the second target residual information and the reconstruction value of the current block; or, When the block-level usage flag is used to represent usage, a first target reconstruction value of the current block is determined based on the first target residual information and the reconstruction value of the current block.

12. The method according to claim 11, wherein The method further comprises: When the block-level usage flag indicates that the block is not used, the reconstruction value of the current block is determined as the first target reconstruction value.

13. The method according to claim 1, wherein After obtaining the frame-level switch flag based on the network filtering model, the method further includes: Obtaining at least one of a prediction value, block partition information, and a deblocking filter boundary strength of a current block, and a reconstructed value of the current block; When the frame-level switch flag indicates that it is not turned on, and the obtained frame-level usage flag indicates that any video component of the current block is used, obtaining a block-level usage flag; When the block-level usage flag indicates that any video component of the current block is in use, obtaining an adjusted block-level input parameter; The adjusted block-level input parameter includes: a parameter after any one of the predicted value, block partition information and deblocking filter boundary strength is adjusted; Based on the adjusted block-level input parameters, the obtained original quantization parameters and the network filtering model, filtering is performed on the current block of the current frame to obtain third residual information of the current block.

14. The method according to claim 1, wherein Before obtaining the frame-level switch flag based on the network filtering model, the method further includes: Parse the sequence-level permission flag; When the sequence level permission is allowed using an identification bit to represent permission, the frame level switch identification bit based on the network filtering model is parsed.

15. A filtering method, applied to an encoder, comprising: When the obtained sequence level permission is indicated by a flag, an original value of a current block in a current frame, a reconstructed value of the current block, and a first quantization parameter are obtained; The first quantization parameter includes: an original quantization parameter; Performing at least two filter estimations on the current block based on a network filter model, a reconstructed value of the current block, at least one frame-level quantization offset parameter, and the original quantization parameter to determine at least two first sub-rate-distortion costs of the current block, and traversing the current frame to determine at least two first rate-distortion costs of the current frame; determining a first optimal rate-distortion cost based on at least two of the first rate-distortion costs; Determine a block-level usage flag and a first sub-optimal rate-distortion cost of a current block based on at least two of the first sub-rate-distortion costs, and traverse the current frame to determine a second optimal rate-distortion cost of the current frame; A frame-level switch flag is determined based on the first best rate-distortion cost and the second best rate-distortion cost.

16. The method according to claim 15, wherein The performing at least two filter estimations on the current block based on the network filtering model, the reconstructed value of the current block, at least one frame-level quantization offset parameter, and the original quantization parameter, determining at least two first sub-rate-distortion costs of the current block, and traversing the current frame to determine the at least two first rate-distortion costs of the current frame, includes: Performing filter estimation on the current block based on the network filter model, the reconstructed value of the current block, and the original quantization parameter to determine a first reconstructed value; Performing rate-distortion cost estimation on the first reconstructed value and the original value of the current block to obtain a first sub-rate-distortion cost of the current block, and traversing the current frame to determine a first rate-distortion cost of the current frame; Based on the network filtering model, the at least one frame-level quantization offset parameter, the original quantization parameter, and the reconstructed value of the current block in the current frame, at least one filtering estimation is performed on the current block, at least one first sub-rate-distortion cost of the current block is determined, and the current frame is traversed to determine at least one first rate-distortion cost of the current frame.

17. The method according to claim 15, wherein: The first quantization parameter includes: a frame-level optimal quantization parameter; the frame-level optimal quantization parameter is an optimal quantization parameter directly calculated by the encoder; After obtaining the original value of the current block in the current frame, the reconstructed value of the current block, and the first quantization parameter, the method further includes: Performing filter estimation on the current block based on the network filtering model, the reconstruction value of the current block and the frame-level optimal quantization parameter to determine a second reconstruction value; Performing rate-distortion cost estimation on the second reconstructed value and the original value of the current block to obtain a second sub-rate-distortion cost of the current block, and traversing the current frame to determine a third rate-distortion cost of the current frame; performing a rate-distortion cost estimation based on the original value of the current block and the reconstructed value of the current block in the current frame, determining a third sub-rate-distortion cost of the current block, and traversing the current frame to determine a fourth rate-distortion cost; Determining a block-level usage flag and a second sub-optimal rate-distortion cost of the current block based on the second sub-rate-distortion cost and the third sub-rate-distortion cost of the current block, and traversing the current frame to determine a third optimal rate-distortion cost of the current frame; A frame-level switch flag is determined based on the third rate-distortion cost, the fourth rate-distortion cost, and the third best rate-distortion cost.

18. The method according to claim 15 or 16, wherein Before determining a first optimal rate-distortion cost based on at least two of the first rate-distortion costs, the method further includes: A rate-distortion cost is estimated based on the original value of the current block and the reconstructed value of the current block in the current frame to determine a third sub-rate-distortion cost of the current block, and a fourth rate-distortion cost is determined by traversing the current frame.

19. The method according to claim 18, wherein The determining, based on at least two of the first sub-rate-distortion costs, a block-level usage flag and a first sub-optimal rate-distortion cost of a current block includes: Determining the first sub-optimal rate-distortion cost with the minimum rate-distortion cost from the third sub-rate-distortion cost of the current block and at least two of the first sub-rate-distortion costs; If the first sub-optimal rate-distortion cost is the third sub-rate-distortion cost, determining that the block-level usage flag is unused; If the first sub-optimal rate-distortion cost is any one of at least two first sub-rate-distortion costs, the block-level usage flag is determined to be used.

20. The method according to claim 19, wherein If the first sub-best rate-distortion cost is any one of the at least two first sub-rate-distortion costs, after determining that the block-level usage flag is used, the method further includes: A block-level quantization parameter offset index corresponding to the first sub-optimal rate-distortion cost is obtained; the block-level quantization parameter offset index is determined based on the at least one frame-level quantization offset parameter and the original quantization parameter.

21. The method according to any one of claims 18 to 20, wherein: The determining the frame-level switch flag based on the first best rate-distortion cost and the second best rate-distortion cost includes: Determining a first minimum rate-distortion cost having the minimum rate-distortion cost from the first optimal rate-distortion cost, the second optimal rate-distortion cost, and the fourth rate-distortion cost; If the first minimum rate-distortion cost is the fourth rate-distortion cost, determining that the frame-level switch flag is not enabled; If the first minimum rate-distortion cost is the first optimal rate-distortion cost, determining that the frame-level switch flag is on; If the first minimum rate-distortion cost is the second best rate-distortion cost, it is determined that the frame-level switch flag is not enabled.

22. The method according to claim 21, wherein The method further comprises: If the first minimum rate-distortion cost is the second best rate-distortion cost, determining that the frame-level switch flag is disabled and determining that the frame-level usage flag is enabled; The frame-level usage flag, the frame-level switch flag, the block-level quantization parameter offset index, and the block usage flag are written into a bitstream.

23. The method according to claim 21, wherein If the first minimum rate-distortion cost is the first optimal rate-distortion cost, after determining that the frame-level switch flag is on, the method further includes: The frame-level switch flag and the frame-level quantization parameter offset index are written into the bitstream.

24. The method according to claim 15, wherein After determining a first optimal rate-distortion cost based on at least two of the first rate-distortion costs, the method further includes: A frame-level quantization parameter offset index corresponding to the first optimal rate-distortion cost is obtained; the frame-level quantization parameter offset index is determined based on the at least one frame-level quantization offset parameter and the original quantization parameter.

25. The method according to claim 16, wherein The performing filtering estimation on the current block based on the network filtering model, the reconstructed value of the current block, and the original quantization parameter to determine the first reconstructed value includes: For the current frame, performing filtering estimation on the current block based on the network filtering model, the reconstructed value of the current block and the original quantization parameter to determine first estimation residual information; determining a first residual scaling factor; Scaling the first estimated residual value using the first residual scaling factor to obtain first scaled residual information; The first scaled residual information is combined with the reconstruction value of the current block to determine the first reconstruction value.

26. The method according to claim 25, wherein Before determining the first residual scaling factor, the method further includes: For a current frame, obtaining at least one of a prediction value, block partition information, and a deblocking filter boundary strength of a current block, and a reconstructed value of the current block; Utilizing the network filtering model, combined with the predicted value of the current block, the block partitioning information and at least one of the deblocking filter boundary strength, and the frame-level quantization parameter, a filtered estimate is performed on the reconstructed value of the current block to obtain the first residual information of the current block.

27. The method according to claim 25 or 26, wherein After determining the first residual scaling factor, the method further includes: The first residual scaling factor is written into the bitstream.

28. The method according to claim 16, wherein The performing at least one filter estimation on the current block based on the network filter model, the at least one frame-level quantization offset parameter, the original quantization parameter, and the reconstructed value of the current block in the current frame to determine at least one first sub-rate-distortion cost of the current block includes: Obtaining an i-th frame-level quantization bias parameter, and adjusting the original quantization parameter based on the i-th frame-level quantization bias parameter to obtain an i-th adjusted frame-level quantization parameter, where i is a positive integer greater than or equal to 1; Performing filtering estimation on the current block based on the network filtering model, the reconstruction value of the current block, and the i-th adjusted frame-level quantization parameter to obtain an i-th third reconstruction value; A rate-distortion cost is estimated by performing a calculation on the i-th third reconstruction value and the original value of the current block, and after traversing all blocks of the current frame, an i-th first sub-rate-distortion cost is obtained. Then, an i+1-th filtering estimation is performed based on the i+1-th frame-level quantization bias parameter until at least one calculation is completed and at least one third reconstruction value is obtained, thereby determining at least one first sub-rate-distortion cost of the current block.

29. The method according to claim 28, wherein The performing filtering estimation on the current block based on the network filtering model, the reconstruction value of the current block, and the i-th adjusted frame-level quantization parameter to obtain the i-th third reconstruction value includes: performing filtering estimation on the current block based on the network filtering model, the reconstructed value of the current block and the i-th adjusted frame-level quantization parameter to obtain i-th second estimated residual information; Determining i-th second residual scaling factors respectively corresponding to the i-th adjusted frame-level quantization parameters; Using the i-th second residual scaling factor, scaling the i-th second estimated residual information to obtain i-th second scaled residual information; The i-th second scaled residual information is correspondingly combined with the reconstructed value of the current block to determine the i-th third reconstructed value.

30. The method according to claim 29, wherein After determining the frame-level switch flag based on the first best rate-distortion cost and the second best rate-distortion cost, the method further includes: If the first minimum rate-distortion cost is the first optimal rate-distortion cost, a second residual scaling factor corresponding to the first minimum rate-distortion cost is written into the bitstream.

31. The method of claim 17, wherein: The obtaining of the original value of the current block in the current frame, the reconstructed value of the current block, and the first quantization parameter includes: Obtaining an original value of the current block and a reconstructed value of the current block in a current frame; Calculate a frame-level optimal quantization parameter of the current frame.

32. The method according to claim 17 or 31, wherein The determining, based on the second sub-rate-distortion cost and the third sub-rate-distortion cost of the current block, a block-level usage flag and a second sub-optimal rate-distortion cost of the current block includes: If the third sub-rate-distortion cost is less than the second sub-rate-distortion cost, determining that the block-level usage flag is unused, and the second sub-optimal rate-distortion cost is the third sub-rate-distortion cost; If the third sub-rate-distortion cost is greater than or equal to the second sub-rate-distortion cost, the block-level usage flag is determined to be used, and the second sub-optimal rate-distortion cost is determined to be the second sub-rate-distortion cost.

33. The method of claim 17, 31 or 32, wherein: The determining of the frame-level switch flag based on the third rate-distortion cost, the fourth rate-distortion cost, and the third best rate-distortion cost includes: determining a third minimum rate-distortion cost from the third rate-distortion cost, the fourth rate-distortion cost, and the third best rate-distortion cost; If the third minimum rate-distortion cost is the fourth rate-distortion cost, determining that the frame-level switch flag is not enabled; If the third minimum rate-distortion cost is the third rate-distortion cost, determining that the frame-level switch flag is on; If the third minimum rate-distortion cost is the third optimal rate-distortion cost, it is determined that the frame-level switch flag is not enabled.

34. The method according to claim 33, wherein The method further comprises: If the third minimum rate-distortion cost is the third optimal rate-distortion cost, determining that the frame-level switch flag is disabled and determining that the frame-level usage flag is enabled; The frame-level usage flag, the frame-level switch flag, the frame-level optimal quantization parameter, and the block usage flag are written into the bitstream.

35. The method of claim 33, wherein: If the third minimum rate-distortion cost is the third rate-distortion cost, after determining that the frame-level switch flag is on, the method further includes: The frame-level switch flag and the frame-level optimal quantization parameter are written into the bitstream.

36. The method of claim 15, wherein: The method further comprises: When the sequence level allows the use of a flag to indicate permission, obtaining a prediction value of the current block, at least one of block partition information and a deblocking filter boundary strength, a reconstructed value of the current block, and a first quantization parameter; performing filter estimation on the current block based on the prediction value of the current block, at least one of the block partition information and the deblocking filter boundary strength, the network filter model, the reconstructed value of the current block and the first quantization parameter to determine a fourth reconstructed value; Performing rate-distortion cost estimation on the fourth reconstructed value and the original value of the current block to obtain a fourth sub-rate-distortion cost of the current block, and traversing the current frame to determine a fifth rate-distortion cost of the current frame; performing at least one filter estimation on the current frame based on at least one of a prediction value of the current block, the block partition information and the deblocking filter boundary strength, the network filter model, at least one frame-level input bias parameter, and a reconstructed value of the current block in the current frame, to determine at least one sixth rate-distortion cost of the current frame; A frame-level switch flag is determined based on the fifth rate-distortion cost and the at least one sixth rate-distortion cost.

37. The method of claim 15, wherein: The method further comprises: The frame-level switch flag is written into the bitstream.

38. A decoder, comprising: The parsing part is configured to parse the bitstream and obtain the frame-level switch flag based on the network filtering model; The frame-level switch flag is used to determine whether all blocks in the current frame are filtered; The first determining part is configured to obtain a block-level usage flag when the frame-level switch flag indicates that the switch is not turned on and the obtained frame-level usage flag indicates that any video component of the current block is used; as well as, When the block-level usage flag indicates that any video component of the current block is in use, a quantization parameter is obtained; wherein the quantization parameter includes: an adjusted block-level quantization parameter or a frame-level optimal quantization parameter; The first filtering part is configured to filter the current block of the current frame based on the quantization parameter and the network filtering model to obtain first residual information of the current block.

39. An encoder, comprising: The second determining part is configured to obtain an original value of a current block in a current frame, a reconstructed value of the current block, and a first quantization parameter when the obtained sequence level allows the use of a flag to represent the permission; The first quantization parameter includes: an original quantization parameter; a second filtering part, configured to perform at least two filter estimates on the current block based on the network filtering model, the reconstructed value of the current block, at least one frame-level quantization offset parameter, and the original quantization parameter, determine at least two first sub-rate-distortion costs of the current block, and traverse the current frame to determine the at least two first rate-distortion costs of the current frame; The second determination part is further configured to determine a first optimal rate-distortion cost based on at least two of the first rate-distortion costs; determine a block-level usage flag and a first sub-optimal rate-distortion cost of a current block based on at least two of the first sub-rate-distortion costs, and traverse the current frame to determine a second optimal rate-distortion cost of the current frame; and determine a frame-level switch flag based on the first optimal rate-distortion cost and the second optimal rate-distortion cost.

40. A decoder, comprising: a first memory configured to store a computer program executable on the first processor; The first processor is configured to execute the method according to any one of claims 1 to 14 when running the computer program.

41. An encoder, comprising: a second memory configured to store a computer program executable on the second processor; The second processor is configured to perform the method according to any one of claims 15 to 37 when running the computer program.

42. A computer-readable storage medium, wherein: The computer-readable storage medium stores a computer program, which implements the method according to any one of claims 1 to 14 when executed by a first processor, or implements the method according to any one of claims 15 to 37 when executed by a second processor.

43. A code stream, the code stream being generated by bit encoding based on information to be encoded; wherein, The information to be encoded includes at least one of the following: first syntax element identification information of a current video sequence, second syntax element identification information of a current frame, third syntax element identification information of the current frame, fourth syntax element identification information of a current block, fifth syntax element identification information of the current block, sixth syntax element identification information of the current frame, a residual scaling factor, and an initial residual value of at least one partition block included in the current frame; wherein, the current frame includes at least one partition block, and the current block is any one of the at least one partition block.

Images