Filtering method, encoder, decoder and storage medium

Abstract

The embodiment of the application discloses a filtering method, an encoder, a decoder and a storage medium. The method comprises the following steps: acquiring a frame-level use identification bit by analyzing a code stream; acquiring a frame-level switch identification bit and a frame-level quantization parameter adjustment identification bit when the frame-level use identification bit represents that a neural network filter is enabled for a current frame; acquiring an adjusted frame-level quantization parameter when the frame-level switch identification bit represents that the neural network filter is performed on each block in the current frame and the frame-level quantization parameter adjustment identification bit represents that a quantization parameter is adjusted for the current frame; and performing filtering on a current block of the current frame based on the adjusted frame-level quantization parameter and a neural network filter model, so as to obtain first residual information of the current block.

Description

[0001] This application is a divisional application of patent application filed on April 13, 2022, with application number 202280094789.4 and title "Filtering Method, Encoder, Decoder and Storage Medium". Technical Field

[0002] This application relates to the field of image processing technology, and in particular to a filtering method, encoder, decoder, and storage medium. Background Technology

[0003] In video encoding and decoding systems, most video coding employs a block-based hybrid coding framework. Each frame in the video is divided into several Coding Tree Units (CTUs), and each CTU can be further divided into several rectangular Coding Units (CUs). These CUs can be rectangular or square blocks. Because adjacent CUs use different coding parameters, such as different transform processes, different quantization parameters (QPs), different prediction methods, and different reference image frames, and because the magnitude and distribution characteristics of the errors introduced by each CU are independent, the discontinuity at the boundaries of adjacent CUs produces block artifacts. This affects the subjective and objective quality of the reconstructed image and even the prediction accuracy of subsequent encoding and decoding.

[0004] Thus, during encoding and decoding, loop filters are used to improve the subjective and objective quality of the reconstructed image. The neural network-based loop filtering method exhibits the most outstanding encoding performance. In related technologies, on the one hand, a neural network filtering model is switched at the coding tree unit level. Different neural network filtering models are trained based on different sequence-level quantization parameter values ​​(BaseQP). The encoder tries these different neural network filtering models, and the neural network filtering model with the lowest rate-distortion cost is used as the optimal network model for the current coding tree unit. Through the usage flag bits and network model index information at the coding tree unit level, the decoder can use the same network model as the encoder for filtering. On the other hand, for different test conditions and quantization parameters, a simplified, low-complexity neural network filtering model can be used for loop filtering. When using a low-complexity neural network filtering model, quantization parameter information is added as an additional input, that is, the quantization parameter information is used as input to the network to improve the generalization ability of the neural network filtering model, so as to achieve good encoding performance without switching neural network filtering models.

[0005] However, when using a neural network filtering model that switches between coding tree units for filtering, the hardware implementation is complex and costly because each coding tree unit corresponds to a different neural network filtering model. Conversely, using a low-complexity neural network filtering model results in less flexibility in filtering choices due to the influence of quantization parameters, limiting the options available for encoding and decoding, and ultimately failing to achieve satisfactory encoding and decoding results. Summary of the Invention

[0006] This application provides a filtering method, encoder, decoder, and storage medium, which can make the selection of input parameters for filtering more flexible without increasing complexity, thereby improving encoding and decoding efficiency.

[0007] The technical solution of this application embodiment can be implemented as follows:

[0008] In a first aspect, embodiments of this application provide a filtering method applied to a decoder, the method comprising:

[0009] Parse the bitstream to obtain the frame-level usage flags based on the neural network filtering model;

[0010] When the frame-level usage flag indicates that it is in use, the frame-level switch flag and the frame-level quantization parameter adjustment flag are obtained; the frame-level switch flag is used to determine whether each block in the current frame is filtered.

[0011] When the frame-level switch flag indicates that it is enabled and the frame-level quantization parameter adjustment flag indicates that it is used, the adjusted frame-level quantization parameter is obtained.

[0012] Based on the adjusted frame-level quantization parameters and the neural network filtering model, the current block of the current frame is filtered to obtain the first residual information of the current block.

[0013] Secondly, embodiments of this application provide a filtering method applied to an encoder, the method comprising:

[0014] Enabling sequence-level access allows the use of flags;

[0015] When the sequence-level permission flag indicates permission, obtain the original value of the current block, the reconstructed value of the current block, and the frame-level quantization parameters in the current frame;

[0016] The first reconstruction value is determined by filtering and estimating the current block based on the neural network filtering model, the reconstructed value of the current block, and the frame-level quantization parameters.

[0017] Rate distortion cost is estimated based on the first reconstructed value and the original value of the current block to obtain the rate distortion cost of the current block. The first rate distortion cost of the current frame is determined by traversing the current frame.

[0018] Based on the neural network filtering model, at least one frame-level quantization bias parameter, the frame-level quantization parameter, and the reconstructed value of the current block in the current frame, at least one filtering estimation is performed on the current frame to determine at least one second rate distortion cost of the current frame.

[0019] Based on the first rate-distortion cost and the at least one second rate-distortion cost, a frame-level quantization parameter adjustment flag is determined.

[0020] Thirdly, embodiments of this application provide a decoder, which includes:

[0021] The parsing section is configured to parse the bitstream and obtain the frame-level usage flag bits based on the neural network filtering model;

[0022] The first determining part is configured to acquire a frame-level switch flag and a frame-level quantization parameter adjustment flag when the frame-level use flag indicates that the frame is being used; the frame-level switch flag is used to determine whether each block in the current frame is filtered.

[0023] The first adjustment part is configured to obtain the adjusted frame-level quantization parameters when the frame-level switch flag indicates that it is enabled and the frame-level quantization parameter adjustment flag indicates that it is used.

[0024] The first filtering part is configured to filter the current block of the current frame based on the adjusted frame-level quantization parameters and the neural network filtering model to obtain the first residual information of the current block.

[0025] Fourthly, embodiments of this application provide an encoder, which includes:

[0026] The second determining part is configured to obtain a sequence-level allowed flag bit; and when the sequence-level allowed flag bit indicates that it is allowed, obtain the original value of the current block, the reconstructed value of the current block and the frame-level quantization parameters in the current frame;

[0027] The second filtering part is configured to perform filtering estimation on the current block based on the neural network filtering model, the reconstructed value of the current block, and the frame-level quantization parameters to determine the first reconstructed value;

[0028] The second determining part is further configured to perform rate-distortion cost estimation based on the first reconstructed value and the original value of the current block to obtain the rate-distortion cost of the current block, and traverse the current frame to determine the first rate-distortion cost of the current frame;

[0029] The second filtering part is further configured to perform at least one filtering estimation on the current frame based on the neural network filtering model, at least one frame-level quantization bias parameter, the frame-level quantization parameter, and the reconstructed value of the current block in the current frame, to determine at least one second rate distortion cost of the current frame.

[0030] The second determining portion is further configured to determine a frame-level quantization parameter adjustment flag bit based on the first rate-distortion cost and the at least one second rate-distortion cost.

[0031] Fifthly, embodiments of this application also provide a decoder, which includes:

[0032] The first memory is configured to store computer programs that can run on the first processor;

[0033] The first processor is configured to execute the method described in the first aspect when running the computer program.

[0034] Sixthly, embodiments of this application also provide an encoder, the encoder comprising:

[0035] The second memory is configured to store computer programs that can run on the second processor;

[0036] The second processor is configured to execute the method described in the second aspect when running the computer program.

[0037] In a seventh aspect, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a first processor, implements the method described in the first aspect, or when executed by a second processor, implements the method described in the second aspect.

[0038] This application provides a filtering method, encoder, decoder, and storage medium. By parsing the bitstream, a frame-level usage flag based on a neural network filtering model is obtained. When the frame-level usage flag indicates usage, a frame-level switch flag and a frame-level quantization parameter adjustment flag are obtained. The frame-level switch flag is used to determine whether all blocks within the current frame are filtered. When the frame-level switch flag indicates on and the frame-level quantization parameter adjustment flag indicates usage, the adjusted frame-level quantization parameters are obtained. Based on the adjusted frame-level quantization parameters and the neural network filtering model, the current block of the current frame is filtered to obtain the first residual information of the current block. In this way, based on the frame-level quantization parameter adjustment flag, it is possible to determine whether the quantization parameters input to the neural network filtering model need adjustment, achieving flexible selection and diverse variation processing of quantization parameters (input parameters), thereby improving decoding efficiency. Attached Figure Description

[0039] Figure 1A-1C Exemplary component distribution diagrams in different color formats provided for embodiments of this application;

[0040] Figure 2 A schematic diagram illustrating the division of an exemplary encoding unit provided in an embodiment of this application;

[0041] Figure 3A A structural diagram of an exemplary neural network filtering model provided in this application embodiment is shown in Figure 1.

[0042] Figure 3B The structure of the exemplary neural network filtering model provided in the embodiments of this application Figure 2 ;

[0043] Figure 4 Figure 3 shows the structure of an exemplary neural network filtering model provided in this application embodiment;

[0044] Figure 5 An exemplary video encoding system structure diagram provided for embodiments of this application;

[0045] Figure 6 This application provides an exemplary video decoding system structure diagram;

[0046] Figure 7 A flowchart illustrating a filtering method provided in an embodiment of this application;

[0047] Figure 8 A flowchart illustrating another filtering method provided in an embodiment of this application;

[0048] Figure 9 A schematic diagram of the composition structure of a decoder provided in an embodiment of this application;

[0049] Figure 10 A schematic diagram of the hardware structure of a decoder provided in an embodiment of this application;

[0050] Figure 11 A schematic diagram of the composition structure of an encoder provided in an embodiment of this application;

[0051] Figure 12 This is a schematic diagram of the hardware structure of an encoder provided in an embodiment of this application. Detailed Implementation

[0052] In this application embodiment, digital video compression technology mainly compresses massive amounts of digital video data to facilitate transmission and storage. With the surge in internet video and increasing demands for video clarity, although existing digital video compression standards can save considerable video data, there is still a need to pursue better digital video compression technologies to reduce the bandwidth and traffic pressure of digital video transmission.

[0053] In digital video encoding, the encoder reads unequal pixels from the original video sequence in different color formats, including luminance and chrominance components; that is, the encoder reads a black-and-white or color image. This image is then divided into blocks, which are encoded by the encoder. The encoder typically uses a hybrid frame coding mode, generally including intra-frame and inter-frame prediction, transform and quantization, inverse transform and inverse quantization, loop filtering, and entropy coding. Intra-frame prediction only references information from the same frame, predicting pixel information within the current block to eliminate spatial redundancy. Inter-frame prediction can reference information from different frames, using motion estimation to search for the motion vector information of the best-matching current block to eliminate temporal redundancy. Transform and quantization convert the predicted image blocks to the frequency domain, redistributing energy; combined with quantization, it removes information that is insensitive to the human eye, eliminating visual redundancy. Entropy coding eliminates character redundancy based on the current context model and the probability information of the binary bitstream. Loop filtering mainly processes the pixels after inverse transform and inverse quantization, compensating for distortion information and providing a better reference for subsequent encoded pixels.

[0054] Currently, the scenarios in which filtering can be performed can be either the HPM reference software test platform based on AVS or the VVC reference software test model (VTM) based on Versatile Video Coding (VVC). This application does not impose any limitations on these scenarios.

[0055] In video images, the current coding block (CB) is generally represented by a first video component, a second video component, and a third video component. These three image components are a luminance component, a blue chrominance component, and a red chrominance component, respectively. The luminance component is usually represented by the symbol Y, the blue chrominance component is usually represented by the symbol Cb or U, and the red chrominance component is usually represented by the symbol Cr or V. Thus, video images can be represented in YCbCr format or YUV format.

[0056] Typically, digital video compression technology is applied to image data in YCbCr (YUV) format, where the color encoding method is YCbCr. The YUV ratio is usually 4:2:0, 4:2:2, or 4:4:4. Y represents luminance (Luma), Cb (U) represents blue chroma, Cr (V) represents red chroma, and U and V represent chroma (Chroma), which are used to describe color and saturation. Figures 1A to 1C This shows the component distribution plots under different color formats, where white represents the Y component and black and gray represent the UV components. For example... Figure 1A As shown, in color format, 4:2:0 means that every 4 pixels have 4 luminance components and 2 chrominance components (YYYYCbCr), such as... Figure 1B As shown, 4:2:2 means that every 4 pixels have 4 luminance components and 4 chrominance components (YYYYCbCrCbCr), while... Figure 1C As shown, 4:4:4 represents full pixel display (YYYYCbCrCbCrCbCrCbCr).

[0057] Currently, common video codec standards are based on a block-based hybrid coding framework. Each frame in a video image is divided into a square of equal size (e.g., 128×128, 64×64, etc.) called a Largest Coding Unit (LCU). Each LCU can be further divided into rectangular Coding Units (CUs) according to rules; and the CUs may be further divided into smaller Prediction Units (PUs). Specifically, the hybrid coding framework may include modules such as prediction, transformation, quantization, entropy coding, and in-loop filtering. The prediction module may include intra-prediction and inter-prediction, and inter-prediction may include motion estimation and motion compensation. Since there is a strong correlation between adjacent pixels within a video frame, using intra-prediction in video codec technology can eliminate spatial redundancy between adjacent pixels. Inter-frame prediction can refer to image information from different frames and use motion estimation to search for the motion vector information that best matches the current segmentation block to eliminate temporal redundancy; transformation converts the predicted image block to the frequency domain, redistributes energy, and combined with quantization, can remove information that is not sensitive to the human eye to eliminate visual redundancy; entropy coding can eliminate character redundancy based on the current context model and the probability information of the binary code stream.

[0058] It should be noted that during video encoding, the encoder first reads the image information and divides the image into several Coding Tree Units (CTUs). Each CTU can be further divided into several Coding Units (CUs). These CUs can be rectangular or square blocks; for details, please refer to [reference needed]. Figure 2 As shown.

[0059] During intra-frame prediction, the current coding unit cannot refer to information from different frames; it can only use adjacent coding units within the same frame as reference information for prediction. That is, based on the most common left-to-right, top-to-bottom coding order, the current coding unit can refer to the top-left, top-left, and left-side coding units as reference information to predict its own prediction. The current coding unit then serves as the reference information for the next coding unit, and so on, predicting the entire image. If the input digital video is in color format, i.e., the current mainstream digital video encoder input source is YUV 4:2:0 format (meaning each four pixels of the image consists of four Y components and two UV components), the encoder will encode the Y and UV components separately, using slightly different encoding tools and techniques. Simultaneously, the decoding end will also decode according to the different formats.

[0060] For the intra-frame prediction part of digital video encoding and decoding, the prediction of the current block is mainly based on the image information of the adjacent blocks of the current frame. The residual information is obtained by calculating the residual between the predicted block and the original image block and then transmitting the residual information to the decoder through processes such as transformation and quantization. After receiving and parsing the bitstream, the decoder obtains the residual information through steps such as inverse transformation and inverse quantization. The predicted image block obtained by the decoder is then superimposed with the residual information to obtain the reconstructed image block.

[0061] Currently, common video codec standards (such as H.266 / VVC) all employ a block-based hybrid coding framework. Each frame in the video is divided into largest coding units (LCUs) of the same size (e.g., 128x128, 64x64, etc.). Each LCU can be further divided into rectangular coding units (CUs) according to rules. Coding units may also be divided into prediction units (PUs), transform units (TUs), etc. The hybrid coding framework includes modules such as prediction, transform, quantization, entropy coding, and in-loop filtering. The prediction module includes intra-prediction and inter-prediction. Inter-prediction includes motion estimation and motion compensation. Because there is a strong correlation between adjacent pixels in a video frame, intra-prediction is used in video codec technology to eliminate spatial redundancy between adjacent pixels. Because there is a strong similarity between adjacent frames in a video, inter-frame prediction methods are used in video encoding and decoding technology to eliminate temporal redundancy between adjacent frames, thereby improving encoding efficiency.

[0062] The basic workflow of a video codec is as follows: At the encoding end, a frame is divided into blocks. Intra-frame prediction or inter-frame prediction is used on the current block to generate a prediction block. The prediction block is subtracted from the original image block to obtain a residual block. The residual block is transformed and quantized to obtain a quantization coefficient matrix. The quantization coefficient matrix is ​​then entropy-encoded and output to the bitstream. At the decoding end, intra-frame prediction or inter-frame prediction is used on the current block to generate a prediction block. Simultaneously, the bitstream is parsed to obtain the quantization coefficient matrix. The quantization coefficient matrix is ​​inverse-quantized and inverse-transformed to obtain a residual block. The prediction block and the residual block are added to obtain a reconstructed block. The reconstructed blocks form a reconstructed image. Loop filtering is performed on the reconstructed image based on the image or blocks to obtain the decoded image. The encoding end also requires similar operations to the decoding end to obtain the decoded image. The decoded image can serve as a reference frame for inter-frame prediction in subsequent frames. The block division information, prediction, transform, quantization, entropy coding, loop filtering, and other mode or parameter information determined at the encoding end need to be output to the bitstream if necessary. The decoding end determines the same block partitioning information, prediction, transform, quantization, entropy coding, loop filtering, and other mode or parameter information as the encoding end by parsing and analyzing existing information, thus ensuring that the decoded image obtained by the encoding end is the same as that obtained by the decoding end. The decoded image obtained by the encoding end is usually called the reconstructed image. During prediction, the current block can be divided into prediction units, and during transform, the current block can be divided into transform units. The division of prediction units and transform units can be different. The above is the basic flow of a video codec under a block-based hybrid coding framework. With the development of technology, some modules or steps of this framework or flow may be optimized. The current block can be the current coding unit (CU) or the current prediction unit (PU), etc.

[0063] The international video coding standards organization JVET has established two exploratory experimental groups: one for neural network coding and the other for going beyond VVC, and has also established several corresponding expert discussion groups.

[0064] The aforementioned experimental group that surpasses VVC aims to explore higher coding efficiency based on the latest coding and decoding standard H.266 / VVC with strict performance and complexity requirements. The coding method studied by the group is closer to VVC and can be called a traditional coding method. Currently, the performance of the algorithm reference model of this experimental group has exceeded the coding performance of the latest VVC reference model VTM by about 15%.

[0065] The first exploratory experimental group studied a learning method based on neural networks—an intelligent coding approach. Deep learning and neural networks are currently hot topics across various industries, especially in computer vision, where deep learning-based methods often have overwhelming advantages. Experts from the JVET standards organization brought neural networks into the field of video encoding and decoding. Leveraging the powerful learning capabilities of neural networks, neural network-based coding tools often boast high coding efficiency. In the early stages of VVC standard development, many companies focused on deep learning-based coding tools, proposing methods including intra-frame prediction based on neural networks, inter-frame prediction based on neural networks, and loop filtering based on neural networks. Among these, the loop filtering method based on neural networks exhibited the most outstanding coding performance, achieving over 8% improvement after multiple research conferences. Currently, the loop filtering scheme based on neural networks researched by the first exploratory experimental group at the JVET conference has achieved a coding performance as high as 12%, contributing almost half a generation of coding performance.

[0066] This application improves upon the exploratory experiments presented at the current JVET conference by proposing a neural network-based loop filtering enhancement scheme. The following will first provide a brief overview of the neural network-based loop filtering schemes currently used at the JVET conference, followed by a detailed description of the improvements made in this application.

[0067] Currently, research on neural network-based loop filtering schemes at the JVET conference mainly focuses on two forms: the first is a multi-model intra-frame switchable scheme; the second is an intra-frame non-switchable model scheme. However, regardless of the scheme, the neural network architecture remains largely unchanged, and this tool is used in traditional hybrid coding frameworks for intra-loop filtering. Therefore, the basic processing unit in both schemes is the coding tree unit, i.e., the maximum coding unit size.

[0068] The biggest difference between the first multi-model intra-frame switchable scheme and the second intra-frame non-switchable model scheme is that, during the encoding and decoding of the current frame, the first scheme allows for arbitrary switching of neural network models, while the second scheme does not. Taking the first scheme as an example, when encoding a frame of image, each coding tree unit has multiple candidate neural network models to choose from. The encoding end selects which neural network model will provide the best filtering effect for the current coding tree unit, and then writes the index of that neural network model into the bitstream. That is, if the coding tree unit needs to perform filtering, a coding tree unit-level usage flag must be transmitted first, followed by the neural network model index. If filtering is not required, only a coding tree unit-level usage flag needs to be transmitted. After parsing the index value, the decoding end loads the neural network model corresponding to that index into the current coding tree unit to perform filtering on the current coding tree unit.

[0069] Taking the second scheme as an example, when encoding a frame of image, the neural network model available for each coding tree unit in the current frame is fixed, and each coding tree unit uses the same neural network model. That is, there is no model selection process at the encoding end in the second scheme. At the decoding end, the use flag bit of whether the current coding tree unit uses the neural network-based loop filter is parsed. If the use flag bit is true, the pre-set model (the same as at the encoding end) is used to filter the coding tree unit. If the use flag bit is false, no additional operation is performed.

[0070] The first type of multi-model intra-frame switchable scheme offers strong flexibility at the coding tree unit level, allowing model adjustments based on local details—that is, achieving local optima to achieve better global results. This scheme typically employs numerous neural network models, training different models for different quantization parameters under JVET's general testing conditions. Furthermore, different coding frame types may also require different neural network models to achieve better performance. Taking filter1 of the JVET-Y0080 scheme as an example, this filter uses up to 22 neural network models to cover different coding frame types and different quantization parameters, with model switching performed at the coding tree unit level. This filter can provide more than 10% better coding performance compared to existing VVC methods.

[0071] The second approach, using JVET-Y0078 as an example, employs two neural network models but does not switch between them within a frame. At the encoding end, if the current frame type is an I-frame, the corresponding neural network model is imported, and only that model is used within the current frame. If the current frame type is a B-frame, the corresponding neural network model is imported, and again, only that model is used within the current frame. This approach provides 8.65% better encoding performance than existing VVC methods. While slightly lower than the first approach, its overall performance is nearly unattainable compared to traditional encoding tools.

[0072] Option 1 offers greater flexibility and higher encoding performance, but it suffers from a fatal flaw in hardware implementation. Recent discussions at the JVET conference revealed hardware experts' concerns about the code for intra-frame model switching. Switching models at the encoding tree unit level means that, in the worst-case scenario, the decoding end needs to reload the neural network model every time it processes an encoding tree unit. This is not only an increase in hardware implementation complexity but also an additional burden on current high-performance GPUs. Furthermore, the existence of multiple models also means that a large number of parameters need to be stored, which is a significant overhead in current hardware implementations.

[0073] In contrast, Scheme Two, with its neural network loop filtering, further explores the powerful generalization ability of deep learning. It uses various types of information as input instead of solely relying on reconstructed samples, providing more information to aid the neural network's learning and better demonstrating its generalization ability while eliminating many unnecessary redundant parameters. Continuously updated schemes, up until the last meeting, have resulted in a single, simplified, low-complexity neural network model capable of handling different testing conditions and quantization parameters. Compared to Scheme One, this eliminates the overhead of constantly reloading the model and the need for larger storage spaces for numerous parameters.

[0074] The above provides a simple comparison of the advantages and disadvantages of the two schemes. Next, we will mainly introduce the architecture of the neural network scheme itself.

[0075] The model architecture of Scheme 1, taking JVET-Y0080 as an example, has the following simple network structure: Figure 3B As shown.

[0076] It can be seen that the main body of the network is composed of multiple ResBlocks, and the structure of the ResBlocks is as follows: Figure 3A The given information is provided. A single ResBlock consists of multiple convolutional layers followed by CBAM layers. CBAM (Convolutional Blocks Attention Module) is an attention mechanism module primarily responsible for further extraction of detailed features. Additionally, ResBlocks have a direct skip connection structure between the input and output. There is also a skip connection at the overall network framework, which connects the reconstructed YUV information of the input with the shuffled output.

[0077] The network's inputs mainly consist of reconstructed YUV (rec), predicted YUV (prde), and YUV with partitioning information (par). All inputs undergo simple convolution and activation operations, are then concatenated, and fed into the main network. It's worth noting that the processing of YUVs with partitioning information may differ between I-frames and B-frames; I-frames require YUVs with partitioning information as input, while B-frames do not.

[0078] In summary, for any JVET parameter point in each I-frame and B-frame, Scheme 1 provides a corresponding neural network parameter model. Furthermore, since the YUV components are primarily composed of luminance and chrominance channels, there are differences in the color components.

[0079] The model architecture of Scheme 2, taking JVET-Y0078 as an example, has the following simple network structure: Figure 4 As shown.

[0080] As can be seen, Scheme 1 and Scheme 2 are basically the same in terms of the main network structure. The difference lies in the fact that Scheme 2 adds quantization parameter information as an additional input compared to Scheme 1. Scheme 1 loads different neural network parameter models based on different quantization parameter information to achieve more flexible processing and more efficient encoding results, while Scheme 2 uses the quantization parameter information as the network input to improve the generalization ability of the neural network, enabling the model to adapt to different quantization parameter conditions and provide good filtering performance.

[0081] like Figure 4 As shown, two quantization parameters are input to the network: BaseQP and SliceQP. BaseQP refers to the sequence-level quantization parameters set by the encoder when encoding the video sequence, i.e., the quantization parameter points required for JVET pass testing, and also the parameters used to select the neural network model in Scheme 1. SliceQP is the quantization parameter for the current frame. The quantization parameter for the current frame can differ from the sequence-level parameter because, during video encoding, the quantization conditions for B-frames differ from those for I-frames, and the quantization parameters also differ at different temporal levels. Therefore, SliceQP in B-frames is generally different from BaseQP. Thus, in the JVET-Y0078 scheme design, the neural network model for I-frames only needs SliceQP as input, while the neural network model for B-frames requires both BaseQP and SliceQP as input.

[0082] Scheme 2 differs from Scheme 1 in one key aspect. In Scheme 1, the model output generally doesn't require additional processing. If the output is residual information, it's superimposed with the reconstructed samples of the current coding tree unit and used as the output of the neural network-based loop filter. If the output is a complete reconstructed sample, it's the output of the neural network-based loop filter. However, Scheme 2's output typically requires scaling. Taking residual information as an example, the model infers and outputs the residual information of the current coding tree unit. This residual information is scaled before being superimposed with the reconstructed sample information of the current coding tree unit. This scaling factor is obtained at the encoding end and needs to be written into the code stream before being sent to the decoding end.

[0083] Because quantization parameters serve as additional information input, the reduction in the number of models has become possible, making it the most popular solution at the current JVET conference.

[0084] Furthermore, general neural network-based loop filtering schemes do not have to be exactly the same as the two schemes mentioned above. The specific details of the schemes may differ, but the main ideas are basically the same. For example, the differences in the second scheme may be reflected in the design of the neural network architecture, such as the convolution size of ResBlocks, the number of convolution layers, and whether attention modules are included. They may also be reflected in the input of the neural network, which may even have more additional information, such as the boundary strength value of the deblocking filter.

[0085] Scheme 1 allows switching neural network models at the encoding tree unit level. These different neural network models are trained using different BaseQPs. By trying these different neural network models at the encoder end, the network model with the lowest rate-distortion cost is the optimal network model for the current encoding tree unit. Through the usage flags and network model index information at the encoding tree unit level, the decoder can use the same network model as the encoder end for filtering. Scheme 2, by using input quantization parameters, achieves good encoding performance without switching models, initially addressing hardware implementation concerns. However, Scheme 2's performance is still inferior to Scheme 1. The main drawback is the lack of flexibility in BaseQP switching; Scheme 2 has fewer choices at the encoder end, resulting in suboptimal performance.

[0086] This application provides a video encoding system. Figure 5This is a schematic diagram of the composition structure of a video coding system according to an embodiment of this application. The video coding system 10 includes: a transform and quantization unit 101, an intra-frame estimation unit 102, an intra-frame prediction unit 103, a motion compensation unit 104, a motion estimation unit 105, an inverse transform and inverse quantization unit 106, a filter control and analysis unit 107, a filtering unit 108, an encoding unit 109, and a decoded image buffer unit 110, etc. The filtering unit 108 can implement DBF filtering / SAO filtering / ALF filtering, and the encoding unit 109 can implement header information encoding and context-based adaptive binary arithmetic coding (CABAC). For the input raw video signal, the coding tree unit... The partitioning of a video coding unit (CTU) yields a video coding block. The residual pixel information obtained after intra-frame or inter-frame prediction is then transformed by the transform and quantization unit 101. This transformation involves converting the residual information from the pixel domain to the transform domain and quantizing the resulting transform coefficients to further reduce the bit rate. Intra-frame estimation unit 102 and intra-frame prediction unit 103 perform intra-frame prediction on the video coding block. Specifically, intra-frame estimation unit 102 and intra-frame prediction unit 103 determine the intra-frame prediction mode to be used to encode the video coding block. Motion compensation unit 104 and motion estimation unit 105 perform inter-frame prediction coding of the received video coding block relative to one or more blocks in one or more reference frames to provide temporal prediction information. Motion estimation performed by motion estimation unit 105 is the process of generating motion vectors, which can estimate the motion of the video coding block. Then, motion compensation unit 104 uses the motion vectors determined by motion estimation unit 105 to generate motion vectors. The motion compensation is performed. After determining the intra-prediction mode, the intra-prediction unit 103 is also used to provide the selected intra-prediction data to the coding unit 109, and the motion estimation unit 105 also sends the calculated motion vector data to the coding unit 109. In addition, the inverse transform and inverse quantization unit 106 is used to reconstruct the video coding block, reconstruct the residual block in the pixel domain, and remove the block artifacts by the filter control analysis unit 107 and the filtering unit 108. Then, the reconstructed residual block is added to a predictive block in the frame of the decoding image buffer unit 110 to generate the reconstructed video coding block. The coding unit 109 is used to encode various coding parameters and quantized transform coefficients. In the CABAC-based coding algorithm, the context content can be based on adjacent coding blocks and can be used to encode information indicating the determined intra-prediction mode and output the bitstream of the video signal. The decoding image buffer unit 110 is used to store the reconstructed video coding block for prediction reference.As video image encoding proceeds, new reconstructed video encoding blocks are continuously generated, and these reconstructed video encoding blocks are stored in the decoding image buffer unit 110.

[0087] This application provides a video decoding system. Figure 6 This is a schematic diagram of the composition structure of a video decoding system according to an embodiment of this application. The video decoding system 20 includes: a decoding unit 201, an inverse transform and inverse quantization unit 202, an intra-frame prediction unit 203, a motion compensation unit 204, a filtering unit 205, and a decoding image buffer unit 206, etc. The decoding unit 201 can perform header information decoding and CABAC decoding, and the filtering unit 205 can perform DBF filtering / SAO filtering / ALF filtering. The input video signal is processed... Figure 3A After encoding, the video signal bitstream is output. This bitstream is input into the video decoding system 20, first passing through the decoding unit 201 to obtain the decoded transform coefficients. The transform coefficients are then processed by the inverse transform and inverse quantization unit 202 to generate residual blocks in the pixel domain. The intra-frame prediction unit 203 can generate prediction data for the current video decoding block based on the determined intra-frame prediction mode and data from previously decoded blocks in the current frame or image. The motion compensation unit 204 determines the prediction information for the video decoding block by analyzing motion vectors and other associated syntax elements, and uses this prediction information. The predictive block of the video block being decoded is generated; the decoded video block is formed by summing the residual block from the inverse transform and inverse quantization unit 202 with the corresponding predictive block generated by the intra-prediction unit 203 or the motion compensation unit 204; the decoded video signal is passed through the filtering unit 205 to remove block artifacts, which can improve video quality; then the decoded video block is stored in the decoding image buffer unit 206, which stores reference images for subsequent intra-prediction or motion compensation, and is also used for the output of the video signal, thus obtaining the recovered original video signal.

[0088] It should be noted that the filtering method provided in the embodiments of this application can be applied to, for example, Figure 5 The filter unit 108 shown (represented by a bold black box) can also be applied to, for example... Figure 6 The filtering unit 205 shown is indicated by a bold black box. In other words, the filtering method in this embodiment can be applied to a video encoding system (referred to as an "encoder"), a video decoding system (referred to as a "decoder"), or even simultaneously, but no limitations are imposed here.

[0089] This application's embodiment can be implemented based on the above-described intra-frame non-switching model scheme. The main idea is to leverage the variability of the input book to provide the encoder with more possibilities. The input book of the neural network filtering model contains quantization parameters, which include sequence-level quantization parameter values ​​(BaseQP) or frame-level quantization parameter values ​​(SliceQP). Adjusting the BaseQP and SliceQP as inputs allows the encoder and decoder to have more options to try, thereby improving encoding and decoding efficiency.

[0090] This application provides a filtering method applied to a decoder, such as... Figure 7 As shown, the method may include:

[0091] S101. Parse the bitstream and obtain the frame-level usage flag based on the neural network filtering model;

[0092] In this embodiment, at the decoding end, the decoder uses intra-frame prediction or inter-frame prediction to generate a prediction block for the current block. Simultaneously, the decoder parses the bitstream to obtain a quantization coefficient matrix, performs inverse quantization and inverse transform on the quantization coefficient matrix to obtain a residual block, and adds the prediction block and the residual block to obtain a reconstructed block. The reconstructed blocks then form the reconstructed image. The decoder performs loop filtering on the reconstructed image based on the image or on the block to obtain the decoded image.

[0093] It should be noted that since the original image can be divided into CTUs (Coding Tree Units) or CTUs into CUs, the filtering method of this application embodiment can be applied not only to loop filtering at the CU level (where the block partitioning information is CU partitioning information) but also to loop filtering at the CTU level (where the block partitioning information is CTU partitioning information). This application embodiment does not impose any specific limitations.

[0094] This application will describe the embodiments using CTU as an example.

[0095] In this embodiment, during the loop filtering process of the reconstructed image of the current frame, the decoder can parse the sequence-level enable flag (sps_nnlf_enable_flag) by parsing the bitstream. The sequence-level enable flag is a switch to enable or disable filtering for the entire video sequence. When the sequence-level enable flag indicates that filtering is enabled, the decoder parses the syntax elements of the current frame to obtain the frame-level enable flag based on the neural network filtering model. This frame-level enable flag indicates whether filtering is used in the current frame. When the frame-level enable flag indicates that filtering is used, it means that some or all blocks in the current frame require filtering; when the frame-level enable flag indicates that filtering is not used, it means that all blocks in the current frame do not require filtering. The decoder can then continue to iterate through other filtering methods to output the complete reconstructed image.

[0096] It should be noted that the relevant syntax elements are set to their initial values ​​or are set to no by default.

[0097] It should be noted that the form of the frame-level flag bit used in the neural network filtering model is not limited, and it can be letters or symbols, etc., and the embodiments of this application do not impose any restrictions.

[0098] For example, the value of the frame-level usage flag bit based on the neural network filtering model can be 1 to represent usage and 0 to represent non-use. The embodiments of this application do not limit the representation and meaning of the frame-level usage flag bit value.

[0099] In some embodiments of this application, the frame-level usage flag for the current frame can be represented by one or more flag bits. When multiple flag bits are represented, each color component of the current frame can correspond to its own frame-level usage flag bit, i.e., the frame-level usage flag bit for the color component. The frame-level usage flag bit of a color component indicates whether filtering is required in the block of the current frame under that color component.

[0100] It should be noted that the decoder uses frame-level flags to determine whether to perform filtering on blocks under each color component by traversing each color component of the current frame.

[0101] S102. When the frame-level usage flag indicates that the frame is being used, obtain the frame-level switch flag and the frame-level quantization parameter adjustment flag. The frame-level switch flag is used to determine whether each block in the current frame is filtered.

[0102] In this embodiment, the decoder determines the frame-level usage flag of the current frame and can also parse the frame-level switch flag and frame-level quantization parameter adjustment flag from the bitstream. The frame-level switch flag is used to determine whether each block in the current frame is filtered.

[0103] Each block here can be a coding tree unit of the current frame.

[0104] The frame-level switch flag can be assigned to each color component individually. The frame-level switch flag can also indicate whether to use a neural network-based loop filtering technique for all coding tree units under the current color component.

[0105] In this embodiment, if the frame-level switch flag is enabled, it indicates that all coding tree units under the current color component use neural network-based loop filtering technology for filtering; that is, the coding tree unit-level usage flag of all coding tree units in the current frame under this color component is automatically set to "use". If the frame-level switch flag is disabled, it indicates that some coding tree units under the current color component use neural network-based loop filtering technology, while others do not. If the frame-level switch flag is disabled, it is necessary to further parse the coding tree unit-level usage flag of all coding tree units in the current frame under this color component.

[0106] It should be noted that, in the embodiments of this application, when the coding tree unit is used as a block, the coding tree unit level use flag bit can also be understood as the block level use flag bit.

[0107] For example, the value of the frame-level switch flag can be 1 to indicate that it is enabled and 0 to indicate that it is not enabled. The embodiments of this application do not limit the expression form and meaning of the frame-level switch flag value.

[0108] In this embodiment, the frame-level quantization parameter adjustment flag indicates whether the quantization parameters (BaseQP and SliceQP) have been adjusted in the current frame. If the frame-level quantization parameter adjustment flag is used, it indicates that the quantization parameters of the current frame have been adjusted, and it is necessary to continue parsing and obtaining the frame-level quantization parameter adjustment index for subsequent filtering processing. If the frame-level quantization parameter adjustment flag is not used, it indicates that the quantization parameters of the current frame have been adjusted, and the bitstream can continue to be used. The quantization parameters parsed from the bitstream are then used to implement subsequent processing.

[0109] For example, the value of the frame-level quantization parameter adjustment flag bit can be 1 to indicate use and 0 to indicate unuse. The embodiments of this application do not limit the expression form and meaning of the frame-level quantization parameter adjustment flag bit value.

[0110] In some embodiments of this application, the decoder can select whether to adjust the quantization parameters of the current frame based on the type of the encoded frame. For the first type of frame, quantization parameters need to be adjusted, while for the second type (frames of a type other than the first type), no adjustment is required. Therefore, during decoding, the decoder can obtain the frame-level quantization parameter adjustment flag from the bitstream if the current frame is of the first type and can be filtered.

[0111] In some embodiments of this application, after the decoder obtains the frame-level usage flag bit based on the neural network filtering model and before obtaining the adjusted frame-level quantization parameters, when the frame-level usage flag bit indicates usage and the current frame is a first type frame, the decoder obtains the frame-level switch flag bit and the frame-level quantization parameter adjustment flag bit.

[0112] It should be noted that, in the embodiments of this application, the first type of frame can be a B frame or a P frame, and the embodiments of this application do not impose any restrictions.

[0113] It should be noted that the decoder can simultaneously parse the frame-level switch flag and the frame-level quantization parameter adjustment flag.

[0114] S103. When the frame-level switch flag indicates that it is enabled and the frame-level quantization parameter adjustment flag indicates that it is used, obtain the adjusted frame-level quantization parameters.

[0115] After parsing and obtaining the frame-level switch flag and the frame-level quantization parameter adjustment flag, the decoder obtains the adjusted frame-level quantization parameters when the frame-level switch flag indicates that it is enabled and the frame-level quantization parameter adjustment flag indicates that it is used.

[0116] It should be noted that when the frame-level switch flag indicates that it is enabled, it indicates that there is a coding tree unit that needs to be filtered under the current color component. Therefore, when the frame-level quantization parameter adjustment flag indicates that it is in use, the adjusted frame-level quantization parameters need to be obtained so that they can be used when filtering at the coding tree unit level.

[0117] In this embodiment, the frame-level quantization parameter adjustment flag indicates that it is used. The decoder can obtain the frame-level quantization adjustment index from the bitstream and determine the quantization parameters to be adjusted based on the frame-level quantization adjustment index.

[0118] In some embodiments of this application, the decoder adjusts the index based on the frame-level quantization parameters obtained from the bitstream to determine the frame-level quantization offset parameters; and determines the adjusted frame-level quantization parameters based on the obtained frame-level quantization parameters and frame-level quantization offset parameters.

[0119] Here, all coding tree units in the current frame are adjusted by the same amount, meaning that the quantization parameter inputs for all coding tree units are the same.

[0120] It should be noted that during encoding, if the encoder determines that quantization parameter adjustments are needed, it transmits the sequence number corresponding to the frame-level quantization bias parameter as the frame-level quantization adjustment index to the bitstream. The decoder stores the correspondence between the sequence number and the quantization bias parameter. Thus, the decoder can determine the frame-level quantization bias parameter based on the frame-level quantization adjustment index. The decoder then adjusts the frame-level quantization parameter using the frame-level quantization bias parameter to obtain the adjusted frame-level quantization parameter. The quantization parameter can be obtained from the bitstream.

[0121] For example, if the frame-level quantization parameter index adjustment flag of the current frame is in use, the quantization parameters are adjusted according to the frame-level quantization parameter adjustment index. For instance, if the quantization parameter adjustment index points to offset1, then BaseQP is superimposed with the offset parameter offset1 to obtain BasseQPFinal, which replaces BaseQP as the quantization parameter input to the network model for all coding tree units in the current frame.

[0122] In some embodiments of this application, the decoder obtains the adjusted frame-level quantization parameters from the bitstream.

[0123] In other words, the encoder can directly transmit the adjusted quantization parameters to the decoder via the bitstream for the decoder to use during decoding.

[0124] S104. Based on the adjusted frame-level quantization parameters and neural network filtering model, filter the current block of the current frame to obtain the first residual information of the current block.

[0125] After the decoder obtains the adjusted frame-level quantization parameters, it can perform filtering on all coding tree units of the current frame because the frame-level switch flag indicates that the frame is enabled. Filtering a coding tree unit requires traversing through the filtering of each color component before decoding the next coding tree unit.

[0126] In this embodiment, a neural network filtering model is used to filter the adjusted frame-level quantization parameters for the current block of the current frame, obtaining the first residual information of the current block. Here, the current block is the current coding tree unit.

[0127] In this embodiment, before the decoder filters the current block of the current frame based on adjusted frame-level quantization parameters and a neural network filtering model to obtain the first residual information of the current block, it obtains the reconstructed value of the current block. The neural network filtering model is then used to filter the reconstructed value of the current block and the adjusted frame-level quantization parameters to obtain the first residual information of the current block, thus completing the filtering of the current block.

[0128] In some embodiments of this application, the decoder filters the current block of the current frame based on the adjusted frame-level quantization parameters and the neural network filtering model. Before obtaining the first residual information of the current block, it obtains at least one of the predicted value of the current block, block partitioning information, and deblocking filter boundary strength, as well as the reconstructed value of the current block.

[0129] In some embodiments of this application, the decoder uses a neural network filtering model to filter at least one of the predicted value of the current block, block partitioning information and deblocking filter boundary strength, the reconstructed value of the current block, and the adjusted frame-level quantization parameters to obtain the first residual information of the current block, thereby completing the filtering of the current block.

[0130] It should be noted that during the filtering process, the input parameters to the neural network filtering model may include: the predicted value of the current block, block partitioning information, deblocking filter boundary strength, the reconstructed value of the current block, and adjusted frame-level quantization parameters (or quantization parameters). This application does not limit the types of information in the input parameters. However, the predicted value of the current block, block partitioning information, and deblocking filter boundary strength are not necessarily required every time and need to be determined based on the actual situation.

[0131] In some embodiments of this application, after the decoder filters the current block of the current frame based on the adjusted frame-level quantization parameters and the neural network filtering model to obtain the first residual information of the current block, the decoder can also obtain the second residual scaling factor in the bitstream; based on the second residual scaling factor, the first residual information of the current block is scaled to obtain the first target residual information.

[0132] Based on the first target residual information and the reconstruction value of the current block, determine the first target reconstruction value of the current block.

[0133] It should be noted that when the encoder obtains the residual information, it can scale the first residual information using a second residual scaling factor to obtain the first residual information. Therefore, the decoder needs to scale the first residual information of the current block based on the second residual scaling factor to obtain the first target residual information. Based on the first target residual information and the reconstructed value of the current block, the decoder determines the first target reconstructed value of the current block. However, if the encoder does not use a residual factor during encoding, but requires input of quantization parameters (or adjusted quantization parameters) during filtering, the filtering method provided in this application embodiment is also applicable, except that the residual information does not need to be scaled using a residual factor.

[0134] It should be noted that there are corresponding residual information and residual factors for each color component.

[0135] Understandably, the decoder can adjust the flag bit based on the frame-level quantization parameters to determine whether the quantization parameters of the input neural network filtering model need to be adjusted, thus enabling flexible selection and diverse handling of quantization parameters and improving decoding efficiency.

[0136] In some embodiments of this application, during the encoding and decoding filtering process, some data in the input parameters of the neural network filtering model can be adjusted using the aforementioned principle before filtering.

[0137] In the embodiments of this application, at least one of the input parameters, such as the quantization parameter, the predicted value of the current block, the block partitioning information, and the deblocking filter boundary strength, can be adjusted. The embodiments of this application do not impose any restrictions.

[0138] In some embodiments of this application, when the frame-level usage flag indicates that the frame is being used, the frame-level switch flag and the frame-level input parameter adjustment flag are obtained; the frame-level input parameter adjustment flag indicates whether any one of the parameters, such as the prediction value, block partitioning information, and deblocking filter boundary strength, has been adjusted.

[0139] When the frame-level switch flag indicates that it is enabled and the frame-level input parameter adjustment flag indicates that it is used, the adjusted block-level input parameters are obtained.

[0140] Based on the adjusted block-level input parameters, the acquired frame-level quantization parameters, and the neural network filtering model, the current block of the current frame is filtered to obtain the third residual information of the current block.

[0141] When the frame-level switch flag indicates that the block-level usage flag is not enabled, it is necessary to obtain the block-level usage flag and then determine whether the current block needs to be filtered. When it is determined that filtering is required, the decoder can perform filtering based on the adjusted block-level input parameters.

[0142] Understandably, the decoder can adjust the flag bit based on the frame-level input parameters to determine whether the input parameters of the input neural network filtering model need to be adjusted, thus enabling flexible selection and diverse handling of input parameters and improving decoding efficiency.

[0143] In some embodiments of this application, a filtering method provided in this application may further include:

[0144] S101. Parse the bitstream and obtain the frame-level usage flag based on the neural network filtering model;

[0145] S102. When the frame-level usage flag indicates that the frame is being used, obtain the frame-level switch flag and the frame-level quantization parameter adjustment flag. The frame-level switch flag is used to determine whether each block in the current frame is filtered.

[0146] It should be noted that S101 and S102 have already been described above, and will not be repeated here.

[0147] The current block can be a coding tree unit, which is not limited in this embodiment.

[0148] S105. When the frame-level switch flag indicates that it is not enabled, obtain the block-level usage flag.

[0149] S106. When the block-level usage flag indicates that any color component of the current block is used, and the frame-level quantization parameter adjustment flag indicates that it is used, obtain the adjusted frame-level quantization parameters.

[0150] In this embodiment of the application, when the frame-level switch flag indicates that it is not enabled, it is necessary to obtain the block-level usage flag from the bitstream.

[0151] It should be noted that the block-level usage flags of the current block include the block-level usage flags corresponding to each color component.

[0152] In this embodiment of the application, when the block-level usage flag indicates that any color component of the current block is used, and the frame-level quantization parameter adjustment flag indicates that it is used, the adjusted frame-level quantization parameter is obtained; wherein, the process of obtaining the adjusted frame-level quantization parameter can be consistent with the aforementioned implementation means, and will not be described again here.

[0153] It should be noted that for the current block, if any color component is identified by a block-level usage flag, then decoding requires filtering the current block to obtain the residual information corresponding to each color component. Therefore, for the current block, if any color component is identified by a block-level usage flag, the adjusted frame-level quantization parameters need to be obtained for use during filtering.

[0154] S107. Based on the adjusted frame-level quantization parameters and neural network filtering model, filter the current block of the current frame to obtain the first residual information of the current block.

[0155] In this embodiment, the decoder filters the current block of the current frame based on the adjusted frame-level quantization parameters and the neural network filtering model to obtain the first residual information of the current block.

[0156] The first residual information includes the residual information corresponding to each color component. The decoder determines the reconstructed value of that color component in the current block based on the block-level usage flag corresponding to each color component. If the block-level usage flag corresponding to a color component is "used," then the target reconstructed value corresponding to that color component is the sum of the reconstructed value of that color component in the current block and the residual information of the filtered output under that color component.

[0157] For example, if not all color components of the current coding tree unit have their coding tree unit-level usage flags set to "use," a neural network-based loop filtering technique is applied to the current coding tree unit. The reconstructed sample YUV, predicted sample YUV, partition information YUV, and quantization parameters of the current coding tree unit are used as inputs to obtain the residual information of the current coding tree unit. The quantization parameters are adjusted based on the frame-level quantization parameter adjustment flag and the frame-level quantization parameter adjustment index. This residual information is scaled, with the scaling factor already obtained from the previously analyzed bitstream. The scaled residual is then superimposed onto the reconstructed sample to obtain the reconstructed sample YUV after neural network loop filtering. Based on the coding tree unit usage flags of each color component of the current coding tree unit, a selected reconstructed sample is used as the output of the neural network-based loop filtering technique. If the coding tree unit usage flag of the corresponding color component is set to "use," the reconstructed sample of that color component after neural network loop filtering is used as the output; otherwise, the reconstructed sample without neural network loop filtering is used as the output for that color component. After traversing all the coding tree units of the current frame, the neural network-based loop filtering module ends.

[0158] Understandably, the decoder can adjust the flag bit based on the frame-level quantization parameters to determine whether the quantization parameters of the input neural network filtering model need to be adjusted, thus enabling flexible selection and diverse handling of quantization parameters and improving decoding efficiency.

[0159] In some embodiments of this application, after the decoder obtains the block-level usage flag, it obtains the block-level quantization parameter adjustment flag.

[0160] When the block-level usage flag indicates that any color component of the current block is used, and the block-level quantization parameter adjustment flag indicates that it is used, the adjusted block-level quantization parameters are obtained; based on the adjusted block-level quantization parameters and the neural network filtering model, the current block of the current frame is filtered to obtain the second residual information of the current block.

[0161] In some embodiments of this application, the decoder determines the block-level quantization bias parameter based on the block-level quantization parameter index obtained from the bitstream; and determines the adjusted block-level quantization parameter based on the obtained block-level quantization parameter and the block-level quantization bias parameter.

[0162] It should be noted that the decoder obtains the adjusted block-level quantization parameters by using the block-level quantization parameter index corresponding to the block-level quantization bias parameter parsed from the bitstream. Based on the quantization parameters, the block-level quantization bias parameters corresponding to different blocks are further superimposed to obtain the block-level quantization parameters corresponding to the current block. Then, based on the adjusted block-level quantization parameters and the neural network filtering model, the current block of the current frame is filtered to obtain the second residual information of the current block.

[0163] In the embodiments of this application, the adjustments between different coding tree units can be different, that is, the quantization parameter inputs of different coding tree units can be different.

[0164] In some embodiments of this application, after the decoder obtains the block-level usage flag, when the block-level usage flag indicates that any color component of the current block is used, the decoder obtains the block-level quantization parameters corresponding to the current block; based on the adjusted block-level quantization parameters and the neural network filtering model, the decoder filters the current block of the current frame to obtain the second residual information of the current block.

[0165] It should be noted that each flag bit in this application can be set to 1 for a used or allowed state and 0 for a not used or not allowed state, and this application embodiment does not impose any restrictions.

[0166] It should be noted that the block-level quantization parameters corresponding to the current block can be parsed from the bitstream.

[0167] In some embodiments of this application, the decoder filters the current block of the current frame based on adjusted block-level quantization parameters and a neural network filtering model to obtain the second residual information of the current block, and then obtains the second residual scaling factor in the bitstream; based on the second residual scaling factor, it scales the second residual information of the current block to obtain the second target residual information; when the block-level usage flag indicates that it is used, the second target reconstruction value of the current block is determined based on the second target residual information and the reconstruction value of the current block. When the block-level usage flag indicates that it is not used, the reconstruction value of the current block is determined as the second target reconstruction value.

[0168] It should be noted that the decoding end continues to traverse other loop filtering methods, and outputs the complete reconstructed image after completion.

[0169] Understandably, the decoder can adjust the flag bit based on the frame-level quantization parameters to determine whether the block-level quantization parameters of the input neural network filtering model need to be adjusted. This enables flexible selection and diverse handling of block-level quantization parameters, and the adjustment range of each block can be different, thereby improving decoding efficiency.

[0170] This application provides a filtering method applied to an encoder, such as... Figure 8 As shown, the method may include:

[0171] S201. Obtain the sequence-level allowed flag;

[0172] S202. When the sequence-level enable flag indicates that the sequence is enabled, obtain the original value of the current block, the reconstructed value of the current block, and the frame-level quantization parameters in the current frame.

[0173] S203. Based on the neural network filtering model, the reconstructed value of the current block, and the frame-level quantization parameters, perform filtering estimation on the current block to determine the first reconstructed value;

[0174] In this embodiment, the encoder traverses intra-frame or inter-frame predictions to obtain the prediction blocks of each coding unit. The residual of the coding unit is obtained by subtracting the original image block from the prediction block. The residual is used to obtain frequency domain residual coefficients through various transformation modes, and then quantized and dequantized. After inverse transformation, the distortion residual information is obtained. The distortion residual information is superimposed with the prediction block to obtain the reconstructed block. After the image is encoded, the loop filtering module filters the image at the coding tree unit level as the basic unit. In this embodiment, the coding tree unit is described as a block, but the block is not limited to CTU; it can also be CU. This embodiment does not impose any restrictions. The encoder obtains the sequence-level enable flag, i.e., sps_nnlf_enable_flag, based on the neural network filtering model. If the sequence-level enable flag is enabled, the neural network-based loop filtering technology is allowed; if the sequence-level enable flag is disallowed, the neural network-based loop filtering technology is disallowed. The sequence-level enable flag needs to be written into the bitstream when encoding the video sequence.

[0175] In this embodiment, when the sequence-level allowable flag indicates that it is allowed, the encoder attempts a loop filtering technique based on a neural network filtering model. The encoder obtains the original value of the current block, the reconstructed value of the current block, and the frame-level quantization parameters in the current frame. If the sequence-level allowable flag based on the neural network filtering model is not allowed, the encoder does not attempt a loop filtering technique based on the neural network. It continues to try other loop filtering tools, such as LF filtering, and outputs a complete reconstructed image after completion.

[0176] In this embodiment of the application, for the current frame, the current block is filtered and estimated based on the neural network filtering model, the reconstructed value of the current block and the frame-level quantization parameters to determine the first estimated residual information; the first residual scaling factor is determined; the first estimated residual value is scaled using the first residual scaling factor to obtain the first scaled residual information; the first scaled residual information is combined with the reconstructed value of the current block to determine the first reconstructed value.

[0177] In this embodiment of the application, before the encoder determines the first residual scaling factor, for the current frame, at least one of the predicted value of the current block, block partitioning information, and deblocking filter boundary strength, as well as the reconstructed value of the current block, are obtained; using a neural network filtering model, at least one of the predicted value of the current block, block partitioning information, and deblocking filter boundary strength, the reconstructed value of the current block, and the frame-level quantization parameters are filtered and estimated to obtain the first estimated residual information of the current block.

[0178] It should be noted that the input parameters to the neural network filtering model can be determined according to the actual situation, and this application embodiment does not impose any restrictions.

[0179] S204. Estimate the rate-distortion cost based on the first reconstructed value and the original value of the current block to obtain the rate-distortion cost of the current block, and traverse the current frame to determine the first rate-distortion cost of the current frame.

[0180] In this embodiment of the application, after the encoder obtains the first reconstructed value of the current block, it performs rate-distortion cost estimation based on the first reconstructed value and the original value of the current block to obtain the rate-distortion cost of the current block, and continues to perform encoding processing for the next block until the rate-distortion costs of all blocks of the current frame are obtained. Then, the rate-distortion costs of all blocks are added together to obtain the first rate-distortion cost of the current frame.

[0181] For example, the encoder attempts to use a neural network-based loop filtering technique, inputting the reconstructed sample YUV, predicted sample YUV, YUV with partitioning information, and quantization parameters (BaseQP and SliceQP) of the current coding tree unit into the neural network filtering model for inference. The neural network filtering model outputs the estimated residual information after filtering the current coding tree unit, and scales this estimated residual information. The scaling factor in the scaling operation is calculated based on the original image sample of the current frame, the reconstructed sample without neural network loop filtering, and the reconstructed sample after neural network loop filtering. Different scaling factors for different color components are used, and when necessary, they are all written into the bitstream and transmitted to the decoder. The encoder superimposes the scaled residual information onto the reconstructed sample without neural network loop filtering for output. The encoder calculates the rate-distortion cost based on the coding tree unit sample after neural network loop filtering and the original image sample of that coding tree unit, denoted as the first rate-distortion cost of the current frame, costNN.

[0182] S205. Based on a neural network filtering model, at least one frame-level quantization bias parameter, frame-level quantization parameter, and the reconstructed value of the current block in the current frame, perform at least one filtering estimation on the current frame to determine at least one second rate distortion cost of the current frame.

[0183] The encoder attempts to make at least one filtering estimate by changing the input parameters to the neural network filtering model at least once, and obtains at least one second rate distortion cost (costOffset) for the current frame.

[0184] It should be noted that the input parameters can be at least one of the predicted value of the current block, block partitioning information, and deblocking filter boundary strength, the reconstructed value of the current block, and frame-level quantization parameters, and may also include other information; this application embodiment does not impose any limitations. The encoder can adjust any one of the frame-level quantization parameters, the predicted value of the current block, the block partitioning information, and the deblocking filter boundary strength to perform filter estimation; this application embodiment does not impose any limitations.

[0185] In some embodiments of this application, when the sequence-level allowable flag bit indicates that permission is allowed, at least one of the following is obtained: the predicted value of the current block, the block partitioning information, and the deblocking filter boundary strength, as well as the reconstructed value of the current block and the frame-level quantization parameters.

[0186] The sixth reconstructed value is determined by filtering the current block based on at least one of the predicted value of the current block, block partitioning information, and deblocking filter boundary strength, a neural network filtering model, the reconstructed value of the current block, and frame-level quantization parameters.

[0187] Rate distortion cost is estimated based on the sixth reconstructed value and the original value of the current block to obtain the rate distortion cost of the current block. The seventh rate distortion cost of the current frame is determined by traversing the current frame.

[0188] Based on at least one of the predicted value of the current block, block partitioning information, and deblocking filter boundary strength, a neural network filtering model, at least one frame-level input bias parameter, and the reconstructed value of the current block in the current frame, at least one filtering estimation is performed on the current frame to determine at least one eighth rate distortion cost of the current frame.

[0189] The frame-level input parameter adjustment flag is determined based on the first rate distortion cost and at least one eighth rate distortion cost.

[0190] When the input parameter is a frame-level quantization parameter, the frame-level input parameter adjustment flag can be understood as the frame-level quantization parameter adjustment flag.

[0191] Understandably, the encoder can adjust the flag bit based on the frame-level input parameters to determine whether the input parameters of the input neural network filtering model need to be adjusted, thus enabling flexible selection and diverse handling of input parameters and improving coding efficiency.

[0192] An example implementation of adjusting frame-level quantization parameters is as follows:

[0193] The encoder obtains the i-th frame-level quantization bias parameter, and adjusts the frame-level quantization parameter based on the i-th frame-level quantization bias parameter to obtain the i-th adjusted frame-level quantization parameter; i is a positive integer greater than or equal to 1; the current block is filtered and estimated based on the neural network filtering model, the reconstructed value of the current block, and the i-th adjusted frame-level quantization parameter to obtain the i-th second reconstructed value; rate-distortion cost estimation is performed based on the i-th second reconstructed value and the original value of the current block. After traversing all blocks of the current frame, the i-th second rate-distortion cost is obtained. The i+1-th filtering estimation is then performed based on the (i+1)-th frame-level quantization bias parameter until at least one estimation is completed, thereby determining at least one second rate-distortion cost of the current frame.

[0194] In this embodiment, the encoder estimates the rate-distortion cost based on the i-th second reconstruction value and the original value of the current block. After traversing all blocks of the current frame, the rate-distortion costs of all blocks are added together to obtain the i-th second rate-distortion cost. Then, based on the (i+1)-th frame-level quantization bias parameter, the encoder performs the (i+1)-th filtering estimation until the filtering estimation of all blocks is completed, thus obtaining the first rate-distortion cost of the current frame. This process continues until at least one round of filtering is completed, thus obtaining at least one second rate-distortion cost of the current frame.

[0195] In this embodiment, the encoder performs filtering estimation on the current block based on the neural network filtering model, the reconstructed value of the current block, and the frame-level quantization parameters adjusted in the i-th time to obtain the second reconstructed value in the i-th time. This includes: performing filtering estimation on the current block once each based on the neural network filtering model, the reconstructed value of the current block, and the frame-level quantization parameters adjusted in the i-th time to obtain the i-th second estimation residual information; determining the i-th second residual scaling factor corresponding to the frame-level quantization parameters adjusted in the i-th time; scaling the i-th second estimation residual information using the i-th second residual scaling factor to obtain the i-th second scaling residual information; and combining the i-th second scaling residual information with the reconstructed value of the current block to determine the second reconstructed value in the i-th time.

[0196] In some embodiments of this application, the encoder may also obtain at least one of the predicted value of the current block, block partitioning information, and deblocking filter boundary strength, as well as the reconstructed value of the current block; using a neural network filtering model, frame-level filtering estimation is performed on at least one of the predicted value of the current block, block partitioning information, and deblocking filter boundary strength, the reconstructed value of the current block, and the frame-level quantization parameters adjusted for the i-th time, to obtain the i-th second estimated residual information of the current block.

[0197] In some embodiments of this application, the encoder can select whether to adjust the quantization parameters of the current frame based on the different types of encoded frames. For the first type, the quantization parameters need to be adjusted, while for the second type (frames of types other than the first type), no adjustment is required. Therefore, during encoding, the encoder can adjust the frame-level quantization parameters to perform filter estimation when the current frame is a first-type frame.

[0198] In some embodiments of this application, when the current frame is a first type frame, at least one filtering estimation is performed on the current frame based on a neural network filtering model, at least one frame-level quantization bias parameter, frame-level quantization parameter, and the reconstructed value of the current block in the current frame to determine at least one second rate distortion cost of the current frame.

[0199] It should be noted that, in the embodiments of this application, the first type of frame can be a B frame or a P frame, and the embodiments of this application do not impose any restrictions.

[0200] For example, the encoder can adjust the BaseQP and SliceQP as inputs, giving the encoder more options to try, thereby improving encoding efficiency.

[0201] The aforementioned adjustments to BaseQP and SliceQP include both uniform adjustments to all coding tree units within a frame and individual adjustments to each coding tree unit. For uniform adjustments to all coding tree units within a frame, adjustments can be made regardless of whether the current frame is an I-frame or a B-frame, and the adjustment magnitude for all coding tree units in the current frame is the same, meaning the quantization parameter inputs for all coding tree units are identical. For individual adjustments to each coding tree unit, adjustments can also be made regardless of whether the current frame is an I-frame or a B-frame, but the adjustment magnitude for all coding tree units in the current frame can be selected based on rate-distortion optimization at the encoding end. The adjustments between different coding tree units can differ, meaning the quantization parameter inputs for different coding tree units can be different.

[0202] Understandably, the encoder can adjust the flag bit based on the block-level quantization parameters to determine whether the block-level quantization parameters of the input neural network filtering model need to be adjusted, thus enabling flexible selection and diverse handling of block-level quantization parameters and improving coding efficiency.

[0203] S206. Based on the first rate distortion cost and at least one second rate distortion cost, determine the frame-level quantization parameter adjustment flag bit.

[0204] In this embodiment, the encoder can determine the frame-level quantization parameter adjustment flag bit based on the first rate-distortion cost and at least one second rate-distortion cost, that is, determine whether the frame-level quantization parameters need to be adjusted during filtering.

[0205] For example, the adjustments to BaseQP and SliceQP described above can be controlled by frame-level flags, where there is at least one frame-level flag. For instance, different frame-level quantization parameter adjustment flags can be set for different color components; one frame-level quantization parameter adjustment flag can be set for the luma component, and another for the chroma component. Further extensions of the frame-level quantization parameter adjustment flags can include using one or more flags to indicate whether all coding tree units in the current frame need quantization parameter adjustments, or whether all coding tree units have the same quantization parameter adjustments. This application does not impose such limitations.

[0206] In some embodiments of this application, the implementation of the encoder determining the frame-level quantization parameter adjustment flag bit based on the first rate-distortion cost and at least one second rate-distortion cost includes: determining a first minimum rate-distortion cost (bestCostNN) from the first rate-distortion cost and at least one second rate-distortion cost; if the first minimum rate-distortion cost is the first rate-distortion cost, then determining that the frame-level quantization parameter adjustment flag bit is unused; if the first minimum rate-distortion cost is any one of at least one second rate-distortion cost, then determining that the frame-level quantization parameter adjustment flag bit is used.

[0207] In some embodiments of this application, after the encoder determines the frame-level quantization parameter adjustment flag bit based on the first rate-distortion cost and at least one second rate-distortion cost, if the first minimum rate-distortion cost is any one of the at least one second rate-distortion cost, then the frame-level quantization offset parameter corresponding to the first minimum rate-distortion cost is written into the bitstream from at least one frame-level quantization offset parameter, or the block-level quantization parameter index (offset number) of the frame-level quantization offset parameter corresponding to the first minimum rate-distortion cost is written into the bitstream.

[0208] In some embodiments of this application, if the first minimum rate distortion cost is any one of at least one second rate distortion cost, then a second residual scaling factor corresponding to the first minimum rate distortion cost is written into the bitstream. If the first minimum rate distortion cost is the first rate distortion cost, then the first residual scaling factor is written into the bitstream.

[0209] It should be noted that the "write" here refers to "to be written". The write operation will only be performed after the first minimum rate distortion cost is compared with costOrg and costCTU.

[0210] For example, the encoder continues to try a loop filtering technique based on a neural network, following the same process as the second round, but with adjustments made to the input. This round can be repeated multiple times. If the first attempt involves adjusting the BaseQP quantization parameter, the BaseQP is superimposed with the adjusted bias parameter offset1, resulting in BaseQPFinal, which replaces BaseQP as the input, while other parameters remain unchanged. The rate-distortion cost under offset1 is calculated and denoted as costOffset1. The second bias parameter offset2 is then tried, following the same process, and the rate-distortion cost is calculated and denoted as costOffset2. In this example, the BaseQP bias is tried twice in this round, without adjusting the SliceQP. After obtaining costNN, costOffset1, and costOffset2, the encoder compares them. If costNN is the smallest, the frame-level quantization parameter is adjusted to the unused flag and is ready to be written into the bitstream. If costOffset1 is the smallest, the frame-level quantization parameter is adjusted to the used flag and the frame-level quantization parameter is adjusted to the index representing the current offset1 and is ready to be written into the bitstream. At the same time, the residual scaling factor of the bitstream to be written is replaced with the residual scaling factor under the current offset1.

[0211] Understandably, the encoder can adjust the flag bit based on the frame-level quantization parameters to determine whether the quantization parameters of the input neural network filtering model need to be adjusted, thus enabling flexible selection and diverse handling of quantization parameters and improving coding efficiency.

[0212] In some embodiments of this application, the filtering method provided by the encoder may further include:

[0213] S207. When the sequence-level allowable flag bit indicates that it is allowed, rate-distortion cost is estimated based on the original value and the reconstructed value of the current block in the current frame to obtain the third rate-distortion cost.

[0214] When the sequence-level allowable flag bit indicates that the encoder does not perform filtering, the rate-distortion cost is estimated based on the original value and the reconstructed value of the current block in the current frame, resulting in the third rate-distortion cost (costOrg).

[0215] In some embodiments of this application, after the encoder determines the frame-level quantization parameter adjustment flag bit based on a first rate-distortion cost and at least one second rate-distortion cost, the method further includes:

[0216] S208. Based on the neural network filtering model, the reconstructed value of the current block, and the frame-level quantization parameters, perform filtering estimation on the current block to determine the third reconstructed value;

[0217] It should be noted that the implementation principle of S208 is the same as that of S203, and will not be repeated here.

[0218] S209. Based on the third reconstructed value and the original value of the current block, the rate distortion cost is estimated to obtain the fourth rate distortion cost (costCTUorg) of the current block.

[0219] It should be noted that the implementation principle of S209 is the same as that of S204, and will not be repeated here.

[0220] S210. Based on the neural network filtering model, the target reconstruction value corresponding to the first minimum rate distortion cost, and the frame-level quantization parameters, the current block is filtered and estimated to obtain the fourth reconstruction value.

[0221] It should be noted that the implementation principle of S210 is the same as that of S203. The difference is that the input here is the target reconstruction value corresponding to the first minimum rate distortion cost, instead of the reconstruction value of the current block.

[0222] S211. Based on the fourth reconstructed value and the original value of the current block, the rate distortion cost is estimated to obtain the fifth rate distortion cost (costCTUnn) of the current block.

[0223] It should be noted that the implementation principle of S211 is the same as that of S204, and will not be repeated here.

[0224] S212. Determine the block-level usage flag based on the fourth rate distortion cost and the fifth rate distortion cost;

[0225] In this embodiment of the application, if the fourth rate distortion cost is less than the fifth rate distortion cost, the block-level usage flag is determined to be unused; if the fourth rate distortion cost is greater than or equal to the fifth rate distortion cost, the block-level usage flag is determined to be used.

[0226] It should be noted that the block level uses a flag bit to indicate whether the current block or coding tree unit needs filtering.

[0227] For example, the value of the block-level usage flag bit can be 1 to indicate usage and 0 to indicate unused. This application embodiment does not limit the expression form and meaning of the value of the block-level usage flag bit.

[0228] S213. Traverse the blocks in the current frame and determine the sum of the minimum rate distortion costs of all blocks in the current frame as the sixth rate distortion cost (costCTU) of the current frame.

[0229] In this embodiment of the application, the encoder adds up the minimum rate-distortion cost of each color component corresponding to the block in the current frame to obtain the frame-level rate-distortion cost of each color component, and then adds up the rate-distortion costs of each color component to obtain the sixth rate-distortion cost of the current frame.

[0230] For example, the encoder attempts to optimize the selection at the coding tree unit level, trying combinations of switches at the coding tree unit level, and each component can be controlled independently. The encoder traverses the current coding tree unit, calculates the rate-distortion cost between the reconstructed sample and the original sample of the current coding tree unit without using neural network loop filtering, denoted as costCTUorg; calculates the rate-distortion cost between the reconstructed sample and the original sample of the current coding tree unit with neural network loop filtering, denoted as costCTUnn. If costCTUorg is less than costCTUnn, the block of the coding tree unit with neural network loop filtering is marked as usable and is ready to be written into the bitstream; otherwise, the block of the coding tree unit with neural network loop filtering is marked as usable and is ready to be written into the bitstream; if all coding tree units in the current frame have been traversed, the rate-distortion cost between the reconstructed sample and the original image sample of the current frame is calculated in this case, denoted as costCTU.

[0231] In some embodiments of this application, after the encoder estimates the rate-distortion cost based on the third reconstructed value and the original value of the current block to obtain the fourth rate-distortion cost of the current block, and before determining the block-level usage flag bit based on the fourth rate-distortion cost and the fifth rate-distortion cost, at least one filtering estimation is performed on the current block based on the neural network filtering model, the reconstructed value of the current block, at least one frame-level quantization bias parameter and frame-level quantization parameter to determine at least one fifth reconstructed value; (similar to the principle of the third round) based on at least one fifth reconstructed value and the original value of the current block, the fifth rate-distortion cost with the minimum rate-distortion cost is determined.

[0232] It should be noted that the process of performing at least one filtering estimation on the current block based on the neural network filtering model, the reconstructed value of the current block, at least one frame-level quantization bias parameter, and the frame-level quantization parameter to determine at least one fifth reconstructed value is the same as the principle in S205, and will not be repeated here.

[0233] In some embodiments of this application, when the encoder obtains the third rate distortion cost (costOrg), the first minimum rate distortion cost (bestCostNN), and the sixth rate distortion cost (costCTU), if the minimum rate distortion cost among the third rate distortion cost, the first minimum rate distortion cost, and the sixth rate distortion cost is the third rate distortion cost, then the frame-level usage flag is determined to be unused; and the frame-level usage flag is written into the bitstream.

[0234] If the minimum rate distortion cost among the third rate distortion cost, the first minimum rate distortion cost, and the sixth rate distortion cost is the first minimum rate distortion cost, then the frame-level usage flag is set to "use" and the frame-level switch flag is set to "on"; and the frame-level usage flag and the frame-level switch flag are written to the bitstream. Additionally, the frame-level quantization offset parameter corresponding to the first minimum rate distortion cost is written to the bitstream, or the block-level quantization parameter index (offset number) of the frame-level quantization offset parameter corresponding to the first minimum rate distortion cost is written to the bitstream.

[0235] If the minimum rate distortion cost among the third rate distortion cost, the first minimum rate distortion cost, and the sixth rate distortion cost is the sixth rate distortion cost, then the frame-level usage flag is determined to be in use and the frame-level switch flag is determined to be off; and the frame-level usage flag, the frame-level switch flag, and the block-level usage flag are written into the bitstream.

[0236] For example, iterate through each color component. If the value of costOrg is the smallest, then the frame-level usage flag corresponding to that color component based on neural network loop filtering is set to "use" and written into the bitstream, without performing neural network loop filtering. If the value of bestCostNN is the smallest, then the frame-level usage flag corresponding to that color component based on neural network loop filtering is set to "use", the frame-level switch flag is set to "use", and the frame-level quantization parameter adjustment flag, index information, and residual scaling factor decided in the third round are written into the bitstream. If the value of costCTU is the smallest, then the frame-level usage flag corresponding to that color component based on neural network loop filtering is set to "use", the frame-level switch flag is set to "unuse", and the frame-level quantization parameter adjustment flag, the frame-level quantization parameter adjustment index, and residual scaling factor decided in the third round are written into the bitstream. In addition, the block usage flag of each coding tree unit level also needs to be written into the bitstream.

[0237] Understandably, the encoder can adjust the flag bit based on frame-level quantization parameters to determine whether the quantization parameters of the input neural network filtering model need adjustment. This enables flexible selection and handling of diverse variations of quantization parameters, thereby improving coding efficiency.

[0238] For example, the loop filtering section of the encoder / decoder end is integrated into the reference software of JVET EE1, which is based on VTM10.0 and has the same basic performance as VVC. The test results of the integrated simplification under the test conditions RA (Table 1) and LDB (Table 2) are shown in the table.

[0239] Table 1

[0240]

[0241]

[0242] Table 2

[0243]

[0244] As can be seen from Tables 1 and 2 above, the filtering method provided in this application provides stable performance improvement under both RA and LDB test conditions. From class A1 to class E, RA shows an average performance gain of over 0.2% BD-rate, while LDB performs even better in some classes, with a maximum BD-rate performance gain of 0.57%, mainly depending on Y. The filtering method provided in this application does not introduce additional complexity to the decoding end; there is no additional complexity increase. At the decoding end, only the quantization parameter needs to be adjusted once when decoding the current frame, without increasing complexity, while still providing stable gains.

[0245] This application provides a decoder 1, such as... Figure 9 As shown, the decoder 1 may include:

[0246] The parsing section 10 is configured to parse the bitstream and obtain the frame-level usage flag bits based on the neural network filtering model;

[0247] The first determining part 11 is configured to acquire a frame-level switch flag and a frame-level quantization parameter adjustment flag when the frame-level use flag indicates that the frame is being used; the frame-level switch flag is used to determine whether each block in the current frame is being filtered.

[0248] The first adjustment part 12 is configured to obtain the adjusted frame-level quantization parameters when the frame-level switch flag indicates that it is enabled and the frame-level quantization parameter adjustment flag indicates that it is used.

[0249] The first filtering section 13 is configured to filter the current block of the current frame based on the adjusted frame-level quantization parameters and the neural network filtering model to obtain the first residual information of the current block.

[0250] In some embodiments of this application, the parsing portion 10 is further configured to obtain the block-level usage flag bit when the frame-level switch flag bit indicates that it is not enabled;

[0251] The first determining portion 11 is further configured to obtain the adjusted frame-level quantization parameters when the block-level usage flag indicates that any color component of the current block is used and the frame-level quantization parameter adjustment flag indicates that it is used.

[0252] The first filtering part 13 is further configured to filter the current block of the current frame based on the adjusted frame-level quantization parameters and the neural network filtering model to obtain the first residual information of the current block.

[0253] In some embodiments of this application, the parsing portion 10 is further configured to obtain the block-level quantization parameter adjustment flag after obtaining the block-level usage flag.

[0254] The first determining portion 11 is further configured to obtain the adjusted block-level quantization parameters when the block-level usage flag indicates that any color component of the current block is used, and the block-level quantization parameter adjustment flag indicates that it is used.

[0255] The first filtering part 13 is further configured to filter the current block of the current frame based on the adjusted block-level quantization parameters and the neural network filtering model to obtain the second residual information of the current block.

[0256] In some embodiments of this application, the first determining part 11 is further configured to, after obtaining the block-level usage flag bit, obtain the block-level quantization parameters corresponding to the current block when the block-level usage flag bit indicates that any color component of the current block is used;

[0257] The first filtering part 13 is further configured to filter the current block of the current frame based on the adjusted block-level quantization parameters and the neural network filtering model to obtain the second residual information of the current block.

[0258] In some embodiments of this application, the parsing portion 10 is further configured to, after parsing the bitstream and obtaining the frame-level usage flag based on the neural network filtering model, and before obtaining the adjusted frame-level quantization parameters, obtain the frame-level switch flag and the frame-level quantization parameter adjustment flag when the frame-level usage flag indicates usage and the current frame is a first type frame.

[0259] In some embodiments of this application, the first determining portion 11 is further configured to determine the frame-level quantization offset parameter based on the frame-level quantization parameter adjustment index obtained from the bitstream; and to determine the adjusted frame-level quantization parameter according to the obtained frame-level quantization parameter and the frame-level quantization offset parameter.

[0260] In some embodiments of this application, the parsing portion 10 is further configured to obtain the adjusted frame-level quantization parameters from the bitstream.

[0261] In some embodiments of this application, the first determining portion 11 is further configured to determine a block-level quantization bias parameter based on the block-level quantization parameter index obtained in the bitstream; and to determine the adjusted block-level quantization parameter according to the obtained block-level quantization parameter and the block-level quantization bias parameter.

[0262] In some embodiments of this application, the first determining portion 11 is further configured to obtain the reconstructed value of the current block before filtering the current block of the current frame based on the adjusted frame-level quantization parameters and the neural network filtering model to obtain the first residual information of the current block.

[0263] In some embodiments of this application, the first filtering portion 13 is further configured to use the neural network filtering model to filter the reconstructed value of the current block and the adjusted frame-level quantization parameters to obtain the first residual information of the current block, thereby completing the filtering of the current block.

[0264] In some embodiments of this application, the first determining portion 11 is further configured to obtain at least one of the predicted value of the current block, block partitioning information, and deblocking filter boundary strength, as well as the reconstructed value of the current block, before filtering the current block of the current frame based on the adjusted frame-level quantization parameters and the neural network filtering model to obtain the first residual information of the current block.

[0265] In some embodiments of this application, the first filtering portion 13 is further configured to use the neural network filtering model to filter at least one of the predicted value of the current block, the block partitioning information and the deblocking filtering boundary strength, the reconstructed value of the current block, and the adjusted frame-level quantization parameters to obtain the first residual information of the current block, thereby completing the filtering of the current block.

[0266] In some embodiments of this application, the first determining portion 11 is further configured to, after filtering the current block of the current frame based on the adjusted frame-level quantization parameters and the neural network filtering model to obtain the first residual information of the current block, or, after filtering the current block of the current frame based on the adjusted block-level quantization parameters and the neural network filtering model to obtain the second residual information of the current block, ...

[0267] Obtain the second residual scaling factor in the bitstream; based on the second residual scaling factor, scale the first residual information or the second residual information of the current block to obtain the first target residual information or the second target residual information; based on the first target residual information and the reconstruction value of the current block, determine the first target reconstruction value of the current block; or, when the block-level usage flag indicates usage, determine the second target reconstruction value of the current block based on the second target residual information and the reconstruction value of the current block.

[0268] In some embodiments of this application, the first determining portion 11 is further configured to determine the reconstruction value of the current block as the second target reconstruction value when the block-level usage flag indicates that it is not used.

[0269] In some embodiments of this application, the first determining part 11 is further configured to, after obtaining the frame-level usage flag bit based on the neural network filtering model, obtain at least one of the predicted value of the current block, block partitioning information and deblocking filter boundary strength, as well as the reconstructed value of the current block.

[0270] The parsing section 10 is further configured to acquire a frame-level switch flag and a frame-level input parameter adjustment flag when the frame-level usage flag indicates that the frame is being used; the frame-level input parameter adjustment flag indicates whether any one of the parameters, such as the prediction value, block partitioning information, and deblocking filter boundary strength, has been adjusted.

[0271] The first determining part 11 is further configured to obtain the adjusted block-level input parameters when the frame-level switch flag indicates that it is enabled and the frame-level input parameter adjustment flag indicates that it is used.

[0272] The first filtering section 13 is further configured to filter the current block of the current frame based on the adjusted block-level input parameters, the acquired frame-level quantization parameters, and the neural network filtering model, to obtain the third residual information of the current block.

[0273] In some embodiments of this application, the parsing portion 10 is further configured to parse out a sequence-level allowable flag; when the sequence-level allowable flag indicates that it is allowed, the frame-level use flag based on the neural network filtering model is parsed.

[0274] This application also provides a decoder 1, such as... Figure 10 As shown, the decoder 1 may include:

[0275] The first memory 14 is configured to store computer programs that can run on the first processor 15;

[0276] The first processor 15 is configured to execute the decoder method while running the computer program.

[0277] Understandably, the decoder can adjust the flag bit based on the frame-level quantization parameters to determine whether the quantization parameters of the input neural network filtering model need to be adjusted, thus enabling flexible selection and diverse handling of quantization parameters (input parameters) and improving decoding efficiency.

[0278] The first processor 15 can be implemented by software, hardware, firmware or a combination thereof, and can use circuits, one or more application-specific integrated circuits (ASICs), one or more general-purpose integrated circuits, one or more microprocessors, one or more programmable logic devices, or a combination of the aforementioned circuits or devices, or other suitable circuits or devices, so that the first processor 15 can execute the corresponding steps of the filtering method on the decoder side in the foregoing embodiments.

[0279] This application provides an encoder 2, such as... Figure 11 As shown, the encoder 2 may include:

[0280] The second determining part 20 is configured to obtain a sequence-level allowed flag bit; and when the sequence-level allowed flag bit indicates that it is allowed, obtain the original value of the current block, the reconstructed value of the current block and the frame-level quantization parameters in the current frame;

[0281] The second filtering part 21 is configured to perform filtering estimation on the current block based on the neural network filtering model, the reconstructed value of the current block and the frame-level quantization parameters to determine the first reconstructed value;

[0282] The second determining part 20 is further configured to perform rate-distortion cost estimation based on the first reconstructed value and the original value of the current block to obtain the rate-distortion cost of the current block, and to traverse the current frame to determine the first rate-distortion cost of the current frame;

[0283] The second filtering part 21 is further configured to perform at least one filtering estimation on the current frame based on the neural network filtering model, at least one frame-level quantization bias parameter, the frame-level quantization parameter, and the reconstructed value of the current block in the current frame, to determine at least one second rate distortion cost of the current frame.

[0284] The second determining portion 20 is further configured to determine a frame-level quantization parameter adjustment flag bit based on the first rate-distortion cost and the at least one second rate-distortion cost.

[0285] In some embodiments of this application, the second determining part 20 is further configured to obtain the i-th frame-level quantization bias parameter, and adjust the frame-level quantization parameter based on the i-th frame-level quantization bias parameter to obtain the i-th adjusted frame-level quantization parameter; i is a positive integer greater than or equal to 1;

[0286] Based on the neural network filtering model, the reconstructed value of the current block, and the frame-level quantization parameter adjusted for the i-th time, the current block is filtered and estimated to obtain the second reconstructed value for the i-th time.

[0287] Rate distortion cost estimation is performed based on the i-th second reconstruction value and the original value of the current block. After traversing all blocks of the current frame, the i-th second rate distortion cost is obtained. Then, based on the (i+1)-th frame-level quantization bias parameter, the (i+1)-th filtering estimation is performed until at least one is completed, thereby determining at least one second rate distortion cost of the current frame.

[0288] In some embodiments of this application, the second determining portion 20 is further configured to determine a first minimum rate distortion cost from the first rate distortion cost and the at least one second rate distortion cost;

[0289] If the first minimum rate distortion cost is equal to the first rate distortion cost, then the frame-level quantization parameter adjustment flag is determined to be unused.

[0290] If the first minimum rate distortion cost is any one of the at least one second rate distortion cost, then the frame-level quantization parameter adjustment flag is determined to be used.

[0291] In some embodiments of this application, the second determining portion 20 is further configured to perform rate-distortion cost estimation based on the original value and the reconstructed value of the current block in the current frame when the sequence-level allowable flag bit indicates allowance, to obtain a third rate-distortion cost.

[0292] In some embodiments of this application, the second filtering portion 21 is further configured to, after determining the frame-level quantization parameter adjustment flag bit based on the first rate-distortion cost and the at least one second rate-distortion cost, perform filtering estimation on the current block based on the neural network filtering model, the reconstructed value of the current block, and the frame-level quantization parameters to determine the third reconstructed value;

[0293] The second determining part 20 is further configured to perform rate-distortion cost estimation based on the third reconstructed value and the original value of the current block to obtain a fourth rate-distortion cost for the current block;

[0294] The second filtering section 21 is further configured to perform filtering estimation on the current block based on the neural network filtering model, the target reconstruction value corresponding to the first minimum rate distortion cost, and the frame-level quantization parameters to obtain a fourth reconstruction value;

[0295] The second determining part 20 is further configured to perform rate-distortion cost estimation based on the fourth reconstructed value and the original value of the current block to obtain the fifth rate-distortion cost of the current block; determine the block-level usage flag bit based on the fourth rate-distortion cost and the fifth rate-distortion cost; traverse the blocks in the current frame and determine the sum of the minimum rate-distortion costs of all blocks in the current frame as the sixth rate-distortion cost of the current frame.

[0296] In some embodiments of this application, the second determining portion 20 is further configured to determine that the block-level usage flag is unused if the fourth rate distortion cost is less than the fifth rate distortion cost;

[0297] If the fourth rate distortion cost is greater than or equal to the fifth rate distortion cost, then the block-level usage flag is determined to be in use.

[0298] In some embodiments of this application, the encoder 2 further includes: a writing portion 22; the second determining portion 20 is further configured to determine that the frame-level usage flag bit is unused if the minimum rate distortion cost among the third rate distortion cost, the first minimum rate distortion cost, and the sixth rate distortion cost is the third rate distortion cost;

[0299] The writing section 22 is configured to write the frame-level usage flag bit into the bitstream;

[0300] The second determining part 20 is further configured to determine that if the minimum rate distortion cost among the third rate distortion cost, the first minimum rate distortion cost, and the sixth rate distortion cost is the first minimum rate distortion cost, then the frame-level use flag is set to use and the frame-level switch flag is set to enable.

[0301] The writing section 22 is configured to write the frame-level usage flag bit and the frame-level switch flag bit into the bitstream;

[0302] The second determining part 20 is further configured to determine that if the minimum rate distortion cost among the third rate distortion cost, the first minimum rate distortion cost, and the sixth rate distortion cost is the sixth rate distortion cost, then the frame-level use flag is used and the frame-level switch flag is not enabled.

[0303] The writing section 22 is configured to write the frame-level usage flag, the frame-level switch flag, and the block-level usage flag into the bitstream.

[0304] In some embodiments of this application, the writing portion 22 is configured to, after determining the frame-level quantization parameter adjustment flag bit based on the first rate-distortion cost and the at least one second rate-distortion cost, if the first minimum rate-distortion cost is any one of the at least one second rate-distortion cost, then write the frame-level quantization bias parameter corresponding to the first minimum rate-distortion cost into the bitstream from at least one frame-level quantization bias parameter, or write the block-level quantization parameter index of the frame-level quantization bias parameter corresponding to the first minimum rate-distortion cost into the bitstream.

[0305] In some embodiments of this application, the second filtering portion 21 is further configured to perform filtering estimation on the current block based on the neural network filtering model, the reconstructed value of the current block, and the frame-level quantization parameters for the current frame, to determine first estimated residual information; determine a first residual scaling factor; scale the first estimated residual value using the first residual scaling factor to obtain first scaled residual information; and combine the first scaled residual information with the reconstructed value of the current block to determine the first reconstructed value.

[0306] In some embodiments of this application, the second determining portion 20 is further configured to, for the current frame, obtain at least one of the predicted value of the current block, block partitioning information, and deblocking filter boundary strength, as well as the reconstructed value of the current block, before determining the first residual scaling factor.

[0307] The second filtering part 21 is further configured to use the neural network filtering model to perform filtering estimation on at least one of the predicted value of the current block, the block partitioning information and the deblocking filtering boundary strength, the reconstructed value of the current block, and the frame-level quantization parameters to obtain the first estimated residual information of the current block.

[0308] In some embodiments of this application, the writing portion 22 is configured to write the first residual scaling factor into the bitstream if the first minimum rate distortion cost is equal to the first rate distortion cost after the first residual scaling factor is determined.

[0309] In some embodiments of this application, the second filtering section 21 is further configured to: perform filtering estimation on the current block based on the neural network filtering model, the reconstructed value of the current block, and the frame-level quantization parameter adjusted for the i-th time, to obtain the i-th second estimation residual information; determine the i-th second residual scaling factor corresponding to the frame-level quantization parameter adjusted for the i-th time; use the i-th second residual scaling factor to scale the i-th second estimation residual information to obtain the i-th second scaling residual information; and combine the i-th second scaling residual information with the reconstructed value of the current block to determine the i-th second reconstruction value.

[0310] In some embodiments of this application, the writing portion 22 is configured to, after determining the frame-level quantization parameter adjustment flag bit based on the first rate-distortion cost and the at least one second rate-distortion cost, write a second residual scaling factor corresponding to the first minimum rate-distortion cost into the bitstream if the first minimum rate-distortion cost is any one of the at least one second rate-distortion cost.

[0311] In some embodiments of this application, the second determining part 20 is further configured to obtain at least one of the predicted value of the current block, block partitioning information, and deblocking filter boundary strength, as well as the reconstructed value of the current block, before determining the i-th second residual scaling factor corresponding to the frame-level quantization parameter of the i-th adjustment.

[0312] The second filtering part 21 is further configured to use the neural network filtering model to perform frame-level filtering estimation on at least one of the predicted value of the current block, the block partitioning information and the deblocking filtering boundary strength, the reconstructed value of the current block, and the frame-level quantization parameter adjusted for the i-th time, to obtain the i-th second estimated residual information of the current block.

[0313] In some embodiments of this application, the second filtering portion 21 is further configured to, when the current frame is a first type frame, perform at least one filtering estimation on the current frame based on the neural network filtering model, at least one frame-level quantization bias parameter, the frame-level quantization parameter, and the reconstructed value of the current block in the current frame, to determine at least one second rate distortion cost of the current frame.

[0314] In some embodiments of this application, the second filtering section 21 is further configured to, after performing rate-distortion cost estimation based on the third reconstructed value and the original value of the current block to obtain the fourth rate-distortion cost of the current block, and before determining the block-level usage flag bit based on the fourth rate-distortion cost and the fifth rate-distortion cost, perform at least one filtering estimation on the current block based on the neural network filtering model, the reconstructed value of the current block, at least one frame-level quantization bias parameter and the frame-level quantization parameter to determine at least one fifth reconstructed value;

[0315] The second determining portion 20 is further configured to determine the fifth rate distortion cost that minimizes the rate distortion cost based on the at least one fifth reconstruction value and the original value of the current block.

[0316] In some embodiments of this application, the second determining portion 20 is further configured to obtain at least one of the following when the sequence-level allowable flag bit indicates allowance: the predicted value of the current block, block partitioning information, and deblocking filter boundary strength, the reconstructed value of the current block, and frame-level quantization parameters.

[0317] The second filtering part 21 is further configured to perform filtering estimation on the current block based on at least one of the predicted value of the current block, block partitioning information and deblocking filtering boundary strength, the neural network filtering model, the reconstructed value of the current block and the frame-level quantization parameters, to determine the sixth reconstructed value;

[0318] The second determining part 20 is further configured to perform rate-distortion cost estimation based on the sixth reconstructed value and the original value of the current block to obtain the rate-distortion cost of the current block, and to traverse the current frame to determine the seventh rate-distortion cost of the current frame;

[0319] The second filtering section 21 is further configured to perform at least one filtering estimation on the current frame based on at least one of the predicted value of the current block, the block partitioning information and the deblocking filtering boundary strength, the neural network filtering model, at least one frame-level input bias parameter, and the reconstructed value of the current block in the current frame, to determine at least one eighth rate distortion cost of the current frame.

[0320] The second determining portion 20 is further configured to determine a frame-level input parameter adjustment flag bit based on the first rate-distortion cost and the at least one eighth rate-distortion cost.

[0321] This application provides an encoder 2, such as... Figure 12 As shown, the encoder 2 may include:

[0322] The second memory 23 is configured to store computer programs that can run on the second processor 24;

[0323] The second processor 24 is configured to execute the encoder-based method when the computer program is running.

[0324] Understandably, the encoder can adjust the flag bit based on the frame-level quantization parameters to determine whether the quantization parameters of the input neural network filtering model need to be adjusted, thus enabling flexible selection and diverse handling of quantization parameters (input parameters) and improving decoding efficiency.

[0325] This application provides a computer-readable storage medium storing a computer program that, when executed by a first processor, implements the method described in the decoder, or when executed by a second processor, implements the method described in the encoder.

[0326] In the embodiments of this application, the components can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional module.

[0327] If the integrated unit is implemented as a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the method described in this embodiment. The aforementioned computer-readable storage media include various media capable of storing program code, such as magnetic random access memory (FRAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM), etc., and the embodiments disclosed herein are not limited thereto.

[0328] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0329] Industrial applicability

[0330] This application provides a filtering method, encoder, decoder, and storage medium. By parsing the bitstream, a frame-level usage flag based on a neural network filtering model is obtained. When the frame-level usage flag indicates usage, a frame-level switch flag and a frame-level quantization parameter adjustment flag are obtained. The frame-level switch flag is used to determine whether all blocks within the current frame are filtered. When the frame-level switch flag indicates on and the frame-level quantization parameter adjustment flag indicates usage, the adjusted frame-level quantization parameters are obtained. Based on the adjusted frame-level quantization parameters and the neural network filtering model, the current block of the current frame is filtered to obtain the first residual information of the current block. In this way, based on the frame-level quantization parameter adjustment flag, it is possible to determine whether the quantization parameters input to the neural network filtering model need adjustment, achieving flexible selection and diverse variation processing of quantization parameters (input parameters), thereby improving decoding efficiency.

Claims

1. A filtering method applied to a decoder, the method comprising: The bitstream is parsed to obtain the frame-level switch flag and the frame-level quantization parameter adjustment flag. The frame-level switch flag is used to determine whether each block in the current frame is subjected to neural network filtering. The frame-level quantization parameter adjustment flag is used to indicate whether the quantization parameters are adjusted for the current frame. When the frame-level switch flag indicates that neural network filtering is performed on each block in the current frame, and the frame-level quantization parameter adjustment flag indicates that the quantization parameter is adjusted for the current frame, the adjusted frame-level quantization parameter is obtained. Based on the adjusted frame-level quantization parameters and neural network filtering model, the current block of the current frame is filtered to obtain the first residual information of the current block.

2. The method according to claim 1, wherein, The method further includes: When the frame-level switch flag indicates that neural network filtering is performed only on a portion of the blocks within the current frame, the block-level usage flag is obtained. When the block-level use flag indicates that neural network filtering is applied to any color component of the current block, and the frame-level quantization parameter adjustment flag indicates that the quantization parameter is adjusted for the current frame, the adjusted frame-level quantization parameter is obtained. Based on the adjusted frame-level quantization parameters and the neural network filtering model, the current block of the current frame is filtered to obtain the first residual information of the current block.

3. The method according to claim 1 or 2, wherein, The process of obtaining the adjusted frame-level quantization parameters includes: The frame-level quantization offset parameters are determined by adjusting the index based on the frame-level quantization parameters obtained from the bitstream. The adjusted frame-level quantization parameters are determined based on the obtained frame-level quantization parameters and the frame-level quantization bias parameters.

4. The method according to claim 1 or 2, wherein, Before filtering the current block of the current frame based on the adjusted frame-level quantization parameters and the neural network filtering model to obtain the first residual information of the current block, the method further includes: Get the reconstruction value of the current block.

5. The method according to claim 1 or 2, wherein, Before filtering the current block of the current frame based on the adjusted frame-level quantization parameters and the neural network filtering model to obtain the first residual information of the current block, the method further includes: Obtain at least one of the following: the predicted value of the current block, the block partitioning information, and the deblocking filter boundary strength, as well as the reconstructed value of the current block.

6. The method according to claim 5, wherein, The process of filtering the current block of the current frame based on the adjusted frame-level quantization parameters and the neural network filtering model to obtain the first residual information of the current block includes: Using the neural network filtering model, the current block is filtered based on at least one of the predicted value of the current block, the block partitioning information, and the deblocking filtering boundary strength, the reconstructed value of the current block, and the adjusted frame-level quantization parameters to obtain the first residual information of the current block.

7. The method according to any one of claims 1 or 2, wherein, After filtering the current block of the current frame based on the adjusted frame-level quantization parameters and the neural network filtering model to obtain the first residual information of the current block, the method further includes: Obtain the second residual scaling factor in the bitstream; Based on the second residual scaling factor, the first residual information of the current block is scaled to obtain the first target residual information; Based on the first target residual information and the reconstruction value of the current block, the first target reconstruction value of the current block is determined.

8. A filtering method applied to an encoder, the method comprising: Enabling sequence-level access allows the use of flags; When the sequence level allows the use of flag bits to indicate that the use of neural network-based loop filtering technology is allowed, the original value of the current block, the reconstructed value of the current block, and the frame-level quantization parameters in the current frame are obtained. The first reconstruction value is determined by filtering and estimating the current block based on the neural network filtering model, the reconstructed value of the current block, and the frame-level quantization parameters. Rate distortion cost is estimated based on the first reconstructed value and the original value of the current block to obtain the rate distortion cost of the current block. The first rate distortion cost of the current frame is determined by traversing the current frame. Based on the neural network filtering model, at least one frame-level quantization bias parameter, the frame-level quantization parameter, and the reconstructed value of the current block in the current frame, at least one filtering estimation is performed on the current frame to determine at least one second rate distortion cost of the current frame. Based on the first rate-distortion cost and the at least one second rate-distortion cost, a frame-level quantization parameter adjustment flag is determined; wherein, the frame-level quantization parameter adjustment flag is used to characterize whether to adjust the quantization parameters input to the neural network filtering model for the current frame.

9. The method according to claim 8, wherein, The step of performing at least one filtering estimation on the current frame based on the neural network filtering model, at least one frame-level quantization bias parameter, the frame-level quantization parameter, and the reconstructed value of the current block in the current frame, to determine at least one second rate-distortion cost of the current frame, includes: Obtain the i-th frame-level quantization bias parameter, and adjust the frame-level quantization parameter based on the i-th frame-level quantization bias parameter to obtain the i-th adjusted frame-level quantization parameter; i is a positive integer greater than or equal to 1; Based on the neural network filtering model, the reconstructed value of the current block, and the frame-level quantization parameter adjusted for the i-th time, the current block is filtered and estimated to obtain the second reconstructed value for the i-th time. Rate distortion cost estimation is performed based on the i-th second reconstruction value and the original value of the current block. After traversing all blocks of the current frame, the i-th second rate distortion cost is obtained. Then, based on the (i+1)-th frame-level quantization bias parameter, the (i+1)-th filtering estimation is performed until at least one is completed, thereby determining at least one second rate distortion cost of the current frame.

10. The method according to claim 8 or 9, wherein, The step of determining the frame-level quantization parameter adjustment flag bit based on the first rate-distortion cost and the at least one second rate-distortion cost includes: A first minimum rate distortion cost is determined from the first rate distortion cost and the at least one second rate distortion cost; If the first minimum rate distortion cost is the first rate distortion cost, then the frame-level quantization parameter adjustment flag is determined to indicate that the quantization parameters have not been adjusted for the current frame; If the first minimum rate distortion cost is any one of the at least one second rate distortion cost, then the frame-level quantization parameter adjustment flag is determined to indicate that the quantization parameters are adjusted for the current frame.

11. The method according to claim 10, wherein, The method further includes: When the sequence level allows the use of a flag bit to indicate that the use of a neural network-based loop filtering technique is permitted, a rate-distortion cost estimate is performed based on the original value and the reconstructed value of the current block in the current frame to obtain a third rate-distortion cost.

12. The method according to claim 11, wherein, After determining the frame-level quantization parameter adjustment flag based on the first rate-distortion cost and the at least one second rate-distortion cost, the method further includes: Based on the neural network filtering model, the reconstructed value of the current block, and the frame-level quantization parameters, a filtering estimation is performed on the current block to determine the third reconstructed value; Based on the third reconstructed value and the original value of the current block, the rate-distortion cost is estimated to obtain the fourth rate-distortion cost of the current block; Based on the neural network filtering model, the target reconstruction value corresponding to the first minimum rate distortion cost, and the frame-level quantization parameters, the current block is filtered and estimated to obtain the fourth reconstruction value. Based on the fourth reconstructed value and the original value of the current block, the rate-distortion cost is estimated to obtain the fifth rate-distortion cost of the current block; Based on the fourth rate distortion cost and the fifth rate distortion cost, determine the block-level usage flag bit; Traverse the blocks in the current frame and determine the sum of the minimum rate distortion costs of all blocks in the current frame as the sixth rate distortion cost of the current frame.

13. The method according to claim 12, wherein, The method further includes: If the minimum rate distortion cost among the third rate distortion cost, the first minimum rate distortion cost, and the sixth rate distortion cost is the first minimum rate distortion cost, then the frame-level switch flag is determined to indicate that neural network filtering is performed on each block in the current frame, and the frame-level switch flag is written into the bitstream. If the minimum rate distortion cost among the third rate distortion cost, the first minimum rate distortion cost, and the sixth rate distortion cost is the sixth rate distortion cost, then the frame-level switch flag indicates that neural network filtering is performed only on a portion of the blocks within the current frame, and the frame-level switch flag and the block-level usage flag are written into the bitstream.

14. The method of claim 10, wherein, After determining the frame-level quantization parameter adjustment flag based on the first rate-distortion cost and the at least one second rate-distortion cost, the method further includes: If the first minimum rate distortion cost is any one of the at least one second rate distortion cost, then the frame-level quantization bias parameter corresponding to the first minimum rate distortion cost is written into the bitstream from the at least one frame-level quantization bias parameter, or the block-level quantization parameter index of the frame-level quantization bias parameter corresponding to the first minimum rate distortion cost is written into the bitstream.

15. The method according to claim 8, wherein, The step of filtering and estimating the current block based on the neural network filtering model, the reconstructed value of the current block, and the frame-level quantization parameters to determine the first reconstructed value includes: For the current frame, the current block is filtered and estimated based on the neural network filtering model, the reconstructed value of the current block, and the frame-level quantization parameters to determine the first estimation residual information; Determine the first residual scaling factor; The first estimated residual information is scaled using the first residual scaling factor to obtain the first scaled residual information; The first scaling residual information is combined with the reconstruction value of the current block to determine the first reconstruction value.

16. The method according to claim 15, wherein, Before determining the first residual scaling factor, the method further includes: For the current frame, obtain at least one of the following: the predicted value of the current block, the block partitioning information, and the deblocking filter boundary strength, as well as the reconstructed value of the current block; Using the neural network filtering model, the first estimated residual information is obtained by filtering and estimating the current block based on at least one of the predicted value of the current block, the block partitioning information, and the deblocking filtering boundary strength, the reconstructed value of the current block, and the frame-level quantization parameters.

17. The method according to claim 8, wherein, The step of performing at least one filtering estimation on the current frame based on the neural network filtering model, at least one frame-level quantization bias parameter, the frame-level quantization parameter, and the reconstructed value of the current block in the current frame, to determine at least one second rate-distortion cost of the current frame, includes: When the current frame is a first type frame, based on the neural network filtering model, at least one frame-level quantization bias parameter, the frame-level quantization parameter and the reconstructed value of the current block in the current frame, at least one filtering estimation is performed on the current frame to determine at least one second rate distortion cost of the current frame. The first type of frame is either a B-frame or a P-frame.

18. The method according to claim 8, wherein, The method further includes: When the sequence level allows the use of a flag bit to indicate that the use of a neural network-based loop filtering technique is allowed, at least one of the following is obtained: the predicted value of the current block, the block partitioning information, and the deblocking filter boundary strength; the reconstructed value of the current block; and the frame-level quantization parameters. Based on at least one of the predicted value of the current block, block partitioning information, and deblocking filter boundary strength, the neural network filtering model, the reconstructed value of the current block, and the frame-level quantization parameters, the current block is filtered and estimated to determine the sixth reconstructed value. Rate distortion cost is estimated based on the sixth reconstructed value and the original value of the current block to obtain the rate distortion cost of the current block. The seventh rate distortion cost of the current frame is determined by traversing the current frame. Based on at least one of the predicted value of the current block, the block partitioning information, and the deblocking filter boundary strength, the neural network filtering model, at least one frame-level input bias parameter, and the reconstructed value of the current block in the current frame, at least one filtering estimation is performed on the current frame to determine at least one eighth rate distortion cost of the current frame. Based on the first rate distortion cost and the at least one eighth rate distortion cost, a frame-level input parameter adjustment flag is determined; wherein, the frame-level input parameter adjustment flag is used to characterize whether any one of the parameters, the predicted value, the block partitioning information, and the deblocking filter boundary strength, has been adjusted.

19. A computer-readable storage medium, wherein, The computer-readable storage medium stores a computer program that, when executed by a first processor, implements the method as described in any one of claims 1 to 7, or when executed by a second processor, implements the method as described in any one of claims 8 to 18.

Images