A filtering method, decoder, encoder and computer-readable storage medium

Abstract

The embodiments of the present application provide a filtering method, a decoder, an encoder, and a computer-readable storage medium, which obtain a model adaptive selection identifier by parsing a bitstream; if the model adaptive selection identifier representation allows, obtain a frame-level model selection consistency identifier of the current image frame; determine a filtering model selection type based on the frame-level model selection consistency identifier; the filtering model selection type includes any one of a preset selection network type and a rate-distortion cost selection type; determine a filtering model for the current block based on the filtering model selection type; and use the filtering model to filter the current block to obtain a filtered first image.

Description

Technical Field

[0001] The embodiments of the present application relate to video coding technology, and relate to but are not limited to a filtering method, a decoder, an encoder, and a computer-readable storage medium. Background Art

[0002] In the Audio Video Standard (AVS) codec framework 3, the traditional loop filter module primarily includes a deblocking filter (DBF), sample adaptive offset (SAO), and an adaptive loop filter (ALF). In the AVS3 High-Performance Modular Intelligent Coding Test Model (HPM-ModAI), a residual neural network-based loop filter (CNNLF) is used to implement the intelligent loop filter module, located between the SAO and ALF. During the CNNLF training phase, four different CNNLF models were trained for QPs (quantity parameters) divided into four ranges: 27-31, 32-37, 38-44, and 45-50. Four I-frame luminance component models, four non-I-frame luminance component models, four chrominance U component models, and four chrominance V component models, totaling 16 different CNNLF models. When encoding and decoding an image frame, the CNNLF model selected from these 16 different models is not necessarily the one that produces the best filtering effect for that frame.

[0003] Currently, deep learning-based models are used to select CNNLF models, or a rate-distortion cost is used to select the CNNLF model with the lowest rate-distortion, and filtering is then performed based on the selected CNNLF model. However, because deep learning models are trained, the accuracy of the CNNLF model selected through prediction is not necessarily completely accurate. Furthermore, when using a rate-distortion proxy to select a CNNLF model, the model number must be transmitted in the bitstream, increasing the number of transmitted bits and thus affecting encoding and decoding efficiency. Summary of the Invention

[0004] The embodiments of the present application provide a filtering method, a decoder, an encoder, and a computer-readable storage medium, which can reduce transmission bits and improve encoding and decoding efficiency while ensuring encoding and decoding accuracy.

[0005] In a first aspect, an embodiment of the present application provides a filtering method, applied to a decoder, the method comprising:

[0006] Parse the code stream and obtain the model adaptation selection flag;

[0007] If the model adaptive selection identifier representation allows, obtaining a frame-level model selection consistency identifier for the current image frame;

[0008] Determining the filtering model selection type according to the frame-level model selection consistency flag; the filtering model selection type includes any one of a preset selection network type and a rate-distortion cost selection type;

[0009] Determining a filtering model for a current block based on the filtering model selection type;

[0010] The current block is filtered using the filtering model to obtain a filtered first image.

[0011] In a second aspect, an embodiment of the present application further provides a filtering method, applied to an encoder, the method comprising:

[0012] Using a preset selection network model, performing filter model pre-estimation on multiple candidate filter models of a current block of a current image frame to be filtered to determine a first candidate filter model; and using the multiple candidate filter models, performing rate-distortion cost estimation on the current block of the current image frame to be filtered to determine a second candidate filter model from the multiple candidate filter models;

[0013] Determining a model accuracy of a current block based on the first candidate filtering model and the second candidate filtering model;

[0014] Continue to estimate the filter model and rate-distortion cost of the next block until the last block of the current image frame is traversed, and combine the model accuracy of the current block to obtain the total model accuracy of the last block;

[0015] Determining a filter model selection type for the current image frame based on the statistical number of blocks of the current image frame and the total model accuracy;

[0016] According to the filtering model selection type, a filtering model is determined from the first candidate filtering model and the second candidate model, and the filtering model is used to perform filtering processing on the current image frame to be filtered to obtain a filtered second image.

[0017] In a third aspect, an embodiment of the present application provides a decoder, including:

[0018] The parsing part is configured to parse the code stream and obtain the model adaptation selection flag;

[0019] A first acquisition part is configured to acquire a frame-level model selection consistency flag of the current image frame if the model adaptive selection flag representation allows it;

[0020] The first determining part is configured to determine the filtering model selection type according to the frame-level model selection consistency flag; and determine the filtering model of the current block based on the filtering model selection type; the filtering model selection type includes any one of a preset selection network type and a rate-distortion cost selection type;

[0021] The first filtering part is configured to use the filtering model to filter the current block to obtain a filtered first image.

[0022] In a fourth aspect, an embodiment of the present application provides an encoder, including:

[0023] The model determination part is configured to use a preset selection network model to perform filter model estimation on multiple candidate filter models for the current block of the current image frame to be filtered, and determine a first candidate filter model; and use the multiple candidate filter models to perform rate-distortion cost estimation on the current block of the current image frame to be filtered, and determine a second candidate filter model from the multiple candidate filter models;

[0024] The first determining portion is configured to determine a model accuracy of a current block based on the first candidate filtering model and the second candidate filtering model; continue to estimate the filtering model and rate-distortion cost of a next block until the last block of the current image frame is traversed, and obtain a total model accuracy of the last block based on the model accuracy of the current block; and determine a filtering model selection type for the current image frame based on a statistical number of blocks in the current image frame and the total model accuracy;

[0025] The model determination part is further configured to determine a filtering model from the first candidate filtering model and the second candidate model according to the filtering model selection type;

[0026] The second filtering part is configured to use the filtering model to perform filtering processing on the current image frame to be filtered to obtain a filtered second image.

[0027] In a fifth aspect, an embodiment of the present application further provides a decoder, including:

[0028] a first memory and a first processor;

[0029] The first memory stores a computer program that can be run on the first processor, and the first processor implements the filtering method of the decoder when executing the program.

[0030] In a sixth aspect, an embodiment of the present application further provides an encoder, including:

[0031] a second memory and a second processor;

[0032] The second memory stores a computer program that can be run on the second processor, and the filtering method of the encoder is performed when the second processor executes the program.

[0033] An embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a first processor, the computer program implements the decoding compensation method of the claim decoder; or, when the computer program is executed by a second processor, the computer program implements the filtering method of the claim encoder.

[0034] The present application provides a filtering method decoder, an encoder, and a computer-readable storage medium. The method comprises parsing a bitstream to obtain a model adaptive selection flag; if the model adaptive selection flag indicates that it is allowed, obtaining a frame-level model selection consistency flag of the current image frame; determining a filtering model selection type based on the frame-level model selection consistency flag; the filtering model selection type includes any one of a preset selection network type and a rate-distortion cost selection type; and determining a filtering model for the current block based on the filtering model selection type.

[0035] The current block is filtered using a filter model to obtain a filtered first image. Using the above technical implementation scheme, since the decoder can determine the filter model selection type by parsing the bitstream, it can determine which method to use for adaptive selection of the filter model based on the filter model selection type. The filter model selection type is selected by taking into account both a preset selection network type with low transmission volume (using a deep learning-based model to select the CNNLF model) and an accurate rate-distortion cost selection type (rate-distortion proxy selection). This can achieve the goal of reducing transmission bits and improving decoding efficiency while ensuring encoding and decoding accuracy.

[0036] During the encoding process, the encoder selects models based on two methods: a preset network type (using a deep learning-based model to select the CNNLF model) and a rate-distortion cost selection type (rate-distortion proxy selection). The encoder then compares the model accuracy and selects a frame-level model based on the model accuracy. The filter model is then derived from a comprehensive analysis of accuracy and the number of transmitted bits. Therefore, when the encoder uses this filter model for filtering, it can adaptively select the filter model, reducing transmitted bits while ensuring codec accuracy and improving coding efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figures 1A-1C Schematic diagrams of component distribution in different color formats provided by the embodiments of the present application;

[0038] Figure 2 A schematic diagram of the division of an exemplary coding unit provided in an embodiment of the present application;

[0039] Figure 3 This is a diagram showing the luminance component network structure of an exemplary CNNLF provided in an embodiment of the present application;

[0040] Figure 4 A diagram showing the chrominance component network structure of an exemplary CNNLF provided in an embodiment of the present application;

[0041] Figure 5 A network structure diagram of an exemplary residual block provided in an embodiment of the present application;

[0042] Figure 6 An exemplary video coding system structure diagram provided in an embodiment of the present application;

[0043] Figure 7 An exemplary video decoding system structure diagram provided in an embodiment of the present application;

[0044] Figure 8 Flowchart 1 of a filtering method provided in an embodiment of the present application;

[0045] Figure 9 A filtering method according to an embodiment of the present invention Figure 2 ;

[0046] Figure 10 Flowchart 1 of another filtering method provided in an embodiment of the present application;

[0047] Figure 11 Another filtering method provided in this embodiment of the present invention Figure 2 ;

[0048] Figure 12 This is a diagram of an exemplary AMS network structure provided in an embodiment of the present application;

[0049] Figure 13 A schematic diagram of an exemplary loop filter provided in an embodiment of the present application;

[0050] Figure 14 A first structural diagram of a decoder provided in an embodiment of the present application;

[0051] Figure 15 A schematic diagram of the structure of a decoder provided in an embodiment of the present application Figure 2 ;

[0052] Figure 16 A first structural diagram of an encoder provided in an embodiment of the present application;

[0053] Figure 17 A schematic diagram of the structure of an encoder provided in an embodiment of the present application Figure 2 . DETAILED DESCRIPTION

[0054] The following will be combined with the accompanying drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. It should be understood that the specific embodiments described herein are only used to explain the related applications and are not intended to limit the applications. It should also be noted that for ease of description, only the parts relevant to the related applications are shown in the drawings.

[0055] It should be noted that the terms “first”, “second”, “third”, etc. mentioned throughout the specification are only used to distinguish different features and do not have the function of limiting priority, sequence, size relationship, etc.

[0056] In the embodiments of the present application, digital video compression technology primarily compresses large amounts of digital video data for easier transmission and storage. With the surge in Internet video usage and increasing demand for higher video clarity, while existing digital video compression standards can save significant amounts of video data, there is still a need for better digital video compression technologies to reduce bandwidth and traffic pressures associated with digital video transmission.

[0057] During digital video encoding, the encoder reads unequal pixels from raw video sequences in different color formats, including luminance and chrominance components. This means the encoder reads a black-and-white or color image. The image is then divided into blocks and encoded by the encoder, which typically uses a hybrid frame coding scheme, typically involving intra-frame and inter-frame prediction, transform and quantization, inverse transform and inverse quantization, loop filtering, and entropy coding. Intra-frame prediction refers only to information from the same frame, predicting the pixels within the current block to eliminate spatial redundancy. Inter-frame prediction can reference information from different frames, using motion estimation to search for the motion vector that best matches the current block, eliminating temporal redundancy. Transform and quantization convert the predicted image blocks to the frequency domain, redistributing the energy. Combined with quantization, this process removes information that is insensitive to the human eye, eliminating visual redundancy. Entropy coding eliminates character redundancy based on the current context model and the probabilistic information of the binary bitstream. Loop filtering primarily processes the inverse-transformed and inverse-quantized pixels to compensate for distortion and provide a better reference for subsequent pixel encoding.

[0058] Currently, the scenario in which filtering processing can be performed can be the reference software test platform HPM based on AVS or the VVC reference software test platform (VVC TEST MODEL, VTM) based on Versatile Video Coding (VVC), which is not limited in the embodiments of the present application.

[0059] In a video image, a first video component, a second video component, and a third video component are generally used to represent a current block (Coding Block, CB); wherein, the three image components are a luminance component, a blue chrominance component, and a red chrominance component, respectively. The luminance component is usually represented by the symbol Y, the blue chrominance component is usually represented by the symbol Cb or U, and the red chrominance component is usually represented by the symbol Cr or V; thus, the video image can be represented in either the YCbCr format or the YUV format.

[0060] Typically, digital video compression technology operates on image data in the YCbCr (YUV) color encoding format. The YUV ratio is generally 4:2:0, 4:2:2, or 4:4:4. Y represents brightness (Luma), Cb (U) represents blue chroma, Cr (V) represents red chroma, and U and V represent chroma, which is used to describe color and saturation. Figures 1A to 1C The distribution diagram of each component in different color formats is shown, where white is the Y component and dark gray is the UV component. Figure 1A As shown in the color format, 4:2:0 means that every 4 pixels have 4 brightness components and 2 chrominance components (YYYYCbCr), such as Figure 1B As shown, 4:2:2 means that every 4 pixels have 4 brightness components and 4 chrominance components (YYYYCbCrCbCr), and as Figure 1C As shown, 4:4:4 indicates full pixel display (YYYYCbCrCbCrCbCrCbCr).

[0061] Currently, common video codec standards all use a block-based hybrid coding framework. Each video frame is divided into square Largest Coding Units (LCUs) of the same size (e.g., 128×128, 64×64, etc.). Each LCU can be further divided into rectangular Coding Units (CUs) according to a rule. Coding Units may also be divided into smaller Prediction Units (PUs). Specifically, the hybrid coding framework may include modules such as prediction, transform, quantization, entropy coding, and in-loop filtering. The prediction module may include intra-frame prediction and inter-frame prediction, and inter-frame prediction may include motion estimation and motion compensation. Since there is a strong correlation between adjacent pixels within a video frame, the use of intra-frame prediction in video codec technology can eliminate spatial redundancy between adjacent pixels. Inter-frame prediction can refer to image information from different frames and use motion estimation to search for the motion vector information that best matches the current partition block to eliminate temporal redundancy; the transformation converts the predicted image block to the frequency domain, redistributes the energy, and combines it with quantization to remove information that the human eye is not sensitive to, thereby eliminating visual redundancy; entropy coding can eliminate character redundancy based on the current context model and the probability information of the binary code stream.

[0062] It should be noted that during the video encoding process, the encoder first reads the image information and divides the image into several Coding Tree Units (CTUs). A Coding Tree Unit can be further divided into several Coding Units (CUs). These Coding Units can be rectangular blocks or square blocks. The specific relationship can be referred to Figure 2 shown.

[0063] During intra-frame prediction, the current coding unit cannot reference information from different frames and can only use adjacent coding units from the same frame as reference information for prediction. This means that, based on the current left-to-right, top-to-bottom coding order, the current coding unit can reference the upper-left coding unit, the upper coding unit, and the left coding unit as reference information to predict the current coding unit. The current coding unit then serves as reference information for the next coding unit, thus predicting the entire image. If the input digital video is in color format, the current mainstream digital video encoder input source is in YUV 4:2:0 format, meaning that every four pixels in the image are composed of four Y components and two UV components. The encoder encodes the Y and UV components separately, using slightly different encoding tools and techniques. The decoder also decodes the video according to the different formats.

[0064] Intra-frame prediction in digital video encoding and decoding primarily uses information from adjacent blocks in the current frame to predict the current block. The residual information is then calculated between the predicted block and the original image block. This residual information is then transformed and quantized before being transmitted to the decoder. After receiving and parsing the bitstream, the decoder performs an inverse transform and quantization to obtain the residual information. This residual information is then superimposed on the predicted image block obtained by the decoder to create the reconstructed image block.

[0065] The following takes AVS3 as an example for explanation.

[0066] AVS3's traditional loop filter module mainly includes a deblocking filter (DBF), sample adaptive offset (SAO), and an adaptive correction filter (ALF). In AVS3's high-performance modular intelligent coding test model HPM-ModAI, a loop filter based on a residual neural network (CNNLF) is used as the baseline solution for the intelligent loop filter module, located between SAO and ALF. During the encoding test, according to the general test conditions for intelligent coding, for All Intra configurations, ALF is turned on, DBF and SAO are turned off; for Random Access and Low Delay configurations, DBF for I frames is turned on, ALF is turned on, and SAO is turned off.

[0067] The loop filter based on the residual neural network designs different network structures or network models for the luminance component (video component) and the chrominance component (video component), such as Figure 3 The brightness component network structure of CNNLF shown in Figure 4 The chrominance component network structure of CNNLF is shown in Figure 2. For the luminance component, the entire luminance component network structure consists of convolutional layers, activation layers, residual blocks, jump connections, etc. Among them, the network structure of the residual block is as follows: Figure 5As shown, it consists of convolutional layers, activation layers, and skip connections. The luminance component network structure also includes a global skip connection from input to output, which allows the network to focus on learning the residual and accelerates the network convergence process. For the chrominance component, the luminance component is introduced as one of the inputs to guide the filtering of the chrominance component. The entire chrominance component network structure consists of convolutional layers, activation layers, residual blocks, pooling layers, skip connections, and other parts. Due to the inconsistency of resolution, upsampling is first performed in the chrominance component network structure. In order to avoid introducing other noise during the upsampling process, the resolution is expanded by directly copying adjacent pixels. At the end of the chrominance component network structure, a pooling layer is used to complete the downsampling of the chrominance component. In HPM-ModAI, the number of residual blocks of the luminance component network structure is N = 20, and the number of residual blocks of the chrominance component network structure is N = 10.

[0068] It should be noted that the use of CNNLF involves two phases: offline training and inference testing. During the offline training phase, 16 models were trained: four I-frame luma component models, four non-I-frame luma component models, four chroma U component models, and four chroma V component models. Using the DIV2K image dataset, images were converted from RGB to single-frame video sequences in YUV 4:2:0 format, which served as label data. The sequences were then encoded using HPM in the All Intra configuration, disabling traditional filters such as DBF, SAO, and ALF, and setting quantization steps between 27 and 50 to obtain reconstructed sequences. The reconstructed sequences were partitioned into four intervals with QP values ​​ranging from 27 to 31, 32 to 37, 38 to 44, and 45 to 50, and then cut into 128x128 image blocks as training data. This training data was used to train the models for four I-frame luma component models, four chroma U component models, and four chroma V component models. Furthermore, using the BVI-DVC video dataset, HPM-ModAI was used to encode under the RandomAccess configuration, turning off traditional filters such as DBF, SAO and ALF, and turning on CNNLF for I frames. The encoded and reconstructed non-I frame data was collected, and four non-I frame luminance component models were trained respectively.

[0069] During the inference test phase, HPM-ModAI sets a frame-level switch and a coding tree unit (CTU)-level switch for the luminance component to control whether to turn on CNNLF, and a frame-level switch for the chrominance component to control whether to turn on CNNLF. The frame-level switch is determined by formula (1), where D = D net -D rec Refers to the distortion reduced after CNNLF processing (D net is the distortion after filtering, D recis the distortion before filtering), R is the number of CTUs in the current frame, and λ is consistent with the λ of the adaptive correction filter. When Rdcost is negative, the frame-level CNNLF is turned on, otherwise it is turned off.

[0070] RDcost=D+λ*R (1)

[0071] When the frame-level switch is on, for the luminance component, rate-distortion optimization can also be used to determine whether to enable CNNLF for each CTU. The CTU-level switch is determined by formula (2). When Rdcost is negative, the frame-level CNNLF is enabled, otherwise it is disabled.

[0072] RDcost=D (2)

[0073] The embodiment of the present application is based on the implementation of the inference test phase to perform CNNLF filtering processing.

[0074] In the embodiment of the present application, the encoding and decoding framework is described in the AVS3 encoder framework.

[0075] This application provides a video coding system, such as Figure 6As shown, the video coding system 1 includes: a transform unit 111, a quantization unit 112, a mode selection and coding control logic unit 113, an intra-frame prediction unit 114, an inter-frame prediction unit 115 (including: motion compensation and motion estimation), an inverse quantization unit 116, an inverse transform unit 117, a loop filter unit 118, a coding unit 119 and a decoded image buffer unit 110; for the input original video signal, the coding tree block (Coding A video reconstruction block can be obtained by dividing the video tree unit (CTU), and the coding mode is determined by the mode selection and coding control logic unit 113. Then, the residual pixel information obtained after intra-frame or inter-frame prediction is transformed by the transformation unit 111 and the quantization unit 112 to transform the video reconstruction block, including transforming the residual information from the pixel domain to the transform domain and quantizing the obtained transform coefficients to further reduce the bit rate; the intra-frame prediction unit 114 is used to perform intra-frame prediction on the video reconstruction block; wherein, the intra-frame prediction unit 114 is used to determine the optimal intra-frame prediction mode of the video reconstruction block; the inter-frame prediction unit 115 is used to perform inter-frame prediction coding of the received video reconstruction block relative to one or more blocks in one or more reference frames to provide time prediction information; wherein, motion estimation is to generate motion direction The motion vector can estimate the motion of the video reconstructed block. Motion compensation is then performed based on the motion vector determined by motion estimation. After determining the inter-frame prediction mode, the inter-frame prediction unit 115 is further configured to provide the selected inter-frame prediction data to the encoding unit 119, and also to send the calculated motion vector data to the encoding unit 119. In addition, the inverse quantization unit 116 and the inverse transform unit 117 are configured to reconstruct the video reconstructed block, reconstructing a residual block in the pixel domain. This reconstructed residual block is passed through the loop filter unit 118 to remove blocking artifacts. The reconstructed residual block is then added to a predictive block in the frame of the decoded image buffer unit 110 to generate a reconstructed video reconstructed block. The encoding unit 119 is configured to encode various coding parameters and quantized transform coefficients. The decoded image buffer unit 110 is configured to store the reconstructed video reconstructed block for prediction reference. As video encoding progresses, new reconstructed video reconstructed blocks are continuously generated and stored in the decoded image buffer unit 110. The loop filter unit 118 includes: DBF, SAO, Adaptive Model Selection (AMS) module CNNLF and ALF.

[0076] The embodiment of the present application provides a video decoding system, Figure 7 This is a schematic diagram of the composition structure of the video decoding system according to an embodiment of the present application. Figure 5 As shown, the video decoding system 2 includes:

[0077] The decoding unit 121, the inverse transform unit 127, the inverse quantization unit 122, the intra-frame prediction unit 123, the motion compensation unit 124, the loop filter unit 125 and the decoded image cache unit 126 are configured; after the input video signal is encoded by the video encoding system 11, a code stream of the video signal is output; the code stream is input to the video decoding system 12, and first passes through the decoding unit 121 to obtain a decoded transform coefficient; the transform coefficient is processed by the inverse transform unit 127 and the inverse quantization unit 122 to generate a residual block in the pixel domain; the intra-frame prediction unit 123 can be used to generate prediction data of the current video decoding block based on the determined intra-frame prediction direction and data from a previously decoded block of the current frame or picture; the motion compensation unit 124 determines prediction information for a video decoding block by analyzing motion vectors and other associated syntax elements, and uses this prediction information to generate a predictive block for the video decoding block being decoded. A decoded video block is formed by summing the residual block from the inverse transform unit 127 and the inverse quantization unit 122 with the corresponding predictive block generated by the intra-frame prediction unit 123 or the motion compensation unit 124. The decoded video signal passes through the loop filter unit 125 to remove blocking artifacts, thereby improving video quality. The decoded video block is then stored in the decoded image buffer unit 126, which stores reference images for subsequent intra-frame prediction or motion compensation and is also used for outputting the video signal to obtain the restored original video signal. The loop filter unit 125 includes: DBF, SAO, AMS, CNNLF, and ALF.

[0078] Based on this, when the filtering method is implemented, filtering processing is performed on the current block, which primarily affects the AMS in the loop filter unit 118 of the video encoding system 1; when the filtering method is implemented, filtering processing is performed on the current block, which primarily affects the AMS in the loop filter unit 125 of the video decoding system 2. The AMS implements the adaptive selection process of the filtering model of the filtering method provided in the embodiments of the present application.

[0079] The present application provides a filtering method for use in a video decoding device, i.e., a decoder. The functions implemented by this method can be implemented by a first processor in the video decoding device invoking program code. The program code can also be stored in a computer-readable storage medium. Therefore, the video decoding device includes at least a first processor and a first storage medium. The current decoding block and the current encoding block are both referred to as "current block" below.

[0080] Figure 8 The following is a flowchart of a filtering method according to an embodiment of the present application, which includes:

[0081] S101: Parse the code stream to obtain a model adaptation selection identifier.

[0082] S102: If the model adaptive selection identifier representation allows, obtain a frame-level model selection consistency identifier for the current image frame.

[0083] In an embodiment of the present application, when performing loop filtering on a reconstructed image of a current image frame, the decoder can first parse the bitstream to obtain a model adaptive selection flag. The model adaptive selection flag is used to determine whether the model adaptive selection function is enabled for the current image frame. If the model adaptive selection function is enabled, a filtering model is selected using an adaptive model selection method, and the filtering method of the present application is implemented using the filtering model.

[0084] It should be noted that the decoder will pre-set the model adaptive selection flag during the decoding process. If the model adaptive selection flag indicates that the model adaptive selection function is not enabled, the filter model will be selected and filtered according to a preset model selection method.

[0085] In the embodiment of the present application, whether the model adaptive selection function is enabled or not is reflected by a set model adaptive selection flag. If the model adaptive selection flag indicates permission, the model adaptive selection function is enabled; if the model adaptive selection flag indicates disallowance, the model adaptive selection function is disabled.

[0086] The model adaptive selection flag can be implemented in various forms such as numerical values, graphics, and marks, and is not limited in the embodiments of the present application. For example, when the model adaptive selection flag is a first value (e.g., 1), the representation is allowed; when the model adaptive selection flag is a second value (e.g., 0), the representation is not allowed.

[0087] In the embodiment of the present application, the model adaptive selection flag can be represented by model_adaptive_selection_enable_flag.

[0088] In this embodiment of the present application, if the model adaptive selection flag indicates that it allows, the filter model is selected using an adaptive model selection method. The decoder then continues to obtain a frame-level model selection consistency flag for the current image frame. This frame-level model selection consistency flag indicates the accuracy of the models selected using different model selection methods when decoding the current image frame, reflecting the accuracy of the selected model.

[0089] It should be noted that the decoder performs decoding processing on the current block, and the current block is a decoded block in the current image frame.

[0090] In an embodiment of the present application, the decoder processes the video component block of each video component, and the decoder can obtain the frame-level model selection consistency identifier of the video component of the current image frame; wherein different video components have their own corresponding frame-level model selection consistency identifiers.

[0091] In the embodiment of the present application, the video components may be: a luminance component and a chrominance component. A video component block corresponding to each video component corresponds to its own frame-level model selection consistency identifier.

[0092] Exemplarily, the frame-level model selection consistency flag of the luminance component is represented by luma_nn_rdo_equal_flag, and the frame-level model selection consistency flag of the chrominance component is represented by chroma_nn_rdo_equal_flag.

[0093] It should be noted that the frame-level model selection consistency flag is determined by the encoder during encoding, transmitted to the decoder through the bitstream, and parsed by the decoder during decoding.

[0094] In the embodiment of the present application, the frame-level model selection consistency flag for the same video component of each current block in a current image frame is consistent. For example, the frame-level model selection consistency flag for the luminance component of each current block in image frame 1 is the same, and the frame-level model selection consistency flag for the chrominance component of each current block is the same.

[0095] S103 , determining a filter model selection type according to a frame-level model selection consistency flag; the filter model selection type includes any one of a preset selection network type and a rate-distortion cost selection type.

[0096] In an embodiment of the present application, the frame-level model selection consistency identifier can be implemented in various forms such as numerical values, graphics, and marks, and the embodiment of the present application does not limit this.

[0097] The decoder can determine the filter model selection type based on the frame-level model selection consistency flag. The decoder can determine the current filter model selection type based on the value of the frame-level model selection consistency flag and the correspondence between the preset value and the filter model selection type.

[0098] In an embodiment of the present application, the filter model selection type includes: a preset selection network type and a rate-distortion cost selection type. Exemplarily, when the model adaptation selection flag is a third value (e.g., 1), it indicates the preset selection network type; when the model adaptation selection flag is a fourth value (e.g., 0), it indicates the rate-distortion cost selection type.

[0099] It should be noted that the preset selection network type is a method of using a preset selection network to predict and select multiple candidate filter models, and the rate-distortion cost selection type is a method of pre-filtering multiple candidate filter models to obtain their respective rate-distortion costs, and then selecting the filter model based on the rate-distortion cost.

[0100] In an embodiment of the present application, the decoder can select a consistency identifier based on the frame-level model transmitted in the code stream, so as to know how to adaptively select the corresponding filtering model for each video component in an image frame.

[0101] It should be noted that the multiple candidate filter models are trained filters to be selected.

[0102] Taking AVS3 as an example, in the HPM-ModAI of AVS3, for the CNNLF model, multiple candidate filter models can be 16 models in total, including 4 I-frame luminance component models, 4 non-I-frame luminance component models, 4 chrominance U component models, and 4 chrominance V component models trained offline. This application is not limited to the CNNLF model, and can also perform adaptive model selection for other efficient neural network filter models without limitation.

[0103] S104: Determine a filtering model for the current block based on the filtering model selection type.

[0104] S105 : Using a filtering model, filter the current block to obtain a filtered first image.

[0105] In the embodiment of the present application, after the decoder determines the filter model selection type, it can select the filtering mode according to the filter model selection type.

[0106] In some embodiments of the present application, if the filter model selection type is a preset selection network type, the decoder uses the preset selection network model to perform filter model estimation on multiple candidate filter models of the current block, determines the first candidate filter model, and uses the first candidate filter model as the filter model.

[0107] It should be noted that, since filtering is performed on each video component during decoding of the current block, and a consistency identifier is selected for the frame-level model corresponding to each video component, the filtering model selection type of each video component can be determined. In this way, the decoder can determine the corresponding filtering model for each video component.

[0108] In some embodiments of the present application, the decoder uses a preset selection network model to perform filtering model estimation on multiple candidate filtering models of the video component block of the current block, and determines multiple model probabilities corresponding to the multiple candidate filtering models one by one; the model corresponding to the maximum model probability among the multiple model probabilities is determined as the first candidate filtering model.

[0109] It should be noted that the preset selection network model in the embodiment of the present application can be a deep learning network. When a video component is processed using a deep learning network, at least two candidate filtering models corresponding to a video component are selected and predicted, and multiple model probabilities corresponding to at least two candidate filtering models of a video component are output. From the multiple model probabilities, the model corresponding to the maximum model probability is determined as the first candidate filtering model.

[0110] In some embodiments of the present application, if the filter model selection type is a rate-distortion cost selection type, the filter model index information is obtained; according to the filter model index information, a second candidate filter model is matched from multiple candidate filter models, and the second candidate filter model is used as the filter model.

[0111] luma_cnnlf_model_index is used to indicate the filter model index information of the luma component. chroma_cnnlf_model_index is used to indicate the filter model index information of the chroma component.

[0112] It should be noted that the encoder uses a rate-distortion cost selection model in which at least two candidate models corresponding to each video component are used to pre-filter each video component, determine multiple rate-distortion costs corresponding to each video component, and determine the second candidate filter model corresponding to the minimum rate-distortion cost from the multiple rate-distortion costs. If the filter model selection type of a video component is a rate-distortion cost selection type, the encoder will pass the selected filter model index information into the bitstream. In this way, when the decoder determines that the filter model selection type of a video component is a rate-distortion cost selection type, it can obtain the filter model index information from the bitstream, and based on the filter model index information, match the second candidate filter model from multiple candidate filter models to obtain the second candidate filter model. This second candidate filter model is the filter model of the video component.

[0113] After the decoder determines the filtering model, it can use the filtering model to filter the current block to obtain a filtered first image.

[0114] It should be noted that the decoder uses a filtering model to process the reconstructed block obtained by decoding the current block, thereby obtaining a filtered reconstructed block.

[0115] It can be understood that since the decoder can determine the filter model selection type by parsing the bit stream, it can determine which method to use for adaptive selection of the filter model based on the filter model selection type. The filter model selection type is selected while taking into account the preset selection network type with small transmission volume (using a deep learning-based model to select the CNNLF model) and the rate-distortion cost selection type with accuracy (rate-distortion proxy selection). It can achieve the goal of reducing transmission bits and improving decoding efficiency while ensuring the accuracy of encoding and decoding.

[0116] In some embodiments of the present application, Figure 9 As shown, since the current block has multiple video component blocks, a filtering method is implemented based on this, including:

[0117] S201: Parse the code stream and obtain a model adaptation selection flag.

[0118] S202: If the model adaptive selection identifier representation allows, obtain a frame-level model selection consistency identifier for the video component of the current image frame.

[0119] S203: Determine the filter model selection type for the video component according to the frame-level model selection consistency flag of the video component.

[0120] S204: Determine a filtering model for the video component of the current block based on the filtering model selection type of the video component.

[0121] S205 : Filter each video component block of the current block using a filtering model of the video component to obtain a filtered first image.

[0122] In an embodiment of the present application, the decoder processes each video component (luminance component and chrominance component) during the process of decoding the current block. Each video component corresponds to its own frame-level model selection consistency identifier and the filtering model selection type of the video component, and then determines the filtering model of the video component. Each video component is filtered using its corresponding filtering model. After the processing of each video component of the current block is completed, the first filtered image is obtained.

[0123] The following example illustrates the filtering method of the decoder.

[0124] For example, taking the video components as luminance component and chrominance component as an example, a filtering method is implemented on the decoder side as follows:

[0125] The decoder obtains and parses the bitstream. When parsing to the loop filter module, it processes it according to the specified filter order. When entering the model adaptive selection module, it executes as follows:

[0126] a) Determine whether the model adaptive selection module can be used in the current image frame based on model_adaptive_selection_enable_flag. If model_adaptive_selection_enable_flag is "1", try to perform model adaptive selection module processing on the current image frame and jump to b); if model_adaptive_selection_enable_flag is "0", jump to f);

[0127] b) For the current image frame, parse out the frame-level flags luma_nn_rdo_equal_flag and chroma_nn_rdo_equal_flag respectively, and then jump to c);

[0128] c) Determine the color component (video component) of the current block in the current image frame. If the current block is a luminance component (i.e., the video component is a luminance component), jump to d); if it is a chrominance component block, jump to e);

[0129] d) For the luminance component, judge according to the luma_nn_rdo_equal_flag flag. If luma_nn_rdo_equal_flag = 1, then for the current block of the current frame, the input of CNNLF is used to reconstruct the luminance image block as the input of the neural network (preset selection network), and the preset selection network output obtains the probability of the CNNLF model of each luminance component, and selects the model with the largest probability value as the CNNLF model (filter model) of the current luminance image block (luminance component block); if luma_nn_rdo_equal_flag = 0, then further parse luma_cnnlf_model_index to obtain the CNNLF model selected by the rate-distortion cost. Perform CNNLF filtering on the current luminance image block according to the above operation to obtain the final output reconstructed image block, and jump to f);

[0130] e) For the chroma component, judge according to the chroma_nn_rdo_equal_flag flag. If chroma_nn_rdo_equal_flag = 1, then for the chroma component of the current frame, the input of CNNLF is used to reconstruct the chroma image block as the input of the neural network. The neural network outputs the probability of each chroma CNNLF model, and the model with the largest probability value is selected as the CNNLF model of the current chroma image block; if chroma_nn_rdo_equal_flag = 0, then further parse chroma_cnnlf_model_index to obtain the CNNLF model selected by the rate-distortion cost. Perform CNNLF filtering on the current chroma image block according to the above operation to obtain the final output reconstructed image block, and jump to f);

[0131] f) If the current image frame has completed processing by the model adaptive selection module, the next image frame is loaded for processing and the process jumps to step a).

[0132] It can be understood that since the decoder can determine the filter model selection type of each video component of the current block by parsing the bit stream, it can determine which method to use for adaptive selection of the filter model based on the filter model selection type of each video component. The filter model selection type is selected while taking into account the preset selection network type with small transmission volume (using a deep learning-based model to select the CNNLF model) and the rate-distortion cost selection type with accuracy (rate-distortion proxy selection). It can achieve the goal of reducing transmission bits and improving decoding efficiency while ensuring encoding and decoding accuracy.

[0133] The present invention provides a filtering method for use in a video encoding device, i.e., an encoder. The functions implemented by the method can be implemented by a second processor in the video encoding device invoking program code. The program code can also be stored in a computer-readable storage medium. Therefore, the video encoding device includes at least a second processor and a second storage medium.

[0134] Figure 10 This is a schematic diagram of an implementation flow of an encoding according to an embodiment of the present application, the method comprising:

[0135] S301. Use a preset selection network model to perform filtering model estimation on multiple candidate filtering models of the current block of the current image frame to be filtered, and determine a first candidate filtering model; and use multiple candidate filtering models to perform rate-distortion cost estimation on the current block of the current image frame to be filtered, and determine a second candidate filtering model from multiple candidate filtering models.

[0136] When the encoder filters the current block of the current image frame, it uses the model adaptive selection flag to determine whether the model adaptive selection function is enabled for the current image frame. When the model adaptive selection function is enabled, the adaptive selection model is used to select the filtering model, and the filtering model is used to implement the filtering method of the present application.

[0137] It should be noted that during the encoding process, the encoder will pre-set the model adaptive selection flag. If the model adaptive selection flag indicates that the model adaptive selection function is not enabled, the filter model is selected and filtered according to a preset model selection method.

[0138] In some embodiments of the present application, the encoder obtains a model adaptive selection identifier; if the model adaptive selection identifier representation allows, a preset selection network model is used to perform filtering model estimation on multiple candidate filtering models of the current block of the current image frame to be filtered, and a first candidate filtering model is determined; and multiple candidate filtering models are used to perform rate-distortion cost estimation on the current block of the current image frame to be filtered, and a second candidate filtering model is determined from the multiple candidate filtering models.

[0139] In the embodiment of the present application, whether the model adaptive selection function is enabled or not is reflected by a set model adaptive selection flag. If the model adaptive selection flag indicates permission, the model adaptive selection function is enabled; if the model adaptive selection flag indicates disallowance, the model adaptive selection function is disabled.

[0140] The model adaptive selection flag can be implemented in various forms such as numerical values, graphics, and marks, and is not limited in the embodiments of the present application. For example, when the model adaptive selection flag is a first value (e.g., 1), the representation is allowed; when the model adaptive selection flag is a second value (e.g., 0), the representation is not allowed.

[0141] In the embodiment of the present application, the model adaptive selection flag can be represented by model_adaptive_selection_enable_flag.

[0142] In some embodiments of the present application, if the model adaptive selection identification representation is not allowed, the encoder performs filtering processing on the current image frame to obtain a filtered third image.

[0143] In some embodiments of the present application, the encoder can use multiple candidate filtering models to estimate the rate-distortion cost of the current block of the current image frame to be filtered, and obtain multiple rate-distortion costs; based on the multiple rate-distortion costs, determine the second candidate filtering model with the smallest rate-distortion cost from the multiple candidate filtering models.

[0144] S302: Determine the model accuracy of the current block based on the first candidate filtering model and the second candidate filtering model.

[0145] S303 , continue to perform filter model estimation and rate-distortion cost estimation for the next block until the last block of the current image frame is traversed, and combine the model accuracy of the current block to obtain the total model accuracy of the last block.

[0146] In an embodiment of the present application, the encoder can adaptively select one of the first and second candidate filter models as the final filter model. The encoder primarily determines the model accuracy of the current block based on the first and second candidate filter models, and determines which type of candidate filter model to select based on the model accuracy.

[0147] In some embodiments of the present application, if the first candidate filtering model and the second candidate filtering model are consistent, the encoder adds a first preset value on the basis of the previous model accuracy to obtain the model accuracy of the current block; wherein the initial model accuracy is the second preset value; if the first candidate filtering model and the second candidate filtering model are inconsistent, the previous model accuracy is used as the model accuracy of the current block.

[0148] It should be noted that the initial accuracy can be set to an initial value, that is, a second preset value, such as 0. The first preset value is a quantitative statistical value of the model selection consistency, which can be set to 1.

[0149] In some embodiments of the present application, after the encoder completes model selection for the current block, it continues to perform filtering model estimation and rate-distortion cost estimation for the next block, and combines the model accuracy of the current block to obtain the model accuracy of the next block, until the last block of the current image frame is traversed, and the total model accuracy of the last block is obtained.

[0150] The embodiments of the present application are not limited. Before performing CNNLF filtering, the encoder needs to adaptively select the filtering model of the current block in two ways, and obtain two selected candidate filtering models, namely the first candidate filtering model and the second candidate filtering model. By comparing whether the two methods of selecting models for each current block in the current image frame are consistent, it is determined whether the accuracy meets the conditions (preset accuracy threshold). If the accuracy meets the conditions, the first candidate filtering model selected by the preset selection network type can be used for filtering, so as to reduce bit transmission while ensuring the accuracy. If the accuracy does not meet the conditions, the second candidate filtering model selected by the rate-distortion cost selection type can be used for filtering to ensure the accuracy of the filtering.

[0151] It should be noted that the encoder needs to use an adaptive model selection method to select the first and second candidate filter models for each block of the current image frame. Then, it needs to perform frame-level statistics on the overall model accuracy to determine the model selection method for the entire frame. The current block has different video components, and the model selection methods for different video components are not necessarily the same. Each video component can determine its own frame-level model selection method (i.e., filter model selection type).

[0152] S304: Determine the filter model selection type for the current image frame based on the statistical number of blocks and the total model accuracy of the current image frame.

[0153] S305 , determining a filtering model from the first candidate filtering model and the second candidate filtering model according to the filtering model selection type, and performing filtering processing on the current image frame to be filtered using the filtering model to obtain a filtered second image.

[0154] After the encoder selects a model for each block of the current image frame, it can count the number of blocks in the current image frame. Based on the number of blocks in the current image frame and the overall model accuracy at the frame level, the type of filtering model for the current image frame can be selected.

[0155] The encoder can generate a frame-level model selection consistency flag for the current image frame based on the filter model selection type, wherein different video components have corresponding frame-level model selection consistency flags, and write the frame-level model selection consistency flag into the bitstream.

[0156] In the embodiment of the present application, the video components may be: a luminance component and a chrominance component. A video component block corresponding to each video component corresponds to its own frame-level model selection consistency identifier.

[0157] In the embodiment of the present application, the video components may be: a luminance component and a chrominance component. A video component block corresponding to each video component corresponds to its own frame-level model selection consistency identifier.

[0158] In the embodiment of the present application, the frame-level model selection consistency flag for the same video component of each current block in a current image frame is consistent. For example, the frame-level model selection consistency flag for the luminance component of each current block in image frame 1 is the same, and the frame-level model selection consistency flag for the chrominance component of each current block is the same.

[0159] In some embodiments of the present application, if the filter model selection type is a rate-distortion cost selection type, the encoder writes the filter model index information of the second candidate filter model into the bitstream; wherein different video components have their own corresponding filter model index information of the second candidate filter model.

[0160] Among them, luma_cnnlf_model_index is used to represent the filter model index information of the luminance component, and chroma_cnnlf_model_index is used to represent the filter model index information of the chrominance component.

[0161] It is understandable that during the encoding process, the encoder will select the model based on the preset network type (using a deep learning-based model to select the CNNLF model) and the rate-distortion cost selection type (rate-distortion proxy selection). After comparing the model accuracy, the frame-level model selection method is selected based on the model accuracy, and the filter model is obtained from the two aspects of accuracy and transmission bit reduction. Therefore, when the encoder uses this filter model for filtering, it can adaptively select the filter model, while ensuring the accuracy of encoding and decoding, reducing the transmission bits and improving the coding efficiency.

[0162] In some embodiments of the present application, Figure 11 As shown, since the current block has multiple video component blocks, a filtering method is implemented based on this, including:

[0163] S401: Using a preset selection network model, perform filter model estimation on multiple video component candidate filter models of a video component block to determine a first candidate filter model corresponding to the video component.

[0164] S402 : Using multiple video component candidate filter models, perform rate-distortion cost estimation on the video component block to obtain a second candidate filter model corresponding to the video component block.

[0165] S403 : Determine the model accuracy of the video component block in the current block based on the first candidate filtering model and the second candidate filtering model of the video component.

[0166] S404. Continue to estimate the filter model and rate-distortion cost of the next video component block until the last block of the current image frame is traversed. Combined with the model accuracy of the current block, the total model accuracy of the last video component block is obtained.

[0167] S405 : Determine the filter model selection type for the current image frame based on the statistical number of video component blocks in the current image frame and the total model accuracy of the video components.

[0168] In an embodiment of the present application, the decoder processes each video component (luminance component and chrominance component) during the process of encoding the current block. Each video component corresponds to its own first candidate filtering model and second candidate filtering model. After completing the model candidate selection for each block of the current image frame, the models selected based on the two model selection methods are compared for consistency to obtain the model accuracy of the video component block, and the total model accuracy of all blocks of a video component (the total model accuracy of the video component block of the last block) is counted. Combined with the number of video component blocks, the frame-level filtering model selection type corresponding to a video component can be determined.

[0169] In some embodiments of the present application, the encoder determines a first ratio of the total model accuracy of a video component to the number of corresponding video component blocks; if the first ratio is greater than or equal to a preset accuracy threshold, the filtering model selection type of the video component of the current image frame is a preset selection network type; if the first ratio is less than the preset accuracy threshold, the filtering model selection type of the video component of the current image frame is a rate-distortion cost selection type.

[0170] Exemplarily, when the video components are luminance components and chrominance components, the encoder determines that the first ratio corresponding to the luminance component is greater than a preset accuracy threshold (i.e., the accuracy threshold of the luminance component neural network selection model), then the luminance components of all blocks of the current image frame are filtered using the second candidate filtering model corresponding to their respective rate-distortion cost selection types. The encoder determines that the first ratio corresponding to the chrominance component is greater than the preset accuracy threshold (i.e., the accuracy threshold of the chrominance component neural network selection model), then the chrominance components of all blocks of the current image frame are filtered using the second candidate filtering model corresponding to their respective rate-distortion cost selection types.

[0171] It should be noted that the luminance component neural network selection model accuracy threshold identifier, i.e., the luminance component neural network selection model accuracy threshold, such as luma_nn_rdo_equal_threshold, can be used to represent the chrominance component neural network selection model accuracy threshold identifier, i.e., the chrominance component neural network selection model accuracy threshold, such as chroma_nn_rdo_equal_threshold.

[0172] In the embodiment of the present application, model_adaptive_selection_enable_flag, luma_nn_rdo_equal_threshold, and chroma_nn_rdo_equal_threshold are defined using syntax elements to describe the sequence header as shown in Table 1 below.

[0173] Table 1

[0174]

[0175] S406 : Determine a filtering model for the video component from a first candidate filtering model and a second candidate filtering model of the video component according to the filtering model selection type of the video component.

[0176] S407 , using a filtering model of a video component, filtering the video component block of the current block of the current image frame to be filtered, determining the filtered current block, and continuing filtering the next block of the current image frame to obtain a filtered second image.

[0177] After the encoder obtains the filter model selection type of each video component, it can determine the filter model of the video component from the first candidate filter model and the second candidate filter model of the video component, and use the filter model of the video component to perform filtering processing.

[0178] In some embodiments of the present application, if the filter model selection type is a preset selection network type, the encoder determines the first candidate filter model as the filter model of the video component; if the filter model selection type is a rate-distortion cost selection type, the encoder determines the second candidate filter model as the filter model of the video component.

[0179] It should be noted that the first candidate filtering model selected by the preset selection network type is used for filtering to achieve the goal of reducing bit transmission while ensuring accuracy. The second candidate filtering model selected by the rate-distortion cost selection type is used for filtering to ensure filtering accuracy.

[0180] For example, taking the video components as luminance component and chrominance component as an example, a filtering method is implemented on the encoder side as follows:

[0181] When the encoder enters the loop filter module, it is processed according to the specified filter order. When it enters the model adaptive selection module, it is executed as follows:

[0182] a) Determine whether the model adaptive selection module can be used in the current image frame based on model_adaptive_selection_enable_flag. If model_adaptive_selection_enable_flag is "1", try to perform model adaptive selection module processing on the current image frame and jump to b); if model_adaptive_selection_enable_flag is "0", jump to k);

[0183] b) Determine the color component (video component) of the current block in the current image frame. If the current block is a luminance component (i.e., the video component is a luminance component), jump to c); if it is a chrominance component block, jump to g);

[0184] c) For the brightness component, the CNNLF input is used to reconstruct the brightness image block as the input of the neural network, and the network output is obtained.

[0185] d) For the luminance component, for the current block of the current frame, the luminance image block reconstructed by the input of CNNLF is used as the input of the neural network (preset selection network), and the preset selection network output obtains the probability of the CNNLF model of each luminance component, and the model with the largest probability value is selected as the CNNLF model (filter model) of the current luminance image block (luminance component block); and compared with the CNNLF model selected by the rate-distortion cost to see whether the CNNLF models selected by the two are consistent. The comparison result is counted as luma_nn_rdo_equal_cnt, whose initial value is 0. If the two selected models are the same, luma_nn_rdo_equal_cnt (previous comparison result) + 1, otherwise unchanged, jump to e);

[0186] e) If all luminance component blocks in the current image frame have been traversed, jump to f), otherwise jump to c);

[0187] f) According to the number of luminance blocks luma_cnt in the current image frame, calculate luma_nn_rdo_equal_cnt / luma_cnt and compare it with the preset accuracy threshold luma_nn_rdo_equal_threshold. The comparison result is represented by the frame-level flag luma_nn_rdo_equal_flag. If luma_nn_rdo_equal_cnt / luma_cnt is greater than or equal to luma_nn_rdo_equal_threshold, then all luminance component blocks of the current image frame use the CNNLF model (first candidate filter model) predicted by the preset selection network for neural network filtering. If luma_nn_rdo_equal_cnt / luma_cnt is less than luma_nn_rdo_equal_threshold, then all luminance component blocks of the current image frame use the CNNLF model selected by the rate-distortion cost for neural network filtering, and at the same time encode the model serial number luma_cnnlf_model_index of the luminance component corresponding to each current block into the bitstream, and jump to k);

[0188] g) For the chroma component, traverse all candidate CNNLF models, calculate the CNNLF model with the best filtering performance (the second candidate filtering model) by rate-distortion cost, and jump to h);

[0189] h) Use the input CNNLF to reconstruct the chroma image block as the input of the neural network, and the neural network outputs the probability of each chroma CNNLF model. Select the filter model with the largest probability value and compare it with the CNNLF model selected by the rate-distortion cost. The comparison result is counted as chroma_nn_rdo_equal_cnt. If the two selected filter models are the same, then you can add 1 to the comparison result obtained last time, that is, chroma_nn_rdo_equal_cnt+1, otherwise it remains unchanged and jump to i);

[0190] i) If all chrominance component blocks in the current image frame have been traversed, jump to j), otherwise jump to g);

[0191] j) Calculate chroma_nn_rdo_equal_cnt / chroma_cnt based on the number of chroma blocks in the current image frame, and compare it with the preset accuracy threshold chroma_nn_rdo_equal_threshold to obtain a comparison result, which can be represented by the frame-level flag chroma_nn_rdo_equal_flag. If chroma_nn_rdo_equal_cnt / chroma_cnt is greater than or equal to chroma_nn_rdo_equal_threshold, then all chroma component blocks of the current image frame are processed by neural network filtering using the CNNLF model (first candidate filter model) predicted by the preset selection network. If chroma_nn_rdo_equal_cnt / chroma_cnt is less than chroma_nn_rdo_equal_threshold, all chroma component blocks of the current image frame are processed by the CNNLF model selected by the rate-distortion cost through neural network filtering, and the model index chroma_cnnlf_model_index of the chroma component corresponding to each current block is encoded into the bitstream, and then jump to k);

[0192] k) If the current image frame has completed processing by the model adaptive selection module, then the next image frame is loaded for processing and the process jumps to step a).

[0193] In the embodiment of the present application, the syntax elements shown in Table 2 are added to the image header definition.

[0194] Table 2

[0195]

[0196] It should be noted that the description of the encoder is consistent with the principle of the decoder and the implementation corresponds to that of the decoder, so they will not be repeated here.

[0197] It is understandable that during the encoding process, the encoder will select the model based on the preset network type of the video component (using a deep learning-based model to select the CNNLF model) and the rate-distortion cost selection type (rate-distortion proxy selection). After comparing the model accuracy, the frame-level model selection method is selected based on the model accuracy, and the filter model is obtained from the two aspects of accuracy and transmission bit reduction. Therefore, when the encoder uses the filter model of the video component for filtering, it can adaptively select the filter model, while ensuring the accuracy of encoding and decoding, reducing the transmission bits and improving the coding efficiency.

[0198] It should be noted that in the embodiment of the present application, the implementation of the filtering method provided by the encoder and decoder is implemented in the model adaptive module AMS. The model adaptive selection module is an adaptive model selection method that combines deep neural network and rate distortion cost. For the deep neural network, it consists of a multi-layer convolutional neural network and a fully connected neural network. Its network structure is as follows: Figure 12 Based on Figure 12 The overall structural diagram of the modified loop filter unit of this application is as follows: Figure 13 As shown. Figure 12 and Figure 13 The AMS network structure consists of K convolutional layers and L fully connected layers. Each convolutional layer is followed by a maximum pooling layer, and the fully connected layers are followed by nonlinear activation functions ReLU and Softmax. The network input is the CNNLF input to reconstruct the luminance image block or chrominance image block. The network output is the probability distribution of each CNNLF model. The CNNLF model with the highest probability value is selected and compared with the CNNLF model selected by the rate-distortion cost to see if they are the same. By traversing all luminance image blocks or chrominance image blocks in each image frame, the accuracy of the CNNLF model selected by the neural network in that frame is obtained and compared with a preset threshold. If the accuracy is less than the preset accuracy threshold, the CNNLF model selected by the rate-distortion cost is used in that frame to perform neural network filtering on the luminance image block or chrominance image block, and the model sequence information (filter model index information) is transmitted in the bitstream. If the accuracy is greater than or equal to the preset accuracy threshold, the CNNLF model selected by the neural network is used in that image frame, and the model sequence information does not need to be transmitted.

[0199] It should be noted that the network structure of AMS may also include modules or structures such as the number of convolutional layers, the number of fully connected layers, and nonlinear activation functions.

[0200] In the embodiment of the present application, the four non-I frame luminance component models in HPM-ModAI are taken as an example, and the corresponding luminance component accuracy threshold is set to 60%, which realizes the training and testing of the model adaptive selection scheme. The filtering method implemented in this application is implemented on the AVS3 intelligent coding reference software HPM11.1-ModAI6.1. As shown in Tables 3 and 4, the test sequence required by AVS3 is tested under the Random Access configuration of the general test conditions for intelligent coding. The comparison anchor is HPM11.1-ModAI6.1. The average BD-rate changes on the Y, U, and V components are -1.20%, 0.47%, and 0.50%, respectively. Under the Low Delay B configuration of the general test conditions for intelligent coding, the average BD-rate changes are -0.91%, 0.28%, and 0.09%, respectively. This data shows that the encoding and decoding performance has been improved.

[0201] Table 3

[0202]

[0203]

[0204] Table 4

[0205]

[0206] By introducing the model adaptive selection optimization technology of joint deep learning (preset selection network) and rate-distortion cost, the AVS3 intelligent coding reference software HPM-ModAI has achieved good performance gains. By introducing the model adaptive selection optimization technology based on joint deep learning and rate-distortion cost, the CNNLF model is input into the reconstructed image block into a neural network with multiple convolution layers plus a fully connected layer, and the probability distribution of each CNNLF model is output. The CNNLF model selected by the network is compared with the CNNLF model selected by the rate-distortion cost. By traversing all image blocks in each image frame, the accuracy of the CNNLF model selected by the neural network in the image frame is obtained, and compared with the preset accuracy threshold value. The appropriate CNNLF model is adaptively selected for filtering processing for the reconstructed image block, and the final output is a reconstructed image block that is closer to the original image block, which improves the encoding and decoding performance.

[0207] Based on the implementation basis of the above embodiments, Figure 14 As shown, an embodiment of the present application provides a decoder 1, comprising:

[0208] The parsing part 10 is configured to parse the code stream and obtain the model adaptation selection flag;

[0209] The first acquisition part 11 is configured to acquire a frame-level model selection consistency flag of the current image frame if the model adaptive selection flag representation allows it;

[0210] The first determining part 12 is configured to determine the filtering model selection type according to the frame-level model selection consistency flag; and determine the filtering model of the current block based on the filtering model selection type; the filtering model selection type includes any one of a preset selection network type and a rate-distortion cost selection type;

[0211] The first filtering part 13 is configured to use the filtering model to filter the current block to obtain a filtered first image.

[0212] In some embodiments of the present application, the first determination part 12 is further configured to use the preset selection network model to perform filtering model estimation on multiple candidate filtering models of the current block if the filtering model selection type is the preset selection network type, and determine the first candidate filtering model, and the first candidate filtering model is used as the filtering model.

[0213] In some embodiments of the present application, the first determination part 12 is further configured to obtain filtering model index information if the filtering model selection type is the rate-distortion cost selection type; and according to the filtering model index information, match a second candidate filtering model from multiple candidate filtering models, and the second candidate filtering model is used as the filtering model.

[0214] In some embodiments of the present application, the first acquisition part 11 is further configured to obtain a frame-level model selection consistency identifier of a video component of a current image frame; wherein different video components have their own corresponding frame-level model selection consistency identifiers.

[0215] In some embodiments of the present application, the first determining part 12 is further configured to determine the filtering model selection type of the video component according to the frame-level model selection consistency identifier of the video component.

[0216] In some embodiments of the present application, the first determining part 12 is further configured to determine the filtering model of the video component of the current block based on the filtering model selection type of the video component.

[0217] In some embodiments of the present application, the current block includes: a plurality of video component blocks;

[0218] The first filtering part 13 is further configured to filter each video component block of the current block using the filtering model of the video component to obtain the first filtered image.

[0219] In some embodiments of the present application, the first determination part 12 is further configured to use the preset selection network model to perform filtering model estimation on multiple candidate filtering models of the video component block of the current block, and determine multiple model probabilities corresponding to the multiple candidate filtering models one by one; and determine the model corresponding to the maximum model probability among the multiple model probabilities as the first candidate filtering model.

[0220] It can be understood that since the decoder can determine the filter model selection type by parsing the bit stream, it can determine which method to use for adaptive selection of the filter model based on the filter model selection type. The filter model selection type is selected while taking into account the preset selection network type with small transmission volume (using a deep learning-based model to select the CNNLF model) and the rate-distortion cost selection type with accuracy (rate-distortion proxy selection). It can achieve the goal of reducing transmission bits and improving decoding efficiency while ensuring the accuracy of encoding and decoding.

[0221] In the practical application of this application, Figure 15 As shown, the embodiment of the present application further provides a decoder, including:

[0222] A first memory 14 and a first processor 15;

[0223] The first memory 14 stores a computer program that can be run on the first processor 15 , and the first processor 15 implements a filtering method corresponding to the decoder when executing the program.

[0224] Among them, the first processor 15 can be implemented by software, hardware, firmware or a combination thereof, and can use circuits, single or multiple application specific integrated circuits (ASICs), single or multiple general integrated circuits, single or multiple microprocessors, single or multiple programmable logic devices, or a combination of the aforementioned circuits or devices, or other suitable circuits or devices, so that the first processor 15 can execute the corresponding steps of the filtering method on the decoder side in the aforementioned embodiment.

[0225] The embodiment of the present application provides an encoder 2, such as Figure 16 As shown, including:

[0226] The model determination part 20 is configured to use a preset selection network model to perform filter model estimation on multiple candidate filter models for the current block of the current image frame to be filtered, and determine a first candidate filter model; and use the multiple candidate filter models to perform rate-distortion cost estimation on the current block of the current image frame to be filtered, and determine a second candidate filter model from the multiple candidate filter models;

[0227] The second determining portion 21 is configured to determine a model accuracy of a current block based on the first candidate filtering model and the second candidate filtering model; continue to estimate the filtering model and rate-distortion cost of the next block until the last block of the current image frame is traversed, and obtain a total model accuracy of the last block based on the model accuracy of the current block; and determine a filtering model selection type for the current image frame based on the number of blocks in the current image frame and the total model accuracy.

[0228] The model determination part 20 is further configured to determine a filtering model from the first candidate filtering model and the second candidate model according to the filtering model selection type;

[0229] The second filtering part 22 is configured to use the filtering model to perform filtering processing on the current image frame to be filtered to obtain a filtered second image.

[0230] In some embodiments of the present application, the model determination part 20 is further configured to use the multiple candidate filtering models to estimate the rate-distortion cost of the current block of the current image frame to be filtered to obtain the multiple rate-distortion costs; and based on the multiple rate-distortion costs, determine the second candidate filtering model with the smallest rate-distortion cost from the multiple candidate filtering models.

[0231] In some embodiments of the present application, the second determination part 21 is further configured to add a first preset value to the previous model accuracy to obtain the model accuracy of the current block if the first candidate filtering model and the second candidate filtering model are consistent; wherein the initial model accuracy is the second preset value; if the first candidate filtering model and the second candidate filtering model are inconsistent, the previous model accuracy is used as the model accuracy of the current block.

[0232] In some embodiments of the present application, the second determination part 21 is further configured to continue to perform filtering model estimation and rate-distortion cost estimation for the next block, and combine the model accuracy of the current block to obtain the model accuracy of the next block, until the last block of the current image frame is traversed, and the total model accuracy of the last block is obtained.

[0233] In some embodiments of the present application, the current block of the current image frame to be filtered includes: a video component block; the multiple candidate filtering models include: multiple video component candidate filtering models corresponding to the video components;

[0234] The model determination part 20 is further configured to use a preset selection network model to perform filter model estimation on the multiple video component candidate filter models of the video component block, and determine the first candidate filter model corresponding to the video component.

[0235] In some embodiments of the present application, the model determination part 20 is further configured to use the multiple video component candidate filtering models to perform rate-distortion cost estimation on the video component block to obtain the second candidate filtering model corresponding to the video component block.

[0236] In some embodiments of the present application, the second determining part 21 is further configured to determine the model accuracy of the video component block in the current block based on the first candidate filtering model of the video component and the second candidate filtering model thereof.

[0237] In some embodiments of the present application, the second determination part 21 is further configured to determine the filtering model selection type of the current image frame based on the statistical number of video component blocks in the current image frame and the total model accuracy of the video components.

[0238] In some embodiments of the present application, the second determination part 21 is further configured to determine a first ratio of the total model accuracy of the video component to the number of corresponding video component blocks; if the first ratio is greater than a preset accuracy threshold, the filtering model selection type of the video component of the current image frame is a preset selection network type; if the first ratio is less than or equal to the preset accuracy threshold, the filtering model selection type of the video component of the current image frame is a rate-distortion cost selection type.

[0239] In some embodiments of the present application, the model determination part 20 is further configured to determine the filtering model of the video component from the first candidate filtering model and the second candidate filtering model of the video component according to the filtering model selection type of the video component;

[0240] The second filtering part 22 is further configured to adopt the filtering model of the video component to filter the video component block of the current block of the current image frame to be filtered, determine the filtered current block, continue filtering the next block of the current image frame, and then obtain the filtered second image.

[0241] In some embodiments of the present application, the model determination part 20 is further configured to determine the first candidate filtering model as the filtering model of the video component if the filtering model selection type is a preset selection network type; and to determine the second candidate filtering model as the filtering model of the video component if the filtering model selection type is a rate-distortion cost selection type.

[0242] In some embodiments of the present application, the encoder 2 further includes: a generating part 23 and a writing part 24;

[0243] The generating part 23 is configured to generate a frame-level model selection consistency identifier of the current image frame based on the filtering model selection type; wherein different video components have their own corresponding frame-level model selection consistency identifiers;

[0244] The writing part 24 is configured to write the frame-level model selection consistency flag into the bitstream.

[0245] In some embodiments of the present application, the writing part 24 is also configured to write the filtering model index information of the second candidate filtering model into the bitstream if the filtering model selection type is a rate-distortion cost selection type; wherein different video components have their own corresponding filtering model index information of the second candidate filtering model.

[0246] In some embodiments of the present application, the model determination part 20 is further configured to obtain a model adaptive selection identifier; if the model adaptive selection identifier indicates that it is allowed, the preset selection network model is used to perform filtering model estimation on multiple candidate filtering models of the current block of the current image frame to be filtered, and the first candidate filtering model is determined; and the multiple candidate filtering models are used to perform rate-distortion cost estimation on the current block of the current image frame to be filtered, and the second candidate filtering model is determined from the multiple candidate filtering models.

[0247] It is understandable that during the encoding process, the encoder will select the model based on the preset network type (using a deep learning-based model to select the CNNLF model) and the rate-distortion cost selection type (rate-distortion proxy selection). After comparing the model accuracy, the frame-level model selection method is selected based on the model accuracy, and the filter model is obtained from the two aspects of accuracy and transmission bit reduction. Therefore, when the encoder uses this filter model for filtering, it can adaptively select the filter model, while ensuring the accuracy of encoding and decoding, reducing the transmission bits and improving the coding efficiency.

[0248] In practical applications, such as Figure 17 As shown, the embodiment of the present application further provides an encoder, including:

[0249] a second memory 25 and a second processor 26;

[0250] The second memory 25 stores a computer program that can be run on the second processor 26. When the second processor 26 executes the program, the encoder corresponds to the illumination compensation method.

[0251] An embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a first processor, the filtering method corresponding to the decoder of the claim is implemented; or, when the computer program is executed by a second processor, the filtering method corresponding to the encoder of the claim is implemented.

[0252] The various components in the embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or in the form of software functional modules.

[0253] If the integrated unit is implemented in the form of a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the method described in this embodiment. The aforementioned computer-readable storage medium includes: ferromagnetic random access memory (FRAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, magnetic surface storage, optical disk, or compact disc read-only memory (CD-ROM), etc. Various media that can store program codes are not limited in the embodiments of the present disclosure.

[0254] The above is merely an embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0255] Industrial Applicability

[0256] The embodiments of the present application provide a filtering method, an encoder, a decoder, and a computer-readable storage medium. The method comprises: obtaining a model adaptive selection flag by parsing a bitstream; obtaining a frame-level model selection consistency flag of a current image frame if the model adaptive selection flag indicates that the flag allows the selection; determining a filtering model selection type based on the frame-level model selection consistency flag; the filtering model selection type including any one of a preset selection network type and a rate-distortion cost selection type; and determining a filtering model for a current block based on the filtering model selection type.

[0257] The current block is filtered using a filter model to obtain a filtered first image. Using the above technical implementation scheme, since the decoder can determine the filter model selection type by parsing the bitstream, it can determine which method to use for adaptive selection of the filter model based on the filter model selection type. The filter model selection type is selected by taking into account both a preset selection network type with low transmission volume (using a deep learning-based model to select the CNNLF model) and an accurate rate-distortion cost selection type (rate-distortion proxy selection). This can achieve the goal of reducing transmission bits and improving decoding efficiency while ensuring encoding and decoding accuracy.

[0258] During the encoding process, the encoder selects models based on two methods: a preset network type (using a deep learning-based model to select the CNNLF model) and a rate-distortion cost selection type (rate-distortion proxy selection). The encoder then compares the model accuracy and selects a frame-level model based on the model accuracy. The filter model is then derived from a comprehensive analysis of accuracy and the number of transmitted bits. Therefore, when the encoder uses this filter model for filtering, it can adaptively select the filter model, reducing transmitted bits while ensuring codec accuracy and improving coding efficiency.

Claims

1. A filtering method, applied to a decoder, comprising: Parse the code stream and obtain the model adaptation selection flag; If the model adaptive selection identifier representation allows, obtaining a frame-level model selection consistency identifier for the current image frame; The frame-level model selection consistency flag represents the flag corresponding to the accuracy of the model selected by different model selection methods when decoding the current image frame; Determining a filter model selection type according to the frame-level model selection consistency flag; The filtering model selection type includes any one of a preset selection network type and a rate-distortion cost selection type; Determining a filtering model for a current block based on the filtering model selection type; The current block is filtered using the filtering model to obtain a filtered first image.

2. The method according to claim 1, wherein The determining the filtering model of the current block based on the filtering model selection type includes: If the filtering model selection type is the preset selection network type, the preset selection network model is used to perform filtering model estimation on multiple candidate filtering models of the current block, and a first candidate filtering model is determined, and the first candidate filtering model is used as the filtering model.

3. The method according to claim 1, wherein The determining the filtering model of the current block based on the filtering model selection type includes: If the filtering model selection type is the rate-distortion cost selection type, obtaining filtering model index information; According to the filtering model index information, a second candidate filtering model is obtained by matching from multiple candidate filtering models, and the second candidate filtering model is used as the filtering model.

4. The method according to any one of claims 1 to 3, wherein: The obtaining of a frame-level model selection consistency identifier of the current image frame includes: Obtain a frame-level model selection consistency identifier of a video component of a current image frame; wherein different video components have respective corresponding frame-level model selection consistency identifiers.

5. The method according to claim 4, wherein The determining the filtering model selection type according to the frame-level model selection consistency flag includes: The filtering model selection type of the video component is determined according to the frame-level model selection consistency identifier of the video component.

6. The method according to claim 5, wherein: The determining the filtering model of the current block based on the filtering model selection type includes: A filtering model for the video component of the current block is determined based on the filtering model selection type for the video component.

7. The method according to claim 6, wherein: The current block includes: a plurality of video component blocks; The filtering model is used to filter the current block to obtain a filtered first image, including: Each video component block of the current block is filtered using the filtering model of the video component to obtain the first filtered image.

8. The method according to claim 2, wherein: The method of using a preset selection network model to perform filter model estimation on multiple candidate filter models of the current block to determine a first candidate filter model includes: Using the preset selection network model, performing filter model estimation on multiple candidate filter models of the video component block of the current block, and determining multiple model probabilities corresponding to the multiple candidate filter models one by one; The model corresponding to the maximum model probability among the multiple model probabilities is determined as the first candidate filtering model.

9. A filtering method, applied to an encoder, comprising: Using a preset selection network model, a plurality of candidate filtering models of a current block of a current image frame to be filtered are estimated to determine a first candidate filtering model; and performing rate-distortion cost estimation on a current block of a current image frame to be filtered using a plurality of candidate filtering models, and determining a second candidate filtering model from the plurality of candidate filtering models; Determining a model accuracy of a current block based on the first candidate filtering model and the second candidate filtering model; Continue to estimate the filter model and rate-distortion cost of the next block until the last block of the current image frame is traversed, and combine the model accuracy of the current block to obtain the total model accuracy of the last block; Determining a filter model selection type for the current image frame based on the statistical number of blocks of the current image frame and the total model accuracy; Determine a filtering model from the first candidate filtering model and the second candidate filtering model according to the filtering model selection type, and perform filtering processing on the current image frame to be filtered using the filtering model to obtain a filtered second image; Based on the filtering model selection type, a frame-level model selection consistency identifier for the current image frame is generated; wherein different video components have corresponding frame-level model selection consistency identifiers; the frame-level model selection consistency identifier represents an identifier corresponding to the accuracy of the models selected by different model selection methods when encoding the current image frame; The frame-level model selection consistency flag is written into the bitstream.

10. The method according to claim 9, wherein: The method of using a plurality of candidate filtering models to perform rate-distortion cost estimation on a current block of a current image frame to be filtered, and determining a second candidate filtering model from the plurality of candidate filtering models, comprises: Using the multiple candidate filtering models, estimating the rate-distortion cost of the current block of the current image frame to be filtered to obtain multiple rate-distortion costs; Based on the multiple rate-distortion costs, the second candidate filtering model with the minimum rate-distortion cost is determined from the multiple candidate filtering models.

11. The method according to claim 9, wherein: The determining, based on the first candidate filtering model and the second candidate filtering model, a model accuracy of the current block includes: If the first candidate filtering model and the second candidate filtering model are consistent, then adding a first preset value to the previous model accuracy to obtain the model accuracy of the current block; wherein the initial model accuracy is the second preset value; If the first candidate filtering model and the second candidate filtering model are inconsistent, the previous model accuracy is used as the model accuracy of the current block.

12. The method according to claim 11, wherein The filter model estimation and rate-distortion cost estimation of the next block are continued until the last block of the current image frame is traversed, and the total model accuracy of the last block is obtained by combining the model accuracy of the current block, including: Continue to estimate the filter model and rate-distortion cost of the next block, and combine the model accuracy of the current block to obtain the model accuracy of the next block, until the last block of the current image frame is traversed, and the total model accuracy of the last block is obtained.

13. The method according to any one of claims 9 to 12, wherein: The current block of the current image frame to be filtered includes: a video component block; the multiple candidate filtering models include: multiple video component candidate filtering models corresponding to the video components; The method of using a preset selection network model to pre-estimate a plurality of candidate filtering models of a current block of a current image frame to be filtered and determining a first candidate filtering model includes: A preset selection network model is used to perform filter model pre-estimation on the multiple video component candidate filter models of the video component block to determine the first candidate filter model corresponding to the video component.

14. The method according to claim 13, wherein The method of using a plurality of candidate filtering models to perform rate-distortion cost estimation on a current block of a current image frame to be filtered, and determining a second candidate filtering model from the plurality of candidate filtering models, comprises: The multiple video component candidate filter models are used to perform rate-distortion cost estimation on the video component block to obtain the second candidate filter model corresponding to the video component block.

15. The method according to claim 14, wherein The determining, based on the first candidate filtering model and the second candidate filtering model, a model accuracy of the current block includes: Based on the first candidate filtering model and the second candidate filtering model of the video component, a model accuracy of the video component block in the current block is determined.

16. The method according to claim 13, wherein: The determining of the filter model selection type for the current image frame based on the number of blocks of the current image frame and the total model accuracy includes: Based on the statistical number of video component blocks in the current image frame and the total model accuracy of the video components, the filter model selection type of the current image frame is determined.

17. The method according to claim 16, wherein The determining of the filtering model selection type for the current image frame based on the number of video component blocks in the current image frame and the total model accuracy of the video components includes: determining a first ratio of a total model accuracy of a video component to a number of corresponding video component blocks; If the first ratio is greater than a preset accuracy threshold, the filter model selection type of the video component of the current image frame is a preset selection network type; If the first ratio is less than or equal to a preset accuracy threshold, the filtering model selection type of the video component of the current image frame is a rate-distortion cost selection type.

18. The method according to claim 17, wherein The step of selecting a filtering model according to the filtering model type, determining a filtering model from the first candidate filtering model and the second candidate filtering model, and performing filtering processing on the current image frame to be filtered using the filtering model to obtain a filtered second image, comprising: Determining a filtering model for the video component from the first candidate filtering model and the second candidate filtering model of the video component according to the filtering model selection type of the video component; The filtering model of the video component is used to filter the video component block of the current block of the current image frame to be filtered, determine the filtered current block, and continue filtering the next block of the current image frame to obtain the filtered second image.

19. The method according to claim 18, wherein The step of determining the filtering model of the video component from the first candidate filtering model and the second candidate filtering model of the video component according to the filtering model selection type of the video component includes: If the filter model selection type is a preset selection network type, determining the first candidate filter model as the filter model of the video component; If the filtering model selection type is a rate-distortion cost selection type, the second candidate filtering model is determined as the filtering model of the video component.

20. The method according to claim 9, wherein The method further comprises: If the filtering model selection type is a rate-distortion cost selection type, the filtering model index information of the second candidate filtering model is written into the bitstream; wherein different video components have their own corresponding filtering model index information of the second candidate filtering model.

21. The method according to claim 9, wherein The preset selection network model is used to pre-estimate the filter model of multiple candidate filter models of the current block of the current image frame to be filtered, and a first candidate filter model is determined; and performing rate-distortion cost estimation on a current block of a current image frame to be filtered using a plurality of candidate filtering models, and determining a second candidate filtering model from the plurality of candidate filtering models, comprising: Get the model adaptive selection flag; If the model adaptive selection identification representation allows, the preset selection network model is used to perform filtering model estimation on multiple candidate filtering models of the current block of the current image frame to be filtered, and the first candidate filtering model is determined; and the multiple candidate filtering models are used to perform rate-distortion cost estimation on the current block of the current image frame to be filtered, and the second candidate filtering model is determined from the multiple candidate filtering models.

22. A decoder comprising: The parsing part is configured to parse the code stream and obtain the model adaptation selection flag; The first acquisition part is configured to obtain a frame-level model selection consistency flag of the current image frame if the model adaptive selection flag indicates that it is allowed; the frame-level model selection consistency flag indicates the flag corresponding to the accuracy of the model selected by different model selection methods when decoding the current image frame; A first determining part is configured to determine a filtering model selection type according to the frame-level model selection consistency flag; and determining a filtering model for a current block based on the filtering model selection type; The filtering model selection type includes any one of a preset selection network type and a rate-distortion cost selection type; The first filtering part is configured to use the filtering model to filter the current block to obtain a filtered first image.

23. An encoder comprising: The model determination part is configured to use a preset selection network model to perform filter model estimation on multiple candidate filter models for the current block of the current image frame to be filtered, and determine a first candidate filter model; and use the multiple candidate filter models to perform rate-distortion cost estimation on the current block of the current image frame to be filtered, and determine a second candidate filter model from the multiple candidate filter models; A first determining part is configured to determine a model accuracy of a current block based on the first candidate filtering model and the second candidate filtering model; Continue to estimate the filter model and rate-distortion cost of the next block until the last block of the current image frame is traversed, and combine the model accuracy of the current block to obtain the total model accuracy of the last block; and determining a filter model selection type for the current image frame based on the statistical number of blocks of the current image frame and the total model accuracy; The model determination part is further configured to determine a filtering model from the first candidate filtering model and the second candidate filtering model according to the filtering model selection type; A second filtering part is configured to perform filtering processing on the current image frame to be filtered using the filtering model to obtain a filtered second image; A generating part is configured to generate a frame-level model selection consistency identifier of the current image frame based on the filtering model selection type; wherein different video components have their own corresponding frame-level model selection consistency identifiers; The writing part is configured to write the frame-level model selection consistency identifier into the code stream.

24. A decoder comprising: a first memory and a first processor; The first memory stores a computer program that can be run on the first processor, and the first processor implements the method according to any one of claims 1 to 8 when executing the program.

25. An encoder comprising: a second memory and a second processor; The second memory stores a computer program that can be run on the second processor, and when the second processor executes the program, the method according to any one of claims 9 to 21 is implemented.

26. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a first processor, the method described in any one of claims 1 to 8 is implemented; or when the computer program is executed by a second processor, the method described in any one of claims 9 to 21 is implemented.

Images