Coding method, code stream, encoder, decoder and storage medium

Abstract

The embodiment of the application discloses a coding and decoding method, a code stream, an encoder, a decoder and a storage medium. The method comprises the following steps: analyzing a code stream, determining the value of at least one syntax element identification information; when the at least one syntax element identification information indicates that the current frame or the current block uses a weight matrix for filtering processing, determining the weight matrix network model of the current block, and determining the weight matrix of the current block according to the weight matrix network model; and determining the target reconstructed image block of the current block by using the weight matrix. In this way, the weight matrix technology based on deep learning can be realized by using the weight matrix network model, the pixel-level weighting processing can be provided for the use of the output reconstructed image block, the coding performance is improved, and the coding and decoding efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an encoding / decoding method, a bitstream, an encoder, a decoder, and a storage medium. Background Technology

[0002] In video encoding and decoding systems, loop filters are used to improve the subjective and objective quality of reconstructed images. Traditional loop filters mainly include deblocking filters, sample adaptive compensation filters, and adaptive correction filters. In the High Performance-Modular Artificial Intelligence Model (HPM-ModAI) of the 3rd Audio Video coding Standard (AVS3), a loop filter based on a residual neural network (CNNLF) is also used as the baseline scheme for the intelligent loop filtering module, positioned between the sample adaptive compensation filter and the adaptive correction filter.

[0003] In the current HPM-ModAI of AVS3, when the rate-distortion cost method determines that the current block should use CNNLF, the reconstructed pixel values ​​of the current block will be updated to the pixel values ​​processed by CNNLF. However, a reduction in the overall distortion value of the block does not mean that the distortion of each pixel value in the block will decrease, thus limiting the improvement in encoding performance. Summary of the Invention

[0004] This application provides an encoding / decoding method, a bitstream, an encoder, a decoder, and a storage medium, which can improve encoding performance and thus increase encoding / decoding efficiency.

[0005] The technical solution of this application embodiment can be implemented as follows:

[0006] In a first aspect, embodiments of this application provide a decoding method applied to a decoder, the method comprising:

[0007] Parse the bitstream to determine the value of at least one syntax element identifier.

[0008] When at least one syntax element identifier indicates that the current frame or current block is filtered using a weight matrix, the weight matrix network model of the current block is determined, and the weight matrix of the current block is determined based on the weight matrix network model.

[0009] The target image block for the current block is determined using the weight matrix.

[0010] Secondly, embodiments of this application provide an encoding method applied to an encoder, the method comprising:

[0011] Determine the weight matrix network model for the current block, and determine the weight matrix for the current block based on the weight matrix network model;

[0012] Determine the value of at least one syntax element identification information;

[0013] When at least one syntax element identifies that the current frame or current block is to be filtered using a weight matrix, the target image block to be reconstructed for the current block is determined using the weight matrix.

[0014] Thirdly, embodiments of this application provide a bitstream that is generated by bit encoding based on the value of at least one syntax element identifier information;

[0015] Wherein, at least one syntax element identification information includes at least: first syntax element identification information, first luminance syntax element identification information, second luminance syntax element identification information, and chrominance syntax element identification information;

[0016] The first syntax element identifier is used to indicate whether the video sequence is filtered using a weight matrix; the first luminance syntax element identifier is used to indicate whether the luminance component of the current frame is filtered using a weight matrix; the second luminance syntax element identifier is used to indicate whether the luminance component of the current block is filtered using a weight matrix; and the chroma syntax element identifier is used to indicate whether the chroma component of the current frame is filtered using a weight matrix. The video sequence includes the current frame, and the current frame includes the current block.

[0017] Fourthly, embodiments of this application provide an encoder, which includes a first determining unit and a first filtering unit; wherein,

[0018] The first determining unit is configured to determine the weight matrix network model of the current block, and determine the weight matrix of the current block based on the weight matrix network model; and is further configured to determine the value of at least one syntax element identification information.

[0019] The first filtering unit is configured to determine the target reconstructed image block of the current block using the weight matrix when at least one syntax element identification information indicates that the current frame or the current block is filtered using the weight matrix.

[0020] Fifthly, embodiments of this application provide an encoder, which includes a first memory and a first processor; wherein,

[0021] A first memory for storing computer programs that can run on a first processor;

[0022] A first processor is configured to execute the method described in the second aspect when running the computer program.

[0023] Sixthly, embodiments of this application provide a decoder, which includes a parsing unit, a second determining unit, and a second filtering unit; wherein,

[0024] The parsing unit is configured to parse the code stream and determine the value of at least one syntax element identification information.

[0025] The second determining unit is configured to determine the weight matrix network model of the current block and determine the weight matrix of the current block based on the weight matrix network model when at least one syntax element identification information indicates that the current frame or the current block is filtered using a weight matrix.

[0026] The second filtering unit is configured to use a weight matrix to determine the target image block to be reconstructed for the current block.

[0027] In a seventh aspect, embodiments of this application provide a decoder, which includes a second memory and a second processor; wherein,

[0028] The second memory is used to store computer programs that can run on the second processor;

[0029] A second processor is configured to execute the method described in the first aspect when running the computer program.

[0030] Eighthly, embodiments of this application provide a computer storage medium storing a computer program that, when executed, implements the method described in the first aspect or the method described in the second aspect.

[0031] This application provides an encoding / decoding method, a bitstream, an encoder, a decoder, and a storage medium. On the encoder side, a weight matrix network model for the current block is determined, and the weight matrix for the current block is determined based on the weight matrix network model. The value of at least one syntax element identifier is determined. When the at least one syntax element identifier indicates that the current frame or current block uses the weight matrix for filtering, the target reconstructed image block for the current block is determined using the weight matrix. On the decoder side, the value of at least one syntax element identifier is determined by parsing the bitstream. When the at least one syntax element identifier indicates that the current frame or current block uses the weight matrix for filtering, a weight matrix network model for the current block is determined, and the weight matrix for the current block is determined based on the weight matrix network model. The target reconstructed image block for the current block is determined using the weight matrix. Thus, by using a weight matrix network model to determine the weight matrix, not only can deep learning-based weight matrix technology be implemented, providing pixel-level weighting for the use of the output reconstructed image block, but encoding performance can also be improved, thereby increasing encoding / decoding efficiency. Furthermore, this application can make the output of the neural network loop filter closer to the original image, improving video image quality. Attached Figure Description

[0032] Figure 1 An application diagram illustrating a coding framework provided for related technologies;

[0033] Figure 2 An application diagram illustrating another coding framework provided for related technologies;

[0034] Figure 3A A detailed framework diagram of a video encoding system provided in this application embodiment;

[0035] Figure 3B A detailed framework diagram of a video decoding system provided in this application embodiment;

[0036] Figure 4 A flowchart illustrating a decoding method provided in an embodiment of this application;

[0037] Figure 5A This is a schematic diagram of the network structure composition of the luminance component provided in an embodiment of this application;

[0038] Figure 5B A schematic diagram of the network structure composition of chromaticity components provided in an embodiment of this application;

[0039] Figure 6 A schematic diagram of the network structure of a residual block provided in an embodiment of this application;

[0040] Figure 7This is an application diagram of an encoding framework provided in an embodiment of this application;

[0041] Figure 8 A schematic diagram illustrating the composition structure of a weight matrix network model provided in an embodiment of this application;

[0042] Figure 9 A schematic diagram of the overall framework of a weight matrix network model provided in this application embodiment;

[0043] Figure 10 A flowchart illustrating an encoding method provided in an embodiment of this application;

[0044] Figure 11 A schematic diagram of the composition structure of an encoder provided in an embodiment of this application;

[0045] Figure 12 This is a schematic diagram of the specific hardware structure of an encoder provided in an embodiment of this application;

[0046] Figure 13 A schematic diagram of the composition structure of a decoder provided in an embodiment of this application;

[0047] Figure 14 This is a schematic diagram of the specific hardware structure of a decoder provided in an embodiment of this application. Detailed Implementation

[0048] In order to gain a more detailed understanding of the features and technical content of the embodiments of this application, the implementation of the embodiments of this application will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for reference and illustration only and are not intended to limit the embodiments of this application.

[0049] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0050] In the following description, references to "some embodiments" refer to a subset of all possible embodiments. It is understood that "some embodiments" may be the same or different subsets of all possible embodiments and may be combined with each other without conflict. It should also be noted that the terms "first, second, third" used in the embodiments of this application are merely for distinguishing similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0051] In video images, a coding block (CB) is generally represented by a first image component, a second image component, and a third image component. These three image components are a luminance component, a blue chrominance component, and a red chrominance component, respectively. Specifically, the luminance component is usually represented by the symbol Y, the blue chrominance component is usually represented by the symbol Cb or U, and the red chrominance component is usually represented by the symbol Cr or V. Thus, video images can be represented in YCbCr format or YUV format.

[0052] Before providing a further detailed description of the embodiments of this application, the nouns and terms used in the embodiments of this application will be explained. The nouns and terms used in the embodiments of this application shall be interpreted as follows:

[0053] Moving Picture Experts Group (MPEG)

[0054] International Organization for Standardization (ISO)

[0055] International Electrotechnical Commission (IEC)

[0056] Joint Video Experts Team (JVET)

[0057] Alliance for Open Media (AOM)

[0058] The next-generation video coding standard H.266 / Versatile Video Coding (VVC)

[0059] VVC Reference Software Testing Platform (VVC Test Model, VTM)

[0060] Audio Video Coding Standard (AVS)

[0061] AVS High-Performance Model (HPM)

[0062] AVS's High Performance-Modular Artificial Intelligence Model (HPM-ModAI)

[0063] Convolutional Neural Network Based In-Loop Filter (CNNLF)

[0064] Deblocking Filter (DBF)

[0065] Sample adaptive offset (SAO)

[0066] Adaptive Loop Filter (ALF)

[0067] Quantization Parameter (QP)

[0068] Coding Unit (CU)

[0069] Coding Tree Unit (CTU)

[0070] It's understandable that digital video compression technology primarily compresses massive amounts of digital video data to facilitate transmission and storage. With the surge in internet video and increasing demands for video clarity, while existing digital video compression standards can save considerable video data, there is still a need to pursue better digital video compression technologies to reduce the bandwidth and traffic pressure on digital video transmission.

[0071] In digital video encoding, the encoder reads unequal amounts of pixels from raw video sequences of different color formats, including luminance and chrominance components; that is, the encoder reads a black-and-white or color image. This image is then divided into blocks, and the block data is passed to the encoder for encoding. Modern encoders typically use a hybrid frame coding mode, generally including operations such as intra-frame prediction and inter-frame prediction, transform / quantization, inverse quantization / inverse transform, loop filtering, and entropy coding. For a detailed processing flow, please refer to [reference needed]. Figure 1As shown in the diagram, intra-frame prediction only references information from the same frame to predict pixel information within the current block, thus eliminating spatial redundancy. Inter-frame prediction can include motion estimation and motion compensation, referencing image information from different frames and using motion estimation to search for the motion vector information that best matches the current block, thus eliminating temporal redundancy. Transform converts the predicted image block to the frequency domain, redistributing energy, and combined with quantization, removes information that is insensitive to the human eye, thus eliminating visual redundancy. Entropy coding can eliminate character redundancy based on the current context model and the probability information of the binary code stream. Loop filtering mainly processes the pixels after inverse transform and inverse quantization to compensate for distortion information and provide a better reference for subsequent encoded pixels.

[0072] For AVS3, the traditional loop filtering module mainly includes a deblocking filter (DBF), a sample adaptive compensation filter (SAO), and an adaptive correction filter (ALF). In the HPM-ModAI application, a loop filter based on a residual neural network (CNNLF) is also used as the baseline scheme for the intelligent loop filtering module, positioned between the SAO and ALF filters. For details, see [link to HPM-ModAI application]. Figure 2 As shown. During encoding testing, following the general test conditions for intelligent encoding, for the All Intra configuration, ALF is enabled, and DBF and SAO are disabled; for the Random Access and Low Delay configurations, DBF for I-frames is enabled, ALF is enabled, and SAO is disabled.

[0073] In the current HPM-ModAI of AVS3, when using CNNLF is determined by the rate-distortion cost method, the reconstructed pixel values ​​of the current CTU will be updated to the pixel values ​​processed by CNNLF. Enabling CNNLF at this point indicates that the mean squared error of the current CTU after CNNLF processing will be smaller compared to the original CTU before CNNLF processing; however, a reduction in the overall distortion of the CTU does not necessarily mean that the distortion of each pixel value within that CTU will decrease. Current solutions lack the ability to optimally select each pixel value in the CNNLF-processed CTU, thus limiting the improvement in encoding performance.

[0074] This application provides an encoding method in which, on the encoder side, a weight matrix network model of the current block is determined, and a weight matrix of the current block is determined based on the weight matrix network model; the value of at least one syntax element identification information is determined; when at least one syntax element identification information indicates that the current frame or the current block is filtered using the weight matrix, the target reconstructed image block of the current block is determined using the weight matrix.

[0075] This application provides a decoding method. On the decoder side, the bitstream is parsed to determine the value of at least one syntax element identification information. When the at least one syntax element identification information indicates that the current frame or the current block is filtered using a weight matrix, the weight matrix network model of the current block is determined, and the weight matrix of the current block is determined based on the weight matrix network model. Using the weight matrix, the target reconstructed image block of the current block is determined.

[0076] In this way, by using a weight matrix network model to determine the weight matrix, not only can deep learning-based weight matrix technology be implemented, providing pixel-level weighting processing for the use of output reconstructed image patches, but coding performance can also be improved, thereby increasing encoding and decoding efficiency; in addition, this application can also make the output of the neural network loop filter closer to the original image, thereby improving video image quality.

[0077] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0078] See Figure 3A This illustrates a detailed framework diagram of a video encoding system provided in an embodiment of this application. For example... Figure 3AAs shown, the video coding system 10 includes a transform and quantization unit 101, an intra-frame estimation unit 102, an intra-frame prediction unit 103, a motion compensation unit 104, a motion estimation unit 105, an inverse transform and inverse quantization unit 106, a filter control and analysis unit 107, a filtering unit 108, an encoding unit 109, and a decoded image buffer unit 110. The filtering unit 108 can implement DBF filtering / SAO filtering / ALF filtering, and the encoding unit 109 can implement header information encoding and context-based adaptive binary arithmetic coding (CABAC). For the input raw video signal, the coding tree unit... The partitioning of a video coding unit (CTU) yields a video coding block. The residual pixel information obtained after intra-frame or inter-frame prediction is then transformed by the transform and quantization unit 101. This transformation involves converting the residual information from the pixel domain to the transform domain and quantizing the resulting transform coefficients to further reduce the bit rate. Intra-frame estimation unit 102 and intra-frame prediction unit 103 perform intra-frame prediction on the video coding block. Specifically, intra-frame estimation unit 102 and intra-frame prediction unit 103 determine the intra-frame prediction mode to be used to encode the video coding block. Motion compensation unit 104 and motion estimation unit 105 perform inter-frame prediction coding of the received video coding block relative to one or more blocks in one or more reference frames to provide temporal prediction information. Motion estimation performed by motion estimation unit 105 is the process of generating motion vectors, which can estimate the motion of the video coding block. Then, motion compensation unit 104 uses the motion vectors determined by motion estimation unit 105 to generate motion vectors. The motion compensation is performed. After determining the intra-prediction mode, the intra-prediction unit 103 is also used to provide the selected intra-prediction data to the coding unit 109, and the motion estimation unit 105 also sends the calculated motion vector data to the coding unit 109. In addition, the inverse transform and inverse quantization unit 106 is used to reconstruct the video coding block, reconstruct the residual block in the pixel domain, and remove the block artifacts by the filter control analysis unit 107 and the filtering unit 108. Then, the reconstructed residual block is added to a predictive block in the frame of the decoding image buffer unit 110 to generate the reconstructed video coding block. The coding unit 109 is used to encode various coding parameters and quantized transform coefficients. In the CABAC-based coding algorithm, the context content can be based on adjacent coding blocks and can be used to encode information indicating the determined intra-prediction mode and output the bitstream of the video signal. The decoding image buffer unit 110 is used to store the reconstructed video coding block for prediction reference.As video image encoding proceeds, new reconstructed video encoding blocks are continuously generated, and these reconstructed video encoding blocks are stored in the decoding image buffer unit 110.

[0079] See Figure 3B This illustrates a detailed framework diagram of a video decoding system provided in an embodiment of this application. Figure 3B As shown, the video decoding system 20 includes a decoding unit 201, an inverse transform and inverse quantization unit 202, an intra-frame prediction unit 203, a motion compensation unit 204, a filtering unit 205, and a decoding image buffer unit 206. The decoding unit 201 can perform header information decoding and CABAC decoding, and the filtering unit 205 can perform DBF filtering / SAO filtering / ALF filtering. The input video signal is processed... Figure 3A After encoding, the video signal bitstream is output. This bitstream is input into the video decoding system 20, first passing through the decoding unit 201 to obtain the decoded transform coefficients. The transform coefficients are then processed by the inverse transform and inverse quantization unit 202 to generate residual blocks in the pixel domain. The intra-frame prediction unit 203 can generate prediction data for the current video decoding block based on the determined intra-frame prediction mode and data from previously decoded blocks in the current frame or image. The motion compensation unit 204 determines the prediction information for the video decoding block by analyzing motion vectors and other associated syntax elements, and uses this prediction information. The predictive block of the video block being decoded is generated; the decoded video block is formed by summing the residual block from the inverse transform and inverse quantization unit 202 with the corresponding predictive block generated by the intra-prediction unit 203 or the motion compensation unit 204; the decoded video signal is passed through the filtering unit 205 to remove block artifacts, which can improve video quality; then the decoded video block is stored in the decoding image buffer unit 206, which stores reference images for subsequent intra-prediction or motion compensation, and is also used for the output of the video signal, thus obtaining the recovered original video signal.

[0080] It should be noted that the method provided in the embodiments of this application can be applied to, for example, Figure 3A The filter unit 108 shown (represented by a bold black box) can also be applied to, for example... Figure 3B The filter unit 205 shown is indicated by a bold black box. In other words, the method in this embodiment can be applied to a video encoding system (referred to as an "encoder"), a video decoding system (referred to as a "decoder"), or even simultaneously, but no limitations are imposed here.

[0081] It should also be noted that when the embodiments of this application are applied to an encoder, the "current block" specifically refers to the block to be encoded in the video image (which can also be simply referred to as the "encoded block"); when the embodiments of this application are applied to a decoder, the "current block" specifically refers to the block to be decoded in the video image (which can also be simply referred to as the "decoded block").

[0082] In one embodiment of this application, see Figure 4 The diagram illustrates a flowchart of a decoding method provided in an embodiment of this application. Figure 4 As shown, the method may include:

[0083] S401: Parse the code stream and determine the value of at least one syntax element identification information.

[0084] It should be noted that a video image can be divided into multiple image blocks, and each image block to be decoded can be called a decoding block. Here, each decoding block may include a first image component, a second image component, and a third image component; while the current block is the decoding block in the video image that is currently undergoing loop filtering processing on the first image component, the second image component, or the third image component.

[0085] Here, from a color classification perspective, the first image component, the second image component, and the third image component can be divided into two color component types: luminance component and chrominance component. In this case, if the current block performs operations such as luminance component prediction, inverse transform and inverse quantization, and loop filtering, then the current block can also be called a luminance block; or, if the current block performs operations such as chrominance component prediction, inverse transform and inverse quantization, and loop filtering, then the current block can also be called a chrominance block.

[0086] It should also be noted that, on the decoder side, this application embodiment specifically provides a loop filtering method, which is applied to applications such as... Figure 3B The filtering unit 205 shown is an example. Here, the filtering unit 205 may include a deblocking filter (LBF), a sample adaptive compensation filter (SAO), a loop filter based on a residual neural network (CNNLF), and an adaptive correction filter (ALF). The loop filtering method described in this embodiment is specifically applied between CNNLF and ALF so that the optimal selection can be made for each pixel value in the current block after CNNLF filtering.

[0087] Understandably, in a specific example, CNNLF designed different network structures for the luminance and chrominance components, as detailed in [link to example]. Figure 5A and Figure 5B .

[0088] For the luminance component, such as Figure 5AAs shown, the entire network structure can be composed of convolutional layers, activation layers, residual blocks, and jump connection layers. Here, the convolutional kernels of the convolutional layers can be 3×3, which can be represented by 3×3Conv; the activation layers can use linear activation functions, which can be represented by the rectified linear unit (ReLU), also known as the corrected linear unit, a commonly used activation function in artificial neural networks, usually referring to nonlinear functions represented by the ramp function and its variants. The network structure of the residual block (ResBlock) is as follows... Figure 6 As shown in the dashed box, a network can consist of convolutional layers (Conv), activation layers (ReLU), and jump connection layers. In the network structure, a jump connection layer (Concat) refers to a global jump connection from the input to the output, which enables the network to focus on learning residuals and accelerates the network's convergence process.

[0089] For chromaticity components, such as Figure 5B As shown, the luminance component is introduced as one of the inputs to guide the filtering of the chrominance component. The entire network structure can be composed of convolutional layers, activation layers, residual blocks, pooling layers, and jump connection layers. Due to the inconsistency in resolution, the chrominance component first needs to be upsampled. To avoid introducing other noise during the upsampling process, the resolution can be enlarged by directly copying neighboring pixels to obtain an enlarged chroma frame. In addition, at the end of the network structure, a pooling layer (such as 2×2 AvgPool) is used to downsample the chrominance component. Specifically, in the application of HPM-ModAI, the number of residual blocks in the luminance component network can be set to N=20, and the number of residual blocks in the chrominance component network can be set to N=10.

[0090] Here, the use of CNNLF can include two phases: offline training and inference testing. In the offline training phase, 16 models can be trained offline, including 4 I-frame luma component models, 4 non-I-frame luma component models, 4 chroma U component models, and 4 chroma V component models. Specifically, a pre-defined image dataset (e.g., DIV2K, which contains 1000 high-resolution images (2K resolution), with 800 used for training, 100 for validation, and 100 for testing) is used. The images are converted from RGB to YUV4:2:0 format single-frame video sequences as label data. Then, HPM is used to encode the sequences in All Intra configuration, disabling traditional filters such as DBF, SAO, and ALF, and setting the quantization step size to 27 to 50. The reconstructed sequence obtained from the encoding was divided into four intervals according to QP ranges of 27–31, 32–37, 38–44, and 45–50. These intervals were then cut into 128×128 image blocks as training data, and four I-frame luma component models, four chroma U component models, and four chroma V component models were trained respectively. Furthermore, using a pre-defined video dataset (e.g., BVI-DVC), HPM-ModAI was used for encoding under Random Access configuration. Traditional filters such as DBF, SAO, and ALF were disabled, and CNNLF for I-frames was enabled. Non-I-frame data from the encoded reconstruction was collected, and four non-I-frame luma component models were trained respectively.

[0091] During the inference testing phase, HPM-ModAI sets on / off frame-level and CTU-level flags for the luminance component to control whether CNNLF is enabled, and sets on / off frame-level flags for the chrominance component to control whether CNNLF is enabled. Here, the flags are typically represented by the symbol "flag". Furthermore, the frame-level flags are determined by equation (1), where D = D net -D rec This indicates the reduction in distortion (D) after CNNLF processing. net D represents the distortion after filtering. rec (where RDcost is the distortion before filtering), R represents the number of CTUs in the current frame, and λ is consistent with the λ of the adaptive correction filter. When RDcost is negative, the frame-level flag is turned on; otherwise, the frame-level flag is turned off.

[0092] RDcost=D+λ*R (1)

[0093] When the frame-level flag is enabled, it is still necessary to further determine whether each CTU should enable CNNLF using a rate-distortion cost approach. Here, a CTU-level flag is set to control whether CNNLF is enabled. Specifically, the CTU-level flag is determined by equation (2).

[0094] RDcost=D (2)

[0095] In other words, in AVS3's HPM-ModAI, when the rate-distortion cost method determines to use CNNLF, the reconstructed pixel values ​​of the current CTU will be updated to the pixel values ​​processed by CNNLF. At this point, the CNNLF flag is turned on, indicating that the mean squared error of the current CTU after CNNLF processing will be smaller compared to the original CTU before CNNLF processing. However, a reduction in the overall distortion of the CTU does not mean that the distortion of each pixel value within that CTU will decrease, thus limiting the improvement in coding performance.

[0096] To further improve encoding performance, this application provides a loop filtering method based on a deep learning weight matrix to weight the output of CNNLF, making the CNNLF output closer to the original image. Figure 7 As shown, in Figure 2 Based on the optimization, and with Figure 2 In contrast, the loop filter, in addition to DBF, SAO, CNNLF, and ALF, may also include a weighting matrix module, which is located between the CNNLF filter and the ALF filter. Furthermore, the use of the weighting matrix module does not depend on the flags of DBF, SAO, CNNLF, and ALF; it is simply positioned after CNNLF and before ALF.

[0097] Here, in order to facilitate the decoder in determining whether to use the loop filtering method of the present application embodiment (i.e. whether the weight matrix module is enabled), a first syntax element identifier can be set to indicate whether the current video sequence uses the loop filtering method of the present application embodiment.

[0098] In other words, the identification information of at least one syntax element obtained through decoding includes at least the identification information of a first syntax element. In some embodiments, parsing the bitstream to determine the value of the identification information of at least one syntax element may include:

[0099] The bitstream is parsed to determine the value of the first syntax element identifier information; wherein, the first syntax element identifier information is used to indicate whether the video sequence is filtered using a weight matrix.

[0100] Furthermore, in some embodiments, the method may further include:

[0101] If the value of the first syntax element identifier is the first value, then the first syntax element identifier indicates that the video sequence should be filtered using a weight matrix; or...

[0102] If the value of the first syntax element identifier is the second value, then it is determined that the first syntax element identifier indicates that the video sequence does not use the weight matrix for filtering.

[0103] Here, the first value and the second value are different, and the first value and the second value can be in parameter form or in numeric form. Specifically, the first syntax element identification information can be a parameter written in the profile or a value of a flag, and this application embodiment does not limit it in any way.

[0104] Taking a flag as the first syntax element identifier, the first and second values ​​can be set to 1 and 0 respectively; or, the first value can be set to true and the second value to false; or, the first value can be set to 0 and the second value to 1; or, the first value can be set to false and the second value to true. For example, generally, for a flag, the first value can be 1 and the second value can be 0, but there are no restrictions.

[0105] It should also be noted that the video sequence includes at least one frame, which may include the current frame. Here, when the video sequence is enabled to use the loop filtering method of this application embodiment, that is, when it is determined that the weight matrix module is enabled for the video sequence, this application embodiment also needs to further determine whether the current frame in the video sequence is filtered using the weight matrix, that is, it also needs to set a second syntax element identification information.

[0106] In other words, in some embodiments, parsing the code stream to determine the identification information of at least one syntax element may further include:

[0107] The bitstream is parsed to determine the value of the second syntax element identifier; the second syntax element identifier is used to indicate whether the current frame is filtered using a weight matrix.

[0108] In a specific example, parsing the code stream to determine the identification information of at least one syntax element may include:

[0109] Parse the bitstream to determine the value of the first syntax element identifier information;

[0110] When the first syntax element identifier information indicates that the video sequence is filtered using a weight matrix, the bitstream is parsed to determine the value of the second syntax element identifier information.

[0111] In other words, this at least one syntax element identification information may include at least a first syntax element identification information and a second syntax element identification information. Here, the meaning of the second syntax element identification information varies depending on the color component type of the current frame. Furthermore, different color component types have different weight matrices; for example, the weight matrix corresponding to the luminance component can be called the luminance weight matrix, and the weight matrix corresponding to the chrominance component can be called the chrominance weight matrix. Specifically, in some embodiments, the parsing of the bitstream to determine the second syntax element identification information may include:

[0112] When the color component type of the current frame is a luminance component, the second syntax element identifier is determined to be the first luminance syntax element identifier. The first luminance syntax element identifier is used to indicate whether the luminance component of the current frame is filtered using a luminance weight matrix; or...

[0113] When the color component type of the current frame is chroma component, the second syntax element identification information is determined to be chroma syntax element identification information. The chroma syntax element identification information is used to indicate whether the chroma component of the current frame is filtered using the chroma weight matrix.

[0114] In this embodiment, the color component type may include a luminance component and a chrominance component. If the color component type is a luminance component, then the second syntax element identification information can be referred to as the first luminance syntax element identification information to indicate whether the luminance component of the current frame is filtered using a luminance weight matrix. If the color component type is a luminance component, then the second syntax element identification information can be referred to as the first luminance syntax element identification information to indicate whether the luminance component of the current frame is filtered using a chrominance weight matrix.

[0115] S402: When at least one syntax element identification information indicates that the current frame or the current block is to be filtered using a weight matrix, determine the weight matrix network model of the current block, and determine the weight matrix of the current block based on the weight matrix network model.

[0116] It should be noted that the at least one syntax element identification information described in the embodiments of this application is not limited to the first syntax element identification information and the second syntax element identification information, and may even include other syntax element identification information, without any limitation here.

[0117] It should also be noted that after the current frame is divided into blocks, at least one block is obtained, and this block includes the current block. Here, if the current frame is filtered using a weight matrix, it does not mean that every block within the current frame is filtered using the weight matrix. CTU-level syntax element identification information may also be involved to determine whether the current block is filtered using the weight matrix. The following description will use the luma component and chroma component as examples to illustrate these points.

[0118] Furthermore, the weight matrix network model differs depending on the type of color component. In this embodiment, it is assumed that the weight matrix network model corresponding to the luminance component can be called the luminance weight matrix network model, and the weight matrix network model corresponding to the chrominance component can be called the chrominance weight matrix network model.

[0119] In one possible implementation, when the color component type of the current frame is a luminance component, determining the weight matrix network model of the current block when the identification information of at least one syntax element indicates that filtering processing is performed using a weight matrix may include:

[0120] When the first luminance syntax element identifier information indicates that the luminance component of the current frame is filtered using a luminance weight matrix, the bitstream is parsed to determine the value of the second luminance syntax element identifier information.

[0121] When the second luminance syntax element identifier information indicates that the luminance component of the current block is filtered using a luminance weight matrix, the luminance weight matrix network model of the current block is determined.

[0122] It should be noted that for the luma component, two types of syntax elements are involved: frame-level syntax elements and CTU-level syntax elements. The frame-level syntax element can be referred to as the first luma syntax element identifier, denoted by `luma_frame_weighting_matrix_flag`; the CTU-level syntax element can be referred to as the second luma syntax element identifier, denoted by `luma_ctu_weighting_matrix_flag`.

[0123] It should also be noted that, for the luminance component, embodiments of this application can further set a luminance frame-level flag and a luminance CTU-level flag, and determine whether to use the luminance weight matrix for filtering by controlling whether the weight matrix module is enabled. Therefore, in some embodiments, the method may further include: setting a luminance frame-level flag and a luminance CTU-level flag.

[0124] Here, the current block is located within the current frame. The luminance frame-level flag controls whether the luminance component of the current frame is filtered using the luminance weight matrix, while the luminance CTU-level flag controls whether the luminance component of the current block is filtered using the luminance weight matrix.

[0125] Furthermore, regarding the first luminance syntax element identification information, in some embodiments, the method may further include:

[0126] If the value of the first luminance syntax element identifier is the first value, then the first luminance syntax element identifier indicates that the luminance component of the current frame is filtered using a luminance weight matrix; or,

[0127] If the value of the first luminance syntax element identifier is the second value, then it is determined that the first luminance syntax element identifier indicates that the luminance component of the current frame is not filtered using the luminance weight matrix.

[0128] In some embodiments, the method may further include:

[0129] If the value of the first luma syntax element identifier is the first value, then the luma frame-level flag is enabled; or,

[0130] If the value of the first luminance syntax element identifier is the second value, then the luminance frame-level flag is turned off.

[0131] It should be noted that the first value and the second value are different, and the first value and the second value can be in parameter form or in numerical form. Specifically, both the first luminance syntax element identification information and the first luminance syntax element identification information can be parameters written in the profile or a flag value. This application embodiment does not impose any limitations on this.

[0132] In this embodiment of the application, taking the first luminance syntax element identifier information as a flag as an example, the first value can be set to 1 and the second value can be set to 0; or, the first value can be set to true and the second value can be set to false; or, the first value can be set to 0 and the second value can be set to 1; or, the first value can be set to false and the second value can be set to true. This embodiment of the application does not impose any limitations on this.

[0133] Taking a first value of 1 and a second value of 0 as an example, if the value of the first luminance syntax element identifier obtained through decoding is 1, then the luminance frame-level flag can be enabled, i.e., the frame-level weight matrix module can be enabled. In this case, it can be determined that the luminance component of the current frame will be filtered using the luminance weight matrix. Otherwise, if the value of the first luminance syntax element identifier is 0, then the luminance frame-level flag can be disabled, i.e., the frame-level weight matrix module can be disabled. In this case, it can be determined that the luminance component of the current frame will not be filtered using the luminance weight matrix. At this point, the next frame can be obtained from the video sequence, the next frame can be identified as the current frame, and then the process of parsing the bitstream and determining the value of the first luminance syntax element identifier can continue.

[0134] Furthermore, regarding the second luminance syntax element identification information, in some embodiments, the method may further include:

[0135] If the value of the second luminance syntax element identifier is the first value, then the second luminance syntax element identifier indicates that the luminance component of the current block is filtered using a luminance weight matrix; or,

[0136] If the value of the second luminance syntax element identifier is the second value, then it is determined that the second luminance syntax element identifier indicates that the luminance component of the current block is not filtered using the luminance weight matrix.

[0137] In some embodiments, the method may further include:

[0138] If the value of the second luminance syntax element identifier is the first value, then the luminance CTU level flag is enabled; or,

[0139] If the value of the second brightness syntax element identifier is the second value, then the brightness CTU level flag bit is turned off.

[0140] It should be noted that the first value and the second value are different.

[0141] In this embodiment of the application, taking the second luminance syntax element identifier information as another flag information as an example, for the first value and the second value, the first value can be set to 1 and the second value can be set to 0; or, the first value can also be set to true and the second value can also be set to false; or, the first value can also be set to 0 and the second value can also be set to 1; or, the first value can also be set to false and the second value can also be set to true. This embodiment of the application does not impose any limitations on this.

[0142] Taking a first value of 1 and a second value of 0 as an example, if the value of the first luminance syntax element identifier is 1 after decoding, and the value of the second luminance syntax element identifier is 1, then the luminance CTU-level flag can be turned on, i.e., the CTU-level weight matrix module can be turned on. At this time, it can be determined that the luminance component of the current block is filtered using the luminance weight matrix. Otherwise, if the value of the second luminance syntax element identifier is 0, then the luminance CTU-level flag can be turned off, i.e., the CTU-level weight matrix module can be turned off. At this time, it can be determined that the luminance component of the current block is not filtered using the luminance weight matrix. Then, the next block can be obtained from the current frame, the next block is determined as the current block, and then the steps of parsing the bitstream and determining the value of the second luminance syntax element identifier are continued until all blocks included in the current frame are processed. Then, the next frame is loaded and processing continues.

[0143] In another possible implementation, when the color component type of the current frame is chroma component, determining the weight matrix network model of the current block when the identification information of at least one syntax element indicates that filtering processing is performed using a weight matrix may include:

[0144] When the chroma syntax element identification information indicates that the chroma components of the current frame are filtered using the chroma weight matrix, the chroma weight matrix network model of the current block is determined.

[0145] It should be noted that for the chroma component, frame-level syntax elements are involved. These frame-level syntax elements can be referred to as chroma syntax element identifiers, denoted by `chroma_frame_weighting_matrix_flag`.

[0146] It should also be noted that, considering encoding performance and computational complexity, if the chroma syntax element identifier indicates that the chroma components of the current frame are filtered using a chroma weight matrix, then all blocks included in the current frame will be filtered using the chroma weight matrix by default; conversely, if the chroma syntax element identifier indicates that the chroma components of the current frame are not filtered using a chroma weight matrix, then all blocks included in the current frame will not be filtered using the chroma weight matrix by default. Therefore, it is no longer necessary to set CTU-level syntax elements for chroma components, and similarly, it is not necessary to set CTU-level flags. In other words, for chroma components, this embodiment of the application may only set frame-level flags. Therefore, in some embodiments, the method may further include: setting chroma frame-level flags. The chroma frame-level flags can be used to control whether the chroma components of the current frame are filtered using a chroma weight matrix.

[0147] Furthermore, regarding the chroma syntax element identification information, in some embodiments, the method may further include:

[0148] If the chroma syntax element identifier information takes the first value, then the chroma syntax element identifier information indicates that the chroma components of the current frame are filtered using the chroma weight matrix; or,

[0149] If the value of the chroma syntax element identifier is the second value, then it is determined that the chroma syntax element identifier indicates that the chroma components of the current frame are not filtered using the chroma weight matrix.

[0150] In some embodiments, the method may further include:

[0151] If the chroma syntax element identifier information takes the first value, then the chroma frame-level flag is enabled; or...

[0152] If the chroma syntax element identifier information takes the second value, then the chroma frame-level flag is turned off.

[0153] It should be noted that the first value and the second value are different, and both the first value and the second value can be in parameter form or in numerical form. Specifically, the chroma syntax element identification information can be a parameter written in the profile or a flag value; this application embodiment does not impose any limitations on this.

[0154] In this embodiment of the application, taking the chroma syntax element identifier information as another flag information as an example, for the first value and the second value, the first value can be set to 1 and the second value can be set to 0; or, the first value can also be set to true and the second value can also be set to false; or, the first value can also be set to 0 and the second value can also be set to 1; or, the first value can also be set to false and the second value can also be set to true. This embodiment of the application does not impose any limitations on this.

[0155] Taking a first value of 1 and a second value of 0 as an example, if the decoded chroma syntax element identifier is 1, then the chroma frame-level flag can be enabled, i.e., the frame-level weight matrix module can be enabled. In this case, it can be determined that the chroma components of the current frame are filtered using the chroma weight matrix, and by default, the chroma components of each block within the current frame are filtered using the chroma weight matrix. Otherwise, if the chroma syntax element identifier is 0, then the chroma frame-level flag can be disabled, i.e., the frame-level weight matrix module can be disabled. In this case, it can be determined that the chroma components of the current frame are not filtered using the chroma weight matrix. At this point, the next frame can be obtained from the video sequence, designated as the current frame, and the process of parsing the bitstream and determining the value of the chroma syntax element identifier can continue.

[0156] It should also be noted that if it is determined that the luminance component of the current block is filtered using a chroma weight matrix, then the luminance weight matrix network model for the current block also needs to be determined. In some embodiments, determining the luminance weight matrix network model for the current block may include:

[0157] If the color component type of the current block is a luminance component, determine at least one candidate luminance weight matrix network model;

[0158] Determine the quantization parameters of the current block, and select the candidate brightness weight matrix network model corresponding to the quantization parameters from at least one candidate brightness weight matrix network model;

[0159] The selected candidate brightness weight matrix network model is determined as the brightness weight matrix network model of the current block.

[0160] Furthermore, when determining the chroma components of the current block and performing filtering using a weight matrix, it is necessary to determine the chroma weight matrix network model for the current block. In some embodiments, determining the chroma weight matrix network model for the current block may include:

[0161] If the color component type of the current block is chromaticity component, determine at least one candidate chromaticity weight matrix network model;

[0162] Determine the quantization parameters of the current block, and select the candidate chromaticity weight matrix network model corresponding to the quantization parameters from at least one candidate chromaticity weight matrix network model;

[0163] The selected candidate chromaticity weight matrix network model is determined as the chromaticity weight matrix network model of the current block.

[0164] It should be noted that the weight matrix network model of the current block is related not only to the quantization parameters but also to the color component type. Different color component types correspond to different weight matrix network models. For example, for the luminance component, the weight matrix network model can be a luminance weight matrix network model related to the luminance component; for the chrominance component, the weight matrix network model can be a chrominance weight matrix network model related to the chrominance component.

[0165] It should also be noted that, depending on different quantization parameters, such as QP values ​​of 27–31, 32–37, 38–44, 45–50, at least one candidate luminance weight matrix network model and at least one candidate chrominance weight matrix network model can be pre-trained. Thus, after determining the quantization parameters of the current block, the candidate luminance weight matrix network model corresponding to that quantization parameter can be selected from the at least one candidate luminance weight matrix network model; similarly, the candidate chrominance weight matrix network model corresponding to that quantization parameter can be selected from the at least one candidate chrominance weight matrix network model; that is, the chrominance weight matrix network model for the current block.

[0166] Furthermore, for training at least one candidate luminance weight matrix network model and at least one candidate chrominance weight matrix network model, in some embodiments, the method may further include:

[0167] Determine at least one training sample, wherein the training sample is obtained based on at least one quantization parameter;

[0168] The pre-defined neural network model is trained using the luminance component of at least one training sample to obtain at least one candidate luminance weight matrix network model.

[0169] The preset neural network model is trained using the chromaticity components of at least one training sample to obtain at least one candidate chromaticity weight matrix network model.

[0170] Among them, at least one candidate luminance weight matrix network model has a correspondence with the luminance component and quantization parameter, and at least one candidate chrominance weight matrix network model has a correspondence with the chrominance component and quantization parameter.

[0171] In the embodiments of this application, the preset neural network model may include at least one of the following: at least one convolutional layer, at least one activation layer, and a jump connection layer.

[0172] In other words, the preset neural network model can be a multi-layer convolutional neural network, and then deep learning can be performed using training samples to obtain a weight matrix network model, such as a luminance weight matrix network model or a chrominance weight matrix network model.

[0173] It should be noted that deep learning is a type of machine learning, and machine learning is an essential path to achieving artificial intelligence. The concept of deep learning originates from the research of artificial neural networks; a multilayer perceptron with multiple hidden layers is a type of deep learning structure. Deep learning can discover distributed feature representations of data by combining low-level features to form more abstract high-level representations of attribute categories or features. In this embodiment, Convolutional Neural Networks (CNNs) are used as an example. CNNs are a type of feedforward neural network that includes convolutional computation and has a deep structure, and are one of the representative algorithms of deep learning. The preset neural network model here can be a convolutional neural network structure.

[0174] It should also be noted that, in the embodiments of this application, whether it is the luminance weight matrix network model or the chrominance weight matrix network model, the weight matrix network model can also be regarded as being composed of a multi-layer convolutional neural network. Specifically, the weight matrix network model may also include at least one of the following: at least one convolutional layer, at least one activation layer, and a jump connection layer.

[0175] Furthermore, after determining the weight matrix network model, the weight matrix can be determined accordingly. In some embodiments, determining the weight matrix of the current block based on the weight matrix network model may include:

[0176] Determine the input and output reconstructed image blocks of the neural network loop filter;

[0177] The output reconstructed image block is input into the weight matrix network model to obtain the weight matrix of the current block.

[0178] Here, the neural network loop filter specifically refers to the aforementioned CNNLF. After determining the input and output reconstructed image blocks of the CNNLF, the output reconstructed image block is used as the input to the weight matrix network model, and the output of the weight matrix network model is the weight matrix of the current block.

[0179] S403: Use the weight matrix to determine the target image block for the current block.

[0180] It should be noted that after obtaining the weight matrix, if the current block is used for filtering using the weight matrix, then the input and output of CNNLF can be weighted.

[0181] In some embodiments, determining the target reconstructed image block of the current block using a weight matrix may include: weighting the input reconstructed image block and the output reconstructed image block using a weight matrix to obtain the target reconstructed image block.

[0182] In a specific example, the weight matrix is ​​represented by the Weighting Matrix. Then, for the target reconstructed image patch, the target reconstructed image patch = output reconstructed image patch × Weighting Matrix + input reconstructed image patch × (1 - Weighting Matrix).

[0183] In this way, by weighting the input and output of the neural network loop filter using a weight matrix, the output of the neural network loop filter can be made closer to the original image.

[0184] It should also be noted that, depending on the type of color component, the weight matrix includes a luminance weight matrix and a chrominance weight matrix; thus, for a target reconstructed image patch, it can also include a target reconstructed image patch with luminance components and a target reconstructed image patch with chrominance components. In some embodiments, determining the weight matrix of the current block based on the weight matrix network model may include:

[0185] When the color component type of the current block is a luminance component, the luminance weight matrix is ​​determined using the luminance weight matrix network model.

[0186] When the color component type of the current block is chrominance component, the chrominance weight matrix is ​​determined using the chrominance weight matrix network model.

[0187] In a specific example, when the color component type of the current block is a luminance component, determining the luminance weight matrix using the luminance weight matrix network model may include:

[0188] Determine the input and output brightness reconstruction image blocks of the neural network loop filter;

[0189] The output brightness reconstructed image patch is input into the brightness weight matrix network model to obtain the brightness weight matrix;

[0190] Furthermore, determining the target reconstructed image block for the current block using the weight matrix may include:

[0191] The input and output luminance reconstructed image blocks are weighted using a luminance weight matrix to obtain the target reconstructed image block of the current block's luminance component.

[0192] In another specific example, when the color component type of the current block is chroma component, the step of determining the chroma weight matrix using the chroma weight matrix network model may include:

[0193] Determine the input and output chroma reconstructed image blocks of the neural network loop filter;

[0194] The output chroma reconstructed image patch is input into the chroma weight matrix network model to obtain the chroma weight matrix;

[0195] Furthermore, determining the target reconstructed image block for the current block using the weight matrix may include:

[0196] The input and output chromaticity reconstructed image blocks are weighted using a chromaticity weight matrix to obtain the target reconstructed image block of the current block's chromaticity components.

[0197] It should be noted that, depending on the type of color component, the input reconstructed image block can refer to either the input luminance reconstructed image block or the input chrominance reconstructed image block; similarly, the output reconstructed image block can also refer to either the output luminance reconstructed image block or the output chrominance reconstructed image block. Thus, the target reconstructed image block for the luminance component can be calculated by weighting the input and output luminance reconstructed image blocks; similarly, the target reconstructed image block for the chrominance component can be calculated by weighting the input and output chrominance reconstructed image blocks.

[0198] It should also be noted that, for the input reconstructed image block (including the input luminance reconstructed image block or the input chrominance reconstructed image block), the input reconstructed image block can be obtained after filtering by a deblocking filter and a sample adaptive compensation filter.

[0199] In addition, in some embodiments, the method may further include: after determining the target reconstructed image block of the current block, continuing to filter the target reconstructed image block using an adaptive correction filter.

[0200] In other words, if the current block is filtered using a weight matrix, then the input to the adaptive correction filter is the target reconstructed image block; if the current block is not filtered using a weight matrix, then there is no need to weight the output of the neural network loop filter, meaning the input to the adaptive correction filter is the output reconstructed image block.

[0201] For example, see Figure 8 This illustrates a schematic diagram of the composition structure of a weight matrix network model provided in an embodiment of this application. Figure 8 As shown, the weight matrix network model is set in the weight matrix module. The weight matrix network model can be composed of a multi-layer convolutional neural network. Its network structure can be composed of K convolutional layers, J activation layers and L jump connection layers; where K, J and L are all integers greater than or equal to 1.

[0202] In a specific example, K=5, J=4, L=1, but no restrictions are imposed.

[0203] Specifically, Figure 8 The network structure shown includes five convolutional layers. Except for the third convolutional layer, each convolutional layer is followed by an activation layer. The network structure also includes a jump connection layer connecting the input of the second convolutional layer to the output of the third convolutional layer. It is important to note that the activation function can be a linear activation function or a non-linear activation function, etc.

[0204] Furthermore, the input to this network structure is the output reconstructed image patch of CNNLF (which can be either the output luminance reconstructed image patch or the output chrominance reconstructed image patch), and the output of this network structure is a weight matrix, used to weight the output reconstructed image patch of CNNLF and the input reconstructed image patch of CNNLF, specifically as follows: Figure 9 As shown, it illustrates an example of the overall framework of a weighted matrix network model. From Figure 9 As can be seen, the input reconstructed image block can be obtained by filtering through a deblocking filter (DBF) and a sample adaptive compensation filter (SAO), while the weighted target reconstructed image block can be further filtered by an adaptive correction filter (ALF).

[0205] It should also be noted that the input reconstructed image patch in the weighted processing can also be multiple hidden layer feature image patches extracted from CNNLF and used in the weighted processing; in addition, the output reconstructed image patch of CNNLF can also be the output image patch using other efficient neural network filters, and the embodiments of this application do not make any limitations.

[0206] Understandably, the technical solution of this application embodiment operates in the loop filtering module of the decoder, and its specific process is as follows:

[0207] The decoder acquires and parses the bitstream. When it encounters the loop filter module, it processes the filters according to a preset filter order. Here, the preset filter order is: DBF filtering -> SAO filtering -> CNNLF filtering -> weight matrix module -> ALF filtering. When entering the weight matrix module...

[0208] (a) For the luma component, determine whether the current frame should be processed using the weighting matrix module based on the decoded luma_frame_weighting_matrix_flag. Then, determine whether each CTU in the current frame should be processed using the weighting matrix module based on the decoded luma_ctu_weighting_matrix_flag. If luma_frame_weighting_matrix_flag is "0", proceed to (c). Otherwise, if luma_frame_weighting_matrix_flag is "1", and luma_ctu_weighting_matrix_flag is "0", proceed to (c); otherwise, if luma_ctu_weighting_matrix_flag is "1", then the current CTU in the current frame should be processed using the weighting matrix module.

[0209] (b) For the chroma component, determine whether the current frame should be processed using the weighting matrix module based on the decoded chroma_frame_weighting_matrix_flag. If chroma_frame_weighting_matrix_flag is "0", proceed to (c); otherwise, if chroma_frame_weighting_matrix_flag is "1", then the current frame should be processed using the weighting matrix module.

[0210] (c) If the current frame has completed the processing of the weight matrix module, load the next frame for processing, and then jump to (a).

[0211] It is important to note that for the weight matrix module, the corresponding weight matrix network model needs to be selected based on the color component type and quantization parameters of the current block. The output reconstructed image block of CNNLF is used as the input to the weight matrix network model, and the output of this model yields the weight matrix. The output reconstructed image block of CNNLF and the input reconstructed image block of CNNLF are weighted according to the weight matrix to obtain the final output reconstructed image block (i.e., the target reconstructed image block described in the aforementioned embodiment).

[0212] In the implementation, the modifications to the syntax elements for the luminance component are shown in Table 1.

[0213] Table 1

[0214]

[0215] In the implementation, the modifications to the syntax elements for the chroma component are shown in Table 2.

[0216] Table 2

[0217]

[0218] Here, the frame-level flag for the luminance component of the weight matrix based on the neural network is luma_frame_weighting_matrix_flag; the CTU-level flag for the luminance component of the weight matrix based on the neural network is luma_ctu_weighting_matrix_flag; and the frame-level flag for the chroma component of the weight matrix based on the neural network is chroma_frame_weighting_matrix_flag.

[0219] In short, in this embodiment of the application, by inputting the reconstructed image patch of the CNNLF output of HPM-ModAI into the weight matrix network model (multilayer convolutional neural network), the feature information is extracted and the weight matrix is ​​output, thereby providing pixel-level weighted processing for the use of CNNLF, which improves the encoding performance.

[0220] For example, after the loop filtering method of this application embodiment is implemented on the AVS3 intelligent coding reference software HPM10.0-ModAI5.0, the test sequence required by AVS3 is tested under the general intelligent coding test condition All Intra configuration. The average changes in BD-rate for the three components Y, U, and V are -0.36%, -1.26%, and -0.38%, respectively, as shown in Table 3. Under the general intelligent coding test condition Random Access configuration, the test sequence required by AVS3 is tested. The average changes in BD-rate for the three components Y, U, and V are -0.16%, -1.04%, and -0.79%, respectively, as shown in Table 4. The data in Tables 3 and 4 can illustrate that the method of this application embodiment improves coding performance. Specifically, by introducing a weight matrix technology based on deep learning, this application embodiment can bring significant performance gains to the existing AVS3 intelligent coding reference software HPM-ModAI.

[0221] Table 3

[0222]

[0223] Table 4

[0224]

[0225] This embodiment provides a decoding method, specifically a loop filtering method applied to a decoder. By parsing the bitstream, the value of at least one syntax element identifier is determined. When the at least one syntax element identifier indicates that the current frame or current block should be filtered using a weight matrix, the weight matrix network model of the current block is determined, and the weight matrix of the current block is determined based on the weight matrix network model. Using the weight matrix, the target reconstructed image block of the current block is determined. Thus, by using a weight matrix network model to determine the weight matrix, not only can deep learning-based weight matrix technology be implemented, providing pixel-level weighting for the use of the output reconstructed image block, but coding performance can also be improved, thereby increasing encoding and decoding efficiency. Furthermore, this application can make the output of the neural network loop filter closer to the original image, improving video image quality.

[0226] In another embodiment of this application, see Figure 10 This illustrates a flowchart of an encoding method provided in an embodiment of this application. Figure 10 As shown, the method may include:

[0227] S1001: Determine the weight matrix network model of the current block, and determine the weight matrix of the current block based on the weight matrix network model.

[0228] It should be noted that a video image can be divided into multiple image blocks, and each image block to be encoded can be called a coding block. Here, each coding block may include a first image component, a second image component, and a third image component; and the current block is the coding block in the video image that is currently undergoing loop filtering processing on the first image component, the second image component, or the third image component.

[0229] Here, from a color classification perspective, the first image component, the second image component, and the third image component can be divided into two color component types: luminance component and chrominance component. In this case, if the current block performs operations such as luminance component prediction, transformation and quantization, and loop filtering, then the current block can also be called a luminance block; or, if the current block performs operations such as chrominance component prediction, transformation and quantization, and loop filtering, then the current block can also be called a chrominance block.

[0230] It should also be noted that, on the encoder side, this application embodiment specifically provides a loop filtering method, which is applied to applications such as... Figure 3AThe filtering unit 108 shown is an example. Here, the filtering unit 108 may also include a deblocking filter (LBF), a sample adaptive compensation filter (SAO), a loop filter based on a residual neural network (CNNLF), and an adaptive correction filter (ALF). The loop filtering method described in this embodiment is specifically applied between CNNLF and ALF so that the optimal selection can be made for each pixel value in the current block after CNNLF filtering.

[0231] In this embodiment, the loop filtering method mainly introduces a weight matrix network model, which can be a multi-layer convolutional neural network. The weight matrix network model differs for different color component types. In some embodiments, determining the weight matrix network model for the current block may include:

[0232] When the color component type of the current block is a luminance component, determine the luminance weight matrix network model of the current block;

[0233] When the color component type of the current block is chroma component, determine the chroma weight matrix network model of the current block.

[0234] It should be noted that the weight matrix network model is related to the type of color component. Here, the weight matrix network model corresponding to the luminance component can be called the luminance weight matrix network model, and the weight matrix network model corresponding to the chrominance component can be called the chrominance weight matrix network model. The following descriptions will use the luminance component and the chrominance component as examples respectively.

[0235] In one possible implementation, the network model for determining the brightness weight matrix of the current block may include:

[0236] If the color component type of the current block is a luminance component, determine at least one candidate luminance weight matrix network model;

[0237] Determine the quantization parameters of the current block, and select the candidate brightness weight matrix network model corresponding to the quantization parameters from at least one candidate brightness weight matrix network model;

[0238] The selected candidate brightness weight matrix network model is determined as the brightness weight matrix network model of the current block.

[0239] In another possible implementation, the network model for determining the chroma weight matrix of the current block may include:

[0240] If the color component type of the current block is chromaticity component, determine at least one candidate chromaticity weight matrix network model;

[0241] Determine the quantization parameters of the current block, and select the candidate chromaticity weight matrix network model corresponding to the quantization parameters from at least one candidate chromaticity weight matrix network model;

[0242] The selected candidate chromaticity weight matrix network model is determined as the chromaticity weight matrix network model of the current block.

[0243] It should be noted that the weight matrix network model of the current block is related to the quantization parameters and the color component type. Different color component types correspond to different weight matrix network models. For example, for the luminance component, the weight matrix network model can be a luminance weight matrix network model related to the luminance component; for the chrominance component, the weight matrix network model can be a chrominance weight matrix network model related to the chrominance component.

[0244] Furthermore, based on different quantization parameters, such as QP values ​​of 27–31, 32–37, 38–44, 45–50, at least one candidate luminance weight matrix network model and at least one candidate chrominance weight matrix network model can be pre-trained. Thus, after determining the quantization parameters of the current block, the candidate luminance weight matrix network model corresponding to that quantization parameter can be selected from the at least one candidate luminance weight matrix network model; similarly, the candidate chrominance weight matrix network model corresponding to that quantization parameter can be selected from the at least one candidate chrominance weight matrix network model; this is the chrominance weight matrix network model for the current block.

[0245] It should also be noted that, for model training of at least one candidate luminance weight matrix network model and at least one candidate chrominance weight matrix network model, in some embodiments, the method may further include:

[0246] Determine at least one training sample, wherein the training sample is obtained based on at least one quantization parameter;

[0247] The luminance components of the at least one training sample are used to train a preset neural network model to obtain at least one candidate luminance weight matrix network model.

[0248] The preset neural network model is trained using the chromaticity components of the at least one training sample to obtain at least one candidate chromaticity weight matrix network model;

[0249] Specifically, the at least one candidate luminance weight matrix network model has a corresponding relationship with the luminance component and the quantization parameter, and the at least one candidate chrominance weight matrix network model has a corresponding relationship with the chrominance component and the quantization parameter.

[0250] In the embodiments of this application, the preset neural network model may include at least one of the following: at least one convolutional layer, at least one activation layer, and a jump connection layer.

[0251] In other words, the preset neural network model can be a multi-layer convolutional neural network, and then deep learning can be performed using training samples to obtain a weight matrix network model, such as a luminance weight matrix network model or a chrominance weight matrix network model. Thus, the weight matrix network model can also be viewed as being composed of multi-layer convolutional neural networks. Specifically, the weight matrix network model can also include at least one of the following: at least one convolutional layer, at least one activation layer, and a jump connection layer.

[0252] Furthermore, after determining the weight matrix network model, the weight matrix can be determined accordingly. In some embodiments, determining the weight matrix of the current block based on the weight matrix network model may include:

[0253] Determine the input and output reconstructed image blocks of the neural network loop filter;

[0254] The output reconstructed image patch is input into the weight matrix network model to obtain the weight matrix of the current patch.

[0255] Here, the neural network loop filter specifically refers to the aforementioned CNNLF. After determining the input and output reconstructed image blocks of the CNNLF, the output reconstructed image block is used as the input to the weight matrix network model, and the output of the weight matrix network model is the weight matrix of the current block.

[0256] It should also be noted that, depending on the type of color component, the weight matrix includes a luminance weight matrix and a chrominance weight matrix. In some embodiments, determining the weight matrix of the current block based on the weight matrix network model may include:

[0257] When the color component type of the current block is a luminance component, the luminance weight matrix is ​​determined using the luminance weight matrix network model.

[0258] When the color component type of the current block is chrominance component, the chrominance weight matrix is ​​determined using the chrominance weight matrix network model.

[0259] In a specific example, when the color component type of the current block is a luminance component, determining the luminance weight matrix using the luminance weight matrix network model may include:

[0260] Determine the input and output brightness reconstruction image blocks of the neural network loop filter;

[0261] The output brightness reconstructed image patch is input into the brightness weight matrix network model to obtain the brightness weight matrix.

[0262] In another specific example, when the color component type of the current block is chroma component, the step of determining the chroma weight matrix using the chroma weight matrix network model may include:

[0263] Determine the input and output chroma reconstructed image blocks of the neural network loop filter;

[0264] The output chroma reconstructed image patch is input into the chroma weight matrix network model to obtain the chroma weight matrix.

[0265] It should be noted that, depending on the type of color component, the input reconstructed image patch can refer to either the input luminance reconstructed image patch or the input chrominance reconstructed image patch; similarly, the output reconstructed image patch can also refer to either the output luminance reconstructed image patch or the output chrominance reconstructed image patch. Thus, the luminance weight matrix can be obtained from the luminance weight matrix network model and the output luminance reconstructed image patch, and the chrominance weight matrix can be obtained from the chrominance weight matrix network model and the output chrominance reconstructed image patch.

[0266] S1002: Determine the value of at least one syntax element identification information.

[0267] It should be noted that there is no specific order between steps S1001 and S1002. Step S1001 can be executed first, followed by step S1002; or step S1002 can be executed first, followed by step S1001; or even both can be executed in parallel. This application embodiment does not impose any limitations.

[0268] It should also be noted that whether to use a weight matrix for filtering can be indicated by at least one syntax element identifier. Thus, in the encoder, once the value of this at least one syntax element identifier is determined, it is written into the bitstream for transmission to the decoder, allowing the decoder to determine whether to use a weight matrix for filtering by parsing the bitstream.

[0269] In some embodiments, determining the value of at least one syntax element identification information may include:

[0270] If it is determined that the video sequence will be filtered using a weight matrix, then the value of the first syntax element identifier information will be determined to be the first value; or,

[0271] If it is determined that the video sequence will not be filtered using a weight matrix, then the value of the first syntax element identifier information is determined to be the second value.

[0272] Furthermore, the method also includes: encoding the value of the first syntax element identification information and writing the encoded bits into the bitstream.

[0273] In other words, a first syntax element identifier can be set to indicate whether the current video sequence uses the loop filtering method of this application embodiment (i.e., whether the weight matrix module is enabled). Here, if the weight matrix module of the video sequence is enabled, that is, it is determined that the video sequence uses the weight matrix for filtering, then the value of the first syntax element identifier can be a first value; if the weight matrix module of the video sequence is not enabled, that is, it is determined that the video sequence does not use the weight matrix for filtering, then the value of the first syntax element identifier can be a second value.

[0274] It should also be noted that the first value and the second value are different.

[0275] Taking a flag as the first syntax element identifier, the first and second values ​​can be set to 1 and 0 respectively; or, the first value can be set to true and the second value to false; or, the first value can be set to 0 and the second value to 1; or, the first value can be set to false and the second value to true. For example, generally, for a flag, the first value can be 1 and the second value can be 0, but there are no restrictions.

[0276] It should also be noted that the video sequence includes at least one frame, which may include the current frame. Here, when the video sequence enables the loop filtering method of this application embodiment, i.e., the weight matrix module is enabled for the video sequence, this application embodiment also needs to further determine whether the current frame in the video sequence is filtered using the weight matrix, i.e., it also needs to set a second syntax element identification information. The meaning of the second syntax element identification information varies depending on the color component type.

[0277] In this embodiment, the color component type may include a luminance component and a chrominance component. If the color component type is a luminance component, it is assumed that the second syntax element identifier information is the first luminance syntax element identifier information, to indicate whether the luminance component of the current frame is filtered using a luminance weight matrix; if the color component type is a chrominance component, it is assumed that the second syntax element identifier information is the chrominance syntax element identifier information, to indicate whether the chrominance component of the current frame is filtered using a chrominance weight matrix.

[0278] Thus, after determining that the video sequence is filtered using a weight matrix, when the color component type of the current frame is a luma component, the weight matrix is ​​determined to be a luma weight matrix. In this case, in one possible implementation, determining the value of at least one syntax element identifier information may further include:

[0279] The first rate-distortion cost is determined by filtering the luminance component of the current frame within the video sequence using a luminance weight matrix.

[0280] Determine the second rate distortion cost of the luminance component of the current frame within the video sequence that was not filtered using the luminance weight matrix;

[0281] If the cost of the first rate distortion is less than the cost of the second rate distortion, then the value of the first luma syntax element identifier is determined to be the first value; or,

[0282] If the first rate distortion value is greater than or equal to the second rate distortion value, then the value of the first luminance syntax element identifier information is determined to be the second value.

[0283] Furthermore, the method also includes: encoding the value of the first luminance syntax element identifier information and writing the encoded bits into the bitstream.

[0284] It should be noted that, for the luminance component, if the value of the first luminance syntax element identifier is the first value, i.e., it is determined that the luminance component of the current frame is filtered using the luminance weight matrix, then it is still necessary to further determine whether the luminance component of the current block within the current frame is filtered using the luminance weight matrix. Therefore, in some embodiments, when the first rate-distortion cost is less than the second rate-distortion cost, determining the value of at least one syntax element identifier may further include:

[0285] The third rate-distortion arithmetic value is determined for the luminance components of the current block that have been filtered using the luminance weight matrix; and the fourth rate-distortion arithmetic value is determined for the luminance components of the current block that have not been filtered using the luminance weight matrix.

[0286] If the cost of the third rate distortion is less than the cost of the fourth rate distortion, then the value of the second luminance syntax element identifier is determined to be the first value; or...

[0287] If the third rate distortion value is greater than or equal to the fourth rate distortion value, then the value of the second luminance syntax element identifier information is determined to be the second value.

[0288] Furthermore, the method also includes: encoding the value of the second luminance syntax element identifier information and writing the encoded bits into the bitstream.

[0289] In other words, for the luma component, two types of syntax elements are involved: frame-level syntax elements and CTU-level syntax elements. The frame-level syntax element can be referred to as the first luma syntax element identifier, and the CTU-level syntax element can be referred to as the second luma syntax element identifier. Assuming that the first and second luma syntax element identifiers are flag information, then the first luma syntax element identifier can be represented by `luma_frame_weighting_matrix_flag`; and the second luma syntax element identifier can be represented by `luma_ctu_weighting_matrix_flag`. Here, the values ​​of both the first and second luma syntax element identifiers can be determined using rate-distortion costing.

[0290] Taking the first luminance syntax element identification information as an example, for the current frame, in some embodiments, the method may further include: dividing the current frame into blocks to determine at least one partition block; wherein, the at least one partition block includes the current block;

[0291] Accordingly, the first rate-distortion cost of determining the luminance component of the current frame within the video sequence using a luminance weight matrix for filtering may include:

[0292] Calculate the third rate distortion cost of the luminance components of each of the at least one segmented block by filtering them using the luminance weight matrix;

[0293] The calculated cost of third-rate distortion is summed to obtain the cost of first-rate distortion.

[0294] The calculation of the second rate-distortion cost of the luminance component of the current frame within the video sequence before filtering using the luminance weight matrix may include:

[0295] Calculate the fourth rate distortion cost of the luminance components of each of the at least one partitioned block without filtering using the luminance weight matrix;

[0296] The calculated fourth rate distortion cost is summed to obtain the second rate distortion cost.

[0297] In other words, the third rate-distortion cost can be calculated by using a luminance weight matrix to filter the luminance component of each block, and then the first rate-distortion cost of the current frame can be obtained by accumulating these values. In a specific example, the distortion value can be determined based on the mean squared error in the calculation of the rate-distortion cost.

[0298] In one possible implementation, the luminance component of each block is filtered using a luminance weight matrix to obtain the target reconstructed image block for each block's luminance component. Then, the mean square error (MSE) between the target reconstructed image block and the original image block is calculated to obtain the MSE value for each block. The third rate-distortion cost for each block is calculated using the rate-distortion cost formula RDcost = D + λ*R, where D is the MSE value for each block, R is 1, and λ is consistent with the λ of the adaptive correction filter. Finally, the third rate-distortion costs of each block are summed to obtain the first rate-distortion cost for the current frame.

[0299] In another possible implementation, the luminance component of each block is filtered using a luminance weight matrix to obtain the target reconstructed image block of the luminance component of each block; then the mean square error (MSE) value between the target reconstructed image block and the original image block is calculated to obtain the MSE value of each block, and the MSE value of the current frame is obtained by accumulating the MSE values; then the first rate distortion cost is calculated using the rate distortion cost formula RDcost=D+λ*R; where D is the MSE value of the current frame, R is the number of blocks included in the current frame, and λ is consistent with the λ of the adaptive correction filter.

[0300] It should also be noted that the second rate-distortion cost of the current frame can be obtained by calculating the fourth rate-distortion cost of the luminance component of each block without filtering using the luminance weight matrix, and then accumulating these values. Here, in the calculation of the rate-distortion cost, the distortion value can also be determined based on the mean square error; in this case, the mean square error refers to the mean square error between the output reconstructed image block without the weight matrix module and the original image block. Other calculation operations are the same as those for calculating the first rate-distortion cost, and will not be detailed here.

[0301] Thus, taking a first value of 1 and a second value of 0 as an example, after obtaining the first rate-distortion cost value and the second rate-distortion cost value, the first rate-distortion cost value can be compared with the second rate-distortion cost value. If the first rate-distortion cost value is greater than or equal to the second rate-distortion cost value, then it can be determined that the value of the first luminance syntax element identifier information is 0, which means that the luminance component of the current frame does not need to be filtered using the luminance weight matrix. At this time, the next frame can be obtained from the video sequence, the next frame is determined as the current frame, and the calculation of the first rate-distortion cost value and the second rate-distortion cost value can continue. Otherwise, if the first rate-distortion cost is less than the second rate-distortion cost, then the value of the first luminance syntax element identifier is determined to be 1, meaning that the luminance component of the current frame needs to be filtered using the luminance weight matrix. In this case, it is necessary to further determine whether the luminance component of the current block within the current frame needs to be filtered using the luminance weight matrix; that is, to compare the third rate-distortion cost with the fourth rate-distortion cost. If the third rate-distortion cost is less than the fourth rate-distortion cost, then the value of the second luminance syntax element identifier is determined to be 1, meaning that the luminance component of the current block needs to be filtered using the luminance weight matrix. Otherwise, if the third rate-distortion cost is greater than or equal to the fourth rate-distortion cost, then the value of the second luminance syntax element identifier is determined to be 0, meaning that the luminance component of the current block does not need to be filtered using the luminance weight matrix. In this case, the next block can be obtained from the current frame, and the next block is determined as the current block. The calculation of the third rate-distortion cost and the fourth rate-distortion cost continues.

[0302] In addition, for the luminance component, embodiments of this application can also set a luminance frame-level flag and a luminance CTU-level flag, and determine whether to use the luminance weight matrix for filtering by controlling whether the weight matrix module is turned on.

[0303] In some embodiments, the method may further include setting a luminance frame-level flag bit, which is used to control whether the luminance component of the current frame is filtered using a luminance weight matrix.

[0304] Accordingly, the method may further include:

[0305] If the cost of the first rate distortion is less than the cost of the second rate distortion, then enable the luminance frame-level flag; or...

[0306] If the first rate distortion cost is greater than or equal to the second rate distortion cost, then the luminance frame-level flag is turned off.

[0307] For the luminance CTU level flag bit, in some embodiments, the method may further include: setting the luminance CTU level flag bit, which is used to control whether the luminance component of the current block is filtered using a luminance weight matrix;

[0308] Accordingly, the method may further include:

[0309] If the cost of third-rate distortion is less than the cost of fourth-rate distortion, then enable the luminance CTU level flag; or,

[0310] If the third rate distortion cost is greater than or equal to the fourth rate distortion cost, then the luminance CTU level flag is turned off.

[0311] It should be noted that both the luminance frame-level flag and the luminance CTU-level flag can be enabled or disabled based on the rate-distortion cost. One possible implementation involves determining this based on the calculated rate-distortion cost.

[0312] In another possible implementation, the luma frame-level flag is still determined according to RDcost = D + λ*R. Here, D represents the distortion reduction value after the current frame is processed by the weight matrix module, D = D out -D rec (D out The distortion after processing by the weight matrix module, D rec (This refers to the distortion before processing by the weight matrix module), where R is the number of blocks in the current frame, and λ is consistent with the λ of the adaptive correction filter. At this point, when RDcost is negative, the luma frame-level flag is turned on, i.e., the frame-level weight matrix module is enabled; otherwise, the luma frame-level flag is turned off, i.e., the frame-level weight matrix module is disabled.

[0313] When the luminance frame-level flag is enabled, the luminance CTU-level flag can be determined based on RDcost = D. Here, D represents the distortion reduction value after the current block is processed by the weight matrix module, D = D out -D rec (D out The distortion after processing by the weight matrix module, D rec Distortion before processing by the weight matrix module).

[0314] Furthermore, after determining that the video sequence is filtered using a weight matrix, when the color component type of the current frame is chroma component, the weight matrix is ​​determined to be a chroma weight matrix. In this case, in another possible implementation, determining the value of at least one syntax element identifier information may further include:

[0315] The fifth rate distortion cost is determined by filtering the chroma components of the current frame within the video sequence using a chroma weight matrix.

[0316] The sixth rate distortion cost is determined for the chroma components of the current frame in the video sequence that have not been filtered using the chroma weight matrix.

[0317] If the distortion cost of the fifth rate is less than the distortion cost of the sixth rate, then the value of the chroma syntax element identifier is determined to be the first value; or...

[0318] If the value of the fifth rate distortion is greater than or equal to the value of the sixth rate distortion, then the value of the chroma syntax element identifier information is determined to be the second value.

[0319] Furthermore, the method may also include: encoding the value of the chroma syntax element identification information and writing the encoded bits into the bitstream.

[0320] It should be noted that for the chroma component, frame-level syntax elements are involved. These frame-level syntax elements can be called chroma syntax element identifiers. Assuming the chroma syntax element identifier is a flag, it can be represented as `chroma_frame_weighting_matrix_flag`.

[0321] It should also be noted that, for the first and second values, the first value can be set to 1 and the second value can be set to 0; or, the first value can also be set to true and the second value can also be set to false; or, the first value can also be set to 0 and the second value can also be set to 1; or, the first value can also be set to false and the second value can also be set to true. For example, in general, the first value can be 1 and the second value can be 0, but there are no restrictions.

[0322] Furthermore, considering encoding performance and computational complexity, if the chroma syntax element identifier indicates that the chroma components of the current frame are filtered using a chroma weight matrix, then all blocks included in the current frame will be filtered using the chroma weight matrix by default. Conversely, if the chroma syntax element identifier indicates that the chroma components of the current frame are not filtered using a chroma weight matrix, then all blocks included in the current frame will not be filtered using the chroma weight matrix by default. Therefore, it is no longer necessary to set CTU-level syntax elements for chroma components, and similarly, it is also unnecessary to set CTU-level flags.

[0323] In other words, for the chroma components, embodiments of this application may only need to set a frame-level flag. Therefore, in some embodiments, the method may further include: setting a chroma frame-level flag, which is used to control whether the chroma components of the current frame are filtered using a chroma weight matrix;

[0324] Accordingly, the method may further include:

[0325] If the fifth rate distortion value is less than the sixth rate distortion value, then enable the chroma frame-level flag; or...

[0326] If the fifth rate distortion value is greater than or equal to the sixth rate distortion value, then the chroma frame-level flag is turned off.

[0327] It should be noted that, for the fifth and sixth rate distortion values, in a specific example, the distortion value can also be determined based on the mean square error. Other calculation operations are the same as those for calculating the first and second rate distortion values, and will not be detailed here. Furthermore, the chroma frame-level flag is handled in the same way as determining whether the luminance frame-level flag is on, and will not be detailed here either.

[0328] Taking a first value of 1 and a second value of 0 as an example, after obtaining the fifth and sixth rate-distortion values, we can compare them. If the fifth rate-distortion value is less than the sixth rate-distortion value, the chroma frame-level flag can be turned on, and the value of the chroma syntax element identifier information can be determined to be 1, meaning that the chroma components of the current frame need to be filtered using the chroma weight matrix. After processing the current frame, we continue loading the next frame for processing. Otherwise, if the fifth rate-distortion value is greater than or equal to the sixth rate-distortion value, the chroma frame-level flag can be turned off, and the value of the chroma syntax element identifier information can be determined to be 0, meaning that the chroma components of the current frame do not need to be filtered using the chroma weight matrix. In this case, we can obtain the next frame from the video sequence, determine the next frame as the current frame, and continue loading the next frame for processing to determine the value of the syntax element identifier information of the next frame.

[0329] S1003: When at least one syntax element identification information indicates that the current frame or the current block is filtered using a weight matrix, the target reconstructed image block of the current block is determined using the weight matrix.

[0330] It should be noted that color component types can include luma and chroma components. If the current frame uses a weight matrix for filtering, it doesn't mean that every block within the current frame uses the weight matrix for filtering. CTU-level syntax element identification information may also be involved to further determine whether the current block uses the weight matrix for filtering. The following descriptions will use luma and chroma components as examples.

[0331] In some embodiments, for the luminance component, determining the target reconstructed image block of the current block using the weight matrix when at least one syntax element identification information indicates that the current frame or current block is filtered using the weight matrix may include:

[0332] If the value of the third rate distortion is less than the value of the fourth rate distortion, then the target image block for the current block is determined using the luminance weight matrix.

[0333] In some embodiments, for the chroma component, determining the target reconstructed image block of the current block using the weight matrix when at least one syntax element identification information indicates that the current frame or current block is filtered using the weight matrix may include:

[0334] If the fifth rate distortion value is less than the sixth rate distortion value, then the target reconstructed image block for the current block is determined using the chroma weight matrix.

[0335] It should be noted that two types of syntax elements are required for the luma component: frame-level syntax elements and CTU-level syntax elements. Only when the CTU-level syntax element (i.e., the second luma syntax element identifier information) indicates that the current block is filtered using the weight matrix, that is, the third rate distortion cost is less than the fourth rate distortion cost, can the luma weight matrix be used to determine the target reconstructed image block of the current block (specifically, the target reconstructed image block of the luma component of the current block).

[0336] For the chroma component, only one syntax element is needed: the frame-level syntax element. Only when the frame-level syntax element (i.e., the chroma syntax element identification information) indicates that the current frame is filtered using the weight matrix, that is, the fifth rate distortion cost is less than the sixth rate distortion cost, will all blocks in the current frame be filtered using the chroma weight matrix by default. At this time, the weight matrix can also be used to determine the target reconstructed image block of the current block (specifically, the target reconstructed image block of the chroma component of the current block).

[0337] Furthermore, in some embodiments, determining the target reconstructed image block for the current block using a weight matrix may include:

[0338] The input and output reconstructed image blocks are weighted using a weight matrix to obtain the target reconstructed image block.

[0339] In a specific example, the weight matrix is ​​represented by the Weighting Matrix. Then, for the target reconstructed image patch, the target reconstructed image patch = output reconstructed image patch × Weighting Matrix + input reconstructed image patch × (1 - Weighting Matrix).

[0340] In this way, by weighting the input and output of CNNLF with a weight matrix, the output of the neural network loop filter can be made closer to the original image.

[0341] It should also be noted that, depending on the type of color component, the weight matrix includes a luminance weight matrix and a chrominance weight matrix; thus, for a target reconstructed image patch, it can also include a target reconstructed image patch with luminance components and a target reconstructed image patch with chrominance components. In a specific example, for the luminance component, determining the luminance weight matrix using the luminance weight matrix network model can include:

[0342] Determine the input and output brightness reconstruction image blocks of the neural network loop filter;

[0343] The output brightness reconstructed image patch is input into the brightness weight matrix network model to obtain the brightness weight matrix;

[0344] Furthermore, determining the target reconstructed image block for the current block using the weight matrix may include:

[0345] The input and output luminance reconstructed image blocks are weighted using the luminance weight matrix to obtain the target reconstructed image block of the luminance component of the current block.

[0346] In another specific example, for the chromaticity components, determining the chromaticity weight matrix using the chromaticity weight matrix network model may include:

[0347] Determine the input and output chroma reconstructed image blocks of the neural network loop filter;

[0348] The output chroma reconstructed image patch is input into the chroma weight matrix network model to obtain the chroma weight matrix;

[0349] Furthermore, determining the target reconstructed image block for the current block using the weight matrix may include:

[0350] The input and output chromaticity reconstructed image blocks are weighted using a chromaticity weight matrix to obtain the target reconstructed image block of the current block's chromaticity components.

[0351] It should be noted that, depending on the type of color component, the input reconstructed image block can refer to either the input luminance reconstructed image block or the input chrominance reconstructed image block; similarly, the output reconstructed image block can also refer to either the output luminance reconstructed image block or the output chrominance reconstructed image block. Thus, the target reconstructed image block for the luminance component can be calculated by weighting the input and output luminance reconstructed image blocks; similarly, the target reconstructed image block for the chrominance component can be calculated by weighting the input and output chrominance reconstructed image blocks.

[0352] It should also be noted that, for the input reconstructed image block (including the input luminance reconstructed image block or the input chrominance reconstructed image block), the input reconstructed image block can be obtained after filtering by a deblocking filter and a sample adaptive compensation filter.

[0353] In addition, in some embodiments, the method may further include: after determining the target reconstructed image block of the current block, continuing to filter the target reconstructed image block using an adaptive correction filter.

[0354] In other words, if the current block is filtered using a weight matrix, then the input to the adaptive correction filter is the target reconstructed image block; if the current block is not filtered using a weight matrix, then there is no need to weight the output of CNNLF, meaning the input to the adaptive correction filter is the output reconstructed image block.

[0355] In short, this application proposes a deep learning-based weight matrix to weight the output of a neural network loop filter, making the output of the neural network loop filter closer to the original image. Here, the network structure corresponding to the weight matrix network model can be a multi-layer convolutional neural network. The input is the reconstructed image patch from the output of the neural network loop filter, and the output is a new reconstructed image patch after weighting (i.e., the "target reconstructed image patch" mentioned in the previous embodiment). The weight matrix module is located in the encoder / decoder as follows: Figure 7 As shown, the Weighting Matrix is ​​the weight matrix module in the figure. The use of this weight matrix module does not depend on the flags of DBF, SAO, ALF, and CNNLF. It is simply placed after CNNLF and before ALF.

[0356] Furthermore, whether the deep learning-based weight matrix is ​​selected can be determined by the encoder. On the encoder side, when entering the loop filtering module, the input reconstructed image patch of CNNLF can be marked as I. rec The reconstructed image patch from the output of CNNLF is labeled as I. net , will I netAs input to the weight matrix network model, the output can be obtained as the weight matrix W, and the final output I. out It can be obtained from equation (3), as shown below.

[0357] I out =W*I net +(1-W)*I rec (3)

[0358] Furthermore, in this embodiment, a frame-level flag and a CTU-level flag are set for the luminance component to control whether it is enabled, and a frame-level flag is set for the chrominance component to control whether it is enabled. Specifically, the frame-level flag can be determined by equation (4), where D = D out -D rec This represents the reduction in distortion (D) after processing by the weight matrix module. out The distortion after processing by the weight matrix module, D rec (This represents the distortion before processing by the weight matrix module), where R is the number of CTUs in the current frame, and λ is consistent with the λ of the adaptive correction filter. At this point, when RDcost is negative, the frame-level weight matrix module is enabled; otherwise, it is disabled.

[0359] RDcost=D+λ*R (4)

[0360] When the frame-level flag is enabled, the decision on whether each CTU should enable the weight matrix module can be further determined by the rate-distortion cost method. Specifically, the CTU-level flag can be determined by equation (5).

[0361] RDcost = D (5)

[0362] Here, the flag information of the weight matrix module is encoded into the bitstream for the decoder to read. On the decoder side, when the weight matrix module is turned on, the input and output of CNNLF are weighted according to equation (3); when the weight matrix module is turned off, no weighting is performed.

[0363] In a specific example, the technical solution of this application embodiment is applied to the loop filtering module of the encoder, and its specific process is as follows:

[0364] When the encoder enters the loop filtering module, it processes the filters according to a preset filter order. Here, the preset filter order is: DBF filtering -> SAO filtering -> CNNLF filtering -> weight matrix module -> ALF filtering. When entering the weight matrix module...

[0365] (a) First, select the corresponding weight matrix network model based on the color component type and quantization parameters of the current block. Use the output reconstructed image block of CNNLF as the input to this weight matrix network model, and obtain the weight matrix from the output of the model. Weight the output reconstructed image block of CNNLF and the input reconstructed image block of CNNLF according to the obtained weight matrix to obtain the final target reconstructed image block;

[0366] (b) For the luminance component, first calculate the mean square error (MSE) between the output reconstructed image block and the original image block, and accumulate the MSE value of the current frame. Then, make a decision using the rate-distortion cost method. If it is determined that the rate-distortion cost after using the weight matrix module is less than that without using the weight matrix module, then set luma_frame_weighting_matrix_flag to "1", otherwise set it to "0". If luma_frame_weighting_matrix_flag is "0", then jump to (d). Otherwise, further make a decision based on the MSE values ​​of each CTU in the current frame using the rate-distortion cost method. If it is determined that the rate-distortion cost after using the weight matrix module is less than that without using the weight matrix module, then set luma_ctu_weighting_matrix_flag to "1", otherwise set it to "0".

[0367] (c) For the chroma component, first calculate the mean square error between the output reconstructed image patch and the original image patch, and accumulate the mean square error of the current frame. Then, make a decision based on the rate-distortion cost. If it is determined that the rate-distortion cost after using the weight matrix module is less than that without using the weight matrix module, then set chroma_frame_weighting_matrix_flag to "1", otherwise set it to "0".

[0368] (d) If the current frame has completed the decision of the weight matrix module, then load the next frame for processing, and then jump to (a).

[0369] In the implementation, the modifications to the syntax elements for the luminance component are shown in Table 1; the modifications to the syntax elements for the chrominance component are shown in Table 2.

[0370] In other words, in this embodiment of the application, by inputting the reconstructed image patch of the CNNLF output of HPM-ModAI into the weight matrix network model (multilayer convolutional neural network), the feature information is extracted and the weight matrix is ​​output, thereby providing pixel-level weighted processing for the use of CNNLF, which improves the encoding performance.

[0371] In another embodiment of this application, a bitstream is provided, which is generated by bit encoding based on the value of at least one syntax element identifier information.

[0372] In the application embodiments, at least one syntax element identification information may include at least: first syntax element identification information, first luminance syntax element identification information, second luminance syntax element identification information, and chrominance syntax element identification information.

[0373] The first grammar element identifier is used to indicate whether the video sequence is filtered using a weight matrix; the first luminance grammar element identifier is used to indicate whether the luminance component of the current frame is filtered using a weight matrix; the second luminance grammar element identifier is used to indicate whether the luminance component of the current block is filtered using a weight matrix; and the chroma grammar element identifier is used to indicate whether the chroma component of the current frame is filtered using a weight matrix. The video sequence may include the current frame, and the current frame may include the current block.

[0374] It should also be noted that the first luma syntax element identifier and the chroma syntax element identifier can be frame-level flags, while the second luma syntax element identifier can be a CTU-level flag. Here, the frame-level flag for the luma component of the neural network-based weight matrix (i.e., the first luma syntax element identifier) ​​is luma_frame_weighting_matrix_flag; the CTU-level flag for the luma component of the neural network-based weight matrix (i.e., the second luma syntax element identifier) ​​is luma_ctu_weighting_matrix_flag; and the frame-level flag for the chroma component of the neural network-based weight matrix (i.e., the chroma syntax element identifier) ​​is chroma_frame_weighting_matrix_flag.

[0375] This embodiment provides an encoding method, specifically a loop filtering method applied to an encoder. It determines the weight matrix network model of the current block and the weight matrix of the current block based on the weight matrix network model; it also determines the value of at least one syntax element identifier; when the at least one syntax element identifier indicates that the current frame or current block should use the weight matrix for filtering, the target reconstructed image block of the current block is determined using the weight matrix. Thus, by using the weight matrix network model to determine the weight matrix, not only can deep learning-based weight matrix technology be implemented, providing pixel-level weighting for the use of the output reconstructed image block, but it can also improve encoding performance, thereby increasing encoding and decoding efficiency. Furthermore, this application can make the output of the neural network loop filter closer to the original image, improving video image quality.

[0376] In another embodiment of this application, based on the same inventive concept as the foregoing embodiments, see [link to previous embodiment]. Figure 11 This illustrates a schematic diagram of the composition structure of an encoder 110 provided in an embodiment of this application. Figure 11 As shown, the encoder 110 may include: a first determining unit 1101 and a first filtering unit 1102; wherein,

[0377] The first determining unit 1101 is configured to determine the weight matrix network model of the current block and determine the weight matrix of the current block according to the weight matrix network model; and is also configured to determine the value of at least one syntax element identification information.

[0378] The first filtering unit 1102 is configured to determine the target reconstructed image block of the current block using the weight matrix when at least one syntax element identification information indicates that the current frame or the current block is filtered using the weight matrix.

[0379] In some embodiments, the first determining unit 1101 is further configured to determine the luminance weight matrix network model of the current block when the color component type of the current block is luminance component; and to determine the chrominance weight matrix network model of the current block when the color component type of the current block is chrominance component.

[0380] In some embodiments, the first determining unit 1101 is further configured to: determine at least one candidate luminance weight matrix network model when the color component type of the current block is luminance component; determine the quantization parameter of the current block; select the candidate luminance weight matrix network model corresponding to the quantization parameter from the at least one candidate luminance weight matrix network model; and determine the selected candidate luminance weight matrix network model as the luminance weight matrix network model of the current block.

[0381] The first determining unit 1101 is further configured to: determine at least one candidate chromaticity weight matrix network model when the color component type of the current block is chromaticity component; determine the quantization parameter of the current block; select the candidate chromaticity weight matrix network model corresponding to the quantization parameter from the at least one candidate chromaticity weight matrix network model; and determine the selected candidate chromaticity weight matrix network model as the chromaticity weight matrix network model of the current block.

[0382] In some embodiments, see Figure 11 The encoder 110 may also include a first training unit 1103;

[0383] The first determining unit 1101 is further configured to determine at least one training sample, wherein the training sample is obtained based on at least one quantization parameter;

[0384] The first training unit 1103 is configured to train a preset neural network model using the luminance component of at least one training sample to obtain at least one candidate luminance weight matrix network model; and to train the preset neural network model using the chrominance component of at least one training sample to obtain at least one candidate chrominance weight matrix network model; wherein, at least one candidate luminance weight matrix network model has a correspondence with the luminance component and the quantization parameter, and at least one candidate chrominance weight matrix network model has a correspondence with the chrominance component and the quantization parameter.

[0385] In some embodiments, the preset neural network model includes at least one of the following: at least one convolutional layer, at least one activation layer, and a jump connection layer.

[0386] In some embodiments, the first determining unit 1101 is further configured to determine the value of the first syntax element identification information as a first value if it is determined that the video sequence is filtered using a weight matrix; or, if it is determined that the video sequence is not filtered using a weight matrix, determine the value of the first syntax element identification information as a second value.

[0387] In some embodiments, see Figure 11 The encoder 110 may also include an encoding unit 1104 configured to encode the value of the first syntax element identification information and write the encoded bits into the bit stream.

[0388] In some embodiments, after determining that the video sequence is filtered using a weight matrix, the first determining unit 1101 is further configured to: determine that the weight matrix is ​​a luminance weight matrix when the color component type of the current frame is a luminance component; determine a first rate-distortion value for the luminance component of the current frame in the video sequence that is filtered using the luminance weight matrix; determine a second rate-distortion value for the luminance component of the current frame in the video sequence that is not filtered using the luminance weight matrix; and if the first rate-distortion value is less than the second rate-distortion value, determine that the value of the first luminance syntax element identification information is a first value; or, if the first rate-distortion value is greater than or equal to the second rate-distortion value, determine that the value of the first luminance syntax element identification information is a second value.

[0389] In some embodiments, the encoding unit 1104 is further configured to encode the value of the first luminance syntax element identification information and write the encoded bits into the bitstream.

[0390] In some embodiments, see Figure 11 The encoder 110 may also include a first setting unit 1105, configured to set a luminance frame-level flag bit, which is used to control whether the luminance component of the current frame is filtered using a weight matrix.

[0391] Accordingly, the first determining unit 1101 is further configured to turn on the luminance frame-level flag if the first rate-distortion value is less than the second rate-distortion value; or to turn off the luminance frame-level flag if the first rate-distortion value is greater than or equal to the second rate-distortion value.

[0392] In some embodiments, the first determining unit 1101 is further configured to divide the current frame into blocks and determine at least one block; wherein the at least one block includes the current block; and to calculate the third rate-distortion value of the luminance components of the at least one block after filtering with a luminance weight matrix, and to accumulate the calculated third rate-distortion value to obtain a first rate-distortion value; and to calculate the fourth rate-distortion value of the luminance components of the at least one block after filtering without using a luminance weight matrix, and to accumulate the calculated fourth rate-distortion value to obtain a second rate-distortion value.

[0393] In some embodiments, the first determining unit 1101 is further configured to: determine a third rate-distortion value for the luminance component of the current block when the first rate-distortion value is less than the second rate-distortion value; and determine a fourth rate-distortion value for the luminance component of the current block when it is not filtered by the luminance weight matrix; and determine the value of the second luminance syntax element identification information as a first value if the third rate-distortion value is less than the fourth rate-distortion value; or determine the value of the second luminance syntax element identification information as a second value if the third rate-distortion value is greater than or equal to the fourth rate-distortion value.

[0394] In some embodiments, the encoding unit 1104 is further configured to encode the value of the second luminance syntax element identification information and write the encoded bits into the bitstream.

[0395] In some embodiments, the first setting unit 1105 is further configured to set a luminance CTU level flag bit, which is used to control whether the luminance component of the current block is filtered using a luminance weight matrix.

[0396] Accordingly, the first determining unit 1101 is further configured to turn on the luminance CTU level flag bit if the third rate distortion value is less than the fourth rate distortion value; or to turn off the luminance CTU level flag bit if the third rate distortion value is greater than or equal to the fourth rate distortion value.

[0397] In some embodiments, the first determining unit 1101 is further configured to determine the target reconstructed image block of the current block using a luminance weight matrix if the third rate distortion value is less than the fourth rate distortion value.

[0398] In some embodiments, after determining that the video sequence is filtered using a weight matrix, the first determining unit 1101 is further configured to: determine that the weight matrix is ​​a chroma weight matrix when the color component type of the current frame is a chroma component; determine the fifth rate distortion value of the chroma component of the current frame in the video sequence filtered using the chroma weight matrix; determine the sixth rate distortion value of the chroma component of the current frame in the video sequence not filtered using the chroma weight matrix; and if the fifth rate distortion value is less than the sixth rate distortion value, determine that the chroma syntax element identification information takes a first value; or, if the fifth rate distortion value is greater than or equal to the sixth rate distortion value, determine that the chroma syntax element identification information takes a second value.

[0399] In some embodiments, the encoding unit 1104 is further configured to encode the value of the chroma syntax element identification information and write the encoded bits into the bitstream.

[0400] In some embodiments, the first setting unit 1105 is further configured to set a chroma frame level flag bit, which is used to control whether the chroma components of the current frame are filtered using a weight matrix.

[0401] Accordingly, the first determining unit 1101 is further configured to turn on the chroma frame level flag if the fifth rate distortion value is less than the sixth rate distortion value; or to turn off the chroma frame level flag if the fifth rate distortion value is greater than or equal to the sixth rate distortion value.

[0402] In some embodiments, the first value is 1 and the second value is 0.

[0403] In some embodiments, the first determining unit 1101 is further configured to determine the target reconstructed image block of the current block using a chroma weight matrix if the fifth rate distortion value is less than the sixth rate distortion value.

[0404] In some embodiments, the first determining unit 1101 is further configured to determine the input reconstructed image block and the output reconstructed image block of the neural network loop filter; and to input the output reconstructed image block into the weight matrix network model to obtain the weight matrix of the current block.

[0405] In some embodiments, the first filtering unit 1102 is specifically configured to use a weight matrix to perform weighted processing on the input reconstructed image block and the output reconstructed image block to obtain the target reconstructed image block.

[0406] In some embodiments, the first determining unit 1101 is further configured to determine a luminance weight matrix using a luminance weight matrix network model when the color component type of the current block is a luminance component; and to determine a chrominance weight matrix using a chrominance weight matrix network model when the color component type of the current block is a chrominance component.

[0407] In some embodiments, the first determining unit 1101 is further configured to determine the input luminance reconstruction image block and the output luminance reconstruction image block of the neural network loop filter when the color component type of the current block is a luminance component; and to input the output luminance reconstruction image block into the luminance weight matrix network model to obtain the luminance weight matrix.

[0408] The first filtering unit 1102 is further configured to use a luminance weight matrix to perform weighted processing on the input luminance reconstructed image block and the output luminance reconstructed image block to obtain the target reconstructed image block of the luminance component of the current block.

[0409] In some embodiments, the first determining unit 1101 is further configured to determine the input chroma reconstructed image block and the output chroma reconstructed image block of the neural network loop filter when the color component type of the current block is a chroma component; and to input the output chroma reconstructed image block into the chroma weight matrix network model to obtain the chroma weight matrix.

[0410] The first filtering unit 1102 is further configured to use a chromaticity weight matrix to perform weighted processing on the input chromaticity reconstructed image block and the output chromaticity reconstructed image block to obtain the target reconstructed image block of the chromaticity component of the current block.

[0411] In some embodiments, the input reconstructed image patch is obtained by filtering via a deblocking filter and a sample adaptive compensation filter.

[0412] In some embodiments, the first filtering unit 1102 is further configured to continue filtering the target reconstructed image block using an adaptive correction filter after determining the target reconstructed image block of the current block.

[0413] Understandably, in the embodiments of this application, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and can also be a module or a non-modular one. Furthermore, the components in this embodiment can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit described above can be implemented in hardware or as a software functional module.

[0414] If the integrated unit is implemented as a software functional module and not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the method described in this embodiment. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0415] Therefore, this application provides a computer storage medium applied to an encoder 110, the computer storage medium storing a computer program that, when executed by a first processor, implements the method described in any of the foregoing embodiments.

[0416] Based on the composition of the encoder 110 and the computer storage medium described above, see [link to documentation]. Figure 12 This illustrates a schematic diagram of the specific hardware structure of the encoder 110 provided in an embodiment of this application. Figure 12 As shown, it may include: a first communication interface 1201, a first memory 1202, and a first processor 1203; the various components are coupled together through a first bus system 1204. It is understood that the first bus system 1204 is used to realize the connection and communication between these components. In addition to a data bus, the first bus system 1204 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 12 The general designated all buses as the first bus system 1204. Among them,

[0417] The first communication interface 1201 is used for receiving and sending signals during the process of sending and receiving information with other external network elements;

[0418] The first memory 1202 is used to store computer programs that can run on the first processor 1203;

[0419] The first processor 1203 is configured to, when running the computer program, execute:

[0420] Determine the weight matrix network model for the current block, and determine the weight matrix for the current block based on the weight matrix network model;

[0421] Determine the value of at least one syntax element identification information;

[0422] When at least one syntax element identifies that the current frame or current block is to be filtered using a weight matrix, the target image block to be reconstructed for the current block is determined using the weight matrix.

[0423] It is understood that the first memory 1202 in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The first memory 1202 of the system and method described in this application is intended to include, but is not limited to, these and any other suitable types of memory.

[0424] The first processor 1203 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the first processor 1203 or by software instructions. The first processor 1203 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in the first memory 1202. The first processor 1203 reads the information in the first memory 1202 and completes the steps of the above method in conjunction with its hardware.

[0425] It is understood that the embodiments described in this application can be implemented using hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in this application, or combinations thereof. For software implementation, the technology described in this application can be implemented through modules (e.g., procedures, functions, etc.) that perform the functions described in this application. Software code can be stored in memory and executed by a processor. The memory can be implemented in the processor or external to the processor.

[0426] Alternatively, as another embodiment, the first processor 1203 is further configured to execute the method described in any of the foregoing embodiments when running the computer program.

[0427] This embodiment provides an encoder, which may include a first determining unit and a first filtering unit. By utilizing a weighted matrix network model to determine the weight matrix, not only can deep learning-based weighted matrix technology be implemented, providing pixel-level weighting processing for the use of output reconstructed image patches, but coding performance can also be improved, thereby increasing encoding and decoding efficiency. Furthermore, this application can make the output of the neural network loop filter closer to the original image, thus improving video image quality.

[0428] In another embodiment of this application, based on the same inventive concept as the foregoing embodiments, see [link to previous embodiment]. Figure 13 This illustrates a schematic diagram of the composition structure of a decoder 130 provided in an embodiment of this application. Figure 13 As shown, the decoder 130 may include: a parsing unit 1301, a second determining unit 1302, and a second filtering unit 1303; wherein,

[0429] Parsing unit 1301 is configured to parse the code stream and determine the value of at least one syntax element identification information;

[0430] The second determining unit 1302 is configured to determine the weight matrix network model of the current block and determine the weight matrix of the current block based on the weight matrix network model when at least one syntax element identification information indicates that the current frame or the current block is filtered using a weight matrix.

[0431] The second filtering unit 1303 is configured to use a weight matrix to determine the target reconstructed image block of the current block.

[0432] In some embodiments, the parsing unit 1301 is specifically configured to parse the bitstream to determine the value of the first syntax element identification information; and when the first syntax element identification information indicates that the video sequence is filtered using a weight matrix, to parse the bitstream to determine the value of the second syntax element identification information; wherein the second syntax element identification information is used to indicate whether the current frame in the video sequence is filtered using a weight matrix, and the current frame includes the current block.

[0433] In some embodiments, the second determining unit 1302 is further configured to determine that if the value of the first syntax element identification information is a first value, the first syntax element identification information indicates that the video sequence is filtered using a weight matrix; or, if the value of the first syntax element identification information is a second value, the first syntax element identification information indicates that the video sequence is not filtered using a weight matrix.

[0434] In some embodiments, the second determining unit 1302 is further configured to determine the second syntax element identification information as the first luminance syntax element identification information when the color component type of the current frame is a luminance component, the first luminance syntax element identification information being used to indicate whether the luminance component of the current frame is filtered using a luminance weight matrix; or, when the color component type of the current frame is a chrominance component, determine the second syntax element identification information as the chrominance syntax element identification information, the chrominance syntax element identification information being used to indicate whether the chrominance component of the current frame is filtered using a chrominance weight matrix.

[0435] In some embodiments, when the color component type is a luminance component, the parsing unit 1301 is further configured to parse the bitstream and determine the value of the second luminance syntax element identification information when the first luminance syntax element identification information indicates that the luminance component of the current frame is filtered using a luminance weight matrix.

[0436] The second determining unit 1302 is further configured to determine the luminance weight matrix network model of the current block when the second luminance syntax element identification information indicates that the luminance component of the current block is filtered using a luminance weight matrix.

[0437] In some embodiments, see Figure 13 The decoder 130 may further include a second setting unit 1304, configured to set a luminance frame-level flag bit and a luminance CTU-level flag bit; wherein, the luminance frame-level flag bit is used to control whether the luminance component of the current frame is filtered using a luminance weight matrix, and the luminance CTU-level flag bit is used to control whether the luminance component of the current block is filtered using a luminance weight matrix.

[0438] In some embodiments, the second determining unit 1302 is further configured to turn on the luminance frame level flag if the value of the first luminance syntax element identifier information is a first value; or to turn off the luminance frame level flag if the value of the first luminance syntax element identifier information is a second value.

[0439] In some embodiments, the second determining unit 1302 is further configured to turn on the luminance CTU level flag if the value of the second luminance syntax element identification information is a first value; or to turn off the luminance CTU level flag if the value of the second luminance syntax element identification information is a second value.

[0440] In some embodiments, the second determining unit 1302 is further configured to determine that if the value of the first luminance syntax element identifier information is a first value, the first luminance syntax element identifier information indicates that the luminance component of the current frame is filtered using a luminance weight matrix; or, if the value of the first luminance syntax element identifier information is a second value, the first luminance syntax element identifier information indicates that the luminance component of the current frame is not filtered using a luminance weight matrix.

[0441] In some embodiments, the second determining unit 1302 is further configured to determine that if the value of the second luminance syntax element identifier information is a first value, the second luminance syntax element identifier information indicates that the luminance component of the current block is filtered using a luminance weight matrix; or, if the value of the second luminance syntax element identifier information is a second value, the second luminance syntax element identifier information indicates that the luminance component of the current block is not filtered using a luminance weight matrix.

[0442] In some embodiments, when the color component type of the current frame is a chroma component, the second determining unit 1302 is further configured to determine the chroma weight matrix network model of the current block when the chroma syntax element identification information indicates that the chroma component of the current frame is filtered using a chroma weight matrix.

[0443] In some embodiments, the second setting unit 1304 is further configured to set a chroma frame level flag bit; wherein the chroma frame level flag bit is used to control whether the chroma components of the current frame are filtered using a weight matrix.

[0444] In some embodiments, the second determining unit 1302 is further configured to turn on the chroma frame level flag if the value of the chroma syntax element identification information is a first value; or to turn off the chroma frame level flag if the value of the chroma syntax element identification information is a second value.

[0445] In some embodiments, the second determining unit 1302 is further configured to determine that if the value of the chroma syntax element identifier information is a first value, the chroma syntax element identifier information indicates that the chroma components of the current frame are filtered using a chroma weight matrix; or, if the value of the chroma syntax element identifier information is a second value, the chroma syntax element identifier information indicates that the chroma components of the current frame are not filtered using a chroma weight matrix.

[0446] In some embodiments, the first value is 1 and the second value is 0.

[0447] In some embodiments, the second determining unit 1302 is further configured to: determine at least one candidate luminance weight matrix network model when the color component type of the current block is luminance component; determine the quantization parameter of the current block; select the candidate luminance weight matrix network model corresponding to the quantization parameter from the at least one candidate luminance weight matrix network model; and determine the selected candidate luminance weight matrix network model as the luminance weight matrix network model of the current block.

[0448] In some embodiments, the second determining unit 1302 is further configured to: determine at least one candidate chromaticity weight matrix network model when the color component type of the current block is chromaticity component; determine the quantization parameter of the current block; select the candidate chromaticity weight matrix network model corresponding to the quantization parameter from the at least one candidate chromaticity weight matrix network model; and determine the selected candidate chromaticity weight matrix network model as the chromaticity weight matrix network model of the current block.

[0449] In some embodiments, see Figure 13 The decoder 130 may also include a second training unit 1305;

[0450] The second determining unit 1302 is further configured to determine at least one training sample, wherein the training sample is obtained based on at least one quantization parameter;

[0451] The second training unit 1305 is configured to train a preset neural network model using the luminance component of at least one training sample to obtain at least one candidate luminance weight matrix network model; and to train the preset neural network model using the chrominance component of at least one training sample to obtain at least one candidate chrominance weight matrix network model; wherein, at least one candidate luminance weight matrix network model has a correspondence with the luminance component and the quantization parameter, and at least one candidate chrominance weight matrix network model has a correspondence with the chrominance component and the quantization parameter.

[0452] In some embodiments, the preset neural network model includes at least one of the following: at least one convolutional layer, at least one activation layer, and a jump connection layer.

[0453] In some embodiments, the second determining unit 1302 is further configured to determine the input reconstructed image block and the output reconstructed image block of the neural network loop filter; and to input the output reconstructed image block into the weight matrix network model to obtain the weight matrix of the current block.

[0454] In some embodiments, the second filtering unit 1303 is further configured to perform weighted processing on the input reconstructed image block and the output reconstructed image block using a weight matrix to obtain the target reconstructed image block.

[0455] In some embodiments, the second determining unit 1302 is further configured to determine a luminance weight matrix using a luminance weight matrix network model when the color component type of the current block is a luminance component; and to determine a chrominance weight matrix using a chrominance weight matrix network model when the color component type of the current block is a chrominance component.

[0456] In some embodiments, the second determining unit 1302 is further configured to determine the input luminance reconstruction image block and the output luminance reconstruction image block of the neural network loop filter when the color component type of the current block is a luminance component; and to input the output luminance reconstruction image block into the luminance weight matrix network model to obtain the luminance weight matrix.

[0457] The second filtering unit 1303 is further configured to use a luminance weight matrix to perform weighted processing on the input luminance reconstructed image block and the output luminance reconstructed image block to obtain the target reconstructed image block of the luminance component of the current block.

[0458] In some embodiments, the second determining unit 1302 is further configured to determine the input chroma reconstructed image block and the output chroma reconstructed image block of the neural network loop filter when the color component type of the current block is chroma component; and to input the output chroma reconstructed image block into the chroma weight matrix network model to obtain the chroma weight matrix.

[0459] The second filtering unit 1303 is further configured to use a chromaticity weight matrix to perform weighted processing on the input chromaticity reconstructed image block and the output chromaticity reconstructed image block to obtain the target reconstructed image block of the chromaticity component of the current block.

[0460] In some embodiments, the input reconstructed image patch is obtained by filtering via a deblocking filter and a sample adaptive compensation filter.

[0461] In some embodiments, the second filtering unit 1303 is further configured to continue filtering the target reconstructed image block using an adaptive correction filter after the target reconstructed image block of the current block has been determined.

[0462] Understandably, in this embodiment, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and can also be a module or a non-modular component. Furthermore, the components in this embodiment can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional module.

[0463] If the integrated unit is implemented as a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, this embodiment provides a computer storage medium applied to the decoder 130. This computer storage medium stores a computer program, which, when executed by a second processor, implements the method described in any of the foregoing embodiments.

[0464] Based on the composition of the decoder 130 and the computer storage medium described above, see [link to documentation]. Figure 14 This illustrates a schematic diagram of the specific hardware structure of the decoder 130 provided in an embodiment of this application. Figure 14 As shown, it may include: a second communication interface 1401, a second memory 1402, and a second processor 1403; the various components are coupled together via a second bus system 1404. It is understood that the second bus system 1404 is used to implement communication between these components. In addition to a data bus, the second bus system 1404 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 14 The various buses are all labeled as the second bus system 1404. Among them,

[0465] The second communication interface 1401 is used for receiving and sending signals during the process of sending and receiving information with other external network elements;

[0466] The second memory 1402 is used to store computer programs that can run on the second processor 1403;

[0467] The second processor 1403 is configured to, when running the computer program, perform:

[0468] Parse the bitstream to determine the value of at least one syntax element identifier.

[0469] When at least one syntax element identifier indicates that the current frame or current block is filtered using a weight matrix, the weight matrix network model of the current block is determined, and the weight matrix of the current block is determined based on the weight matrix network model.

[0470] The target image block for the current block is determined using the weight matrix.

[0471] Alternatively, as another embodiment, the second processor 1403 is also configured to perform the method described in any of the foregoing embodiments when running the computer program.

[0472] It is understood that the second memory 1402 has similar hardware functions to the first memory 1202, and the second processor 1403 has similar hardware functions to the first processor 1203; these will not be described in detail here.

[0473] This embodiment provides a decoder, which may include a parsing unit, a second determining unit, and a second filtering unit. By utilizing a weighted matrix network model to determine the weight matrix, not only can deep learning-based weighted matrix technology be implemented, providing pixel-level weighting processing for the use of output reconstructed image patches, but coding performance can also be improved, thereby increasing encoding and decoding efficiency. Furthermore, this application can make the output of the neural network loop filter closer to the original image, thus improving video image quality.

[0474] It should be noted that, in this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0475] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0476] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0477] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0478] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0479] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0480] Industrial applicability

[0481] In this embodiment, on the encoder side, the weight matrix network model of the current block is determined, and the weight matrix of the current block is determined based on the weight matrix network model; the value of at least one syntax element identifier is determined; when the at least one syntax element identifier indicates that the current frame or the current block uses the weight matrix for filtering, the target reconstructed image block of the current block is determined using the weight matrix. On the decoder side, the value of at least one syntax element identifier is determined by parsing the bitstream; when the at least one syntax element identifier indicates that the current frame or the current block uses the weight matrix for filtering, the weight matrix network model of the current block is determined, and the weight matrix of the current block is determined based on the weight matrix network model; the target reconstructed image block of the current block is determined using the weight matrix. Thus, by using the weight matrix network model to determine the weight matrix, not only can deep learning-based weight matrix technology be implemented, providing pixel-level weighting processing for the use of the output reconstructed image block, but coding performance can also be improved, thereby increasing encoding and decoding efficiency; furthermore, this application can make the output of the neural network loop filter closer to the original image, improving video image quality.

Claims

1. A decoding method applied to a decoder, the method comprising: Parse the bitstream to determine the value of at least one syntax element identifier. When the at least one syntax element identification information indicates that the current frame or the current block is filtered using a weight matrix, the weight matrix network model of the current block is determined, and the weight matrix of the current block is determined according to the weight matrix network model; wherein, the filtering process is applied between a loop filter CNNLF based on a residual neural network and an adaptive correction filter ALF; Using the weight matrix, the target reconstructed image block for the current block is determined; Wherein, determining the weight matrix of the current block according to the weight matrix network model includes: Determine the input and output reconstructed image blocks of the neural network loop filter; The output reconstructed image block is input into the weight matrix network model to obtain the weight matrix of the current block.

2. The method according to claim 1, wherein, The parsing of the code stream determines the value of the identification information of at least one syntax element, including: Parse the bitstream to determine the value of the first syntax element identifier information; When the first syntax element identification information indicates that the video sequence is filtered using the weight matrix, the bitstream is parsed to determine the value of the second syntax element identification information; wherein, the second syntax element identification information is used to indicate whether the current frame in the video sequence is filtered using the weight matrix, and the current frame includes the current block.

3. The method according to claim 2, wherein, The method further includes: If the value of the first syntax element identifier is a first value, then it is determined that the first syntax element identifier indicates that the video sequence uses the weight matrix for filtering; or... If the value of the first syntax element identifier information is the second value, then it is determined that the first syntax element identifier information indicates that the video sequence does not use the weight matrix for filtering.

4. The method according to claim 2, wherein, The parsed code stream determines the second syntax element identification information, including: When the color component type of the current frame is a luminance component, the second syntax element identifier information is determined to be the first luminance syntax element identifier information. The first luminance syntax element identifier information is used to indicate whether the luminance component of the current frame is filtered using a luminance weight matrix; or... When the color component type of the current frame is chroma component, the second syntax element identification information is determined to be chroma syntax element identification information. The chroma syntax element identification information is used to indicate whether the chroma component of the current frame is filtered using a chroma weight matrix.

5. The method according to claim 4, wherein, When the color component type of the current frame is a luminance component, the step of determining the weight matrix network model of the current block when the identification information of the at least one syntax element indicates that a weight matrix is ​​used for filtering includes: When the first luminance syntax element identifier information indicates that the luminance component of the current frame is filtered using the luminance weight matrix, the bitstream is parsed to determine the value of the second luminance syntax element identifier information. When the second luminance syntax element identification information indicates that the luminance component of the current block is filtered using the luminance weight matrix, the luminance weight matrix network model of the current block is determined.

6. The method according to claim 5, wherein, The method further includes: If the value of the first luminance syntax element identifier is a first value, then it is determined that the first luminance syntax element identifier indicates that the luminance component of the current frame is filtered using the luminance weight matrix; or, If the value of the first luminance syntax element identifier is the second value, then it is determined that the first luminance syntax element identifier indicates that the luminance component of the current frame is not filtered using the luminance weight matrix.

7. The method according to claim 5, wherein, The method further includes: If the value of the second luminance syntax element identifier is the first value, then it is determined that the second luminance syntax element identifier indicates that the luminance component of the current block is filtered using the luminance weight matrix; or, If the value of the second luminance syntax element identifier is the second value, then it is determined that the second luminance syntax element identifier indicates that the luminance component of the current block is not filtered using the luminance weight matrix.

8. The method according to claim 4, wherein, When the color component type of the current frame is chroma component, the step of determining the weight matrix network model of the current block when the identification information of the at least one syntax element indicates that a weight matrix is ​​used for filtering includes: When the chroma syntax element identification information indicates that the chroma components of the current frame are filtered using the chroma weight matrix, the chroma weight matrix network model of the current block is determined.

9. The method according to claim 8, wherein, The method further includes: If the value of the chroma syntax element identifier information is the first value, then it is determined that the chroma syntax element identifier information indicates that the chroma components of the current frame are filtered using the chroma weight matrix; or, If the value of the chroma syntax element identifier information is the second value, then it is determined that the chroma syntax element identifier information indicates that the chroma components of the current frame are not filtered using the chroma weight matrix.

10. The method according to any one of claims 3, 6, 7 and 9, wherein, The first value is 1, and the second value is 0.

11. The method according to claim 5, wherein, The network model for determining the brightness weight matrix of the current block includes: If the color component type of the current block is a luminance component, at least one candidate luminance weight matrix network model is determined. Determine the quantization parameters of the current block, and select the candidate brightness weight matrix network model corresponding to the quantization parameters from the at least one candidate brightness weight matrix network model; The selected candidate brightness weight matrix network model is determined as the brightness weight matrix network model of the current block.

12. The method according to claim 8, wherein, The process of determining the chroma weight matrix network model for the current block includes: If the color component type of the current block is chromaticity component, at least one candidate chromaticity weight matrix network model is determined; Determine the quantization parameters of the current block, and select the candidate chromaticity weight matrix network model corresponding to the quantization parameters from the at least one candidate chromaticity weight matrix network model; The selected candidate chroma weight matrix network model is determined as the chroma weight matrix network model of the current block.

13. The method according to claim 11 or 12, wherein, The method further includes: Determine at least one training sample, wherein the training sample is obtained based on at least one quantization parameter; The luminance components of the at least one training sample are used to train a preset neural network model to obtain at least one candidate luminance weight matrix network model. The preset neural network model is trained using the chromaticity components of the at least one training sample to obtain at least one candidate chromaticity weight matrix network model; Specifically, the at least one candidate luminance weight matrix network model has a corresponding relationship with the luminance component and the quantization parameter, and the at least one candidate chrominance weight matrix network model has a corresponding relationship with the chrominance component and the quantization parameter.

14. The method according to claim 13, wherein, The preset neural network model includes at least one of the following: at least one convolutional layer, at least one activation layer, and a jump connection layer.

15. The method according to claim 1, wherein, The step of determining the target reconstructed image block for the current block using the weight matrix includes: The input reconstructed image block and the output reconstructed image block are weighted using the weight matrix to obtain the target reconstructed image block.

16. The method according to claim 1, wherein, Determining the weight matrix of the current block based on the weight matrix network model includes: When the color component type of the current block is a luminance component, the luminance weight matrix is ​​determined using the luminance weight matrix network model. When the color component type of the current block is chromaticity component, the chromaticity weight matrix is ​​determined using the chromaticity weight matrix network model.

17. The method according to claim 16, wherein, When the color component type of the current block is a luminance component, determining the luminance weight matrix using the luminance weight matrix network model includes: Determine the input and output brightness reconstruction image blocks of the neural network loop filter; The output brightness reconstructed image block is input into the brightness weight matrix network model to obtain the brightness weight matrix; The step of determining the target reconstructed image block for the current block using the weight matrix includes: The input luminance reconstructed image block and the output luminance reconstructed image block are weighted using the luminance weight matrix to obtain the target reconstructed image block of the luminance component of the current block.

18. The method according to claim 16, wherein, When the color component type of the current block is chroma component, the step of determining the chroma weight matrix using the chroma weight matrix network model includes: Determine the input and output chroma reconstructed image blocks of the neural network loop filter; The output chroma reconstructed image patch is input into the chroma weight matrix network model to obtain the chroma weight matrix; The step of determining the target reconstructed image block for the current block using the weight matrix includes: The input chromaticity reconstructed image block and the output chromaticity reconstructed image block are weighted using the chromaticity weight matrix to obtain the target reconstructed image block of the chromaticity components of the current block.

19. The method according to claim 1, wherein, The input reconstructed image block is obtained by filtering through a deblocking filter and a sample adaptive compensation filter.

20. The method according to claim 1, wherein, The method further includes: After the target reconstructed image block is determined, an adaptive correction filter is used to continue filtering the target reconstructed image block.

21. An encoding method applied to an encoder, the method comprising: Determine the weight matrix network model of the current block, and determine the weight matrix of the current block based on the weight matrix network model; Determine the value of at least one syntax element identification information; When the at least one syntax element identification information indicates that the current frame or the current block is filtered using a weight matrix, the target reconstructed image block of the current block is determined using the weight matrix; wherein, the filtering process is applied between a residual neural network-based loop filter CNNLF and an adaptive correction filter ALF; Wherein, determining the weight matrix of the current block according to the weight matrix network model includes: Determine the input and output reconstructed image blocks of the neural network loop filter; The output reconstructed image block is input into the weight matrix network model to obtain the weight matrix of the current block.

22. The method according to claim 21, wherein, The network model for determining the weight matrix of the current block includes: When the color component type of the current block is a luminance component, determine the luminance weight matrix network model of the current block; When the color component type of the current block is chroma component, determine the chroma weight matrix network model of the current block.

23. The method according to claim 22, wherein, The network model for determining the brightness weight matrix of the current block includes: If the color component type of the current block is a luminance component, at least one candidate luminance weight matrix network model is determined. Determine the quantization parameters of the current block, and select the candidate brightness weight matrix network model corresponding to the quantization parameters from the at least one candidate brightness weight matrix network model; The selected candidate brightness weight matrix network model is determined as the brightness weight matrix network model of the current block.

24. The method according to claim 22, wherein, The process of determining the chroma weight matrix network model for the current block includes: If the color component type of the current block is chromaticity component, at least one candidate chromaticity weight matrix network model is determined; Determine the quantization parameters of the current block, and select the candidate chromaticity weight matrix network model corresponding to the quantization parameters from the at least one candidate chromaticity weight matrix network model; The selected candidate chroma weight matrix network model is determined as the chroma weight matrix network model of the current block.

25. The method according to claim 23 or 24, wherein, The method further includes: Determine at least one training sample, wherein the training sample is obtained based on at least one quantization parameter; The luminance components of the at least one training sample are used to train a preset neural network model to obtain at least one candidate luminance weight matrix network model. The preset neural network model is trained using the chromaticity components of the at least one training sample to obtain at least one candidate chromaticity weight matrix network model; Specifically, the at least one candidate luminance weight matrix network model has a corresponding relationship with the luminance component and the quantization parameter, and the at least one candidate chrominance weight matrix network model has a corresponding relationship with the chrominance component and the quantization parameter.

26. The method of claim 25, wherein, The preset neural network model includes at least one of the following: at least one convolutional layer, at least one activation layer, and a jump connection layer.

27. The method according to claim 21, wherein, Determining the value of at least one syntax element identification information includes: If it is determined that the video sequence will be filtered using the weight matrix, then the value of the first syntax element identifier information is determined to be a first value; or, If it is determined that the video sequence will not be filtered using the weight matrix, then the value of the first syntax element identifier information is determined to be the second value.

28. The method according to claim 27, wherein, The method further includes: The value of the first syntax element identifier information is encoded, and the encoded bits are written into the bitstream.

29. The method according to claim 27, wherein, After determining that the video sequence is filtered using the weight matrix, when the color component type of the current frame is a luminance component, the weight matrix is ​​determined to be a luminance weight matrix. Determining the value of at least one syntax element identifier further includes: The first rate-distortion cost is determined by filtering the luminance component of the current frame within the video sequence using the luminance weight matrix. The second rate-distortion cost is determined for the luminance component of the current frame within the video sequence that has not been filtered using the luminance weight matrix. If the first rate-distortion cost is less than the second rate-distortion cost, then the value of the first luma syntax element identifier is determined to be the first value; or... If the first rate distortion value is greater than or equal to the second rate distortion value, then the value of the first luminance syntax element identifier information is determined to be the second value.

30. The method according to claim 29, wherein, The method further includes: The value of the first luminance syntax element identifier information is encoded, and the encoded bits are written into the bitstream.

31. The method according to claim 29, wherein, The method further includes: The current frame is divided into blocks to determine at least one block; wherein the at least one block includes the current block; The determination of the first rate-distortion cost of filtering the luminance component of the current frame within the video sequence using the luminance weight matrix includes: Calculate the third rate-distortion cost of the luminance components of each of the at least one segmented block using the luminance weight matrix for filtering. The calculated third rate distortion cost is summed to obtain the first rate distortion cost. The calculation of the second rate-distortion cost of the luminance component of the current frame within the video sequence before filtering using the luminance weight matrix includes: Calculate the fourth rate distortion cost of the luminance component of each of the at least one segmented block before filtering using the luminance weight matrix; The calculated fourth rate distortion cost is summed to obtain the second rate distortion cost.

32. The method according to claim 31, wherein, When the first rate-distortion cost is less than the second rate-distortion cost, determining the value of at least one syntax element identifier information further includes: The third rate-distortion arithmetic value is determined for the luminance component of the current block after filtering using the luminance weight matrix; and the fourth rate-distortion arithmetic value is determined for the luminance component of the current block after filtering using the luminance weight matrix. If the third rate distortion cost is less than the fourth rate distortion cost, then the value of the second luma syntax element identifier information is determined to be the first value; or, If the third rate distortion value is greater than or equal to the fourth rate distortion value, then the value of the second luminance syntax element identifier information is determined to be the second value.

33. The method according to claim 32, wherein, The method further includes: The value of the second luminance syntax element identifier information is encoded, and the encoded bits are written into the bitstream.

34. The method according to claim 32, wherein, When the at least one syntax element identification information indicates that the current frame or current block is filtered using a weight matrix, determining the target reconstructed image block of the current block using the weight matrix includes: If the third rate distortion value is less than the fourth rate distortion value, then the target reconstructed image block of the current block is determined using the luminance weight matrix.

35. The method according to claim 27, wherein, After determining that the video sequence is filtered using the weight matrix, when the color component type of the current frame is chroma component, the weight matrix is ​​determined to be a chroma weight matrix. Determining the value of at least one syntax element identifier further includes: The fifth rate distortion cost is determined by filtering the chroma components of the current frame within the video sequence using the chroma weight matrix. The sixth rate distortion cost is determined for the chroma components of the current frame in the video sequence that have not been filtered using the chroma weight matrix. If the fifth rate distortion cost is less than the sixth rate distortion cost, then the value of the chroma syntax element identifier information is determined to be the first value; or... If the fifth rate distortion value is greater than or equal to the sixth rate distortion value, then the value of the chroma syntax element identifier information is determined to be the second value.

36. The method according to claim 35, wherein, The method further includes: The values ​​of the chroma syntax element identifier information are encoded, and the encoded bits are written into the bitstream.

37. The method according to any one of claims 27, 29, 32, and 35, wherein, The first value is 1, and the second value is 0.

38. The method according to claim 35, wherein, When the at least one syntax element identification information indicates that the current frame or current block is filtered using a weight matrix, determining the target reconstructed image block of the current block using the weight matrix includes: If the fifth rate distortion value is less than the sixth rate distortion value, then the target reconstructed image block of the current block is determined using the chroma weight matrix.

39. The method according to claim 21, wherein, The step of determining the target reconstructed image block of the current block using the weight matrix includes: The input reconstructed image block and the output reconstructed image block are weighted using the weight matrix to obtain the target reconstructed image block.

40. The method according to claim 21, wherein, Determining the weight matrix of the current block based on the weight matrix network model includes: When the color component type of the current block is a luminance component, the luminance weight matrix is ​​determined using the luminance weight matrix network model. When the color component type of the current block is chromaticity component, the chromaticity weight matrix is ​​determined using the chromaticity weight matrix network model.

41. The method according to claim 40, wherein, When the color component type of the current block is a luminance component, determining the luminance weight matrix using the luminance weight matrix network model includes: Determine the input and output brightness reconstruction image blocks of the neural network loop filter; The output brightness reconstructed image block is input into the brightness weight matrix network model to obtain the brightness weight matrix; The step of determining the target reconstructed image block for the current block using the weight matrix includes: The input luminance reconstructed image block and the output luminance reconstructed image block are weighted using the luminance weight matrix to obtain the target reconstructed image block of the luminance component of the current block.

42. The method according to claim 40, wherein, When the color component type of the current block is chroma component, the step of determining the chroma weight matrix using the chroma weight matrix network model includes: Determine the input and output chroma reconstructed image blocks of the neural network loop filter; The output chroma reconstructed image patch is input into the chroma weight matrix network model to obtain the chroma weight matrix; The step of determining the target reconstructed image block for the current block using the weight matrix includes: The input chromaticity reconstructed image block and the output chromaticity reconstructed image block are weighted using the chromaticity weight matrix to obtain the target reconstructed image block of the chromaticity components of the current block.

43. The method according to claim 21, wherein, The input reconstructed image block is obtained by filtering through a deblocking filter and a sample adaptive compensation filter.

44. The method according to claim 21, wherein, The method further includes: After the target reconstructed image block of the current block is determined, the target reconstructed image block is further filtered using an adaptive correction filter.

45. A method for transmitting a code stream, wherein, The encoding method described in any one of claims 21 to 44 is used to generate a bitstream; and the bitstream is transmitted.

46. ​​An encoder, the encoder comprising a first determining unit and a first filtering unit; wherein, The first determining unit is configured to determine the weight matrix network model of the current block, and determine the weight matrix of the current block according to the weight matrix network model; And determine the value of at least one syntax element identification information; The first filtering unit is configured to determine the target reconstructed image block of the current block using the weight matrix when the at least one syntax element identification information indicates that the current frame or the current block is filtered using the weight matrix; wherein the filtering is applied between a loop filter CNNLF based on a residual neural network and an adaptive correction filter ALF. The first determining unit is further configured to determine the input reconstructed image block and the output reconstructed image block of the neural network loop filter; and to input the output reconstructed image block into the weight matrix network model to obtain the weight matrix of the current block.

47. An encoder, the encoder comprising a first memory and a first processor; wherein, The first memory is used to store computer programs that can run on the first processor; The first processor is configured to, when running the computer program, perform the method as described in any one of claims 21 to 44.

48. A decoder, the decoder comprising a parsing unit, a second determining unit, and a second filtering unit; wherein, The parsing unit is configured to parse the code stream and determine the value of at least one syntax element identification information. The second determining unit is configured to determine the weight matrix network model of the current block and determine the weight matrix of the current block according to the weight matrix network model when the at least one syntax element identification information indicates that the current frame or the current block is filtered using a weight matrix; wherein the filtering process is applied between a loop filter CNNLF based on a residual neural network and an adaptive correction filter ALF. The second filtering unit is configured to use the weight matrix to determine the target reconstructed image block of the current block; The second determining unit is further configured to determine the input reconstructed image block and the output reconstructed image block of the neural network loop filter; and to input the output reconstructed image block into the weight matrix network model to obtain the weight matrix of the current block.

49. A decoder, the decoder comprising a second memory and a second processor; wherein, The second memory is used to store computer programs that can run on the second processor; The second processor is configured to perform the method as described in any one of claims 1 to 20 when running the computer program.

50. A computer storage medium, wherein, The computer storage medium stores a computer program that, when executed, implements the method as described in any one of claims 1 to 20, or the method as described in any one of claims 21 to 44.

Images