Loop filtering implementation method and device and computer storage medium

Abstract

The embodiment of the invention discloses a loop filtering implementation method and device and a computer storage medium. The method comprises the steps of obtaining a to-be-filtered image; wherein the to-be-filtered image is generated by an original image in a to-be-coded video in a video coding process, the to-be-coded video comprises an original image frame, and the original image frame comprises the original image; determining fusion information of the to-be-filtered image; wherein the fusion information is obtained by fusing at least two image components of the to-be-filtered image and corresponding auxiliary information; and performing loop filtering processing on the to-be-filtered image based on the fusion information to obtain at least one image component of the to-be-filtered image after filtering.

Description

[0001] This application is a divisional application of Chinese national phase patent application 201980075008.5 for international patent application PCT / CN2019 / 077371 filed on March 7, 2019. Technical Field

[0002] The embodiments of the present application relate to the field of image processing technologies, and in particular, to a method, apparatus, and computer storage medium for implementing loop filtering. Background Art

[0003] In a video codec system, most video encodings adopt a hybrid coding framework based on block-shaped coding units (CodingUnits, CUs). Since adjacent CUs adopt different coding parameters, such as different transformation processes, different quantization parameters (Quantization Parameter, QP), different prediction methods, different reference image frames, etc., and the error sizes and their distribution characteristics introduced by each CU are independent of each other, discontinuities at the boundaries of adjacent CUs generate blocking effects, thereby affecting the subjective and objective quality of the reconstructed image and even affecting the prediction accuracy of subsequent encoding and decoding.

[0004] Thus, during the encoding and decoding process, a loop filter is used to improve the subjective and objective quality of the reconstructed image. Traditional loop filters usually artificially summarize the characteristics of distorted images and artificially design the filter structure and configure the filter coefficients, such as deblocking filtering, sample adaptive compensation, and adaptive loop filtering, etc. These filters that rely on artificial design do not fit the optimal filter well, have poor adaptive ability and filtering effect, and the encoder needs to write filter-related parameters that depend on local statistical information into the bitstream to ensure consistency between the encoding and decoding ends, which increases the encoding bit rate.

[0005] With the rapid development of deep learning theory, convolutional neural networks (ConvolutionalNeural Networks, CNNs) have been proposed in the industry to perform filtering processing on reconstructed images to remove image distortion, and have obtained obvious improvements in subjective and objective quality compared with traditional loop filters. However, the current CNN filters do not fully utilize relevant information, resulting in limited improvement in the subjective and objective quality of the reconstructed image. Summary of the Invention

[0006] The embodiments of the present application provide a method, apparatus, and computer storage medium for implementing loop filtering. By fusing coding parameters such as block partition information and / or QP information as auxiliary information with multiple input image components, not only the relationship between multiple image components is fully utilized, but also the computational complexity is reduced and the encoding bit rate is saved; at the same time, the subjective and objective quality of the video reconstructed image during the encoding and decoding process is further improved.

[0007] The technical solution of the embodiment of the present application can be implemented as follows:

[0008] In a first aspect, an embodiment of the present application provides a method for implementing loop filtering. The method includes:

[0009] Obtain an image to be filtered; wherein, the image to be filtered is generated during video encoding of an original image in a video to be encoded, the video to be encoded includes original image frames, and the original image frames include the original image;

[0010] Determine the fusion information of the image to be filtered; wherein, the fusion information is obtained by fusing at least two image components of the image to be filtered and corresponding auxiliary information;

[0011] Perform loop filtering processing on the image to be filtered based on the fusion information to obtain at least one image component after filtering the image to be filtered.

[0012] In a second aspect, an embodiment of the present application provides a device for implementing loop filtering. The device for implementing loop filtering includes: an acquisition unit, a determination unit, and a filtering unit, wherein,

[0013] The acquisition unit is configured to obtain an image to be filtered; wherein, the image to be filtered is generated during video encoding of an original image in a video to be encoded, the video to be encoded includes original image frames, and the original image frames include the original image;

[0014] The determination unit is configured to determine the fusion information of the image to be filtered; wherein, the fusion information is obtained by fusing at least two image components of the image to be filtered and corresponding auxiliary information;

[0015] The filtering unit is configured to perform loop filtering processing on the image to be filtered based on the fusion information to obtain at least one image component after filtering the image to be filtered.

[0016] In a third aspect, an embodiment of the present application provides a device for implementing loop filtering. The device for implementing loop filtering includes: a memory and a processor, wherein,

[0017] The memory is used to store a computer program that can run on the processor;

[0018] The processor is configured to execute the steps of the method described in the first aspect when running the computer program.

[0019] Fourthly, an embodiment of the present application provides a computer storage medium, which stores a loop filtering implementation program. When the loop filtering implementation program is executed by at least one processor, the steps of the method described in the first aspect are implemented.

[0020] An embodiment of the present application provides a loop filtering implementation method, device and computer storage medium. First, an image to be filtered is obtained, and the image to be filtered is an original image in a video to be encoded generated during the video encoding process; then, fusion information of the image to be filtered is determined; wherein, the fusion information is obtained by fusing at least two image components of the image to be filtered and corresponding auxiliary information; finally, loop filtering processing is performed on the image to be filtered based on the fusion information to obtain at least one image component after filtering the image to be filtered; in this way, by using coding parameters such as block partitioning information and / or QP information as auxiliary information to perform fusion processing with multiple input image components, not only the relationship between multiple image components is fully utilized, but also the problem of multiple complete network forward calculations required for these multiple image components is effectively avoided, thereby reducing the computational complexity and saving the coding bit rate; in addition, by incorporating auxiliary information such as block partitioning information and / or QP information, filtering can be further assisted, and the subjective and objective quality of the video reconstructed image during the encoding and decoding process is improved. Description of the Drawings

[0021] Figure 1 A schematic diagram of the composition structure of a traditional coding block diagram provided for related technical solutions;

[0022] Figure 2 A schematic diagram of the composition structure of an improved coding block diagram provided for an embodiment of the present application;

[0023] Figure 3 A schematic flowchart of a loop filtering implementation method provided for an embodiment of the present application;

[0024] Figure 4 A schematic diagram of the structure of a block partitioning matrix provided for an embodiment of the present application;

[0025] Figure 5 A schematic diagram of the composition structure of a traditional CNN filter provided for an embodiment of the present application;

[0026] Figure 6A and Figure 6B A schematic diagram of the composition structure of another traditional CNN filter provided for an embodiment of the present application;

[0027] Figure 7 A schematic diagram of the composition structure of a loop filtering framework provided for an embodiment of the present application;

[0028] Figure 8Schematic diagram of the composition structure of another loop filter framework provided by an embodiment of the present application;

[0029] Figure 9 Schematic diagram of the composition structure of a loop filter implementation device provided by an embodiment of the present application;

[0030] Figure 10 Schematic diagram of the specific hardware structure of a loop filter implementation device provided by an embodiment of the present application. Detailed implementation manners

[0031] In order to be able to understand the features and technical content of the embodiments of the present application in more detail, the implementation of the embodiments of the present application will be described in detail below with reference to the accompanying drawings. The accompanying drawings are only for reference and explanation, and are not used to limit the embodiments of the present application.

[0032] In a video coding and decoding system, the video to be encoded includes original image frames, and the original image frames include original images. The original images are subjected to various processes such as prediction, transformation, quantization, reconstruction, and filtering. During these processes, the processed video images may have pixel value offsets relative to the original images, resulting in visual impairments or artifacts. In addition, in the hybrid coding framework based on block-shaped CUs adopted by most video coding and decoding systems, since adjacent coding blocks adopt different coding parameters (such as different transformation processes, different QPs, different prediction methods, different reference image frames, etc.), the error magnitudes and their distribution characteristics introduced by each coding block are independent of each other, and the discontinuity at the boundaries of adjacent coding blocks generates blocking effects. These distortions not only affect the subjective and objective quality of the reconstructed images, but if the reconstructed images are used as reference images for subsequent coding pixels, they will even affect the prediction accuracy of subsequent coding and decoding, and further affect the size of the bits in the video bitstream. Therefore, in a video coding and decoding system, an in-loop filter is often added to improve the subjective and objective quality of the reconstructed images.

[0033] See Figure 1 which shows a traditional coding frame provided by a related technical solution Figure 10 of the composition structure diagram. As Figure 1 shown, this traditional coding frame Figure 10It may include components such as a transform and quantization unit 101, an inverse transform and inverse quantization unit 102, a prediction unit 103, a filtering unit 104, and an entropy coding unit 105. Among them, the prediction unit 103 further includes an intra prediction unit 1031 and an inter prediction unit 1032. For the input original image, coding tree units (CTUs) can be obtained through preliminary partitioning, and by further performing content-adaptive partitioning on a CTU, coding units (CUs) can be obtained. A CU generally contains one or more coding blocks (CBs). Intra prediction by the intra prediction unit 1031 or inter prediction by the inter prediction unit 1032 can be performed on the coding block to obtain residual information. The transform and quantization unit 101 is used to perform a transform on the coding block for the residual information, including transforming the residual information from the pixel domain to the transform domain and quantizing the obtained transform coefficients to further reduce the bit rate. After determining the prediction mode, the prediction unit 103 is further used to provide the selected intra prediction data or inter prediction data to the entropy coding unit 105. In addition, the inverse transform and inverse quantization unit 102 is used for the reconstruction of the coding block, reconstructing the residual block in the pixel domain. The reconstructed residual block removes block effect artifacts through the filtering unit 104, and then the reconstructed residual block is added to the decoded image buffer unit to generate a reconstructed reference image. The entropy coding unit 105 is used to encode various coding parameters and the quantized transform coefficients. For example, the entropy coding unit 105 adopts header information coding and context-based adaptive binary arithmetic coding (CABAC) algorithm, which can be used to encode the coding information indicating the determined prediction mode and output the corresponding bitstream.

[0034] For Figure 1 the aforementioned traditional coding frame Figure 10, the filtering unit 104 is a loop filter, also known as an In-Loop Filter. It can include a De-Blocking Filter (DBF) 1041, a Sample Adaptive Offset (SAO) filter 1042, an Adaptive Loop Filter (ALF) 1043, etc. Among them, the de-blocking filter 1041 is used to implement de-blocking filtering. In the next-generation video coding standard H.266 / Versatile Video Coding (VVC), for all coding block boundaries in the original image, first, the boundary strength is determined based on the coding parameters on both sides of the boundary, and whether to perform de-blocking filtering decision is judged according to the calculated block boundary texture degree value. Finally, the pixel information on both sides of the coding block boundary is corrected according to the boundary strength and the filtering decision. In VVC, after the de-blocking filtering is executed, in order to reduce the quantization distortion of high-frequency AC coefficients, the SAO technology is also introduced, that is, the sample adaptive compensation filter 1042; further, starting from the pixel domain, negative values are added to the pixels at the peaks and positive values are added to the pixels at the valleys for compensation processing. In VVC, after the de-blocking filtering and the sample adaptive compensation filtering are executed, the adaptive loop filter 1043 needs to be further used for filtering processing; for the adaptive loop filtering, it calculates the optimal filter in the mean square sense based on the pixel values of the original image and the pixel values of the distorted image. However, these filters (such as the de-blocking filter 1041, the sample adaptive compensation filter 1042, and the adaptive loop filter 1043, etc.) not only require fine manual design and a large number of judgment decisions; but also at the encoding end, the filter-related parameters that depend on local statistical information (such as filter coefficients and the Flag value indicating whether to select this filter, etc.) need to be written into the code stream to ensure the consistency between the encoding end and the decoding end, increasing the encoding bit count; at the same time, the filters designed manually do not fit well with the complex function of the real optimization target, and the filtering effect needs to be enhanced.

[0035] The embodiment of the present application provides a method for implementing loop filtering, which is applied to an improved coding block diagram; compared with Figure 1 the traditional coding block Figure 10In comparison, the main difference is that an improved loop filter is used to replace the deblocking filter 1041, sample adaptive compensation filter 1042, adaptive loop filter 1043, etc. in the related technical solutions. In the embodiments of the present application, the improved loop filter may be a convolutional neural network (CNN) filter or a filter established by other deep learning, which is not specifically limited in the embodiments of the present application.

[0036] Taking the convolutional neural network filter as an example, refer to Figure 2 , which shows a schematic structural diagram of the composition of an improved coding block diagram 20 provided by the embodiments of the present application. As Figure 2 shown, compared with the traditional coding block Figure 10 , the filtering unit 104 in the improved coding block diagram 20 includes a convolutional neural network filter 201. The convolutional neural network filter 201 can not only completely replace Figure 1 the deblocking filter 1041, sample adaptive compensation filter 1042, and adaptive loop filter 1043, but also partially replace Figure 1 any one or two of the deblocking filter 1041, sample adaptive compensation filter 1042, and adaptive loop filter 1043 in Figure 1 , and even can be used in combination with any one or more of the deblocking filter 1041, sample adaptive compensation filter 1042, and adaptive loop filter 1043 in Figure 1 or Figure 2 . It should also be noted that for each component shown in Figure 1 or Figure 2 , such as the transform and quantization unit 101, inverse transform and inverse quantization unit 102, prediction unit 103, filtering unit 104, entropy coding unit 105, or convolutional neural network filter 201, these components may be virtual modules or hardware modules. In addition, those skilled in the art can understand that these units do not limit the coding block diagram, and the coding block diagram may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0037] In the embodiments of the present application, after the convolutional neural network filter 201 undergoes filtering network training, it can be directly deployed at the encoding end and the decoding end, so that there is no need to transmit any filter-related parameters; moreover, the convolutional neural network filter 201 can also fuse auxiliary information such as block partition information and / or QP information with multiple input image components; in this way, not only the relationship between multiple image components is fully utilized, but also the computational complexity is reduced, the coding rate is saved; at the same time, the subjective and objective quality of the video reconstructed image in the encoding and decoding process is further improved.

[0038] It should be noted that the loop filtering implementation method in the embodiments of the present application can be applied not only to the encoding system, but also to the decoding system. Generally speaking, in order to save the encoding bit rate and ensure that the decoding system can perform correct decoding processing, the loop filter in the embodiments of the present application must be synchronously deployed in both the encoding system and the decoding system. The following will take the application in the encoding system as an example for detailed description.

[0039] See Figure 3 , which shows a schematic flowchart of a loop filtering implementation method provided by an embodiment of the present application. The method may include:

[0040] S301: Obtain the image to be filtered; wherein, the image to be filtered is generated during the video encoding process of the original image in the video to be encoded, the video to be encoded includes original image frames, and the original image frames include the original image;

[0041] S302: Determine the fusion information of the image to be filtered; wherein, the fusion information is obtained by fusing at least two image components of the image to be filtered and corresponding auxiliary information;

[0042] S303: Perform loop filtering processing on the image to be filtered based on the fusion information to obtain at least one image component after filtering the image to be filtered.

[0043] It should be noted that since the original image can be divided into CTUs, or the CTUs can be further divided into CUs; that is to say, the block division information in the embodiments of the present application can refer to CTU division information or CU division information; in this way, the loop filtering implementation method in the embodiments of the present application can be applied not only to loop filtering at the CU level, but also to loop filtering at the CTU level, and the embodiments of the present application do not make specific limitations.

[0044] In the embodiments of the present application, by obtaining the image to be filtered; wherein, the image to be filtered is generated during the video encoding process of the original image in the video to be encoded, the video to be encoded includes original image frames, and the original image frames include the original image; determining the fusion information of the image to be filtered; wherein, the fusion information is obtained by fusing at least two image components of the image to be filtered and corresponding auxiliary information; in this way, not only the relationship between multiple image components is fully utilized, but also the problem of multiple complete network forward calculations required for these multiple image components can be avoided, thereby reducing the computational complexity and saving the encoding bit rate; finally, loop filtering processing is performed on the image to be filtered based on the fusion information to obtain at least one image component after filtering the image to be filtered; since the fusion information incorporates auxiliary information such as block division information and / or QP information, it can further assist in filtering, thereby improving the subjective and objective quality of the video reconstructed image during the encoding and decoding processes.

[0045] In some embodiments, the image components include a first image component, a second image component, and a third image component; wherein, the first image component represents a luminance component, the second image component represents a first chrominance component, and the third image component represents a second chrominance component.

[0046] It should be noted that in video images, the first image component, the second image component, and the third image component are generally used to represent the original image or the image to be filtered. Among them, in the luminance-chrominance component representation method, these three image components are respectively a luminance component, a blue chrominance (chrominance difference) component, and a red chrominance (chrominance difference) component; specifically, the luminance component is usually denoted by the symbol Y, the blue chrominance component is usually denoted by the symbol Cb, and can also be denoted by U; the red chrominance component is usually denoted by the symbol Cr, and can also be denoted by V. In the embodiments of the present application, the first image component can be the luminance component Y, the second image component can be the blue chrominance component U, and the third image component can be the red chrominance component V, but the embodiments of the present application do not make specific limitations. Among them, at least one image component represents one or more of the first image component, the second image component, and the third image component, and at least two image components can be the first image component, the second image component, and the third image component, or the first image component and the second image component, or the first image component and the third image component, or even the second image component and the third image component, and the embodiments of the present application do not make specific limitations.

[0047] In the next-generation video coding standard VVC, its corresponding test model is the VVC Test Model (VTM). When conducting tests on VTM, the currently standard test sequences use the YUV 4:2:0 format, and each frame of the video to be encoded in this format can be composed of three image components: a luminance component (denoted by Y) and two chrominance components (denoted by U and V). Assuming that the height of the original image in the video to be encoded is H and the width is W, then the size information corresponding to the first image component is H×W, and the size information corresponding to the second image component or the third image component is It should be noted that the embodiments of the present application will be described by taking the YUV 4:2:0 format as an example, but the loop filtering implementation method of the embodiments of the present application is also applicable to other sampling formats.

[0048] Taking the case where YUV is in the 4:2:0 format as an example, since the size information of the first image component is different from that of the second image component or the third image component, in order to input the first image component and / or the second image component and / or the third image component into the improved loop filter at one time, it is necessary to sample or reorganize these three image components at this time so that the spatial size information of the three image components is the same.

[0049] In some embodiments, pixel rearrangement processing (which can also be referred to as downsampling processing) can be performed on the high-resolution image component so that the spatial size information of the three image components is the same. Specifically, before fusing at least two image components of the image to be filtered and the auxiliary information, the method further includes:

[0050] For at least two image components of the image to be filtered, select the high-resolution image component;

[0051] Perform pixel rearrangement processing on the high-resolution image component.

[0052] It should be noted that if the first image component is the luminance component, the second image component is the first chrominance component, and the third image component is the second chrominance component; then the high-resolution image component is the first image component, and at this time, pixel rearrangement processing needs to be performed on the first image component. Exemplarily, taking a raw image with a size of 2×2 as an example, converting it into 4 channels, that is, arranging the tensor of 2×2×1 into a tensor of 1×1×4; then when the size information of the first image component of the raw image is H×W, it can be converted into before loop filtering; since the size information of both the second image component and the third image component is In this way, the spatial size information of the three image components can be made the same; subsequently, after merging the first image component, the second image component, and the third image component after pixel rearrangement processing, these three image components are transformed into and input into the improved loop filter in this form.

[0053] In some embodiments, upsampling processing can also be performed on the low-resolution image component so that the spatial size information of the three image components is the same. Specifically, before fusing at least two image components of the image to be filtered and the auxiliary information, the method further includes:

[0054] For at least two image components of the image to be filtered, select the low-resolution image component;

[0055] Perform upsampling processing on the low-resolution image component.

[0056] It should be noted that, in addition to performing pixel rearrangement processing on the size information of the high-resolution image component (i.e., downscaling), in the embodiments of the present application, upsampling processing (i.e., upscaling) can also be performed on the low-resolution image component. Additionally, for the low-resolution image component, not only upsampling processing can be performed, but also deconvolution processing can be performed, and even super-resolution processing can be performed, etc. The effects of these three types of processing are the same, and the embodiments of the present application do not make specific limitations.

[0057] It should also be noted that if the first image component is a luminance component, the second image component is the first chrominance component, and the third image component is the second chrominance component; then the low-resolution image component is the second image component or the third image component, and at this time, upsampling processing needs to be performed on the second image component or the third image component. Exemplarily, when the size information of the second image component and the third image component of the original image is both at this time, before performing loop filtering, it can be converted into the form of H×W through upsampling processing; since the size information of the first image component is H×W, this can also make the spatial domain size information of the three image components the same, and the upsampled second image component and the upsampled third image component will be consistent with the resolution of the first image component.

[0058] In some embodiments, the obtaining of the image to be filtered includes:

[0059] Performing video encoding processing on the original image in the video to be encoded, and using the generated reconstructed image as the image to be filtered; or,

[0060] Performing video encoding processing on the original image in the video to be encoded to generate a reconstructed image; performing preset filtering processing on the reconstructed image, and using the preset filtered image as the image to be filtered.

[0061] It should be noted that based on the improved encoding block diagram 20, during the process of performing video encoding on the original image in the video to be encoded, when performing video encoding processing on the original image, it is subjected to CU partitioning, prediction, transformation, quantization, etc. processing, and in order to obtain a reference image for performing video encoding on the subsequent images to be encoded, inverse transformation, inverse quantization, reconstruction, and filtering, etc. processing can also be performed. In this way, the image to be filtered in the embodiments of the present application can be the reconstructed image generated after reconstruction processing during the video encoding process, or the preset filtered image obtained by performing preset filtering on the reconstructed image using other preset filtering methods (such as a deblocking filtering method), and the embodiments of the present application do not make specific limitations.

[0062] In some embodiments, before determining the fusion information of the image to be filtered, the method further includes:

[0063] Determine the auxiliary information corresponding to the image to be filtered; wherein, the auxiliary information at least includes block partitioning information and / or quantization parameter information.

[0064] Understandably, the auxiliary information can be used to assist in filtering and improve the filtering quality. In the embodiments of the present application, the auxiliary information can not only be block partitioning information (such as CU partitioning information and / or CTU partitioning information), but also quantization parameter information, and even motion vector (MV) information, prediction direction information, etc.; these information can be used as auxiliary information alone, or can be combined arbitrarily as auxiliary information. For example, the block partitioning information can be used as auxiliary information alone, or the block partitioning information and quantization parameter information can be used as auxiliary information together, or the block partitioning information and MV information can be used as auxiliary information together, etc. The embodiments of the present application do not make specific limitations.

[0065] It can also be understood that since the original image can be divided into CTUs, or CTUs can be divided into CUs; therefore, the loop filtering implementation method in the embodiments of the present application can not only be applied to loop filtering at the CU level (at this time, the block partitioning information is CU partitioning information), but also be applied to loop filtering at the CTU level (at this time, the block partitioning information is CTU partitioning information). The embodiments of the present application do not make specific limitations. Hereinafter, the CU partitioning information will be used as an example of the block partitioning information for description.

[0066] In some embodiments, the determining the auxiliary information corresponding to the image to be filtered includes:

[0067] Perform CU partitioning on the original image in the video to be encoded to obtain CU partitioning information, and use the CU partitioning information as the block partitioning information corresponding to the image to be filtered.

[0068] Further, in some embodiments, the using the CU partitioning information as the block partitioning information corresponding to the image to be filtered includes:

[0069] For the CU partitioning information, fill a first value at each pixel position corresponding to the CU boundary, and fill a second value at other pixel positions to obtain a first matrix corresponding to the CU partitioning information; wherein, the first value is different from the second value;

[0070] Use the first matrix as the block partitioning information corresponding to the image to be filtered.

[0071] It should be noted that the first value can be a preset numerical value, letter, etc., and the second value can also be a preset numerical value, letter, etc., and the first value is different from the second value; for example, the first value can be set to 2 and the second value can be set to 1, but the embodiments of the present application do not make specific limitations.

[0072] In the embodiments of the present application, the CU partition information can be used as auxiliary information to assist in filtering the image to be filtered. That is to say, in the process of video encoding the original image in the video to be encoded, the CU partition information can be fully utilized, and it can be fused with at least two image components of the image to be filtered to guide the filtering.

[0073] Specifically, the CU partition information is converted into a Coding Unit Map (CUmap), which is represented by a two-dimensional matrix, that is, the CUmap matrix, which is also the first matrix in the embodiments of the present application; that is to say, for the original image, it can be divided into multiple CUs; at each pixel position corresponding to the CU boundary, the first value is filled, and at other pixel positions, the second value is filled, so that a first matrix reflecting the CU partition information can be constructed. Exemplarily, refer to Figure 4 , which shows a schematic structural diagram of a block partition matrix provided by the embodiments of the present application. As Figure 4 shown, if this figure represents a CTU, then the CTU can be divided into 9 CUs; assuming that the first value is set to 2 and the second value is set to 1; in this way, at each pixel position corresponding to the CU boundary, 2 is filled, and at other pixel positions, 1 is filled, that is to say, the pixel positions filled with 2 represent the boundaries of the CUs, so that the CU partition information, that is, the auxiliary information corresponding to the image to be filtered, can be determined.

[0074] In some embodiments, determining the auxiliary information corresponding to the image to be filtered includes:

[0075] Obtaining the quantization parameter corresponding to the original image in the video to be encoded, and using the quantization parameter as the quantization parameter information corresponding to the image to be filtered.

[0076] Further, in some embodiments, using the quantization parameter as the quantization parameter information corresponding to the image to be filtered includes:

[0077] Establishing a second matrix with the same size as the original image; wherein, the normalized value of the quantization parameter corresponding to the original image is filled at each pixel position in the second matrix;

[0078] Using the second matrix as the quantization parameter information corresponding to the image to be filtered.

[0079] It should be noted that for the images to be filtered corresponding to different quantization parameters, the distortion degrees are not the same. If the quantization parameter information is incorporated, then the filtering network can be enabled to adaptively have the ability to process any quantization parameter during the training process.

[0080] In the embodiments of the present application, quantization parameter information can also be used as auxiliary information to assist in filtering the image to be filtered. That is to say, in the process of video encoding the original image in the video to be encoded, the quantization parameter information can be fully utilized and fused with at least two image components of the image to be filtered to guide the filtering. Among them, the quantization parameter information can be normalized, or the quantization parameter information can be non-normalized (such as classification processing, interval partitioning processing, etc.); hereinafter, the case of normalizing the quantization parameter will be described in detail as an example.

[0081] Specifically, the quantization parameter information is converted into a second matrix reflecting the quantization parameter information; that is to say, taking the original image as an example, a matrix with the same size as the original image is established, and the positions of each pixel point in the matrix are filled with the normalized values of the quantization parameters corresponding to the original image; among them, the normalized value of the quantization parameter is represented by QP max (x,y), that is:

[0082]

[0083] In Equation (1), QP represents the quantization parameter value corresponding to the original image, x represents the abscissa value of the position of each pixel point in the CU block, and y represents the ordinate value of the position of each pixel point in the CU block; QP max represents the maximum value of the quantization parameter. Generally speaking, the value of QP max is 51, but QP max can also be other values, such as 29, 31, etc., which are not specifically limited in the embodiments of the present application.

[0084] In some embodiments, the improved loop filter includes a convolutional neural network filter.

[0085] It should be noted that the improved loop filter is used to implement the loop filtering process of the image to be filtered. Among them, the improved loop filter can be a convolutional neural network filter or a filter established by other deep learning, which is not specifically limited in the embodiments of the present application. Here, the convolutional neural network filter, also called the CNN filter, is a type of feedforward neural network that contains convolutional calculations and has a deep structure, and is one of the representative algorithms of deep learning. The input layer of the CNN filter can process multi-dimensional data, such as the three image component (Y / U / V) channels of the original image in the video to be encoded.

[0086] See Figure 5 , which shows a schematic diagram of the composition structure of a traditional CNN filter 50 provided by the embodiments of the present application. As Figure 5As shown, the traditional CNN filter 50 is improved based on the previous-generation video coding standard H.265 / High Efficiency Video Coding (HEVC). It includes a two-layer convolutional network structure and can replace the deblocking filter and the sample adaptive offset filter. After inputting the image to be filtered (denoted by F in ) into the input layer of the traditional CNN filter 50, it sequentially passes through the first-layer convolutional network F1 (assuming the size of the convolutional kernel is 3×3 and it contains 64 feature maps) and the second-layer convolutional network F2 (assuming the size of the convolutional kernel is 5×5 and it contains 32 feature maps), and then a residual information F3 is obtained. Then, the image to be filtered F in and the residual information F3 are subjected to a summation operation, and finally the filtered image output by the traditional CNN filter 50 (denoted by F out ) is obtained. Among them, this convolutional network structure is also called a residual neural network and is used to output the residual information corresponding to the image to be filtered. In this traditional CNN filter 50, the three image components (Y / U / V) of the image to be filtered are processed independently, but share the same filtering network and the relevant parameters of the filtering network.

[0087] See Figure 6A and Figure 6B , which shows a schematic diagram of the composition structure of another traditional CNN filter 60 provided by an embodiment of the present application; this traditional CNN filter 60 uses two filtering networks. For example, the filtering network shown in Figure 6A is dedicated to outputting the first image component, and the filtering network shown in Figure 6B is dedicated to outputting the second image component or the third image component. Assume that the height of the original image in the video to be encoded is H and the width is W. Then the size information corresponding to the first image component is H×W, and pixel rearrangement processing can be performed on the first image component to convert it into the form of ; since the size information corresponding to the second image component or the third image component is both , then after merging these three image components, it is transformed into the form of and input into the traditional CNN filter 60. Based on the filtering network shown in Figure 6A , after the input layer network receives the image to be filtered F in (assuming the size of the convolutional kernel is N×N and the number of channels is 6), it passes through the first-layer convolutional network F 1-Y (assuming the size of the convolutional kernel is L1×L1, the number of convolutional kernels is M, and the number of channels is 6) and the second-layer convolutional network F 2-Y (assuming the size of the convolutional kernel is L2×L2, the number of convolutional kernels is 4, and the number of channels is M), and then a residual information F 3-Y(Assume that the size of the convolutional kernel is N×N and the number of channels is 4); then the input image F to be filtered in and the residual information F 3-Y are subjected to a summation operation, and finally the first filtered image component output by the traditional CNN filter 60 is obtained (denoted by F out-Y ). Based on the filtering network shown in Figure 6B , after the input layer network receives the image F to be filtered in (Assume that the size of the convolutional kernel is N×N and the number of channels is 6), it passes through the first convolutional network F 1-U (Assume that the size of the convolutional kernel is L1×L1, the number of convolutional kernels is M, and the number of channels is 6) and the second convolutional network F 2-U (Assume that the size of the convolutional kernel is L2×L2, the number of convolutional kernels is 2, and the number of channels is M), and then a residual information F 3-U (Assume that the size of the convolutional kernel is N×N and the number of channels is 2) is obtained; then the input image F to be filtered in and the residual information F 3-U are subjected to a summation operation, and finally the second or third filtered image component output by the traditional CNN filter 60 is obtained (denoted by F out-U ).

[0088] For Figure 5 the traditional CNN filter 50 shown, or Figure 6A and Figure 6B the traditional CNN filter 60 shown, since the relationship between different image components is not considered and it is not reasonable to process each image component independently; in addition, encoding parameters such as block partition information and QP information are not fully utilized at the input end. However, the distortion of the reconstructed image mainly comes from block effects, and the boundary information of the block effects is determined by CU partition information; that is to say, the filtering network in the CNN filter should focus on the boundary region; in addition, integrating quantization parameter information into the filtering network helps to improve its generalization ability so that it can filter distorted images of any quality. Therefore, the loop filtering implementation method provided by the embodiments of the present application not only has a reasonable CNN filtering structure, and the same filtering network can receive multiple image components at the same time, but also fully considers the relationship between these multiple image components, and can simultaneously output enhanced images of these image components after filtering; in addition, the loop filtering implementation method can also use encoding parameters such as block partition information and / or QP information as auxiliary information for auxiliary filtering, thereby improving the filtering quality.

[0089] In some embodiments, determining the fusion information of the image to be filtered includes:

[0090] Fuse at least two image components of the image to be filtered and corresponding auxiliary information to obtain the fusion information of the image to be filtered.

[0091] It should be noted that the fusion information in the embodiments of the present application may be to fuse the first image component, the second image component, the third image component of the image to be filtered and the auxiliary information to obtain the fusion information; it may also be to fuse the first image component, the second image component of the image to be filtered and the auxiliary information to obtain the fusion information; it may also be to fuse the first image component, the third image component of the image to be filtered and the auxiliary information to obtain the fusion information; even it may be to fuse the second image component, the third image component of the image to be filtered and the auxiliary information to obtain the fusion information; the embodiments of the present application do not make specific limitations. It should also be noted that "fusing at least two image components of the image to be filtered and corresponding auxiliary information" may be to first fuse at least two image components of the image to be filtered, and then incorporate the auxiliary information; it may also be to first incorporate each of the at least two image components of the image to be filtered with the corresponding auxiliary information respectively, and then fuse the processed at least two image components; that is to say, the embodiments of the present application do not make specific limitations on the specific manner of the fusion process.

[0092] In addition, for "performing loop filtering processing on the image to be filtered based on the fusion information to obtain at least one image component after filtering the image to be filtered" in the embodiments of the present application, specifically, after the multiple image components (such as the first image component, the second image component, and the third image component) of the image to be filtered and the auxiliary information are fused and input into the filtering network, it may be to only output the first image component after filtering the image to be filtered, or the second image component after filtering, or the third image component after filtering, or it may be to output the first image component after filtering and the second image component after filtering, or the second image component after filtering and the third image component after filtering, or the first image component after filtering and the third image component after filtering, and even it may be the first image component after filtering, the second image component after filtering, and the third image component after filtering the image to be filtered; the embodiments of the present application do not make specific limitations.

[0093] Taking the example that three image components of the image to be filtered are input into the filtering network simultaneously, refer to Figure 7 which shows a schematic structural diagram of the composition of a loop filtering framework 70 provided by the embodiments of the present application. As Figure 7As shown in the figure, the loop filter framework 70 may include three image components of the image to be filtered (represented by Y, U, and V respectively) 701, auxiliary information 702, an input fusion unit 703, a joint processing unit 704, a first adder 705, a second adder 706, a third adder 707, and three filtered image components (represented by Out_Y, Out_U, and Out_V respectively) 708. Among them, the input fusion unit 703, the joint processing unit 704, and the first adder 705, the second adder 706, and the third adder 707 together constitute the improved loop filter in the embodiments of the present application; and the input fusion unit 703 is used to fuse all three image components 701 of the image to be filtered and the auxiliary information 702 together, and then input them into the joint processing unit 704; the joint processing unit 704 includes a multi-layer convolutional filter network for performing convolutional calculations on the input information. Since the specific convolutional calculation process is similar to the related technical solutions, the specific execution steps of the joint processing unit 704 will not be described here. After passing through the joint processing unit 704, the residual information of the Y image component, the residual information of the U image component, and the residual information of the V image component can be obtained respectively; the Y image component in the three image components 701 of the image to be filtered and the obtained residual information of the Y image component are jointly input into the first adder 705, and the output of the first adder 705 is the filtered Y image component (represented by Out_Y); the U image component in the three image components 701 of the image to be filtered and the obtained residual information of the U image component are jointly input into the second adder 706, and the output of the second adder 706 is the filtered U image component (represented by Out_U); the V image component in the three image components 701 of the image to be filtered and the obtained residual information of the V image component are jointly input into the third adder 707, and the output of the third adder 707 is the filtered V image component (represented by Out_V). Here, for the output components, if only the filtered Y image component needs to be output, the loop filter framework 70 may not include the second adder 706 and the third adder 707; if only the filtered U image component needs to be output, the loop filter framework 70 may not include the first adder 705 and the third adder 707; if the filtered Y image component and the filtered U image component need to be output, the loop filter framework 70 may not include the third adder 707; the embodiments of the present application do not make specific limitations.

[0094] Taking the example of simultaneously inputting two image components of the image to be filtered into the filter network, see Figure 8 , which shows a schematic structural diagram of the composition of another loop filter framework 80 provided by the embodiments of the present application. As Figure 8As shown, the loop filtering framework 80 includes two image components of the image to be filtered (denoted by Y and U respectively) 801, auxiliary information 702, an input fusion unit 703, a joint processing unit 704, a first adder 705, a second adder 706, and two filtered image components (denoted by Out_Y and Out_U respectively) 802. Different from Figure 7 the loop filtering framework 70 shown, the loop filtering framework 80 fuses all two image components 801 of the image to be filtered and the auxiliary information 702 together and then inputs them into the joint processing unit 704; after passing through the joint processing unit 704, the residual information of the Y image component and the residual information of the U image component can be obtained respectively; the Y image component in the two image components 801 of the image to be filtered and the obtained residual information of the Y image component are jointly input into the first adder 705, and the output of the first adder 705 is the filtered Y image component (denoted by Out_Y); the U image component in the two image components 801 of the image to be filtered and the obtained residual information of the U image component are jointly input into the second adder 706, and the output of the second adder 706 is the filtered U image component (denoted by Out_U). Here, for the output components, if only the filtered Y image component needs to be output, the loop filtering framework 70 may not include the second adder 706; if only the filtered U image component needs to be output, the loop filtering framework 70 may not include the first adder 705; the embodiments of the present application do not make specific limitations. It should be noted that if only a single image component of the image to be filtered is considered for filtering at a time, there is no need to consider the fusion between multiple image components at this time, which is the same as the filtering processing method of the traditional CNN filter in the related technical solutions, and the embodiments of the present application will not be elaborated further.

[0095] Take Figure 7Taking the loop filter framework 70 shown as an example, it uses a deep learning network (such as CNN) for loop filtering. The difference from the traditional CNN filter is that the improved loop filter in the embodiments of the present application can input the three image components of the image to be filtered into the filtering network simultaneously, and also incorporates other coding-related auxiliary information (such as coding parameters like block partition information, quantization parameter information, MV information, etc.). After fusing all this information, it is input into the filtering network at once. In this way, not only the relationship between the three image components is fully utilized, but also other coding-related auxiliary information is used to assist in filtering, improving the filtering quality. Additionally, by processing the three image components simultaneously, the problem of needing to perform three complete network forward calculations for these three image components is effectively avoided, thereby reducing the computational complexity and saving the coding bit rate. For example, taking VTM3.0 as a benchmark, in a certain experimental test, it is found that compared with the related technical solutions, the loop filtering implementation method of the embodiments of the present application can achieve a 6.4% bit rate reduction for the Y image component, a 9.8% bit rate reduction for the U image component, and an 11.6% bit rate reduction for the V image component while maintaining the same restored video quality, thus saving the coding bit rate.

[0096] The above embodiments provide a loop filtering implementation method. By obtaining the image to be filtered, where the image to be filtered is generated during the video coding process of the original image in the video to be encoded; determining the fusion information of the image to be filtered; where the fusion information is obtained by fusing at least two image components of the image to be filtered and the corresponding auxiliary information; performing loop filtering processing on the image to be filtered based on the fusion information to obtain at least one image component of the filtered image to be filtered. In this way, coding parameters such as block partition information and / or QP information are used as auxiliary information to perform fusion processing with the input multiple image components, not only fully utilizing the relationship between the multiple image components, but also reducing the computational complexity and saving the coding bit rate; at the same time, it further improves the subjective and objective quality of the video reconstructed image in the encoding and decoding process.

[0097] Based on the same inventive concept as the foregoing embodiments, refer to Figure 9 , which shows a schematic structural diagram of a loop filtering implementation device 90 provided by the embodiments of the present application. The loop filtering implementation device 90 may include: an obtaining unit 901, a determining unit 902, and a filtering unit 903, where

[0098] The obtaining unit 901 is configured to obtain the image to be filtered; where the image to be filtered is generated during the video coding process of the original image in the video to be encoded, the video to be encoded includes original image frames, and the original image frames include the original image;

[0099] The determining unit 902 is configured to determine the fusion information of the image to be filtered; wherein, the fusion information is obtained by fusing at least two image components of the image to be filtered and corresponding auxiliary information;

[0100] The filtering unit 903 is configured to perform loop filtering processing on the image to be filtered based on the fusion information to obtain at least one image component after filtering the image to be filtered.

[0101] In the above solution, the obtaining unit 901 is specifically configured to perform video encoding processing on the original image in the video to be encoded, and use the generated reconstructed image as the image to be filtered; or,

[0102] The obtaining unit 901 is specifically configured to perform video encoding processing on the original image in the video to be encoded to generate a reconstructed image; perform preset filtering processing on the reconstructed image, and use the preset filtered image as the image to be filtered.

[0103] In the above solution, the determining unit 902 is further configured to determine the auxiliary information corresponding to the image to be filtered; wherein, the auxiliary information at least includes block partitioning information and / or quantization parameter information.

[0104] In the above solution, refer to Figure 9 , the loop filtering implementation device 90 further includes a partitioning unit 904, configured to perform CU partitioning on the original image in the video to be encoded to obtain CU partitioning information, and use the CU partitioning information as the block partitioning information corresponding to the image to be filtered.

[0105] In the above solution, the determining unit 902 is specifically configured to fill a first value at each pixel position corresponding to the CU boundary for the CU partitioning information, and fill a second value at other pixel positions to obtain a first matrix corresponding to the CU partitioning information; wherein, the first value is different from the second value; and use the first matrix as the block partitioning information corresponding to the image to be filtered.

[0106] In the above solution, the obtaining unit 901 is further configured to obtain the quantization parameter corresponding to the original image in the video to be encoded, and use the quantization parameter as the quantization parameter information corresponding to the image to be filtered.

[0107] In the above solution, the determining unit 902 is specifically configured to establish a second matrix having the same size as the original image; wherein, the normalized value of the quantization parameter corresponding to the original image is filled at each pixel position in the second matrix; and use the second matrix as the quantization parameter information corresponding to the image to be filtered.

[0108] In the above solution, refer toFigure 9 The loop filter implementation device 90 further includes a fusion unit 905 configured to perform a fusion process on at least two image components of the image to be filtered and corresponding auxiliary information to obtain the fusion information of the image to be filtered.

[0109] In the above solution, referring to Figure 9 The loop filter implementation device 90 further includes a sampling unit 906 configured to select low-resolution image components for at least two image components of the image to be filtered; and perform upsampling processing on the low-resolution image components.

[0110] It can be understood that in this embodiment, a "unit" may be a part of a circuit, a part of a processor, a part of a program or software, etc. Of course, it may also be a module or non-modular. Moreover, the components in this embodiment may be integrated in one processing unit, or each unit may exist physically alone, or two or more units may be integrated in one unit. The above integrated unit may be implemented in the form of hardware or in the form of a software function module.

[0111] If the integrated unit is implemented in the form of a software function module and is not sold or used as an independent product, it may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, in essence, or the part that contributes to the prior art, or all or part of this technical solution, may be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the method described in this embodiment. The foregoing storage medium includes: various media such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disc that can store program codes.

[0112] Therefore, this embodiment provides a computer storage medium storing a loop filter implementation program, and when the loop filter implementation program is executed by at least one processor, the steps of the method described in the foregoing embodiment are implemented.

[0113] Based on the composition of the loop filter implementation device 90 and the computer storage medium, referring to Figure 10, which shows a specific hardware structure example of the loop filtering implementation device 90 provided in the embodiments of the present application, may include: a network interface 1001, a memory 1002, and a processor 1003; each component is coupled together through a bus system 1004. It can be understood that the bus system 1004 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 1004 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clear illustration, in Figure 10 all kinds of buses are labeled as the bus system 1004. Among them, the network interface 1001 is used for receiving and sending signals during the process of receiving and sending information with other external network elements;

[0114] The memory 1002 is used to store a computer program that can run on the processor 1003;

[0115] The processor 1003 is used to execute, when running the computer program:

[0116] Obtain the image to be filtered; wherein, the image to be filtered is generated during the video encoding process of the original image in the video to be encoded, the video to be encoded includes original image frames, and the original image frames include the original image;

[0117] Determine the fusion information of the image to be filtered; wherein, the fusion information is obtained by fusing at least two image components of the image to be filtered and corresponding auxiliary information;

[0118] Perform loop filtering processing on the image to be filtered based on the fusion information to obtain at least one image component after filtering the image to be filtered.

[0119] It can be understood that the memory 1002 in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchlink dynamic random access memory (SLDRAM), and direct rambus random access memory (DRRAM). The memory 1002 of the systems and methods described herein is intended to include but not be limited to these and any other suitable types of memory.

[0120] The processor 1003 may be an integrated circuit chip with signal processing capabilities. In the implementation process, the steps of the above method can be completed by the integrated logic circuit in the hardware of the processor 1003 or instructions in the form of software. The above-mentioned processor 1003 may be a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as being executed and completed by a hardware decoding processor, or executed and completed by a combination of hardware and software modules in the decoding processor. The software module may be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory 1002, and the processor 1003 reads the information in the memory 1002 and combines its hardware to complete the steps of the above method.

[0121] It can be understood that these embodiments described herein can be implemented using hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, other electronic units for performing the functions described in the present application, or a combination thereof.

[0122] For software implementation, the technologies described herein can be implemented by modules (such as procedures, functions, etc.) that execute the functions described herein. The software code can be stored in a memory and executed by a processor. The memory can be implemented inside or outside the processor.

[0123] Optionally, as another embodiment, the processor 1003 is further configured to execute the steps of the method described in the foregoing embodiments when running the computer program.

[0124] It should be noted that the technical solutions described in the embodiments of the present application can be arbitrarily combined without conflict.

[0125] As described above, the above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present application, and all should be covered by the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

[0126] Industrial Applicability

[0127] In the embodiment of the present application, first, an image to be filtered is obtained, and the image to be filtered is generated during the video encoding process of the original image in the video to be encoded; then, the fusion information of the image to be filtered is determined; wherein, the fusion information is obtained by fusing at least two image components of the image to be filtered and corresponding auxiliary information; finally, loop filtering processing is performed on the image to be filtered based on the fusion information to obtain at least one image component after filtering the image to be filtered; in this way, by using coding parameters such as block partitioning information and / or QP information as auxiliary information to perform fusion processing with multiple input image components, not only the relationship between multiple image components is fully utilized, but also the problem of multiple complete network forward calculations required for these multiple image components is effectively avoided, thereby reducing the computational complexity and saving the coding bit rate; in addition, by incorporating auxiliary information such as block partitioning information and / or QP information, further auxiliary filtering can be performed, and the subjective and objective quality of the video reconstructed image in the encoding and decoding process is improved.

Claims

1. A method for implementing loop filtering, applied to an encoding system, characterized in that The method includes: Obtaining an image to be filtered; wherein, the image to be filtered is generated during the encoding of the original image in the video to be encoded; Determining the fusion information of the image to be filtered; wherein, the fusion information includes at least two image components of the image to be filtered; Obtaining the loop filter output of the image to be filtered based on the fusion information, to obtain at least one image component in the loop filter output of the image to be filtered; Wherein, at least two image components of the image to be filtered include at least two of a first image component, a second image component, and a third image component, and at least one image component in the loop filter output includes one of the first image component, the second image component, and the third image component.

2. The method according to claim 1, wherein The obtaining of the image to be filtered includes: Performing video encoding processing on the original image in the video to be encoded, and using the generated reconstructed image as the image to be filtered.

3. The method according to claim 1, wherein The obtaining of the image to be filtered includes: Performing video encoding processing on the original image in the video to be encoded to generate a reconstructed image; Performing a preset filtering process on the reconstructed image, and using the image after the preset filtering as the image to be filtered.

4. The method according to claim 1, characterized in that, The method further includes: Determining the auxiliary information corresponding to the image to be filtered; wherein, the auxiliary information at least includes block partitioning information, and the block partitioning information includes block size and block position.

5. The method according to claim 4, wherein The determining of the auxiliary information corresponding to the image to be filtered includes: Performing encoding unit CU partitioning on the original image in the video to be encoded to obtain CU partitioning information, and using the CU partitioning information as the block partitioning information corresponding to the image to be filtered.

6. The method according to claim 4 or 5, characterized in that, The determining of the fusion information of the image to be filtered includes: Performing a fusion process on at least two image components of the image to be filtered and the corresponding auxiliary information to obtain the fusion information of the image to be filtered.

7. A method for implementing loop filtering, applied to a decoding system, characterized in that The method includes: Obtaining an image to be filtered; Determining the fusion information of the image to be filtered; wherein, the fusion information includes at least two image components of the image to be filtered; Obtaining the loop filter output of the image to be filtered based on the fusion information, to obtain at least one image component in the loop filter output of the image to be filtered; Wherein, at least two image components of the image to be filtered include at least two of a first image component, a second image component, and a third image component, and at least one image component in the loop filter output includes one of the first image component, the second image component, and the third image component.

8. The method according to claim 7, characterized in that, The image to be filtered is a reconstructed image.

9. The method according to claim 7, wherein The image to be filtered is a reconstructed image after preset filtering processing.

10. The method according to claim 7, characterized in that, The method further includes: Determining the auxiliary information corresponding to the image to be filtered; wherein, the auxiliary information at least includes block partitioning information, and the block partitioning information includes block size and block position.

11. The method according to claim 10, wherein The determining of the auxiliary information corresponding to the image to be filtered includes: Using the CU partitioning information obtained after performing encoding unit CU partitioning on the original image in the video to be encoded as the block partitioning information corresponding to the image to be filtered.

12. The method according to claim 10 or 11, characterized in that, Determining the fusion information of the image to be filtered includes: Performing fusion processing on at least two image components of the image to be filtered and corresponding auxiliary information to obtain the fusion information of the image to be filtered.

13. A computer storage medium, characterized in that, The computer storage medium stores a loop filtering implementation program. When the loop filtering implementation program is applied to an encoding system, it implements the loop filtering method according to any one of claims 1-6 and outputs a corresponding bitstream.

Images