Loop filtering, video encoding, video decoding method, electronic device and medium

Abstract

This application proposes a loop filtering, video encoding, video decoding method, electronic device and medium, which includes obtaining a first reconstructed video unit processed by a loop filter, performing pixel classification on the first reconstructed video unit, determining corresponding intensity factors for each classification, and performing intensity adjustment on the first reconstructed video unit according to the intensity factors to obtain a second reconstructed video unit.

Description

[Technical Field]

[0001] The present application relates to the technical field of image processing, such as loop filtering, video encoding, video decoding methods, electronic devices, and media. [Background technology]

[0002] In-loop filtering technology can improve image compression efficiency, improve image coding quality, and reduce decoding errors through multi-layer image reconstruction. However, the filtering model in related technology generally uses an offline network, and there is no original image to refer to in actual use, so there is a problem that the strength is too strong or too weak in the filtering process. As a result, there is a large error between the reconstructed value and the original value, which affects the filtering effect. Summary of the Invention

[0003] The present application proposes a loop filtering, a video encoding, a video decoding method, an electronic device and a medium.

[0004] The present embodiment is Obtaining a first reconstructed video unit processed by a loop filter; performing pixel classification on the first reconstructed video unit; determining a corresponding intensity factor for each category; performing an intensity adjustment on the first reconstructed video unit in response to the intensity factor to obtain a second reconstructed video unit. A loop filtering method is proposed.

[0005] The present embodiment is filtering the reconstructed image of the current image to obtain a first reconstructed image; filtering the first reconstructed image by a loop filtering method to obtain a second reconstructed image; and marking loop filtering control information of the loop filtering method in a bitstream based on the filtering process, the first reconstructed image includes a first reconstructed video unit, the second reconstructed image includes a second reconstructed video unit, and the loop filtering method is determined according to the loop filtering method described above. A video encoding method is further proposed.

[0006] The present embodiment is obtaining a bitstream including an encoded video sequence and obtaining loop filtering control information associated with a loop filtering method; filtering the reconstructed image of the current image to obtain a first reconstructed image; and filtering the first reconstructed image through a loop filtering method based on the loop filtering control information to obtain a second reconstructed image, the first reconstructed image includes a first reconstructed video unit, the second reconstructed image includes a second reconstructed video unit, and the loop filtering method is determined according to the loop filtering method described above. A video decoding method is further proposed.

[0007] The present embodiment is We further propose an electronic device comprising a memory, a processor and a computer program stored in the memory and operable on the processor, wherein when the processor executes the program, the above-mentioned loop filtering method or video encoding method or video decoding method is realized.

[0008] An embodiment of the present application further proposes a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-mentioned loop filtering method or video encoding method or video decoding method. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a schematic diagram of one loop filtering according to an embodiment. [Figure 2] FIG. 2 is a flow diagram of one loop filtering method according to one embodiment. [Figure 3] FIG. 1 is a schematic diagram of the operation of one multi-intensity factor according to one embodiment. [Figure 4] 1 is a flow diagram of a video encoding method according to one embodiment. [Figure 5] FIG. 1 is a schematic diagram of one encoding-side framework according to one embodiment. [Figure 6] 1 is a schematic diagram of a video encoding process according to an embodiment; [Figure 7] 1 is a schematic diagram of a video encoding process according to an embodiment; [Figure 8] FIG. 1 is a schematic diagram of one ALF filtering according to one embodiment. [Figure 9] FIG. 10 is a schematic diagram of another video encoding process according to an embodiment. [Figure 10] FIG. 10 is a schematic diagram of a further video encoding process according to an embodiment. [Figure 11] FIG. 1 is a schematic diagram of another video encoding process according to an embodiment. [Figure 12] FIG. 1 is a schematic diagram of another video encoding process according to an embodiment. [Figure 13] FIG. 1 is a schematic diagram of another video encoding process according to an embodiment. [Figure 14] 1 is a flow diagram of a video decoding method according to one embodiment. [Figure 15] FIG. 1 is a schematic diagram of one decoding side framework according to one embodiment. [Figure 16] 1 is a structural schematic diagram of a loop filtering device according to an embodiment; [Figure 17] 1 is a structural schematic diagram of a video encoding device according to an embodiment; [Figure 18]1 is a structural schematic diagram of a video decoding device according to an embodiment; [Figure 19] FIG. 1 is a schematic diagram illustrating the hardware structure of an electronic device according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Compressed video using the new generation video coding standard H.266, also known as Versatile Video Coding (VVC), suffers from distortion effects such as blocking, ringing, color deviation, and image blur. To mitigate the impact of these distortions on video quality, video frames can be filtered using in-loop filtering. FIG. 1 is a schematic diagram of one loop filtering technique according to an embodiment. As shown in FIG. 1, an input original image can be processed using luma mapping with chroma scaling (LMCS), deblocking filtering (DBF), sample adaptive offset (SAO), and / or adaptive loop filtering (ALF) to output a reconstructed image. Among these, LMCS can improve compression efficiency by redistributing codewords within the dynamic range, DBF can be used to reduce blocking, SAO can be used to improve ringing, and ALF can reduce decoding errors.

[0011] In the embodiments of the present application, a filtering method based on neural networks (NN) may be adopted in the loop filtering process. NN filtering may be added as a new module or step to the loop filtering, or may replace the DBF and SAO modules or steps to save transmission overhead and ensure image quality. After NN filtering, ALF may be used to further improve image performance.

[0012] In the embodiment of the present application, intensity adjustment can be performed on the image during the loop filtering process, such as adjusting the brightness and / or chromaticity of pixels in a video frame. Furthermore, if there are too many pixel points in a frame of image, fitting a single line can only meet the intensity adjustment needs of some pixel points, which may lead to some adverse effects, such as needing to enhance a certain pixel point, but weakening the intensity of that pixel point based on the fitting result. This introduces more deviations between the reconstructed value and the original value during the intensity adjustment process, thereby reducing image quality and affecting the filtering effect. In the embodiment of the present application, intensity factors for different categories can be determined separately based on pixel classification, and a flexible intensity adjustment method is proposed.

[0013] In an embodiment of the present application, a loop filtering method is proposed, which may be mainly applied to the encoding side or the decoding side, and the loop filtering method can be used to perform filtering on the reconstructed image in the video in either the video encoding or decoding process. Figure 2 is a flow chart of a loop filtering method according to an embodiment, as shown in Figure 2, the method according to this embodiment includes:

[0014] In step 110, a first reconstructed video unit processed by the loop filter is obtained.

[0015] In this embodiment, the loop filter may refer to a model for performing filtering processing on an input image, and supports various filtering methods, such as NN, LMCS, DBF, SAO, and / or ALF filtering. The video unit may be an image of one frame or multiple frames in a video, or one or multiple slices. The first reconstructed video unit is obtained after processing by the loop filter.

[0016] In step 120, pixel classification is performed on the first reconstructed video unit.

[0017] In this embodiment, instead of using the same intensity factor to perform intensity adjustment for all pixels in the first reconstructed video unit, the first reconstructed video unit can be divided into multiple categories according to a set rule, for example, according to the brightness, chromaticity, area and / or loop filter type of the pixels in the first reconstructed video unit, and then intensity adjustment can be performed for different categories using corresponding intensity factors, so that all pixels in the first reconstructed video unit can obtain reasonable intensity adjustment and the filtering effect can be guaranteed.

[0018] In step 130, the corresponding intensity factors for each category are determined.

[0019] In this embodiment, the intensity factor (Scale) is the main variable used to adjust the intensity of the luminance and / or chromaticity of a pixel. Different magnitudes of the intensity factor result in different adjustment widths. The corresponding intensity factors for each category may be the same or different. The corresponding intensity factor for each category can be calculated according to the difference in pixel values ​​before and after filtering of the pixel points in that category. For example, a straight line is obtained by fitting according to the difference in pixel values ​​before and after filtering of the pixel points in that category, and the intensity factor for that category is determined according to the slope of the line. Since fitting is performed for each category to obtain a corresponding straight line, the diversity of intensity factors is increased, and flexible intensity adjustment for different categories can be achieved. For example, at the decoding side, the corresponding intensity factor for each category can be obtained by analyzing the encoded video data rather than by calculation.

[0020] In step 140, an intensity adjustment is performed on the first reconstructed video unit according to the intensity factor to obtain a second reconstructed video unit.

[0021] In this embodiment, intensity adjustment can be performed for different categories using corresponding intensity factors. This operation of adjusting intensity based on pixel classification and the intensity factors of each category may be called a multi-scale factor (MSF) operation, and adjustment can be performed for each luminance and / or chrominance component of pixel points in each category. MSF filtering can be applied to the entire filtered image, to one or more slices in the image, or to the luminance or chrominance component alone. The intensity-adjusted second reconstructed video unit can serve as a reference and basis for subsequent encoding.

[0022] 3 is a schematic diagram of a multi-intensity factor operation according to an embodiment. As shown in FIG. 3, the loop filter is a NN filter. After filtering by the NN filter, a first video reconstruction unit is obtained. When pixel classification is performed for the first video reconstruction unit, pixel classification can be performed according to chromaticity (including blue color difference component Cb and red color difference component Cr) and / or luma component (Luma). The numerical indicator represents the category to which each pixel belongs. For example, pixels marked with the numerical indicator "1" belong to the same category. In FIG. 3, a total of four categories, 1 to 4, are shown. Then, for each category, a corresponding intensity factor (Scale) is calculated, which is denoted as S1, S2, S3, and S4, respectively. The intensity factor corresponding to each category is used to perform intensity adjustment on the pixels of the corresponding category, thereby obtaining a second video reconstruction unit.

[0023] The loop filtering method of this embodiment can be understood as a multi-intensity factor filtering method based on pixel classification, and by performing intensity adjustment on the first reconstructed video unit according to the classification and then obtaining the second reconstructed video unit, the deviation between the reconstructed value and the original value in the second reconstructed video unit is smaller, it has less distortion, higher image quality, and better filtering effect.

[0024] In one embodiment, the loop filter includes a neural network-based loop filter (i.e., an NN filter). In this embodiment, the NN filter analyzes data based on deep learning and establishes a mapping from the distorted image (or reconstructed image) to the original image, thereby improving image quality. NN filters are generally offline networks, and in actual use, there is no original image to refer to, which can cause problems such as overly strong or weak intensity. However, in this embodiment, pixels are classified and intensity factors for each category are introduced, allowing for flexible control of the intensity of pixel points. Data for each term required for input to the neural network, such as reconstructed image samples, quantization parameter (QP) information, coding unit (CU) division information, deblocking filtering information, and predicted samples, can be arranged in the NN filter, and these data can be input to the NN filter, which then outputs a first reconstructed video unit after NN filtering.

[0025] In one embodiment, the reconstructed video unit includes a reconstructed image, a slice, a coding block, or a coding tree block, where the reconstructed video unit may be a first reconstructed video unit or a second reconstructed video unit.

[0026] In one embodiment, performing an intensity adjustment on the first reconstructed video unit in response to the intensity factor comprises: performing an intensity adjustment on the luminance component of the first reconstructed video unit according to the intensity factor (as shown in FIG. 3); performing an intensity adjustment on a first chrominance component (e.g., Cb) of the first reconstructed video unit according to the intensity factor; performing an intensity adjustment on a second chrominance component (eg, Cr) of the first reconstructed video unit in response to the intensity factor.

[0027] In one embodiment, determining the corresponding intensity factors for each category comprises: Step 131: Integrating pixel points of the same category into the same coordinate map, the horizontal coordinate of each pixel point is the output of the pixel point after passing through the loop filter, and the vertical coordinate is the input of the pixel point before passing through the loop filter; and step 133 of fitting the pixel points in the coordinate map based on the least squares method to obtain a straight line, and taking the slope of the line as the intensity factor corresponding to the corresponding classification.

[0028] In this embodiment, the intensity factor for each category can be calculated and applied to the encoding side. Taking FIG. 3 as an example, pixel classification is performed on the first reconstructed video unit after NN filtering. For one of the categories, the output after NN filtering of the pixel points belonging to that category is taken as the abscissa, and the input before NN filtering is taken as the ordinate. In this way, pixel points belonging to the same category in the first reconstructed video unit are placed on the same coordinate map. Then, a line is obtained by fitting using the least squares method, and the slope of this line is the intensity factor corresponding to that category. The number of intensity factors can be obtained as the number of categories increases. By using the least squares method to fit the line, the reconstructed image after processing by the loop filter can be made closer to the original image, thereby achieving the effect of self-adaptation between the filtering intensity and the original image.

[0029]

number

[0030] In one embodiment, determining the corresponding strength factors for each category includes determining the corresponding strength factors for each category by reading loop filtering control information associated with a loop filtering method.

[0031] In this embodiment, the strength factor for each category may be read from the loop filtering control information related to the loop filtering method by parsing the high-level syntax of the bitstream, and then applied on the decoding side.

[0032] In one embodiment, performing an intensity adjustment on the first reconstructed image in response to the intensity factor comprises: For each pixel point in each category, the method includes multiplying the difference between the output of the pixel point after passing through the loop filter and the input of the pixel point before passing through the loop filter by an intensity factor corresponding to the category, and adding the result of the multiplication to the input of the pixel point before passing through the loop filter to obtain an adjusted pixel value of the pixel point.

[0033]

number

[0034] In one embodiment, the pixel classification scheme is: performing pixel classification based on luminance components; performing pixel classification based on a first chromaticity component; performing pixel classification based on a second chromaticity component; performing pixel classification based on a block classification mode; performing pixel classification based on pixel point area; performing pixel classification based on classification of a neural network-based loop filter (NN filter); Performing pixel classification based on a classification mode used for deblocking filtering (DBF); performing pixel classification based on a classification mode used for pixel self-adaptive offset (SAO) filtering; and performing pixel classification using a neural network for pixel classification.

[0035] In this embodiment, pixel classification may be performed on the luminance component, or may be performed based on the ALF block classification mode. Also, various classification methods may be supported by taking into account factors such as the chromaticity component of the image, pixel point area, NN filtering, DBF filtering, and SAO filtering.

[0036] For example, when pixel classification is performed according to the directional factor and / or mobility factor of each pixel block based on the block classification mode of ALF, for example, there are five directional factors and five mobility factors, the first reconstructed video unit can be divided into 25 classes according to the two-class factors, the first reconstructed video unit can be divided into 5 classes according to the mobility factor, or the first reconstructed video unit can be divided into 5 classes according to the directional factor; For example, when pixel classification is performed based on the luminance component area of ​​the pixel point, for example, the first reconstructed video unit is divided into eight areas, that is, eight classes, and the number of coding units in each class is as equal as possible; For example, when pixel classification is performed based on an NN filter, for example, in the loop filtering process, different pixel blocks may select different NN filters, and each pixel block is classified according to the NN filter used for each block; Also, for example, in the case of classification based on other filtering models, the first reconstructed video unit may be divided into different categories according to the classification mode used for, for example, DBF and / or SAO filtering; For example, pixel classification can be performed using a neural network, for example, by training a neural network for pixel classification, whose input includes information such as pixel values ​​of the first reconstructed video unit and parameters of the loop filter, and whose output is the result of pixel classification.

[0037] Also, classification may be performed according to chromaticity components, for example, classification is performed based on the chromaticity component area of ​​the pixel point, for example, the first reconstructed video unit is divided into eight areas, that is, into eight classes, and the number of coding units in each class is as equal as possible; Further, for example, classification is performed according to at least one chromaticity component of Cb and Cr, and pixel points having the same Cb and / or Cr are classified into one category; Also, for example, in combination with the pixel classification information of the luminance component, the first reconstructed video unit is divided into different classes according to the chrominance component.

[0038] In one embodiment, performing pixel classification based on a block classification mode includes: a step 1210 of obtaining a pixel block of a first reconstructed video unit; a step 1220 of calculating the Laplacian gradient of each pixel block and calculating the directionality factor and the mobility factor based on the Laplacian gradient; determining a classification of the pixel blocks according to the directionality and / or mobility factors, and determining a classification of all pixel points in the reconstructed video unit 1230.

[0039]

number

[0040]

number

[0041] In addition, if there are too many pixel classifications, it may lead to an increase in the bitstream transmission overhead of video coding loop filtering, and for some classifications, the intensity adjustment operation may lead to a deterioration in coding performance rather than bringing about performance gain. Therefore, the embodiments of the present application further propose several competitive strategy decision mechanisms and intensity adjustment opening / closing control strategies based on classification.

[0042] In one embodiment, performing an intensity adjustment on the first reconstructed video unit in response to the intensity factor comprises: Step 1401: for each pixel point of each category, calculate the loss between the pixel after intensity adjustment and the original pixel by using the intensity factor corresponding to the category; Step 1403 includes adopting the intensity factors corresponding to some classes according to the losses corresponding to the pixel points of each class to perform intensity adjustment for the pixel points of the corresponding class.

[0043] In this embodiment, for multiple categories, the intensity adjustment for pixel points of each category can be further determined according to the degree of distortion between the intensity-adjusted pixels and the original pixels, i.e., the loss (cost). After performing pixel classification for the first reconstructed video unit to determine the intensity factors for each category, each intensity factor is used to perform intensity adjustment for pixel points of the corresponding category. When the loss corresponding to each category is calculated, the category with the smallest loss can be selected to perform intensity adjustment, but the category with the large loss does not need to perform intensity adjustment. For example, for a category with a loss higher than a threshold, the corresponding intensity factor can be used to perform intensity adjustment for pixel points of the corresponding category. Alternatively, for example, the loss for each category can be sorted in ascending order, and the corresponding intensity factor for the five categories with the smallest loss can be used to perform intensity adjustment for pixel points of the corresponding category, thereby improving the flexibility of intensity adjustment and image quality.

[0044] In one embodiment, performing an intensity adjustment on the first reconstructed video unit in response to the intensity factor comprises: Step 1411: calculating a first loss between the image after intensity adjustment and the original image for each pixel point of each category using a single intensity factor; In step 1413, a second loss between the image after intensity adjustment and the original image is calculated for the pixel points of the corresponding classes using the corresponding intensity factors for each class; Step 1415 includes performing intensity adjustment on the first reconstructed video unit using a single intensity factor according to the first loss and the second loss, or performing intensity adjustment on pixel points of the corresponding category using corresponding intensity factors for each category.

[0045] In this embodiment, it is possible to select whether to use a single intensity factor to perform intensity adjustment for each category, or to use corresponding intensity factors for each category to perform intensity adjustment for pixel points of the corresponding category. For example, an intensity factor is determined for a first reconstructed video unit, and the intensity factor is used to calculate a first loss between the image after intensity adjustment for the first reconstructed video unit and the original image, and then pixel classification is performed for the first reconstructed video unit to determine intensity factors for each category, and intensity adjustment is performed for pixel points of the corresponding category using each intensity factor, and then a second loss is calculated between the image after adjustment for the first reconstructed video unit and the original image, and the first loss is compared with the second loss, and intensity adjustment is performed according to an intensity adjustment method with a small loss, so as to improve the flexibility of intensity adjustment and the image quality.

[0046] In one embodiment, the single intensity factor is a value set based on statistical information, or the single intensity factor is determined according to the slope of a line obtained by fitting each pixel point in the first reconstructed video unit.

[0047] In this embodiment, a single intensity factor may be calculated for the first reconstructed video unit, for example, by placing all pixel points in the first reconstructed video unit in the same coordinate diagram and fitting according to all pixel points to obtain a straight line, and the slope of the line is used as the single intensity factor, and the single intensity factor may be a preset intensity factor, for example, by setting one appropriate intensity factor based on statistical information.

[0048] In one embodiment, performing an intensity adjustment on the first reconstructed video unit in response to the intensity factor comprises: a step 1421 of calculating a second loss between the video unit after intensity adjustment for the pixel points of the corresponding category and the original video unit using the corresponding intensity factors for each category; Step 1422: merge the categories whose difference values ​​of the corresponding intensity factors are within a set range into one category; a step 1423 of determining an intensity factor corresponding to each fused category; Calculating a third loss between the video unit after intensity adjustment and the original video unit for the pixel points of the corresponding fused category using the intensity factors corresponding to each fused category in step 1424; and step 1425, according to the second loss and the third loss, performing intensity adjustment on the pixel points of the corresponding category using the corresponding intensity factors for each category, or performing intensity adjustment on the pixel points of the corresponding fused category using the intensity factors corresponding to each fused category.

[0049] In this embodiment, the corresponding intensity factors of each category can be used to perform intensity adjustment on pixel points of the corresponding category, or fusion can be performed on each category to reduce the number of categories, reduce the amount of calculation, and improve the efficiency and effectiveness of intensity adjustment. For example, pixel classification is performed on the first reconstructed video unit to determine the intensity factors of each category, and then each intensity factor is used to perform intensity adjustment on the pixel points of the corresponding category, and a second loss is calculated, and then fusion is performed on each category. For example, if some intensity factors are close to each other (for example, the difference between each intensity factor does not exceed a specified range), these categories are fused into one category, and then the intensity factors of each fused category are calculated (for example, the pixel points of each category to be fused into one category are placed on the same coordinate map, and the intensity factor of the fused category is obtained by fitting a line and calculating the slope). The intensity factors corresponding to each fused category are used to calculate the third loss after intensity adjustment on the pixel points of the corresponding fused category, and the second loss is compared with the third loss, and intensity adjustment is performed according to an intensity adjustment method with a smaller loss, thereby improving the flexibility of intensity adjustment and image quality.

[0050] In one embodiment, performing an intensity adjustment on the first reconstructed video unit in response to the intensity factor comprises: Step 1431: for each pixel point of each category, calculate the loss between the pixel after intensity adjustment and the original pixel using the intensity factor corresponding to the category; In step 1432, the intensity adjustment indicator is used to indicate the on / off state of the corresponding intensity adjustment for each category according to the loss corresponding to the pixel point of each category; and step 1433, in response to each intensity adjustment on / off indicator, performing intensity adjustment on pixel points of the corresponding category using an intensity factor corresponding to the category for which intensity adjustment is turned on.

[0051] In this embodiment, the intensity adjustment switch indicator (Flag) is set for each category to control the intensity adjustment switch for each category, thereby achieving the purpose of adjusting the intensity for a desired category.

[0052] For multiple categories, it can further determine which categories to turn on intensity adjustment for and which categories to turn off intensity adjustment for according to the degree of distortion between the pixels after intensity adjustment and the original pixels, i.e., the cost. After performing pixel classification for the first reconstructed video unit to determine the intensity factors for each category, each intensity factor is used to perform intensity adjustment on the pixel points of the corresponding category, and the loss corresponding to each category is calculated. If the pixel points of one category show a performance deterioration rather than a gain after intensity adjustment, the intensity adjustment on / off indicator for that category can be set to 0, indicating that no intensity adjustment is performed for that category. Conversely, the intensity adjustment on / off indicator for that category can be set to 1, indicating that intensity adjustment is performed for that category. Then, for the categories for which intensity adjustment is turned on according to the intensity adjustment on / off indicator, the corresponding intensity factor is used to perform intensity adjustment, thereby improving the flexibility of intensity adjustment and image quality.

[0053] In one embodiment, performing an intensity adjustment on the first reconstructed video unit in response to the intensity factor comprises: a step 1441 of calculating a first loss between the video unit after intensity adjustment for each pixel point of each category and the original video unit using a single intensity factor; a step 1443 of calculating a second loss between the video unit after intensity adjustment for the pixel points of the corresponding category and the original video unit using the corresponding intensity factors for each category; In step 1445, the intensity adjustment indicator is used to indicate the intensity adjustment on / off state corresponding to a single intensity factor and the intensity adjustment on / off state corresponding to each category according to the first loss and the second loss; and step 1447, in response to each intensity adjustment on / off indicator, performing intensity adjustment for pixel points of each category using a single intensity factor, or performing intensity adjustment for pixel points of the corresponding category using an intensity factor corresponding to the category for which intensity adjustment is turned on.

[0054] In this embodiment, an intensity adjustment on / off indicator (Flag) is set for each category, and the intensity adjustment on / off indicator is set for a single intensity factor, thereby controlling the intensity adjustment mode. When intensity adjustment is performed for each category, the intensity adjustment for each category can be further controlled to be on or off.

[0055] For example, a single intensity factor is determined for the first reconstructed video unit, and intensity factors for each category are determined according to the pixel classification results. For example, the first reconstructed video unit can be divided into N categories according to the mobility factor A in ALF, corresponding to N intensity factors. When the single intensity factor is added, there are a total of N+1 intensity factors. The single intensity factor is used to perform intensity adjustment on the first reconstructed video unit, calculate a first loss, and use the above-mentioned N intensity factors to perform intensity adjustment on the pixel points of the corresponding category, calculate a second loss compared to the original image after adjustment for the first reconstructed video unit, compare the first loss and the second loss, and for an intensity adjustment method with a small loss, the intensity adjustment on / off indicator is set to 1, indicating that the intensity adjustment is on, and for an intensity adjustment method with a large loss, the intensity adjustment on / off indicator is set to 0, indicating that the intensity adjustment is off. Wherein, when N intensity factors are used to perform intensity adjustment on the pixel points of the corresponding category, an intensity adjustment on / off indicator can be set for each category according to whether or not there is a performance gain after intensity adjustment of the pixel points of each category.

[0056] The loop filtering method of this embodiment proposes a classification-based competitive strategy decision mechanism (for example, competitive strategy decision for intensity adjustment between each category, competitive strategy decision for intensity adjustment between single intensity factor and multiple intensity factors, competitive strategy decision for intensity adjustment between each category and intensity adjustment for fusion category) and an intensity adjustment gate control strategy, which increases the flexibility of intensity adjustment, reduces the loss of intensity adjustment, reduces the loss of reconstructed video units, ensures the image quality of the reconstruction, and improves loop filtering performance.

[0057] In an embodiment of the present application, a video coding method applicable to the coding side is further proposed, which can be implemented based on a hybrid coding framework, such as the H.266 / VVC coding framework, a new generation video coding standard developed by the International Telecommunication Union Telecommunication Standards Branch ITU-T and the International Organization for Standardization (ISO) / International Electrotechnical Commission (IEC) Joint Video Project, and includes modules such as intra-frame prediction, inter-frame prediction, transform, quantization, loop filtering, and entropy coding.

[0058] It should be noted that the method according to the embodiment of the present application may be applied to the H.266 / VVC standard, an Audio Video coding standard (e.g., AVS3), or a next-generation video encoding / decoding standard, and the embodiment of the present application is not limited thereto.

[0059] 4 is a flow diagram of a video encoding method according to an embodiment. As shown in FIG. 4, the method according to this embodiment includes: In step 210, filtering the reconstructed image of the current image to obtain a first reconstructed image; In step 220, filtering the first reconstructed image using a loop filtering method to obtain a second reconstructed image; In step 230, based on the filtering process, marking loop filtering control information of the loop filtering method in a bitstream; Wherein, the first reconstructed image includes a first reconstructed video unit, and the second reconstructed image includes a second reconstructed video unit, and the loop filtering method is determined according to the loop filtering method of any of the above-mentioned embodiments. For example, performing a filtering process on the first reconstructed image by the loop filtering method to obtain the second reconstructed image mainly includes: Step 2301 of obtaining a first reconstructed video unit processed by a loop filter; a step 2303 of performing pixel classification on the first reconstructed video unit; a step 2305 of determining corresponding intensity factors for each category; and performing intensity adjustment on the first reconstructed video unit according to the intensity factors of each category in step 2307 to obtain a second reconstructed video unit.

[0060] 5 is a schematic diagram of an encoding side framework according to an embodiment. As shown in FIG. 5, the overall frame workflow of the encoding side is as follows:

[0061] (1) Dividing the current image of the input video into coding units; (2) sending the divided coding units to an intra-frame prediction module for mainly removing spatial correlation of the image or an inter-frame prediction module for mainly removing temporal correlation of the image to perform predictive coding; (3) subtracting the resulting prediction from the original block to obtain a residual value, which is then transformed and quantized to remove frequency domain correlations and perform lossy compression on the data; (4) performing entropy coding on all coding parameters and residual values ​​to form a binary stream for storage or transmission, and the output data of the entropy coding module is the original video compressed bitstream; (5) Adding the residual value after inverse quantization and inverse transformation to the predicted value to obtain a block reconstruction value to form a reconstructed image, which can be filtered (e.g., by LMCS, DBF, SAO, ALF, and / or NN) to obtain a first reconstructed image; (6) The first reconstructed image is subjected to MSF filtering using a loop filter to obtain a second reconstructed image, which is then stored in the image cache as a reference image.

[0062] (7) Perform video encoding using the second reconstructed image to obtain an encoded video sequence, and during the encoding process, mark loop filtering control information of the loop filtering method in the bitstream according to the above filtering process.

[0063] In an embodiment, in order to reduce the distortion between the reconstructed image and the original image, NN filter and MSF filtering are introduced in the loop filtering process, which can be added as a newly added step or module, or can replace DBF and SAO in the original filtering process, thus not only saving the transmission overhead but also improving the image quality.

[0064] The NN filter analyzes data based on deep learning, and establishes a mapping from the distorted image to the original image to complete the image quality improvement. In this embodiment, pixel classification is performed in the loop filtering process, and intensity factors are introduced for each category, and a competitive strategy determination mechanism and an intensity adjustment opening / closing control strategy are used to flexibly adjust the intensity.

[0065] FIG. 6 is a schematic diagram of a video encoding process according to one embodiment. In this embodiment, MSF is applied to a hybrid coding framework for video encoding for a first reconstructed image that has been subjected to neural network filtering. As shown in FIG. 6, MSF includes two parts: pixel classification and intensity adjustment based on multiple intensity factors. Various filtering methods are supported in the video encoding process. For example, a reconstructed image may be obtained using LMCS. NN filtering is performed on the reconstructed image to obtain a first reconstructed image. Then, pixel classification is performed based on the first reconstructed image (for example, in the block classification mode of ALF, the gradient and directional and / or mobility factors are calculated to determine the category to which the pixel block belongs), and intensity adjustment is performed according to the intensity factors of each category (a competitive strategy determination mechanism and an intensity adjustment opening / closing control strategy may be used in this process), resulting in a second reconstructed image.

[0066] The video encoding method of the present embodiment performs loop filtering and intensity adjustment based on multiple intensity factors on the reconstructed image, thereby providing a high-quality reconstructed image for video encoding and improving the quality of the encoded video. Also, by marking the loop filtering control information of the loop filtering method in the bitstream, it provides a basis for decoding using the loop filtering method.

[0067] In one embodiment, after obtaining the second reconstructed image: a step 2310 of performing block classification on the second reconstructed image; The method further includes step 2313 of performing an adaptive loop filter (ALF) on each pixel point of each category in the second reconstructed image.

[0068] FIG. 7 is a schematic diagram of a video encoding process according to one embodiment. This embodiment applies MSF to a hybrid coding framework for video encoding. As shown in FIG. 7, MSF includes two parts: pixel classification and intensity adjustment based on multiple intensity factors. Various filtering methods are supported in the video encoding process. FIG. 7 takes NN filtering and ALF filtering as examples. The first reconstructed image may further include performing NN filtering on the first reconstructed image before performing MSF filtering on the first reconstructed image. After obtaining the second reconstructed image, ALF filtering may be performed on the second reconstructed image to further improve filtering performance and image quality. The pixel classification may refer to the block classification mode of ALF.

[0069] 8 is a schematic diagram of one ALF filtering according to one embodiment. As shown in FIG. 8, the ALF filtering process mainly includes: (1) Block classification, for example, dividing the second reconstructed image into pixel blocks (or luminance blocks) of 4×4 size; (2) Obtain a filter template, and for the classification result in (1), a set of filtering coefficients corresponds to each category, and four filtering templates, such as no transformation, diagonal transformation, vertical flip, and rotation transformation, are corresponding, and determine a filtering coefficient adjustment manner for each 4x4 pixel block; (3) determining one 7x7 filtering template for each 4x4 luminance block (including the transformation method), then performing a filtering process on each pixel in the 4x4 pixel block, and finally outputting the reconstructed image.

[0070] In one embodiment, after obtaining the second reconstructed image: The method further includes step 2320 of performing self-adaptive loop filtering on pixel points of each category in the second reconstructed image according to the classification result of the pixel classification.

[0071] 9 is a schematic diagram of another video encoding process according to an embodiment. As shown in FIG. 9, the intensity adjustment of the multiple intensity factors can be performed based on the block classification of ALF filtering, i.e., pixel classification in MSF can be achieved through the block classification mode of ALF filtering. The LMCS-filtered reconstructed image can first be subjected to neural network filtering. Then, based on the classification results of ALF block classification, the intensity adjustment of the multiple intensity factors can be performed. Then, a filter template can be selected to determine the optimal filter usage scheme and perform self-adaptive filtering. Furthermore, the classification results can be fully utilized to perform intensity adjustment and self-adaptive filtering without the need for secondary classification, thereby improving coding efficiency and reducing coding complexity.

[0072] In one embodiment, before performing a filtering process on the first reconstructed image by the loop filtering method, a step 2210 of performing block classification on the first reconstructed image; The method further includes step 2213 of performing self-adaptive loop filtering on pixel points of each category in the first reconstructed image according to the classification result of the block classification.

[0073] FIG. 10 is a schematic diagram of another video encoding process according to an embodiment. Various filtering methods are supported in the video encoding process. FIG. 10 illustrates NN filtering and ALF filtering as examples. The process further includes performing NN filtering and ALF filtering on the first reconstructed image before performing MSF filtering on the first reconstructed image to further improve filtering performance and image quality. MSF includes two parts: pixel classification and intensity adjustment based on multiple intensity factors. The pixel classification may refer to the block classification mode of ALF. In this embodiment, placing the intensity adjustment based on multiple intensity factors after ALF filtering fully ensures the effectiveness of the intensity adjustment without the gain obtained from the intensity adjustment being covered by ALF filtering, thereby improving the quality of the reconstructed image.

[0074] In one embodiment, prior to determining the corresponding intensity factors for each category, The method further includes step 2304 of performing self-adaptive loop filtering on pixel points of each category in the first reconstructed image according to the classification result of pixel classification.

[0075] FIG. 11 is a schematic diagram of another video encoding process according to an embodiment. As shown in FIG. 11, intensity adjustment of multiple intensity factors can be performed based on block classification of ALF filtering, i.e., pixel classification in MSF can be achieved through block classification mode of ALF filtering. The LMCS-filtered reconstructed image can first be subjected to NN filtering, and then, according to the classification result of ALF block classification, a filter template can be selected, an optimal filter application scheme can be determined, and self-adaptive filtering can be performed, after which intensity adjustment of multiple intensity factors can be performed. Furthermore, self-adaptive filtering and intensity adjustment can be performed by fully utilizing the classification result without the need for secondary classification, thereby improving coding efficiency and reducing coding complexity. Furthermore, the gain obtained from intensity adjustment is not covered by ALF filtering, fully ensuring the effectiveness of intensity adjustment and improving the quality of the reconstructed image.

[0076] In one embodiment, before performing a filtering process on the first reconstructed image by a loop filtering method, further comprising a step 2220 of performing DBF filtering on the reconstructed image; After obtaining the second reconstructed image, The step 2330 further includes performing SAO filtering on the second reconstructed image.

[0077] 12 is a schematic diagram of another video encoding process according to an embodiment. In this embodiment, when a loop filtering module such as DBF and / or SAO is turned on, MSF filtering may be added. As shown in FIG. 12, for example, when both DBF and SAO are turned on, NN filtering may be between DBF filtering and SAO filtering. (1) After obtaining the LMCS reconstructed image, DBF filtering is first performed to obtain the first reconstructed image; (2) performing a neural network filtering operation to obtain a first reconstructed image processed by a loop filter; (3) Perform MSF filtering on the first reconstructed image, which includes two parts: pixel classification and multi-intensity factor intensity adjustment, where pixel classification can refer to the block classification mode of ALF; (4) SAO filtering is performed on the second reconstructed image after MSF filtering. (5) ALF filtering is performed on the second reconstructed image after SAO filtering, and the final reconstructed image is output.

[0078] In one embodiment, before performing a filtering process on the first reconstructed image by a loop filtering method, The method further includes a step 2230 of performing DBF filtering and SAO filtering on the reconstructed image.

[0079] 13 is a schematic diagram of another video encoding process according to an embodiment. In this embodiment, the loop filter may adopt a filtering model in the related art, such as DBF and / or SAO, and may not adopt an NN filter. As shown in FIG. 13, for example, when both DBF and SAO are enabled, (1) After obtaining the LMCS reconstructed image, first perform DBF and SAO filtering to obtain the first reconstructed image processed by the loop filter; (2) Perform MSF filtering on the first reconstructed image, which includes two parts: pixel classification and multi-intensity factor intensity adjustment, where pixel classification can refer to the block classification mode of ALF; (3) ALF filtering is performed on the second reconstructed image after MSF filtering, and the final reconstructed image is output.

[0080] In the video encoding process of the above embodiment, the loop filtering method can be used in combination with other filtering, the filtering order can be flexibly set, and only primary classification can be performed, or secondary classification can be performed, and the MSF filtering can adopt different classification methods, intensity factor competition strategies, and gate control, so as to provide a high-quality reconstructed image as a reference for video encoding, and ensure the quality of the encoded video.

[0081] In the embodiment of this application, a video decoding method is further proposed: The classification-based multi-intensity factor adjustment can also be applied to video decoding as a loop filtering method.

[0082] FIG. 14 is a flow chart of a video decoding method according to an embodiment. As shown in FIG. 14, the method according to this embodiment includes: In step 310, a bitstream including a coded video sequence is obtained, and loop filtering control information associated with a loop filtering method is obtained; In step 320, filtering the reconstructed image of the current image to obtain a first reconstructed image; In step 330, filtering the first reconstructed image according to the loop filtering control information through a loop filtering method to obtain a second reconstructed image.

[0083] Wherein, the first reconstructed image includes a first reconstructed video unit, and the second reconstructed image includes a second reconstructed video unit, and the loop filtering method is determined according to the loop filtering method of any of the above-mentioned embodiments. For example, performing a filtering process on the first reconstructed image by the loop filtering method to obtain the second reconstructed image mainly includes: Step 3401 of obtaining a first reconstructed video unit processed by a loop filter; a step 3403 of performing pixel classification on the first reconstructed video unit; a step 3405 of determining corresponding intensity factors for each category; and performing an intensity adjustment on the first reconstructed video unit according to the intensity factors of each category to obtain a second reconstructed video unit.

[0084] Figure 15 is a schematic diagram of a decoding framework according to one embodiment. The decoding framework also includes modules or steps such as intra-frame prediction, motion compensation, inverse transform, inverse quantization, and context-based adaptive binary arithmetic coding, among which the intensity control type, pixel classification method, intensity factor, etc. can be obtained by analyzing the high-level syntax of the bitstream. As shown in Figure 15, the entire frame workflow on the decoding side is as follows:

[0085] (1) performing header information decoding and context-based adaptive binary arithmetic coding on a bitstream of an encoded video sequence to obtain loop filtering control information related to a loop filtering method; (2) decoding the bitstream of the coded video sequence to obtain predicted values ​​and residual values, performing inverse transform and inverse quantization on the residual values, and adding the predicted values ​​to the residual values ​​after inverse quantization and inverse transform to obtain block reconstruction values ​​to form a reconstructed image, and using intra-frame prediction or inter-frame prediction to remove spatial correlation or temporal correlation of the image; (3) performing a filtering process such as LMS, DBF, SAO, and / or ALF on the reconstructed image, and then obtaining a first reconstructed image; (4) filtering the first reconstructed image according to the loop filtering control information through a loop filtering method, and after the first reconstructed image is filtered by the loop filter, a second reconstructed image is obtained and stored in the image cache as a reference; (5) The second reconstructed image is used to obtain decoded video data.

[0086] In this embodiment, in order to reduce the distortion between the reconstructed image and the original image, a NN filter is introduced in the loop filtering process, which can be added as a new step or module, or can replace DBF and SAO in the original filtering process, thus not only saving transmission overhead but also improving image quality.In addition, the loop filtering control information can be obtained by reading from the bitstream, providing a reliable basis for video decoding.

[0087] It should be noted that the video decoding method according to this embodiment is only one realization method of classification-based multi-intensity factor adjustment, and various preferred embodiments are further included in practical applications.

[0088] The video decoding method of the present embodiment performs loop filtering and intensity adjustment based on multiple intensity factors on the reconstructed image, so as to provide a high-quality reconstructed image for video decoding, and improve the quality of the encoded / decoded data.

[0089] In one embodiment, the loop filtering control information comprises: a coded video sequence sequence level filtering on / off indicator for indicating whether the loop filtering method is on or off for the video sequence in which the current picture is located; a filtering picture level indicator for indicating whether the loop filtering control information appears in the picture header information or in the slice header information; and loop filter structure information for instructing loop filtering to be performed on the video unit to which loop filtering is applied.

[0090] In this embodiment, the bitstream of the coded video sequence can express loop filtering control information related to MSF filtering using high-level syntax, including information such as whether to open or close MSF filtering, the number of pixel classification categories, and intensity factors. The decoder decodes header information for the input coded video sequence and performs MSF filtering based on the obtained loop filtering control information.

[0091] The loop filtering control information includes a sequence level filtering on / off indicator, and it can be determined whether to turn on loop filtering for the current video sequence (whether to perform MSF filtering) depending on the sequence level filtering on / off indicator, and the sequence level filtering on / off indicator can be represented by a sequence parameter set (SPS).

[0092] Table 1 shows the syntax elements of the sequence-level filtering opening and closing indicators.

[0093] [Table 1]

[0094] The semantics of the syntax elements in Table 1 are as follows: TIFF0007771392000006.tif5170 When equal to 1, it indicates that loop filtering is turned on (MSF is turned on) for the current video sequence, and when equal to 0, it indicates that loop filtering is not turned on (MSF is turned on) for the current video sequence.

[0095] The loop filtering control information includes a filtering picture level indicator, and the MSF filtering level, i.e., the video unit to which the loop filtering control information is applied, can be determined according to the filtering picture level indicator, and may be applied to the reconstructed image or to one or more slices of the reconstructed image, and the filtering picture level indicator is expressed in a picture parameter set (PPS).

[0096] Table 2 shows the syntax elements of the filtering image level indicators.

[0097] [Table 2]

[0098]

number

[0099]

number

[0100] Table 3 shows the syntax elements for image header information.

[0101] [Table 3]

[0102]

number

[0103]

number

[0104] Table 4 shows the syntax elements of the slice header information.

[0105] [Table 4]

[0106] In one embodiment, a video unit comprises an image or a video slice in a video.

[0107] In one embodiment, the loop filter structure information is a video unit filtering on / off indicator for instructing the video unit to turn the loop filtering method on or off; a video unit component open / close indicator for indicating the video unit component to which loop filtering is applied; a video unit category number for indicating the total number of categories of the video unit pixel classification; a video unit category filtering on / off indicator for indicating whether to turn on or off the loop filtering method for each category in the video unit; a video unit intensity factor for indicating an intensity factor corresponding to a video unit or a classification in the video unit; Among them, the video unit component is the luminance component of the video unit; a first chrominance component of the video unit; and the second chromaticity component of the video unit.

[0108] In this embodiment, the loop filtering (or MSF filtering) open / close status, the number of categories, the strength factor of each category, etc. of the current image or slice are obtained based on the loop filter structure information, and the strength of each category can be adjusted based on this information.

[0109] Table 5 shows syntax elements that perform power adjustment for a single category based on loop filter structure information.

[0110] [Table 5]

[0111] The semantics of the syntax elements in Table 5 are as follows:

[0112]

number

[0113] Table 6 shows syntax elements that implement power adjustment for multiple categories based on loop filter structure information.

[0114] [Table 6]

[0115] The semantics of the syntax elements in Table 6 are as follows:

[0116]

number

[0117]

number

[0118]

number

[0119] Table 7 shows syntax elements that perform intensity adjustment for each of multiple categories of luma and chroma components based on loop filter structure information.

[0120] [Table 7]

[0121] The semantics of the syntax elements in Table 7 are as follows:

[0122]

number

[0123]

number

[0124]

number

[0125]

number

[0126]

number

[0127]

number

[0128]

number

[0129]

number

[0130] Table 8 shows syntax elements that perform intensity adjustment for luma and chroma components of some categories based on loop filter structure information.

[0131] [Table 8]

[0132]

number

[0133]

number

[0134]

number

[0135]

number

[0136]

number

[0137]

number

[0138]

number

[0139]

number

[0140]

number

[0141]

number

[0142] In one embodiment, filtering a video unit to which loop filtering is applied according to a loop filtering method based on loop filtering control information includes: a step 3410 of identifying a video unit for which the loop filtering method is to be turned on in response to a video unit filtering open / close indicator; step 3413, in response to the video unit component on / off indicators and the video unit respective category filtering on / off indicators, identifying the video unit components to which loop filtering is to be applied and the categories of video units for which the loop filtering method is to be turned on; and performing 3415 intensity adjustments on the video unit components of the category for which the loop filtering method is turned on by the loop filtering method in response to the video unit intensity factors.

[0143] In this embodiment, based on the loop filtering control information, for video units for which the loop filtering method is turned on, intensity adjustment can be performed on the video unit components (e.g., luminance components, Cr and / or Cb) to which loop filtering is applied for pixel points of a category for which the loop filtering method is turned on by MSF filtering.

[0144] In one embodiment, after obtaining the second reconstructed image: The method further includes step 3420 of performing self-adaptive loop filtering on pixels of each category in the second reconstructed image according to the classification result of the pixel classification.

[0145] In this embodiment, the LMCS filtered reconstructed image can be first subjected to NN filtering, and then intensity adjustment of multiple intensity factors can be performed according to the classification result of pixel classification (which may refer to the block classification mode of ALF), and then a filter template can be selected to determine the optimal filter application scheme and perform self-adaptive filtering, as may be seen in Figure 9. Furthermore, the classification result can be fully utilized to perform intensity adjustment and self-adaptive filtering without the need for secondary classification, thereby improving coding efficiency and reducing coding complexity.

[0146] In one embodiment, before performing a filtering process on the first reconstructed image by a loop filtering method according to the loop filtering control information, a step 3310 of performing block classification on the first reconstructed video unit; The method further includes a step 3313 of performing self-adaptive loop filtering on pixels of each category in the first reconstructed image according to the classification result of the block classification.

[0147] In this embodiment, before performing MSF filtering on the first reconstructed image in the first reconstructed image, NN filtering and ALF filtering may be performed on the first reconstructed image to further improve filtering performance and image quality, see FIG. 10. MSF includes two parts: pixel classification and intensity adjustment based on multiple intensity factors, of which pixel classification may refer to the block classification mode of ALF. In this embodiment, by placing the intensity adjustment based on multiple intensity factors after ALF filtering, the gain obtained in the intensity adjustment is not covered by ALF filtering, so the effectiveness of the intensity adjustment is fully guaranteed and the quality of the reconstructed image is improved.

[0148] In one embodiment, prior to determining the corresponding intensity factors for each category, The method further includes step 3404 of performing self-adaptive loop filtering on pixels of each category in the video unit according to the classification result of the pixel classification.

[0149] In this embodiment, the LMCS filtered reconstructed image can first undergo NN filtering, and then, according to the classification result of pixel classification (which may refer to the block classification mode of ALF), a filter template can be selected to determine the optimal filter application scheme, and self-adaptive filtering can be performed, and then intensity adjustment of multiple intensity factors can be performed, as may be seen in Figure 11. In addition, intensity adjustment and self-adaptive filtering can be performed by fully utilizing the classification result without the need for secondary classification, and the effectiveness of the intensity adjustment gain can be guaranteed.

[0150] In one embodiment, before performing a filtering process on the first reconstructed image according to the loop filtering control information to obtain a second reconstructed image, further comprising a step 3320 of performing DBF filtering on the reconstructed image; After obtaining the second reconstructed image, The method further includes a step 3430 of performing SAO filtering on the decoded reconstructed image.

[0151] In this embodiment, when a loop filtering module such as DBF and / or SAO is turned on, MSF filtering may be added, and the loop filtering process may refer to FIG.

[0152] In one embodiment, before performing a filtering process on the first reconstructed image by a loop filtering method according to the loop filtering control information, The method further includes a step 3330 of performing DBF filtering and SAO filtering on the reconstructed image.

[0153] In this embodiment, the loop filter adopts the filter in the related art, and does not need to adopt the NN filter, and the loop filtering process may refer to FIG.

[0154] In the video decoding process of the above embodiment, the loop filtering method can be used in combination with other filtering, the filtering order can be flexibly set, and only primary classification can be performed, or secondary classification can be performed, and the MSF filtering can adopt different classification methods, intensity factor competition strategies, and gate control, so as to provide a high-quality reconstructed image as a reference for video decoding, and ensure the quality of the decoded video data.

[0155] The embodiment of the present application further proposes a loop filtering device. Figure 16 is a structural schematic diagram of a loop filtering device according to an embodiment. As shown in Figure 16, the loop filtering device includes: a video unit acquisition module 410 configured to acquire a first reconstructed video unit processed by the loop filter; a classification module 420 configured to perform pixel classification on the first reconstructed video unit; a determination module 430 configured to determine a corresponding intensity factor for each category; and an adjustment module 440 configured to perform an intensity adjustment on the first reconstructed video unit according to the intensity factor to obtain a second reconstructed video unit.

[0156] The loop filtering device of this embodiment obtains a second reconstructed video unit after performing intensity adjustment on the first reconstructed video unit according to classification, so that the deviation between the reconstructed value and the original value in the second reconstructed video unit is smaller, there is less distortion, the image quality is higher, and it has a better filtering effect.

[0157] In one embodiment, the loop filter includes a neural network based loop filter.

[0158] In one embodiment, the reconstructed video unit comprises a reconstructed image, or a slice, or a coding block, or a coding tree block.

[0159] In one embodiment, the adjustment module 440 includes: performing an intensity adjustment on a luminance component of the first reconstructed video unit according to the intensity factor; performing an intensity adjustment on a first chrominance component of the first reconstructed video unit according to the intensity factor; and performing an intensity adjustment on the second chrominance component of the first reconstructed video unit in response to the intensity factor.

[0160] In one embodiment, the determination module 430: A merging unit configured to merge pixel points of the same category into the same coordinate map, and the horizontal coordinate of each pixel point is the output of the pixel point after passing through the loop filter, and the vertical coordinate is the input of the pixel point before passing through the loop filter; The fitting unit is configured to obtain a straight line by fitting the pixel points in the coordinate map based on the least squares method, and to use the slope of the straight line as an intensity factor corresponding to the corresponding classification.

[0161] In one embodiment, the determination module 430 is configured to determine the corresponding strength factor for each category by reading loop filtering control information associated with a loop filtering method.

[0162] In one embodiment, the adjustment module 440 is configured to, for each pixel point in each category, multiply the difference between the output of the pixel point after passing through the loop filter and the input of the pixel point before passing through the loop filter by an intensity factor corresponding to the category, and add the result of the multiplication to the input of the pixel point before passing through the loop filter to obtain an adjusted pixel value of the pixel point.

[0163] In one embodiment, the pixel classification scheme is: performing pixel classification based on luminance components; performing pixel classification based on a first chromaticity component; performing pixel classification based on a second chromaticity component; performing pixel classification based on a block classification mode; performing pixel classification based on pixel point area; performing pixel classification based on a neural network-based loop filter classification; performing pixel classification based on the classification mode used for DBF; performing pixel classification based on a classification mode used for SAO filtering; and performing pixel classification using a neural network for pixel classification.

[0164] In one embodiment, performing pixel classification based on a block classification mode includes: Obtaining a pixel block of a first reconstructed video unit; Calculating a Laplacian gradient for each pixel block, and calculating a directionality factor and a mobility factor based on the Laplacian gradient; determining a classification of the pixel blocks according to the directionality and / or mobility factors, and determining a classification of all pixel points in the reconstructed video unit.

[0165] In one embodiment, the adjustment module 440 includes: a first calculation unit configured to calculate, for each pixel point of each category, a loss between the pixel after intensity adjustment and the original pixel by using an intensity factor corresponding to the category; a first adjusting unit configured to, according to the losses corresponding to the pixel points of each category, adopt intensity factors corresponding to some categories to perform intensity adjustment on the pixel points of the corresponding categories.

[0166] In one embodiment, the adjustment module 440 includes: a second calculation unit configured to calculate a first loss between the image after intensity adjustment for each pixel point of each category and the original image using a single intensity factor; a third calculation unit configured to calculate a second loss between the image after intensity adjustment for the pixel points of the corresponding class and the original image using the corresponding intensity factors for each class; and a second adjustment unit configured to perform intensity adjustment on the first reconstructed video unit using a single intensity factor according to the first loss and the second loss, or to perform intensity adjustment on pixel points of the corresponding categories using corresponding intensity factors for each category, respectively.

[0167] In one embodiment, the single intensity factor is a value set based on statistical information, or the single intensity factor is determined according to the slope of a line obtained by fitting each pixel point in the first reconstructed video unit.

[0168] In one embodiment, the adjustment module 440 includes: a fourth calculation unit configured to calculate a second loss between the video unit after intensity adjustment for the pixel points of the corresponding category and the original video unit using the corresponding intensity factors for each category; a fusion unit configured to fuse the categories into one category when the difference values ​​of the corresponding intensity factors are within a set range; a determining unit configured to determine an intensity factor corresponding to each fused category; a fifth calculation unit configured to calculate a third loss between the video unit after intensity adjustment for the pixel points of the corresponding fused category and the original video unit using the intensity factors corresponding to each fused category respectively; and a third adjustment unit configured to, according to the second loss and the third loss, respectively use corresponding intensity factors for each category to perform intensity adjustment on pixel points of the corresponding category, or respectively use intensity factors corresponding to each fused category to perform intensity adjustment on pixel points of the corresponding fused category.

[0169] In one embodiment, the adjustment module 440 includes: a sixth calculation unit configured to calculate, for each pixel point of each category, a loss between the pixel after intensity adjustment and the original pixel by using an intensity factor corresponding to the category; a first marking unit configured to use an intensity adjustment on / off indicator to mark the on / off state of the intensity adjustment corresponding to each category according to the loss corresponding to each pixel point of each category; and a fourth adjusting unit configured to, in response to each intensity adjustment on / off indicator, perform intensity adjustment on pixel points of the corresponding category using an intensity factor corresponding to the category for which intensity adjustment is turned on.

[0170] In one embodiment, the adjustment module 440 includes: a seventh calculation unit configured to calculate a first loss between the video unit after intensity adjustment for each pixel point of each category and the original video unit using a single intensity factor; an eighth calculation unit configured to calculate a second loss between the video unit after intensity adjustment for the pixel points of the corresponding category and the original video unit using the corresponding intensity factors for each category; a second marking unit configured to use an intensity adjustment on / off indicator to mark an on / off state of the intensity adjustment corresponding to a single intensity factor and an on / off state of the intensity adjustment corresponding to each category according to the first loss and the second loss; and a fifth adjustment unit configured to perform intensity adjustment for pixel points of each category using a single intensity factor according to each intensity adjustment on / off indicator, or to perform intensity adjustment for pixel points of the corresponding category using an intensity factor corresponding to the category for which intensity adjustment is turned on.

[0171] The loop filtering apparatus according to this embodiment belongs to the same inventive concept as the loop filtering method according to the above-described embodiment, and technical details not described in detail in this embodiment may refer to any of the above-described embodiments, and this embodiment has the same beneficial effects as implementing the loop filtering method.

[0172] An embodiment of the present application further proposes a video encoding device. Figure 17 is a structural diagram of a video encoding device according to an embodiment. As shown in Figure 17, the video encoding device includes: a filtering module 510 configured to filter the reconstructed image of the current image to obtain a first reconstructed image; a loop filtering module 520 configured to filter the first reconstructed image by a loop filtering method to obtain a second reconstructed image; an encoding module 530 configured to mark loop filtering control information of the loop filtering method in a bitstream based on the filtering process; Wherein, the first reconstructed image includes a first reconstructed video unit, the second reconstructed image includes a second reconstructed video unit, and the loop filtering method is determined according to the loop filtering method of any of the above embodiments.

[0173] The video encoding device of this embodiment performs loop filtering and intensity adjustment based on multiple intensity factors on the reconstructed image, thereby providing a high-quality reconstructed image for video encoding and improving the quality of the encoded video. Also, loop filtering control information of the loop filtering method is marked in the bitstream, providing a basis for using loop filtering to perform video decoding.

[0174] In one embodiment, the apparatus comprises: a first block classification module configured to, after obtaining the second reconstructed image, perform block classification on the second reconstructed image; and a first ALF module configured to perform self-adaptive loop filtering on each category of pixel points in the second reconstructed image.

[0175] In one embodiment, the apparatus comprises: The image processing apparatus further includes a second ALF module configured to, after obtaining the second reconstructed image, perform self-adaptive loop filtering on pixel points of each category in the second reconstructed image according to the classification result of the pixel classification.

[0176] In one embodiment, the apparatus comprises: a second block classification module configured to perform block classification on the first reconstructed image before filtering the first reconstructed image using a loop filtering method; The image processing device further includes a third ALF module configured to perform self-adaptive loop filtering on pixel points of each category in the first reconstructed image according to the classification result of the block classification.

[0177] In one embodiment, the apparatus comprises: The method further comprises a fourth ALF module configured to perform self-adaptive loop filtering on pixel points of each category in the first reconstructed image according to the classification result of the pixel classification before determining the corresponding intensity factors for each category.

[0178] In one embodiment, the apparatus comprises: a DBF module configured to perform DBF filtering on the reconstructed image before performing a filtering process on the first reconstructed image by a loop filtering method; and an SAO module configured to, after obtaining the second reconstructed image, perform SAO filtering on the second reconstructed image.

[0179] In one embodiment, the apparatus comprises: The method further comprises a double filtering module configured to perform DBF filtering and SAO filtering on the reconstructed image before performing a filtering process on the first reconstructed image by the loop filtering method.

[0180] The video encoding device of this embodiment belongs to the same inventive concept as the video encoding method of the above-mentioned embodiment, and technical details not described in detail in this embodiment may refer to any of the above-mentioned embodiments, and this embodiment has the same beneficial effects as implementing the video encoding method.

[0181] An embodiment of the present application further proposes a video decoding device. Figure 18 is a structural diagram of a video decoding device according to an embodiment. As shown in Figure 18, the video decoding device includes: an information obtaining module 610 configured to obtain a bitstream including an encoded video sequence and obtain loop filtering control information related to a loop filtering method; a filtering module 620 configured to filter the reconstructed image of the current image to obtain a first reconstructed image; a loop filtering module 630 configured to filter the first reconstructed image through a loop filtering method based on the loop filtering control information to obtain a second reconstructed image; Wherein, the first reconstructed image includes a first reconstructed video unit, the second reconstructed image includes a second reconstructed video unit, and the loop filtering method is determined according to the loop filtering method described in any of the above embodiments.

[0182] The video decoding device of this embodiment performs loop filtering and intensity adjustment based on multiple intensity factors on the reconstructed image, so as to provide a high-quality reconstructed image for video decoding and improve the quality of the decoded video data. In addition, the loop filtering control information can be read from the bitstream, providing a reliable basis for video decoding.

[0183] In one embodiment, the loop filtering control information comprises: a coded video sequence sequence level filtering on / off indicator for indicating whether the loop filtering method is on or off for the video sequence in which the current picture is located; a filtering picture level indicator for indicating whether the loop filtering control information appears in the picture header information or in the slice header information; and loop filter structure information for instructing loop filtering to be performed on the video unit to which loop filtering is applied.

[0184] In one embodiment, a video unit comprises an image or a video slice in a video.

[0185] In one embodiment, the loop filter structure information is a video unit filtering on / off indicator for instructing the video unit to turn the loop filtering method on or off; a video unit component open / close indicator for indicating the video unit component to which loop filtering is applied; a video unit category number for indicating the total number of categories of the video unit pixel classification; a video unit category filtering on / off indicator for indicating whether to turn on or off the loop filtering method for each category in the video unit; a video unit intensity factor for indicating an intensity factor corresponding to a video unit or a classification in the video unit; Among them, the video unit component is the luminance component of the video unit; a first chrominance component of the video unit; and the second chromaticity component of the video unit.

[0186] In one embodiment, the loop filtering module 640: a first identification unit configured to identify a video unit for which a loop filtering method is turned on according to a video unit filtering open / close indicator; a second identification unit configured to identify the video unit components to which loop filtering is applied and the video unit categories in which the loop filtering method is turned on according to the video unit component on / off indicator and the video unit each category filtering on / off indicator; an adjustment unit configured to perform intensity adjustment on video unit components of a category for which the loop filtering method is turned on by the loop filtering method depending on the video unit intensity factor.

[0187] In one embodiment, the apparatus comprises: The image processing apparatus further includes a fifth ALF module configured to, after obtaining the second reconstructed image, perform self-adaptive loop filtering on pixels of each category in the second reconstructed image according to the classification result of the pixel classification.

[0188] In one embodiment, the apparatus comprises: a second block classification module configured to perform block classification on the first reconstructed video unit before performing a filtering process on the first reconstructed image through a loop filtering method based on the loop filtering control information; and a sixth ALF module configured to perform self-adaptive loop filtering on pixels of each category in the first reconstructed image according to the classification result of the block classification.

[0189] In one embodiment, the apparatus comprises: The seventh ALF module is further configured to perform self-adaptive loop filtering on pixels of each category in the video unit according to the classification result of the pixel classification before determining corresponding intensity factors for each category.

[0190] In one embodiment, the apparatus comprises: a DBF module configured to perform DBF filtering on the reconstructed image before performing a filtering process on the first reconstructed image through a loop filtering method based on the loop filtering control information; and an SAO module configured to perform SAO filtering on the decoded reconstructed image after obtaining the second reconstructed image.

[0191] In one embodiment, the apparatus further comprises a double filtering module configured to perform DBF filtering and SAO filtering on the reconstructed image before performing a filtering process on the first reconstructed image by a loop filtering method based on the loop filtering control information.

[0192] The video decoding device of this embodiment belongs to the same inventive concept as the video decoding method of the above-mentioned embodiment, and technical details not described in detail in this embodiment may refer to any of the above-mentioned embodiments, and this embodiment has the same beneficial effects as implementing the video decoding method.

[0193]

[0063] An embodiment of the present application further proposes an electronic device, which is an image processing device, and may be an encoding device or a decoding device. Figure 19 is a schematic diagram of the hardware structure of an electronic device according to an embodiment. As shown in Figure 19, the electronic device according to the present application includes a memory 720, a processor 710, and a computer program stored in the memory and operable on the processor. When the processor 710 executes the program, the above-mentioned loop filtering method, video encoding method, or video decoding method is realized.

[0194] The electronic device may further include a memory 720, and the processor 710 in the electronic device may be one or more processors 710. Figure 19 takes one processor 710 as an example, and the memory 720 is used to store one or more programs, and the one or more programs are executed by the one or more processors 710, thereby causing the one or more processors 710 to realize the loop filtering method, video encoding method, or video decoding method described in the embodiments of the present application.

[0195] The electronic device further comprises a communication device 730 , an input device 740 and an output device 750 .

[0196] The processor 710, memory 720, communication device 730, input device 740, and output device 750 in the electronic device can be connected by a bus or other method, and FIG. 19 shows them connected by a bus as an example.

[0197] The input device 740 may be used to receive input numeric or textual information and generate key signal inputs related to user settings and function control of the electronic device. The output device 750 may include a display device such as a display screen.

[0198] The communication device 730 may include a receiver and a transmitter and is configured to transmit and receive information under the control of the processor 710.

[0199] The memory 720 may be configured as a computer-readable storage medium to store software programs, computer-executable programs, and modules, such as program instructions / modules corresponding to the loop filtering method described in the embodiments of the present application (e.g., the video unit acquisition module 410, the classification module 420, the determination module 430, and the adjustment module 440 in the loop filtering device). The memory 720 may include a program storage area and a data storage area, of which the program storage area may store an operating system and / or application programs required for at least one function, and the data storage area may store data generated in accordance with the use of the electronic device. The memory 720 may also include high-speed random access memory or non-volatile memory, such as at least one magnetic disk memory device, flash memory device, or other non-volatile solid-state memory device. In some examples, the memory 720 may further include memory configured remotely from the processor 710, and these remote memories may be connected to the electronic device via a network. Examples of the network include, but are not limited to, the Internet, an intranetwork, a local area network, a mobile communication network, and combinations thereof.

[0200] An embodiment of the present application further proposes a storage medium, the storage medium storing a computer program, which, when executed by a processor, realizes the method according to any one of the embodiments of the present application, and the method may include: obtaining a first reconstructed video unit processed by a loop filter, performing pixel classification on the first reconstructed video unit, determining corresponding intensity factors for each classification, and performing intensity adjustment on the first reconstructed video unit according to the intensity factors to obtain a second reconstructed video unit.

[0201] Alternatively, the method may be a video encoding method including: filtering a reconstructed image of a current image to obtain a first reconstructed image; performing a filtering process on the first reconstructed image using a loop filtering method to obtain a second reconstructed image; and marking loop filtering control information of the loop filtering method in a bitstream based on the filtering process, wherein the first reconstructed image includes a first reconstructed video unit, the second reconstructed image includes a second reconstructed video unit, and the loop filtering method is determined according to the loop filtering method of any of the above-mentioned embodiments.

[0202] Alternatively, the method may be a video decoding method including: obtaining a bitstream including a coded video sequence; obtaining loop filtering control information related to a loop filtering method; filtering a reconstructed image of a current image to obtain a first reconstructed image; and performing a filtering process on the first reconstructed image using the loop filtering method based on the loop filtering control information to obtain a second reconstructed image, wherein the first reconstructed image includes a first reconstructed video unit, and the second reconstructed image includes a second reconstructed video unit, and the loop filtering method is determined according to the loop filtering method of any of the above-mentioned embodiments.

[0203] The computer storage medium of the embodiments of the present application may employ any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include an electrical connection having one or more conductors, a portable computer magnetic disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable CD-ROM, an optical memory device, a magnetic memory device, or any suitable combination of the above. The computer-readable storage medium may be any tangible medium that comprises or stores a program, which can be used in or in connection with an instruction execution system, apparatus, or device.

[0204] A computer-readable signal medium may include a propagated data signal, either in baseband or as part of a carrier wave, having computer-readable program code carried therein. Such propagated data signals may take various forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

[0205] The program code contained in the computer readable medium may be transmitted by any suitable medium, including but not limited to wireless, wire, optical cable, radio frequency (RF), etc., or any suitable combination of the above.

[0206] Computer program code for carrying out the operations of the present application may be written in one or a variety of programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as general procedural programming languages ​​such as "C" or similar programming languages. The program code may execute entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. When referring to a remote computer, the remote computer may be connected to the user's computer by any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet Service Provider).

[0207] The above descriptions are only illustrative examples of the present application, and are not intended to limit the protection scope of the present application.

[0208] Those skilled in the art will appreciate that the term user terminal covers any suitable type of wireless user equipment, for example a mobile phone, a portable data processing device, a portable network browser or a mobile station mounted on a vehicle.

[0209] In general, various embodiments of the present application may be implemented in hardware or special purpose circuits, software, logic, or any combination thereof. For example, but not limited to, some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software executable by a controller, microprocessor, or other computing device.

[0210] Embodiments of the present application may be implemented by hardware, for example in a processor entity, or by a combination of software and hardware, by a data processor of a mobile device executing computer program instructions, which may be assembler instructions, Instruction Set Architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or various programming languages.

[0211] Any logic flow block diagrams in the drawings of this application may represent program steps, or may represent interconnected logic circuits, modules, and functions, or may represent a combination of program steps and logic circuits, modules, and functions. Computer programs can be stored in memory. The memory may be of any type suitable for the local technology environment and may be implemented using any suitable data storage technology, such as, but not limited to, Read-Only Memory (ROM), Random Access Memory (RAM), optical memory devices and systems (Digital Versatile Optical Disks (DVDs) or Compact Disks (CDs)), etc. The computer-readable medium may include non-transitory storage media. The data processor may be of any type suitable for the local technology environment, such as, but not limited to, general purpose computers, special purpose computers, microprocessors, digital signal processing (DSPs), application specific integrated circuits (ASICs), programmable logic devices (Field-Programmable Gate Arrays (FGPAs)), and processors based on multi-core processor architectures.

Claims

1. Obtaining a first reconstructed video unit processed by a loop filter; performing pixel classification on the first reconstructed video unit; determining a corresponding intensity factor for each category; performing an intensity adjustment on the first reconstructed video unit according to the intensity factor to obtain a second reconstructed video unit; Determining the corresponding intensity factors for each category includes: Integrate pixel points of the same category into the same coordinate map, and the horizontal coordinate of each pixel point is the output of the pixel point after passing through the loop filter, and the vertical coordinate of each pixel point is the input of the pixel point before passing through the loop filter; According to the least squares method, a straight line is obtained by fitting the pixel points in the coordinate map, and the slope of the straight line is used as the intensity factor corresponding to the corresponding classification; or Determining the corresponding intensity factors for each category includes: determining a corresponding strength factor for each category by reading loop filtering control information associated with the loop filtering method; Loop filtering method.

2. the loop filter comprises a neural network based loop filter; or the first reconstructed video unit and the second reconstructed video unit each include any one of a reconstructed image, a slice, a coding block, and a coding tree block. The method of claim 1.

3. performing an intensity adjustment on the first reconstructed video unit in response to the intensity factor, performing an intensity adjustment on a luminance component of the first reconstructed video unit in response to the intensity factor; performing an intensity adjustment on a first chrominance component of the first reconstructed video unit in response to the intensity factor; performing an intensity adjustment on a second chrominance component of the first reconstructed video unit in response to the intensity factor; Performing intensity adjustment on the first reconstructed video unit according to the intensity factors includes: for each pixel point of each category, calculating a loss between the pixel after intensity adjustment and the original pixel using a corresponding intensity factor of each category; and according to the loss corresponding to pixel points of a plurality of categories, performing intensity adjustment on pixel points of the corresponding category by adopting intensity factors corresponding to some categories among the plurality of categories; or Performing intensity adjustment on the first reconstructed video unit according to the intensity factor includes: calculating a first loss between an image after intensity adjustment on pixel points of a plurality of categories and an original image using a single intensity factor; calculating a second loss between an image after intensity adjustment on pixel points of the corresponding categories and the original image using intensity factors corresponding to the plurality of categories, respectively; performing intensity adjustment on the first reconstructed video unit using the single intensity factor or performing intensity adjustment on pixel points of the corresponding categories using intensity factors corresponding to the plurality of categories, respectively, according to the first loss and the second loss, wherein the single intensity factor is a value set based on statistical information, or the single intensity factor is determined according to a slope of a straight line obtained by fitting each pixel point in the first reconstructed video unit; or performing intensity adjustment on the first reconstructed video unit according to the intensity factors includes: calculating a second loss between the video unit after intensity adjustment on pixel points of the corresponding category and the original video unit using intensity factors corresponding to a plurality of categories, respectively; fusing categories whose difference values ​​of corresponding intensity factors are within a set range as one category; determining intensity factors corresponding to each fused category; calculating a third loss between the video unit after intensity adjustment on pixel points of the corresponding fused category and the original video unit using intensity factors corresponding to the plurality of fused categories, respectively; and performing intensity adjustment on pixel points of the corresponding category using intensity factors corresponding to the plurality of categories according to the second loss and the third loss, or performing intensity adjustment on pixel points of the corresponding fused category using intensity factors corresponding to the plurality of fused categories, respectively. Performing intensity adjustment on the first reconstructed video unit according to the intensity factors includes: for each pixel point of each category, calculating a loss between the pixel after intensity adjustment and the original pixel using a corresponding intensity factor for each category; using intensity adjustment on / off indicators according to losses corresponding to pixel points of multiple categories to respectively indicate the on / off states of intensity adjustment corresponding to the multiple categories; and performing intensity adjustment on pixel points of the corresponding categories using intensity factors corresponding to the categories for which intensity adjustment is turned on according to the intensity adjustment on / off indicators; or Performing intensity adjustment on the first reconstructed video unit according to the intensity factor includes: calculating a first loss between the video unit after intensity adjustment on pixel points of a plurality of categories and the original video unit using a single intensity factor; calculating a second loss between the video unit after intensity adjustment on pixel points of the corresponding categories and the original video unit using intensity factors corresponding to the plurality of categories, respectively; using an intensity adjustment on / off indicator according to the first loss and the second loss to indicate an on / off state of intensity adjustment corresponding to the single intensity factor and an on / off state of intensity adjustment corresponding to the plurality of categories, respectively; and performing intensity adjustment on pixel points of the plurality of categories using the single intensity factor or an intensity factor corresponding to a category for which intensity adjustment is turned on according to the intensity adjustment on / off indicator. The method of claim 1.

4. The pixel classification method is: performing pixel classification based on luminance components; performing pixel classification based on a first chromaticity component; performing pixel classification based on a second chromaticity component; performing pixel classification based on a block classification mode; performing pixel classification based on pixel point area; performing pixel classification based on a neural network-based loop filter classification; performing pixel classification based on the classification mode used for the deblocking filtering DBF; performing pixel classification based on a classification mode used for pixel self-adaptive compensation SAO filtering; performing pixel classification using a neural network for pixel classification; Performing pixel classification based on block classification mode includes: obtaining a pixel block of the first reconstructed video unit; calculating a Laplacian gradient for each pixel block, and calculating a directionality factor and a mobility factor based on the Laplacian gradient; determining a classification of the pixel block based on at least one of the directionality factor and the mobility factor, and determining a classification of all pixel points in the first reconstructed video unit. The method of claim 1.

5. filtering the reconstructed image of the current image to obtain a first reconstructed image; filtering the first reconstructed image by a loop filtering method to obtain a second reconstructed image; marking loop filtering control information of the loop filtering method in a bitstream based on the filtering process, the first reconstructed image comprises a first reconstructed video unit, the second reconstructed image comprises a second reconstructed video unit, and the loop filtering method is determined according to the loop filtering method of claim 1. Video coding methods.

6. After obtaining the second reconstructed image, performing block classification on the second reconstructed image; performing self-adaptive loop filtering on each pixel point of each category in the second reconstructed image; or After obtaining the second reconstructed image, and performing self-adaptive loop filtering on each pixel point of each category in the second reconstructed image according to the classification result of the pixel classification. The method of claim 5.

7. before filtering the first reconstructed image by the loop filtering method, performing block classification on the first reconstructed image; and performing self-adaptive loop filtering on each category of pixel points in the first reconstructed image according to the classification result of the block classification; or Before performing a filtering process on the first reconstructed image by a loop filtering method, the method further includes performing deblocking filtering (DBF) filtering on the reconstructed image, and after obtaining a second reconstructed image, the method further includes performing pixel self-adaptive compensation SAO filtering on the second reconstructed image, or and performing DBF filtering and SAO filtering on the reconstructed image before performing a filtering process on the first reconstructed image using a loop filtering method. The method of claim 5.

8. Prior to determining the corresponding intensity factors for each category, and performing self-adaptive loop filtering on each pixel point of each category in the first reconstructed image according to the classification result of the pixel classification. The method of claim 5.

9. obtaining a bitstream including an encoded video sequence and obtaining loop filtering control information associated with a loop filtering method; filtering the reconstructed image of the current image to obtain a first reconstructed image; filtering the first reconstructed image through a loop filtering method based on the loop filtering control information to obtain a second reconstructed image, the first reconstructed image comprises a first reconstructed video unit, the second reconstructed image comprises a second reconstructed video unit, and the loop filtering method is determined according to the loop filtering method of claim 1. Video decoding methods.

10. The loop filtering control information a sequence level filtering on / off indicator for the coded video sequence for indicating whether the loop filtering method is on or off for the video sequence in which the current picture is located; a filtering picture level indicator for indicating whether the loop filtering control information appears in picture header information or slice header information; and loop filter structure information for instructing the loop filtering to be performed on a video unit to which the loop filtering is applied.

10. The method of claim 9.

11. the video units comprise images or video slices in a video; The method of claim 10.

12. The loop filter structure information is a video unit filtering on / off indicator for instructing a video unit to turn the loop filtering method on or off; a video unit component open / close indicator for indicating the video unit component to which loop filtering is applied; a video unit category number for indicating the total number of categories of the video unit pixel classification; a video unit category filtering on / off indicator for indicating whether the loop filtering method is on or off for a plurality of categories in the video unit; a video unit intensity factor for indicating an intensity factor corresponding to the video unit or a classification in the video unit; The video unit component is the luminance component of the video unit; a first chrominance component of the video unit; a second chrominance component of the video unit; performing a filtering process on the first reconstructed image by a loop filtering method based on the loop filtering control information, identifying a video unit for which the loop filtering method is turned on in the first reconstructed image according to the video unit filtering on / off indicator; Identifying a video unit component to which loop filtering is applied and a category in which the loop filtering method is turned on in response to the video unit component on / off indicator and the video unit category filtering on / off indicator; performing intensity adjustments on the video unit components of the category for which the loop filtering method is turned on by the loop filtering method in response to the video unit intensity factors. The method of claim 10.

13. After obtaining the second reconstructed image, and performing self-adaptive loop filtering on each category of pixels in the second reconstructed image according to a classification result of the pixel classification.

10. The method of claim 9.

14. before performing a filtering process on the first reconstructed image by a loop filtering method based on the loop filtering control information, performing block classification on the first reconstructed video unit; and performing self-adaptive loop filtering on each category of pixels in the first reconstructed image according to the classification result of the block classification.

10. The method of claim 9.

15. Prior to determining the corresponding intensity factors for each category, Further comprising: performing self-adaptive loop filtering on each category of pixels in the first reconstructed video unit according to a classification result of pixel classification; 10. The method of claim 9.

16. before performing a filtering process on the first reconstructed image by a loop filtering method based on the loop filtering control information to obtain a second reconstructed image; further comprising performing deblocking filtering (DBF filtering) on ​​the reconstructed image; After obtaining the second reconstructed image, and further comprising performing pixel self-adaptive compensation SAO filtering on the decoded reconstructed image.

10. The method of claim 9.

17. before performing a filtering process on the first reconstructed image by a loop filtering method based on the loop filtering control information, further comprising performing DBF filtering and SAO filtering on the reconstructed image.

10. The method of claim 9.

18. a memory; a processor; and a computer program stored in the memory and operable on the processor, wherein when the processor executes the program, the loop filtering method according to any one of claims 1 to 4, the video encoding method according to any one of claims 5 to 8, or the video decoding method according to any one of claims 9 to 17 is realized. electronic equipment.

19. A computer program is stored therein, When the computer program is executed by a processor, the loop filtering method according to any one of claims 1 to 4, the video encoding method according to any one of claims 5 to 8, or the video decoding method according to any one of claims 9 to 17 is realized. A computer-readable storage medium.

Images