Filtering method, device and equipment

Abstract

The present application provides a filtering method, device and equipment, the filtering method comprising: when it is determined that a current image needs to be subjected to Wiener filtering, determining an optimal block division method for the Wiener filtering of the current image; and performing Wiener filtering on the current image based on the optimal block division method. The method can optimize the effect of Wiener filtering and improve decoding performance.

Description

Technical Field

[0001] The present application relates to video coding and decoding technology, and in particular to a filtering method, device and equipment. Background Art

[0002] The adaptive loop filter (ALF) technology is used in the coding framework of the new generation of video coding standards (Versatile Video Coding, referred to as VCC). It calculates the optimal filter in the mean square sense based on the original signal and the distorted signal, namely the Wiener filter. The filter is used to perform filtering operations on the reconstructed image (which can be called Wiener filtering) to improve the quality of the reconstructed image.

[0003] However, practice has found that the ALF technology uses very few parameters to filter a large number of pixels, and the pixel complexity is relatively high. Since the essence of Wiener filtering is a linear regression model, when filtering a large number of pixels with high complexity, the linear regression model will be in an under-fitting state, reducing the filtering effect. Summary of the invention

[0004] In view of this, the present application provides a filtering method, device and equipment.

[0005] Specifically, the present application is implemented through the following technical solutions:

[0006] According to a first aspect of an embodiment of the present application, a filtering method is provided, which is applied to a decoding end device, and the method includes:

[0007] When it is determined that the current image needs to be subjected to Wiener filtering, determining an optimal block division method for performing Wiener filtering on the current image;

[0008] Perform Wiener filtering on the current image based on the optimal block division method.

[0009] According to a second aspect of an embodiment of the present application, a filtering method is provided, which is applied to an encoding end device, and the method includes:

[0010] Determine the optimal block partitioning method for Wiener filtering of the current image;

[0011] When it is determined that the current image needs to be subjected to Wiener filtering, the filter coefficients of the classification to which each image block of the current image belongs under the optimal block division method are notified to a decoding end device through the encoding code stream of the current image.

[0012] According to a third aspect of an embodiment of the present application, there is provided a filtering device, applied to a decoding end device, the device comprising: a determining unit and a filtering unit;

[0013] The determining unit is used to determine the optimal block division method for performing Wiener filtering on the current image when it is determined that Wiener filtering needs to be performed on the current image;

[0014] A filtering unit is used to perform Wiener filtering on the current image based on the optimal block division method.

[0015] According to a fourth aspect of an embodiment of the present application, there is provided a filtering device, applied to an encoding end device, the device comprising: a determining unit and a notifying unit;

[0016] The determining unit is used to determine the optimal block division method for performing Wiener filtering on the current image;

[0017] The notification unit is used to notify the decoding end device of the filter coefficients of the classification to which each image block of the current image belongs under the optimal block division method through the encoding code stream of the current image when it is determined that the current image needs to be Wiener filtered.

[0018] According to the fifth aspect of an embodiment of the present application, a decoding end device is provided, comprising a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor, and the processor is used to execute the machine-executable instructions to implement the method described in the first aspect above.

[0019] According to the sixth aspect of the embodiments of the present application, a machine-readable storage medium is provided, wherein the storage medium stores machine-executable instructions that can be executed by the processor, and the machine-executable instructions, when executed by the processor, implement the method described in the first aspect above.

[0020] According to the seventh aspect of an embodiment of the present application, a coding end device is provided, comprising a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor, and the processor is used to execute the machine-executable instructions to implement the method described in the second aspect above.

[0021] According to an eighth aspect of an embodiment of the present application, a machine-readable storage medium is provided, wherein the storage medium stores machine-executable instructions that can be executed by the processor, and when the machine-executable instructions are executed by the processor, the method described in the second aspect above is implemented.

[0022] The technical solution provided by this application can at least bring the following beneficial effects:

[0023] By determining the need to perform Wiener filtering on the current image, determining the optimal block division method for performing Wiener filtering on the current image, and performing Wiener filtering on the current image based on the optimal block division method, the effect of Wiener filtering is optimized and the decoding performance is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1A is a schematic diagram of a coding architecture shown in an exemplary embodiment of the present application;

[0025] Figure 1B is a schematic diagram of a Wiener filter processing flow of a brightness component shown in an exemplary embodiment of the present application;

[0026] Figure 1C is a schematic diagram of a filter shape shown in an exemplary embodiment of the present application;

[0027] Figure 1D is a schematic diagram of a location for calculating a gradient of a 4*4 image block according to an exemplary embodiment of the present application;

[0028] Figure 1E is a schematic diagram of locations for calculating gradients for an 8*8 image block, a 2*2 image block, and a 1*1 image block, shown in an exemplary embodiment of the present application;

[0029] Figure 1F is a schematic diagram of filter coefficients of brightness components under geometric transformations in different directions shown in an exemplary embodiment of the present application;

[0030] Figure 2 is a flowchart of a filtering method shown in an exemplary embodiment of the present application;

[0031] Figure 3 It is a schematic diagram of a flow chart of performing Wiener filtering on a current image based on an optimal block division method, shown in an exemplary embodiment of the present application;

[0032] Figure 4 is a flow chart showing a method of classifying a plurality of image blocks to determine classification results of the plurality of image blocks, as shown in an exemplary embodiment of the present application;

[0033] Figure 5 is a schematic diagram of error analysis of a linear regression model shown in an exemplary embodiment of the present application;

[0034] Figure 6 is a flowchart of a filtering method shown in an exemplary embodiment of the present application;

[0035] Figure 7 It is a schematic diagram of an exemplary embodiment of the present application showing a flow chart of determining a rate-distortion cost of performing Wiener filtering on a current image under the candidate block division method;

[0036] Figure 8An exemplary embodiment of the present application shows a flow chart of splitting an image block in a first category into two parts, and changing the classification of one part of the image blocks to the second category, while keeping the classification of the other part of the image blocks as the first category;

[0037] Fig.9A is a schematic diagram showing four directions of an image block according to an exemplary embodiment of the present application;

[0038] Fig. 9B is a schematic diagram showing a filter coefficient deletion position according to an exemplary embodiment of the present application;

[0039] Fig.10 is a structural schematic diagram of a filtering device according to an exemplary embodiment of the present application;

[0040] Fig.11 is a schematic diagram showing a hardware structure of a decoding end device according to an exemplary embodiment of the present application;

[0041] Fig.12 is a structural schematic diagram of a filtering device according to an exemplary embodiment of the present application;

[0042] Fig.13 The figure is a schematic diagram showing the hardware structure of an encoding terminal device according to an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0043] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0044] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0045] In order to enable those skilled in the art to better understand the technical solution provided by the embodiments of the present application, the technical terms involved in the embodiments of the present application and the implementation of the Wiener filtering technology (ALF technology) are briefly explained below.

[0046] 1. Technical terms

[0047] 1. Rate-Distortion Optimized (RDO): The indicators for evaluating coding efficiency include bit rate and Peak Signal to Noise Ratio (PSNR). The smaller the bit rate, the greater the compression rate; the larger the PSNR, the better the reconstructed image quality. When selecting a mode, the discriminant formula is essentially a comprehensive evaluation of the two.

[0048] The cost corresponding to the mode is: J(mode) = D + λ*R. Where D represents distortion, which is usually measured by the SSE (mean square difference) indicator. SSE refers to the mean square difference between the reconstructed block and the source image block; λ is the Lagrange multiplier; R is the actual number of bits required for encoding the image block in this mode, including the sum of bits required for encoding mode information, motion information, residual, etc.

[0049] When selecting a mode, if you use the RDO principle to make comparison decisions on the encoding mode, you can usually ensure the best encoding performance.

[0050] 2. Tree Coding Unit (CTU): Traditional video coding is based on macroblocks. For videos in 4:2:0 sampling format, a macroblock contains a 16×16 luminance block and two 8×8 chrominance blocks. Considering the characteristics of HD / UHD videos, Versatile Video Coding (VVC) introduces CTU, whose size is specified by the encoder and is allowed to be larger than the macroblock size. A luminance coding tree block (CTB) and two chrominance CTBs at the same location, plus corresponding syntax elements, form a CTU. In VVC, for an L×L luminance CTB, L∈{8, 16, 32, 64, 128}.

[0051] The range of brightness CTB size is: {8×8, 16×16, 32×32, 64×64, 128×128}

[0052] The chroma CTB size range is {4×4, 8×8, 16×16, 32×32, 64×64}

[0053] In the high-resolution video encoding process, using a larger CTB can achieve better compression effect.

[0054] 2. Implementation of Wiener Filtering Technology

[0055] See also Figure 1A , the position of Wiener filter in the video coding framework is as follows Figure 1A As shown, Figure 1A As shown, video coding generally includes processes such as prediction, transformation, quantization and entropy coding.

[0056] Among them, prediction can be divided into intra-frame prediction and inter-frame prediction. Intra-frame prediction uses the surrounding coded blocks as references to predict the current uncoded block, effectively removing redundancy in the spatial domain. Inter-frame prediction uses the neighboring coded images to predict the current image, effectively removing redundancy in the temporal domain.

[0057] Transformation refers to converting an image from the spatial domain to the transform domain, and using transform coefficients to represent the image. Most images contain many flat areas and slowly changing areas. Appropriate transformation can transform the image from a dispersed distribution in the spatial domain to a relatively concentrated distribution in the transform domain, remove the frequency domain correlation between signals, and cooperate with the quantization process to effectively compress the bit stream.

[0058] Entropy coding is a lossless coding method that can transform a series of element symbols into a binary code stream for transmission or storage. The input symbols may include quantized transform coefficients, motion vector information, prediction mode information, transform quantization related syntax, etc. Entropy coding can effectively remove the redundancy of video element symbols.

[0059] The above is introduced using encoding as an example. The processes of video decoding and video encoding are relative, that is, video decoding usually includes entropy decoding, prediction, inverse quantization, inverse transformation, filtering and other processes. The implementation principles of each process are the same or similar to entropy coding.

[0060] The Wiener filtering process is after the DB (Deblock) and SAO (Sample adaptive offset) processes, that is, the decoding end device restores the DB, SAO and filtered reconstructed image.

[0061] Since the human eye is less sensitive to chrominance components than to luminance components, the ALF technology is more detailed in filtering luminance components and the chrominance filtering process is relatively simple.

[0062] In one possible embodiment, see Figure 1B , the brightness Wiener filter processing flow is as follows:

[0063] 1. Divide the brightness component of the current image into multiple 4*4 brightness blocks, and classify the multiple 4*4 brightness blocks. The entire frame image can be divided into 25 categories at most.

[0064] 2. Calculate the gradients in four directions (horizontal, vertical and diagonal) of each 4*4 brightness block to determine the geometric transformation direction of the pixel block.

[0065] 3. Use the Wiener filtering principle to calculate the filtering parameters in each category, that is, determine Figure 1C The values ​​of filter coefficients C = {c1, c2, c3, c4, c5, c6, c7, c8, c9, c10, c11, c12} in (a).

[0066] 4. Merge the 25 categories of brightness blocks.

[0067] 5. Determine the on / off of the CTU level filter and the on / off of the frame level filter.

[0068] The chroma Wiener filter processing flow is relatively simple, because there is no classification division, and it only includes the following two steps:

[0069] 1. Divide the chrominance components of the current image into multiple 4*4 chrominance blocks. All chrominance blocks are of the same type. Use the Wiener filter principle to calculate the filter parameters, that is, determine Figure 1C In (b), filter coefficient C = {c1, c2, c3, c4, c5, c6}.

[0070] 2. Determine the on / off of the CTU-level filter and the frame-level filtering switch.

[0071] The implementation of each step in the Wiener filter processing flow is described in more detail below.

[0072] A. Filter Shape

[0073] Wiener filtering technology uses Figure 1C If the current CTU is marked as requiring Wiener filtering, each pixel R(i, j) in the CTU is filtered according to the following formula.

[0074] Exemplarily, L represents the length of the filter, and f(k, l) is the parameter of the filter to be determined (ie, the filter coefficient), which can be obtained from the encoded bitstream during decoding.

[0075] Exemplarily, the filter size of the luminance component is 7*7, and the filter size of the chrominance component is 5×5.

[0076] B. Block Classification Method

[0077] The Wiener filter technique uses a 1-D Laplacian operator to calculate the horizontal, vertical, and diagonal gradients of a 4*4 luminance block. v , g h , g d0 , g d1 , and further obtain a D (ranging from 0 to 4) representing the direction and a (value range is 0 to 4), and finally get the category to which it belongs (Total 25 categories) For chrominance components, the following classification is not required.

[0078] The method of obtaining D is as follows:

[0079] First, based on the following formula, we obtain The values ​​of the four variables.

[0080]

[0081]

[0082]

[0083]

[0084] Exemplarily, in order to reduce complexity, a down-sampled 1-D Laplacian operator is used, as shown in FIG1D , where every other pixel is sampled, and the number of samples is reduced by half.

[0085] For example, for 8*8, 2*2 and 1*1 image blocks, the schematic diagram of the position of calculating the gradient when classifying can be as follows: Figure 1E shown.

[0086] The four required parameters are obtained based on the following formula:

[0087]

[0088]

[0089] Then, the corresponding D is obtained according to the following process:

[0090] 1) If and Then D = 0;

[0091] 2) If Then proceed to step 3, otherwise proceed to step 4;

[0092] 3) If Then D = 2; if Then D = 1;

[0093] 4) If Then D = 4; if Then D=3.

[0094] First, we get A by the following formula, and then quantize it to the range of 0 to 4, and get

[0095]

[0096] C. Geometric Transformation Adjustment of Filter Parameters

[0097] Before filtering, the following geometric transformation is performed on each type of filter parameter f(k,l) based on the four gradient parameters in step A, so that different blocks are more similar through direction alignment:

[0098] Diagonal transformation: f D (k, l) = f(l, k)

[0099] Vertical flip transformation: f V (k, l) = f(k, Kl-1)

[0100] Rotation transformation: f R (k, l) = f(Kl-1, k)

[0101] For example, K is the filter coefficient length. The mapping table of transformation and gradient is shown in Table 3-1:

[0102] Table 3-1 Filter geometric transformation mapping table

[0103] Gradient parameter conditions Geometric transformation <![CDATA[g d1 <g d0 And g h <g v ]]> No change <![CDATA[g d1 <g d0 And g v <g h ]]> Diagonal Transformation <![CDATA[g d0 <g d1 And g h <g v ]]> Vertical flip transformation <![CDATA[g d0 <g d1 And g v <g h ]]> Rotation Transformation

[0104] The filter coefficients of the brightness component under different directions of geometric transformation can be as follows Figure 1F shown.

[0105] D. Calculate the filter coefficients.

[0106] 25 types of filter coefficients are derived based on the Wiener filter principle.

[0107] Wiener filter algorithm:

[0108] In order to minimize the square loss between the filtered signal and the original image, we can define:

[0109] 1) Pixel position r = (x, y), ||R|| represents the number of pixels in the area to be filtered;

[0110] 2) Original pixel: S(r)

[0111] 3) Pixel before filtering: t(r)

[0112] 4) Filter coefficient: c = [c0, c1, ..., cn] T

[0113] 5) Filter tap coefficient position offset: {p0, p1, ..., pn}

[0114] 6) Filtered pixel: f(r)

[0115] The pixel before filtering and its surrounding pixels form an array: T = [t(r+p0), t(r+p1), ..., t(r+pn)] T

[0116] It can be seen that f(r) = T T c

[0117] En is the square error between f(r) and S(r):

[0118]

[0119] By taking the derivative of c and setting it to zero, we can find the value of c at the minimum point of En.

[0120] T T Tc=S(r)T (3-4)

[0121] The pixel value after filtering is obtained by using the coefficients c and T:

[0122] f(r)=c0·t(r+p0)+c1·t(r+p1)+…cn·t(r+pn)

[0123] Detailed process of brightness Wiener filter operation:

[0124] 1) Divide the brightness component of the current image to obtain multiple 4*4 brightness blocks, and classify the multiple 4*4 brightness blocks based on the gradients of the brightness blocks in four directions to obtain the classification of the multiple 4*4 brightness blocks;

[0125] 2) Based on the gradients of the four directions of each brightness block, determine the geometric transformation direction of each brightness block;

[0126] 3) Obtain each pixel t(r) on the brightness component and its surrounding 12 pixels T (as shown in Figure 1C(a), the average pixel is taken at the symmetrical position) to form a set of 13*13 symmetrical matrix data T T T, and transform the 12 pixels according to the geometric transformation direction of each pixel. Figure 1D Make corresponding changes;

[0127] 4) Get the original pixel S(r);

[0128] 5) Use formula (3-3) to calculate 13 filter coefficients for each category.

[0129] Chroma Wiener filter operation details:

[0130] 1) For each pixel on the chrominance component and its surrounding 6 pairs of pixels T (such as Figure 1C (b) The average pixel is taken at the symmetrical position to form a set of 7x7 symmetric matrix data T T T;

[0131] 2) Get the original pixel S(r);

[0132] 3) Use formula (3-3) to calculate 6 filter coefficients for each category.

[0133] E. Category Merger

[0134] Merge the 25 classes and finally determine the number of classes to be between 1 and 25

[0135] 1. Merge the 25 categories in pairs. Calculate the total error sum after merging every two categories in the 25 categories. The calculation formula is shown in (Formula 3-3). Compare and find the merger with the smallest error sum. Change the 25 categories into 24 categories. Repeat this process. Merge the 24 categories into 23 categories, and finally merge them into 1 category. Record the results of each merger.

[0136] 2. Select the most suitable number of categories from 25 to 1. Calculate the rate-distortion cost of each classification method and replace it with the classification method with the lowest cost, where the compression rate is calculated as the number of bits required to encode all coefficients using Golomb coding, and the distortion loss is the error and loss calculated by formula (3-3).

[0137] F. Determine the on / off of CTU-level filters and frame-level filtering switches

[0138] Calculate the loss of the image before Wiener filtering and the loss of the image after Wiener filtering, and compare the loss of the image after Wiener filtering or canceling Wiener filtering in each CTU to determine the on / off of frame-level filtering and the on / off of CTU-level filters. The loss calculation is consistent with the rate-distortion cost calculation method of category merging in D.

[0139] It can be seen that in the Wiener filtering technology, a large number of pixels are filtered using very few parameters. Although the brightness classification is performed, the model will still be in an underfitting state if the complexity of the pixels after classification is still very high. If you want to alleviate the underfitting situation, the common methods are:

[0140] 1. Increase model parameters;

[0141] 2. Increase the model order;

[0142] 3. Reduce data complexity.

[0143] In the encoding and decoding process, the quality of the reconstructed image is measured by rate distortion, so the model parameters and model order should not be set too large. Therefore, the effect of Wiener filtering can be improved by processing the data and reducing the data complexity.

[0144] In order to make the above-mentioned purposes, features and advantages of the embodiments of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application are further described in detail below with reference to the accompanying drawings.

[0145] In the embodiment of the present application, the data complexity of each category is reduced so that the same-dimensional data in each category approximately satisfies a linear relationship, thereby improving the accuracy of the Wiener filtering result.

[0146] See also Figure 2 , is a flow chart of a filtering method provided in an embodiment of the present application, wherein the filtering method can be applied to a decoding end device, such as Figure 2 As shown, the filtering method may include the following steps:

[0147] Step S200: When it is determined that the current image needs to be subjected to Wiener filtering, an optimal block division method for the Wiener filtering of the current image is determined.

[0148] In an embodiment of the present application, considering that the effect of Wiener filtering on an image may be different when the image is divided into blocks in different ways, in order to optimize the effect of Wiener filtering on the image, the encoding end device can determine the optimal way to divide the image blocks (referred to as the optimal block division method in this article), and determine the relevant parameters of the filtering, such as the filter coefficient, based on the optimal block division method.

[0149] When the decoding end device receives the code stream of the current image and determines that the current image needs to be subjected to Wiener filtering, it can first determine the optimal block division method for the Wiener filtering of the current image.

[0150] Exemplarily, the decoding end device may determine whether Wiener filtering is required for the current image based on the on / off of the frame-level filter carried in the bitstream. When the frame-level filter is on, it is determined that Wiener filtering is required for the current image.

[0151] In one example, the luminance component and the chrominance component may be identified as needing to be subjected to Wiener filtering by turning on / off different frame-level filters.

[0152] The encoding end device needs to determine whether the brightness component and chrominance component of the current image need Wiener filtering respectively, so as to determine the on / off of the corresponding frame-level filter; the decoding end device determines whether the brightness component and chrominance component need Wiener filtering by obtaining the on / off of the frame-level filters corresponding to the brightness component and chrominance component respectively.

[0153] In another example, the luminance component and the chrominance component can be identified by turning on / off the same frame-level filter to determine whether Wiener filtering is required.

[0154] The encoding end device needs to determine whether the brightness component and chrominance component of the current image need Wiener filtering respectively. If both the brightness component and the chrominance component need Wiener filtering, the frame-level filter is determined to be on / off; if neither the brightness component nor the chrominance component needs Wiener filtering, the frame-level filter is determined to be off / on. If one of the brightness component and the chrominance component needs Wiener filtering and the other does not need Wiener filtering, the rate-distortion cost of Wiener filtering for both the brightness component and the chrominance component and the rate-distortion cost of not Wiener filtering for both the brightness component and the chrominance component can be compared. When the former is higher, the frame-level filter is determined to be off / off; when the latter is higher, the frame-level filter is determined to be on / off.

[0155] The decoding end device determines whether the luminance component and the chrominance component require Wiener filtering by obtaining the on / off status of the frame-level filter; if the on / off status of the frame-level filter is on, it is determined that the luminance component and the chrominance component require Wiener filtering; if the on / off status of the frame-level filter is off, the luminance component and the chrominance component do not require Wiener filtering.

[0156] In one example, in step S200, determining the optimal block division method for performing Wiener filtering on the current image may include:

[0157] The optimal block division method for performing Wiener filtering on the current image is obtained from the coded bit stream of the current image.

[0158] Exemplarily, when the encoding end determines the optimal block division method for performing Wiener filtering on the image, the optimal block division method can be indicated to the decoding end device through the encoding bit stream.

[0159] When the decoding end device determines that the current image needs to be subjected to Wiener filtering, the optimal block division method for the Wiener filtering of the current image can be obtained from the coded bit stream of the current image.

[0160] For example, the encoding end device and the decoding end device can pre-negotiate multiple candidate block division methods and the identifiers of each candidate block division method (such as numbers); when the encoding end device determines the optimal block division method for Wiener filtering of the current image, the identifier of the optimal block division method can be carried in the encoded code stream of the current image; when the decoding end device receives the encoded code stream of the current image, it can obtain the identifier of the optimal division method from the encoded code stream, and determine the optimal block division method for Wiener filtering of the current image based on the identifier.

[0161] In another example, in step S200, determining the optimal block division method for performing Wiener filtering on the current image may include:

[0162] Based on the acquired decoding information, the optimal block division method for performing Wiener filtering on the current image is determined.

[0163] Exemplarily, the optimal block division method for performing Wiener filtering on an image may be determined based on the optimal block division method for performing Wiener filtering on an already encoded / decoded image.

[0164] It should be noted that, for the first frame image, the encoding end device can determine the optimal block division method for Wiener filtering based on the rate-distortion cost. The specific implementation can be described below and will not be repeated in the embodiments of the present application.

[0165] When the decoding end device determines that the current image needs to be Wiener filtered, the decoding end device can determine the optimal block division method for Wiener filtering the current image based on the optimal block method for Wiener filtering the previous N frames (N≥1) of the current image.

[0166] For example, when the optimal block division method for the decoding end device to perform Wiener filtering on the first two frames of the current image is 2*2 and 8*8 respectively, the decoding end device can determine that the optimal block division method for Wiener filtering on the current image is to divide the current image into 4*4 image blocks.

[0167] It should be noted that in the embodiment of the present application, when it is necessary to perform Wiener filtering on an image, when dividing the image into blocks, it is not limited to dividing the image into image blocks of the same size, but the image can also be divided into image blocks of multiple different sizes, and its specific implementation will not be described here.

[0168] Step S210: Perform Wiener filtering on the current image based on the optimal block division method.

[0169] In an embodiment of the present application, when the decoding end device determines the optimal block division method for performing Wiener filtering on the current image, the current image can be Wiener filtered based on the optimal block division method.

[0170] For example, the decoding end device can divide the current image into blocks based on the optimal block division method, and for any image block, determine whether Wiener filtering is required for the current image block based on the on / off of the CTU-level filter of the CTU to which the image block belongs.

[0171] It should be noted that the implementation mechanism of turning on / off the CTU-level filter can refer to the implementation of turning on / off the frame-level filter mentioned above, and the embodiments of the present application will not be described in detail here.

[0172] In a possible embodiment, Figure 3 As shown, in step S210, performing Wiener filtering on the current image based on the optimal block division method may include:

[0173] Step S211: divide the current image into blocks based on the optimal block division method to obtain multiple image blocks.

[0174] Step S212: classify the multiple image blocks to determine classification results of the multiple image blocks.

[0175] Step S213: For any image block in the CTU that needs to be subjected to Wiener filtering, determine the geometric transformation direction of the image block based on the gradients of the image block in N1 directions.

[0176] Step S214: Based on the category to which the image block belongs, obtain the filter coefficient of the category from the coded bitstream of the current image.

[0177] Step S215: performing a geometric inverse transformation on the classified filter coefficients based on the geometric transformation direction of the image block, and filtering the image block based on the filter coefficients after the geometric inverse transformation.

[0178] Exemplarily, N1 is a positive integer greater than or equal to 2, and the N1 directions in step S213 may include any N1 directions.

[0179] For example, assuming that N1=2, the two directions may be a horizontal direction and a vertical direction, or a 45° and 135° direction, etc.

[0180] Exemplarily, when the decoding end device determines the optimal block division method for performing Wiener filtering on the current image, the current image can be divided into blocks based on the optimal block division method to obtain multiple image blocks, and the obtained multiple image blocks can be classified to determine the classification results of the multiple image blocks.

[0181] For any image block, the decoding end device may determine whether it is necessary to perform Wiener filtering on the CTU to which the image block belongs based on the on / off status of the CTU-level filter of the CTU to which the image block belongs.

[0182] When the decoding end device determines that the CTU to which the image block belongs needs to be subjected to Wiener filtering, on the one hand, the decoding end device may determine the geometric transformation direction of the image block based on the gradients of the image block in N1 directions.

[0183] In one example, N1=4, the decoding end device can determine the geometric transformation direction of the image block based on the gradient of the image block in four directions (such as vertical direction, horizontal direction, 135° direction and 45° direction). The specific implementation can refer to the relevant implementation in the existing Wiener filtering technology, and the embodiments of the present application will not be repeated here.

[0184] On the other hand, the decoding end device can obtain the filter coefficient of the category from the coded bitstream of the current image based on the category to which the image block belongs.

[0185] The decoding end device may perform a geometric inverse transformation on the filter coefficients of the category to which the image block belongs based on the geometric transformation direction of the image block, and filter the image block based on the filter coefficients after the geometric inverse transformation.

[0186] In one example, if Figure 4 As shown, in step S212, classifying the multiple image blocks to determine the classification results of the multiple image blocks may include:

[0187] Step S2121: classify the multiple image blocks based on the gradients of each image block in N2 directions to determine preliminary classification results of the multiple image blocks.

[0188] Step S2122: Based on the preliminary classification results, determine the number of image blocks included in each classification.

[0189] Step S2123: when the number of image blocks in the first category with the largest number of image blocks is greater than a first threshold, and the number of image blocks in the second category with the smallest number of image blocks is less than a second threshold, the image blocks in the first category are split into two parts, and the classification of one part of the image blocks is modified to the second classification, and the classification of the other part of the image blocks remains in the first classification, so as to determine the final classification results of the multiple image blocks.

[0190] Exemplarily, N2 is a positive integer greater than or equal to 2, N2 and N1 may be the same or different, and the N2 directions in step S2121 may include any N2 directions.

[0191] For example, after classifying multiple image blocks of the current image, the number of image blocks in each classification may vary greatly. For example, the number of image blocks in some classifications may reach thousands, while the number of image blocks in other classifications may be only a few.

[0192] Since the essence of Wiener filtering is a linear regression model, the image blocks in each category approximately satisfy a linear relationship, which can make the results of Wiener filtering more accurate. However, when the number of image blocks contained in the category increases, the possibility of underfitting increases, and the accuracy of the Wiener filtering results may decrease.

[0193] like Figure 5 As shown in the figure, the same fitting line can be obtained for hollow circle data or solid circle data according to the least squares principle. If the data distribution is the distribution of solid circle data, the error will be very small; when the data distribution is the distribution of hollow circle data, the error will be large. The more image blocks included in the same category, the more difficult it is for its data distribution to reach the distribution of solid circle data.

[0194] In order to improve the accuracy of the Wiener filtering result, when classifying multiple image blocks of the current image, the multiple image blocks can be first classified based on the gradients in N2 directions of each image block to determine the preliminary classification results of the multiple image blocks.

[0195] In one example, N2=4. The specific implementation of the decoding end device classifying the image blocks based on the gradients of the image blocks in four directions (such as vertical direction, horizontal direction, 135° direction and 45° direction) can be referred to the relevant implementation in the existing Wiener filtering technology, and the embodiments of the present application will not be described in detail here.

[0196] The decoding end device can determine the number of image blocks included in each classification based on the preliminary classification results of the multiple image blocks, so as to determine the number of image blocks in the classification with the most image blocks (referred to as the first classification in this article) and the number of image blocks in the classification with the least image blocks (referred to as the second classification in this article).

[0197] The decoding end device can compare the number of image blocks in the first category with a first threshold (which can be set according to the actual scenario), and compare the number of image blocks in the second category with a second threshold (which can be set according to the actual scenario, the first threshold>the second threshold).

[0198] For example, the first threshold may be width×height>>7, and the second threshold may be width×height>>14, where width is the width of the current image, height is the height of the current image, and >> is a right shift operation.

[0199] When the number of image blocks in the first category is greater than the first threshold and the number of image blocks in the second category is less than the second threshold, the decoding end device can split the image blocks in the first category into two parts, and modify the classification of one part of the image blocks to the second category, and maintain the classification of the other part of the image blocks as the first category, so as to determine the final classification results of the multiple image blocks.

[0200] It should be noted that in the embodiment of the present application, after processing in the manner described in steps S1121 to S1123, the number of image blocks in each category can be re-counted to determine the category with the most image blocks and the category with the least image blocks. If the number of image blocks in the category with the most image blocks is greater than a first threshold, and the number of image blocks in the category with the least image blocks is less than a second threshold, then processing can be performed again in the manner described in steps S1121 to S1123 until the number of image blocks in the category with the most image blocks is less than or equal to the first threshold, or / and the number of image blocks in the category with the least image blocks is greater than or equal to the second threshold.

[0201] For example, assuming that the decoding end device divides the current image into blocks based on the optimal block division method, and classifies the obtained multiple image blocks, the category with the most image blocks is category A, and the number of image blocks in category A is greater than the first threshold, and the category with the least image blocks is category B, and the number of image blocks in category B is less than the second threshold, then the decoding end device can split the image blocks in category B into two parts, wherein the classification of one part of the image blocks is modified to category B, and the classification of the other part of the image blocks remains as category A, that is, after the classification splitting, category B includes the original image blocks and a part of the image blocks in category A, and category A includes another part of the image blocks in category A.

[0202] After completing this category splitting, the decoding end device can re-determine the category with the most image blocks (assuming it is category C) and the category with the least image blocks (assuming it is category D). If the number of image blocks in category C is greater than the first threshold and the number of image blocks in category D is less than the second threshold, the decoding end device can split category C, and so on, until the number of image blocks in the category with the most image blocks in the category to which the image blocks of the current image belong is less than or equal to the first threshold, or / and the number of image blocks in the category with the least image blocks is greater than or equal to the second threshold.

[0203] In one example, in step S2123, splitting the image blocks in the first category into two parts, and changing the classification of one part of the image blocks to the second category, and keeping the classification of the other part of the image blocks as the first category may include:

[0204] Dividing the current image into a first sub-image and a second sub-image;

[0205] Keeping the classification of the image block belonging to the first classification and located in the first sub-image as the first classification;

[0206] The classification of the image block that belongs to the first classification and is in the second sub-image is changed to the second classification.

[0207] Exemplarily, in order to split the image blocks in the first category, the current image may be divided into two parts (respectively referred to as a first sub-image and a second sub-image).

[0208] Exemplarily, the first sub-image and the second sub-image may not overlap each other, and together constitute a current image, that is, the area of ​​the current image is the sum of the area of ​​the first sub-image and the area of ​​the second sub-image.

[0209] For example, the current image may be divided into two equal parts, such as dividing the current image along a horizontal symmetry axis, a vertical symmetry axis, or a diagonal line of the current image.

[0210] For another example, the current image may be divided into two parts of unequal size.

[0211] When the decoding end device divides the current image into a first sub-image and a second sub-image, for each image block in the first category, it can be determined whether it is in the first sub-image or in the second sub-image.

[0212] For an image block that belongs to the first category and is in the first sub-image, its classification is kept as the first category;

[0213] For an image block that belongs to the first category and is in the second sub-image, its category is changed to the second category.

[0214] It should be noted that in the embodiment of the present application, for any image block in the first category, if part of the image block is in the first sub-image and the other part is in the second sub-image, its classification can be maintained as the first category, or its classification can be modified to the second category.

[0215] In one example, considering that the human eye is less sensitive to chrominance components than to luminance components, the filtering process for luminance components can be more detailed, and the filtering process for chrominance components can be relatively simple. Therefore, the image block in the above embodiment can be a luminance block, that is, the decoding end device can process the luminance block of the current image in the manner described in the above embodiment, and the chrominance block can be processed according to other strategies.

[0216] In a possible embodiment, the filtering method provided in the embodiment of the present application may further include:

[0217] When it is determined that the chrominance component of the current image needs to be Wiener filtered, for any chrominance block in the CTU for which the chrominance component needs to be Wiener filtered, determine the geometric transformation direction of the chrominance block based on the gradients of the chrominance block in N3 directions;

[0218] Obtain filter coefficients of chrominance blocks from the coded bitstream of the current image;

[0219] The obtained filter coefficients are geometrically inversely transformed based on the geometric transformation direction of the chroma block, and the chroma block is filtered based on the filter coefficients after the geometric inverse transformation.

[0220] Exemplarily, N3 is a positive integer greater than or equal to 2, N1, N2 and N3 may be all the same, all different or partially the same, and the above-mentioned N3 directions may include any N3 directions.

[0221] For example, in order to improve the accuracy of Wiener filtering of chrominance components, the encoding end device can rotate the chrominance block according to the gradient direction based on the gradient size and direction of each pixel and its surrounding pixels, so that the directions of all pixel blocks are unified.

[0222] When the decoding end device determines that the chrominance component of the current image needs to be Wiener filtered, for any chrominance block in the CTU that needs to be Wiener filtered, on the one hand, the decoding end device can determine the geometric transformation direction of the chrominance block based on the gradients of the chrominance block in N3 directions.

[0223] In one example, N3=4, the decoding end device determines the specific implementation of the geometric transformation direction of the chrominance block based on the gradient of the chrominance block in four directions (such as vertical direction, horizontal direction, 135° direction and 45° direction). Please refer to the relevant implementation of the luminance block in the existing Wiener filtering technology, and the embodiments of the present application will not be repeated here.

[0224] On the other hand, the decoding end device can obtain the filter coefficients of the chrominance block from the encoded code stream of the current image.

[0225] Exemplarily, when the chrominance blocks are not classified, the filter coefficients of the chrominance blocks in the coded bitstream of the current image are consistent.

[0226] When the chroma blocks are classified (for specific implementation, please refer to the relevant description in the above method embodiment), the filter coefficients of chroma blocks of different classifications can be carried in the encoded code stream of the current image; in this case, the decoding end device also needs to classify the chroma blocks and obtain the corresponding filter coefficients from the encoded code stream based on the classification to which the chroma blocks belong.

[0227] When the decoding end device determines the geometric transformation direction of the chroma block and obtains the filter coefficients of the chroma block, it can perform a geometric inverse transformation on the obtained filter coefficients based on the geometric transformation direction of the chroma block, and filter the chroma block based on the filter coefficients of the geometric inverse transformation.

[0228] See also Figure 6 , is a flow chart of a filtering method provided in an embodiment of the present application, wherein the filtering method can be applied to a coding end device, such as Figure 6 As shown, the filtering method may include the following steps:

[0229] Step S600: determine the optimal block division method for performing Wiener filtering on the current image.

[0230] In an embodiment of the present application, considering that the effect of Wiener filtering on an image may be different when the image is divided into blocks in different ways, in order to optimize the effect of Wiener filtering on the image, the encoding end device can determine the optimal way to divide the image blocks (referred to as the optimal block division method in this article), and determine the relevant parameters of the filtering, such as the filter coefficient, based on the optimal block division method.

[0231] In one example, in step S600, determining the optimal block division method for performing Wiener filtering on the current image may include:

[0232] For any candidate block partitioning method among the multiple candidate block partitioning methods, determine a rate-distortion cost of performing Wiener filtering on the current image under the candidate block partitioning method;

[0233] Based on the rate-distortion cost of Wiener filtering of the current image under each candidate block division method, the candidate block division method with the smallest rate-distortion cost is determined as the optimal block division method.

[0234] Exemplarily, the encoding end device may determine an optimal block division method for performing Wiener filtering on the current image based on the rate-distortion cost of performing Wiener filtering on the current image under different candidate block division methods.

[0235] The encoding end device can respectively determine the rate-distortion cost of performing Wiener filtering on the current image block under each candidate block division method, and based on the rate-distortion cost of performing Wiener filtering on the current image under each candidate block division method, determine the candidate block division method with the smallest rate-distortion cost as the optimal block division method.

[0236] Exemplarily, the candidate block division method may be determined by negotiation between the codec end, or configured by default, or configured in real time according to demand.

[0237] Exemplarily, the multiple candidate block division methods include multiple block division methods for dividing the current image into blocks according to different image block sizes.

[0238] Exemplarily, different image block sizes may include but are not limited to 1*1, 2*2, 4*4 or 8*8.

[0239] In one example, if Figure 7 As shown, the rate-distortion cost of performing Wiener filtering on the current image under the candidate block division method can be determined by the following steps:

[0240] Step S700: classify the image blocks obtained by performing block division according to the candidate block division method to determine a preliminary classification result corresponding to the candidate block division method.

[0241] Step S710: Determine the number of image blocks included in each classification based on the preliminary classification result corresponding to the candidate block division method.

[0242] Step S720: When the number of image blocks in the first category with the largest number of image blocks is greater than a first threshold, and the number of image blocks in the second category with the smallest number of image blocks is less than a second threshold, the image blocks in the first category are split into two parts, and the classification of one part of the image blocks is modified to the second classification, and the classification of the other part of the image blocks is maintained as the first classification, so as to determine the final classification result corresponding to the candidate block division method.

[0243] Step S730: Based on the final classification result corresponding to the candidate block division method, determine the rate-distortion cost of performing Wiener filtering on the current image under the candidate block division method.

[0244] For example, after classifying multiple image blocks of the current image, the number of image blocks in each classification may vary greatly. For example, the number of image blocks in some classifications may reach thousands, while the number of image blocks in other classifications may be only a few.

[0245] Since the essence of Wiener filtering is a linear regression model, the image blocks in each category approximately satisfy a linear relationship, which can make the results of Wiener filtering more accurate. However, when the number of image blocks contained in the category increases, the possibility of underfitting increases, and the accuracy of the Wiener filtering results may decrease.

[0246] In order to improve the accuracy of the result of Wiener filtering, when classifying multiple image blocks of the current image, if there are too many classifications of image blocks and too few classifications of image blocks, the classifications with too many image blocks can be split.

[0247] For any candidate block division method, the encoding end device can divide the current image into blocks based on the candidate block division method, and classify the obtained multiple image blocks to obtain preliminary classification results of the multiple image blocks.

[0248] Exemplarily, the encoding end device may classify the multiple image blocks based on the gradients of each image block in N2 directions to determine preliminary classification results of the multiple image blocks.

[0249] For example, N2=4, such as vertical direction, horizontal direction, 135° direction and 45° direction, the schematic diagram can be as follows: Fig.9A shown.

[0250] When the encoding end device obtains the preliminary classification results of multiple image blocks under the candidate block division method, it can determine the number of image blocks in each classification to determine the number of image blocks in the classification with the most image blocks (referred to as the first classification in this article) and the number of image blocks in the classification with the least image blocks (referred to as the second classification in this article).

[0251] The encoding end device can compare the number of image blocks in the first category with a first threshold (which can be set according to the actual scenario), and compare the number of image blocks in the second category with a second threshold (which can be set according to the actual scenario, the first threshold > the second threshold).

[0252] For example, the first threshold may be width×height>>7, and the second threshold may be width×height>>14, where width is the width of the current image, height is the height of the current image, and >> is a right shift operation.

[0253] When the number of image blocks in the first category is greater than the first threshold and the number of image blocks in the second category is less than the second threshold, the encoding end device can split the image blocks in the first category into two parts, and modify the classification of one part of the image blocks to the second category, and maintain the classification of the other part of the image blocks as the first category, so as to determine the final classification results of the multiple image blocks.

[0254] It should be noted that, in an embodiment of the present application, after the classification splitting processing is performed in the above manner, the number of image blocks in each classification can be re-counted to determine the classification with the most image blocks and the classification with the least image blocks. If the number of image blocks in the classification with the most image blocks is greater than a first threshold, and the number of image blocks in the classification with the least image blocks is less than a second threshold, the classification splitting processing can be performed again in the above manner until the number of image blocks in the classification with the most image blocks is less than or equal to the first threshold, or / and the number of image blocks in the classification with the least image blocks is greater than or equal to the second threshold.

[0255] For example, assuming that the encoding end device divides the current image into blocks based on the candidate block division method, and classifies the obtained multiple image blocks, the category with the most image blocks is category A, and the number of image blocks in category A is greater than the first threshold, and the category with the least image blocks is category B, and the number of image blocks in category B is less than the second threshold, then the encoding end device can split the image blocks in category B into two parts, wherein the classification of one part of the image blocks is modified to category B, and the classification of the other part of the image blocks remains as category A, that is, after the classification splitting, category B includes the original image blocks and a part of the image blocks of category A, and category A includes another part of the image blocks in category A.

[0256] After completing this category splitting, the encoding end device can re-determine the category with the most image blocks (assuming it is category C) and the category with the least image blocks (assuming it is category D). If the number of image blocks in category C is greater than the first threshold and the number of image blocks in category D is less than the second threshold, the encoding end device can split category C, and so on, until the number of image blocks in the category with the most image blocks in the category to which the image blocks of the current image belong is less than or equal to the first threshold, or / and the number of image blocks in the category with the least image blocks is greater than or equal to the second threshold.

[0257] When the encoding end device determines the final classification result corresponding to the candidate block division method, it can determine the rate-distortion cost of performing Wiener filtering on the current image under the candidate block division method based on the final classification result corresponding to the candidate block division method.

[0258] It should be noted that after the encoding end device obtains the classification of each image block, the specific implementation of determining the rate-distortion cost of Wiener filtering for the current image can refer to the relevant implementation in the existing Wiener filtering technology, and the embodiments of this application will not be repeated here.

[0259] When the encoding end device determines the rate-distortion cost of performing Wiener filtering on the current image under each candidate block division method, the candidate block division method with the smallest rate-distortion cost can be determined as the optimal block division method.

[0260] In one example, in step S720, splitting the image blocks in the first category into two parts, and changing the classification of one part of the image blocks to the second category, and keeping the classification of the other part of the image blocks as the first category may include:

[0261] Dividing the current image into a first sub-image and a second sub-image;

[0262] Keeping the classification of the image block belonging to the first classification and located in the first sub-image as the first classification;

[0263] The classification of the image block that belongs to the first classification and is in the second sub-image is changed to the second classification.

[0264] Exemplarily, in order to split the image blocks in the first category, the current image may be divided into two parts (respectively referred to as a first sub-image and a second sub-image).

[0265] For example, the current image may be divided into two equal parts, such as dividing the current image along a horizontal symmetry axis, a vertical symmetry axis, or a diagonal line of the current image.

[0266] For another example, the current image may be divided into two parts of unequal size.

[0267] When the encoding end device divides the current image into a first sub-image and a second sub-image, for each image block in the first category, it can be determined whether it is in the first sub-image or the second sub-image.

[0268] For an image block that belongs to the first category and is in the first sub-image, its classification is kept as the first category;

[0269] For an image block that belongs to the first category and is in the second sub-image, its category is changed to the second category.

[0270] It should be noted that in the embodiment of the present application, for any image block in the first category, if part of the image block is in the first sub-image and the other part is in the second sub-image, its classification can be maintained as the first category, or its classification can be modified to the second category.

[0271] In addition, in an embodiment of the present application, after completing the splitting of the first category (i.e., the number of image blocks in the first category is less than or equal to the first threshold, or / and, the number of image blocks in the second category is greater than or equal to the second threshold), inter-class merging operations can also be performed on each category. The specific implementation can refer to the relevant implementation in the existing Wiener filtering technology, and the embodiment of the present application will not be elaborated here.

[0272] Step S610: When it is determined that the current image needs to be subjected to Wiener filtering, the filter coefficients of the classification to which each image block of the current image belongs under the optimal block division method are notified to the decoding end device through the encoding code stream of the current image.

[0273] In an embodiment of the present application, when the encoding end device determines that Wiener filtering is required for the current image, the filter coefficients of the categories to which each image block of the current image belongs can be determined under the optimal block division method, and the filter coefficients of the categories to which each image block belongs can be notified to the decoding end device through the encoding code stream of the current image.

[0274] It should be noted that the encoding end device can determine whether to perform Wiener filtering on the current image based on the image loss in two situations: performing Wiener filtering on the current image and not performing Wiener filtering on the current image.

[0275] Similarly, for any CTU, the encoding end device can also determine whether to perform Wiener filtering on the CTU based on the image loss in two situations: performing Wiener filtering and not performing Wiener filtering.

[0276] In a possible embodiment, Figure 8 As shown, in step S610, the filter coefficients of the categories to which each image block of the current image belongs under the optimal block division method can be determined by the following steps:

[0277] Step S611: determine the filter coefficients of the categories to which each image block of the current image belongs under the optimal block division method.

[0278] Step S612: For the filter coefficients of any classification, based on the rate-distortion cost of Wiener filtering of the current image in the optimal block division mode, the filter coefficients are adaptively adjusted.

[0279] Step S613: Determine the filter coefficients after the adaptive adjustment as the filter coefficients of the classification.

[0280] For example, considering that there may be a high correlation between coefficients at different positions in the filter coefficients, in this case, the coefficients at certain positions may not be transmitted to the decoding end device, thereby reducing the transmitted bit stream while ensuring the filtering quality, thereby improving the encoding performance.

[0281] When the encoding end device determines the optimal block division method for Wiener filtering of the current image and determines that Wiener filtering is required for the current image, the filter coefficients of the classification to which each image block of the current image belongs under the optimal block division method can be determined.

[0282] It should be noted that the specific implementation of the encoding end device determining the filter coefficients of the category to which each image block of the current image belongs can be found in the relevant description of the existing Wiener filtering technology, and the embodiments of the present application will not be elaborated here.

[0283] For the filter coefficients of any classification, the encoding end device can adaptively adjust the filter coefficients based on the rate-distortion cost of Wiener filtering of the current image in the optimal block division method.

[0284] Exemplarily, for any filter coefficient adjustment, the encoding end device can determine whether the filter coefficient adjustment is effective based on the rate-distortion cost of Wiener filtering of the current image under the optimal block division method before the filter coefficient adjustment, and the rate-distortion cost of Wiener filtering of the current image under the optimal block division method after the filter coefficient adjustment; if the latter is greater than the former, it is determined that it is not effective; otherwise, it is determined that the filter coefficient adjustment is effective.

[0285] When the encoding end device completes the adaptive adjustment of the filter coefficients, the filter coefficients after the adaptive adjustment can be determined as the filter coefficients of the classification.

[0286] In one example, in step S612, adaptively adjusting the filter coefficient based on the rate-distortion cost of Wiener filtering of the current image in the optimal block division manner may include:

[0287] According to the preset order, try to delete the coefficients at the specified position in the filter coefficients in turn;

[0288] For any attempted deletion operation, if the rate-distortion cost of performing Wiener filtering on the current image before deleting the coefficients is greater than or equal to the rate-distortion cost of performing Wiener filtering on the current image after deleting the coefficients, then it is determined that this deletion operation is effective, and attempts are made to delete the coefficients at the specified positions in the filter coefficients that have not been deleted in sequence in the preset order; otherwise, it is determined that this deletion operation is ineffective, and attempts are made to delete the coefficients at the specified positions in the filter that have not been deleted in sequence in the preset order.

[0289] Exemplarily, in order to achieve adaptive adjustment of the filter coefficients, for any category of filter coefficients, the coefficients at specified positions in the filter coefficients may be deleted in sequence according to a preset order.

[0290] For any deletion operation, the encoding end device can compare the rate-distortion cost of performing Wiener filtering on the current image before the coefficients are deleted, and the rate-distortion cost of performing Wiener filtering on the current image after the coefficients are deleted. If the former is greater than or equal to the latter, it is determined that the deletion operation is effective, and the coefficients at the specified positions in the filter coefficients that have not been deleted are tried again in sequence according to the preset order; otherwise, it is determined that the deletion operation is ineffective, and the coefficients at the specified positions in the filter coefficients that have not been deleted are tried again in sequence according to the preset order.

[0291] Repeat the above operation until it is determined that after deleting the coefficients at any position in the filter coefficients that have not been deleted, the rate-distortion cost of Wiener filtering on the current image is greater than the rate-distortion cost of Wiener filtering on the current image before the coefficients are deleted.

[0292] For example, assuming that the preset order is: {[0, 9], [1, 3], [2, 10], [4, 8], [5, 7], [6, 11]}, that is, the encoding end device can first try to delete the coefficients at position 0 and position 9 in the filter coefficients, and compare the rate-distortion cost of performing Wiener filtering on the current image before deleting the coefficients, and the rate-distortion cost of performing Wiener filtering on the current image after deleting the coefficients.

[0293] If the former is greater than or equal to the latter, the encoding end device determines that the deletion operation is effective and deletes the coefficients at positions 0 and 9 from the filter coefficients; then, the encoding end device can try to delete the coefficients at positions 1 and 3 in the filter coefficients and determine whether the deletion operation is effective according to the same strategy. If effective, the coefficients at positions 1 and 3 are deleted, and the coefficients at positions 2 and 10 are tried again; if not effective, the coefficients at positions 1 and 3 are not deleted, and the coefficients at positions 2 and 10 are tried again.

[0294] If the former is less than the latter, the encoding end determines that the deletion operation is not effective. At this time, the encoding end device determines not to delete the coefficients at positions 0 and 9, and tries to delete the coefficients at positions 1 and 3 in the filter coefficients, and determines whether the deletion operation is effective according to the same strategy; if effective, the encoding end device deletes the coefficients at positions 1 and 3, and retry to delete the coefficients at positions 0 and 9 (at this time the coefficients at positions 1 and 3 have been deleted); if ineffective, the coefficients at positions 1 and 3 are not deleted, and the coefficients at positions 2 and 10 are tried.

[0295] That is, the encoding end device tries to delete the coefficients at the corresponding positions in sequence according to the preset order, and determines whether the deletion is effective based on the rate-distortion cost of Wiener filtering of the current image before and after the coefficient deletion; if it is effective, try to delete the coefficients at the specified positions that have not been deleted again according to the preset order; if it is not effective, continue to try backward according to the preset order.

[0296] In one example, considering that the human eye is less sensitive to chrominance components than to luminance components, the filtering process for luminance components can be more detailed, and the filtering process for chrominance components can be relatively simple. Therefore, the image block in the above embodiment can be a luminance block, that is, the encoding end device can process the luminance block of the current image in the manner described in the above embodiment, and the chrominance block can be processed according to other strategies.

[0297] In a possible embodiment, the filtering method provided in the embodiment of the present application may further include:

[0298] For any chromaticity block of a preset size in the current image, determine the geometric transformation direction of the chromaticity block based on the gradients of the chromaticity block in N3 directions;

[0299] Determining a filter coefficient of the chroma block based on a geometric transformation direction of the chroma block;

[0300] Based on the filter coefficients of each chrominance block in the current image, determining on / off of a CTU-level filter and on / off of a frame-level filter of the current image.

[0301] Exemplarily, the preset size may include but is not limited to 1*1, 2*2, 4*4 or 8*8.

[0302] For example, in order to improve the accuracy of Wiener filtering of chrominance components, the encoding end device can rotate the chrominance block according to the gradient direction based on the gradient size and direction of each pixel and its surrounding pixels, so that the directions of all pixel blocks are unified.

[0303] For any chroma block of preset size in the current image, the encoding end device can determine the geometric transformation direction of the chroma block based on the gradients of the chroma block in N3 directions, and determine the filter coefficient of the chroma block based on the geometric transformation direction of the chroma block.

[0304] Exemplarily, N3=4, the encoding end device determines the geometric transformation direction of the chroma block based on the gradients of the chroma block in four directions, and determines the filter coefficients of the chroma block based on the geometric transformation direction of the chroma block. For specific implementation, please refer to the relevant implementation for the luminance block in the existing Wiener filtering technology, and the embodiments of the present application will not be repeated here.

[0305] It should be noted that when the chroma blocks are not classified, the filter coefficients of the chroma blocks are consistent. When the chroma blocks are classified (for specific implementation, see the relevant description in the above method embodiment), the filter coefficients of chroma blocks of different classifications may be different.

[0306] The encoding end device can determine the on / off of the CTU-level filter and the on / off of the frame-level filter of the current image based on the filter coefficient of the chrominance block.

[0307] In order to enable those skilled in the art to better understand the technical solutions provided by the embodiments of the present application, the technical solutions provided by the embodiments of the present application are described below with reference to specific examples.

[0308] Embodiment 1: Multi-size brightness block division method

[0309] By changing the size of the blocks used to divide the image, different classification results can be obtained, and the filtering quality will also be different. Therefore, when dividing the image into blocks, different image block sizes can obtain different classification results, and then the block division method with the least loss can be selected.

[0310] Exemplarily, an optimal block division method is determined from a plurality of candidate block division methods, and based on the optimal block division method, the brightness component of the image block is divided into blocks and Wiener filtering is performed.

[0311] Encoding equipment:

[0312] 1. Based on different brightness block sizes, the brightness component of the current image is divided into blocks.

[0313] Exemplarily, different luminance block sizes include 1*1, 2*2, 4*4 or 8*8.

[0314] 2. For any block division method, multiple brightness blocks obtained by block division of the brightness component of the current image based on the block division method are classified to obtain a classification result under the block division method.

[0315] 3. Determine the rate-distortion cost of performing Wiener filtering on the brightness component of the current image based on the classification result.

[0316] 4. Based on the rate-distortion cost of Wiener filtering on the brightness component of the current image under each block division method, determine the optimal block division method.

[0317] Embodiment 2: Classification Splitting Processing

[0318] The number of luminance blocks included in each category may vary greatly. Some categories may include thousands of luminance blocks, while others may include a few luminance blocks or even no luminance blocks.

[0319] At present, the Wiener filtering process only includes the process of merging different categories, and there is no process of splitting the same category. The more luminance blocks included in a category, the higher the complexity of the luminance blocks of the category, the greater the possibility of underfitting, and thus, the lower the accuracy of the Wiener filtering result.

[0320] Reducing the number of luminance blocks in the same category can effectively reduce data complexity and optimize the Wiener filtering effect of the category; while the category with too few luminance blocks has a good fitting effect, it does not have a significant effect on the overall image. Therefore, the category with a large number of luminance blocks can be split, and the split luminance blocks can be merged with the category with a small number of luminance blocks.

[0321] 1. Block-dividing the brightness component of the current image according to a preset block division method to obtain multiple brightness blocks.

[0322] 2. Based on the gradients of the brightness blocks in four directions, namely horizontal, vertical and diagonal, the classification results of the multiple brightness blocks (ie, preliminary classification results) are obtained.

[0323] 3. Count the number of brightness blocks included in each category.

[0324] 4. When the number of brightness blocks in the category with the most brightness blocks (i.e., the first category) is greater than the first threshold, and the number of brightness blocks in the category with the least brightness blocks (i.e., the second category) is less than the second threshold, go to step 5; otherwise, determine that the current classification result is the final classification result, and go to step 6.

[0325] 5. Divide the current image into a first sub-image and a second sub-image. For any luminance block in the first category, if the luminance block is in the first sub-image, keep the classification of the luminance block as the first category; if the luminance block is in the second sub-image, change the classification of the luminance block to the second category and go to step 3.

[0326] Exemplarily, for any image, when entering step 5 for the first time, the image segmentation operation can be performed, and then the first classification splitting operation can be performed; when it is not the first time entering step 5, the image segmentation operation can be omitted, and then the first classification splitting operation can be performed.

[0327] Exemplarily, after obtaining the final classification results of each brightness block of the current image, an inter-class merging operation may be performed, and its specific implementation may refer to the relevant implementation in the existing Wiener filtering technology, which will not be described in detail in this application.

[0328] Embodiment 3: Adaptive adjustment of filter coefficients

[0329] The filter for the luminance component is a 7*7 filter. The required parameters for each filter are fixed 13 coefficients. Considering that there may be a high correlation between the coefficients at different positions in the filter coefficients, in this case, the coefficients at certain positions do not need to be transmitted to the decoding device. Thus, the transmitted bit stream can be reduced while ensuring the filtering quality, thereby improving the encoding performance.

[0330] Encoding equipment:

[0331] 1. For the filter coefficients of any classification of the luminance block, determine the rate-distortion cost of using 13 coefficients (ie, no coefficient deletion) to perform Wiener filtering on the current image.

[0332] 2. According to the preset order, try to delete the coefficients at the specified positions in the filter in turn, and determine the rate-distortion cost of Wiener filtering on the current image after the coefficients are deleted.

[0333] Exemplarily, the preset order is: {[0, 9], [1, 3], [2, 10], [4, 8], [5, 7], [6, 11]}.

[0334] The schematic diagram can be shown as Fig. 9B As shown, for the brightness component filter, the coefficients at corresponding positions can be deleted according to the same shadow.

[0335] 3. For any attempted deletion operation, if the rate-distortion cost of performing Wiener filtering on the current image before deleting the coefficients is greater than or equal to the rate-distortion cost of performing Wiener filtering on the current image after deleting the coefficients, then it is determined that the deletion operation is effective, and the coefficients at the specified positions in the filter coefficients that have not been deleted are attempted again in sequence according to the preset order; otherwise, it is determined that the deletion operation is ineffective, and attempts are made to delete the coefficients at the specified positions in the filter coefficients that have not been deleted in sequence according to the preset order.

[0336] 4. When it is determined that the rate-distortion cost of performing Wiener filtering on the current image after deleting the coefficients at any position in the filter coefficients that have not been deleted is greater than the rate-distortion cost of performing Wiener filtering on the current image before deleting the coefficients, the current filter coefficients are determined as the filter coefficients of this category.

[0337] Embodiment 4

[0338] Encoding equipment:

[0339] 1. Determine the optimal block division method for performing Wiener filtering on the brightness component of the current image in the manner described in the first embodiment.

[0340] 2. Based on the optimal block division method, the brightness component of the current image is divided into blocks to obtain multiple brightness blocks.

[0341] 3. Classify the multiple brightness blocks based on the gradients in four directions: horizontal, vertical and diagonal.

[0342] 4. For any luminance block, determine the geometric transformation direction of the luminance block based on the gradients in four directions: horizontal, vertical and diagonal.

[0343] 5. For any category, rotate each luminance block in the category according to the geometric transformation direction determined in step 3 to obtain the filter coefficient of the category.

[0344] 6. Determine the on / off of the CTU level filter and the on / off of the frame level filter.

[0345] 7. The optimal block division method, the filter coefficients of each category, the on / off of the CTU-level filter, and the on / off of the frame-level filter are notified to the decoding end device through the encoded bit stream.

[0346] Decoding device:

[0347] 1. Get the on / off status of the frame-level filter from the encoded bitstream of the current image; if it is on, go to step 2; otherwise, end the current process.

[0348] 2. Obtain the optimal block division method for performing Wiener filtering on the brightness component of the current image from the encoded bit stream of the current image.

[0349] 3. Based on the optimal block division method, the brightness component of the current image is divided into blocks to obtain multiple brightness blocks.

[0350] 4. Classify each brightness block based on its gradient in four directions: horizontal, vertical and diagonal.

[0351] 5. Obtain the on / off status of each CTU-level filter from the encoded bitstream in sequence according to the CTU order.

[0352] 6. For any luma block in a CTU with a CTU-level filter turned on, based on the category to which the luma block belongs, obtain the filter coefficient of the category from the coded bitstream.

[0353] 7. Based on the gradients of the brightness block in four directions, namely horizontal, vertical and diagonal, determine the geometric transformation direction of the brightness block.

[0354] 8. Based on the geometric transformation direction of the luminance block, perform an inverse geometric transformation on the filter coefficients of the classification to which the luminance block belongs.

[0355] 9. Filter the luminance block based on the filter coefficients after the geometric inverse transform.

[0356] Embodiment 5

[0357] Encoding equipment:

[0358] 1. Divide the brightness component of the current image into blocks to obtain multiple 4*4 brightness blocks.

[0359] 2. Determine the classification of the multiple 4*4 brightness blocks according to the method described in the second embodiment.

[0360] 3. For any 4*4 brightness block, determine the geometric transformation direction of the 4*4 brightness block based on the gradients in the horizontal, vertical and diagonal directions of the 4*4 brightness block.

[0361] 4. For any category, rotate each luminance block in the category according to the geometric transformation direction determined in step 3 to obtain the filter coefficient of the category.

[0362] 5. Determine the on / off of the CTU level filter and the on / off of the frame level filter.

[0363] 6. The filter coefficients of each category, the on / off of the CTU-level filter, and the on / off of the frame-level filter are notified to the decoding end device through the encoded bitstream.

[0364] Decoding device:

[0365] 1. Get the on / off status of the frame-level filter from the encoded bitstream of the current image; if it is on, go to step 2; otherwise, end the current process.

[0366] 2. Divide the brightness component of the current image into blocks to obtain multiple 4*4 brightness blocks.

[0367] 3. Determine the classification of the multiple 4*4 brightness blocks according to the method described in the second embodiment.

[0368] 4. Obtain the on / off status of each CTU-level filter from the encoded bitstream in CTU order.

[0369] 5. For a 4*4 luminance block in any CTU with the CTU-level filter turned on, based on the category to which the 4*4 luminance block belongs, obtain the filter coefficient of the category from the coded bitstream.

[0370] 6. Based on the gradients of the 4*4 brightness block in the horizontal, vertical and diagonal directions, a geometric transformation direction of the 4*4 brightness block is determined.

[0371] 7. Based on the geometric transformation direction of the 4*4 luminance block, perform an inverse geometric transformation on the filter coefficients of the classification to which the obtained 4*4 luminance block belongs.

[0372] 8. Filter the 4*4 luminance block based on the filter coefficients after the geometric inverse transform.

[0373] Embodiment 6

[0374] Encoding equipment:

[0375] 1. Divide the brightness component of the current image into blocks to obtain multiple 4*4 brightness blocks.

[0376] 2. Classify multiple 4*4 brightness blocks based on the gradients in four directions: horizontal, vertical and diagonal.

[0377] 3. For any 4*4 brightness block, determine the geometric transformation direction of the 4*4 brightness block based on the gradients in the horizontal, vertical and diagonal directions of the 4*4 brightness block.

[0378] 4. For any category, rotate each 4*4 brightness block in the category according to the geometric transformation direction determined in step 3 to obtain the filter coefficient of the category.

[0379] 5. Adaptively adjust the filter coefficients of the classification in the manner described in the third embodiment.

[0380] 6. Determine the on / off of the CTU level filter and the on / off of the frame level filter.

[0381] 7. The adaptively adjusted filter coefficients of each classification, the on / off of the CTU-level filter, and the on / off of the frame-level filter are notified to the decoding end device through the encoded bit stream.

[0382] Decoding equipment:

[0383] 1. Get the on / off status of the frame-level filter from the encoded bitstream of the current image; if it is on, go to step 2; otherwise, end the current process.

[0384] 2. Divide the brightness component of the current image into blocks to obtain multiple 4*4 brightness blocks.

[0385] 3. Classify the brightness blocks based on their gradients in four directions: horizontal, vertical and diagonal.

[0386] 4. Obtain the on / off status of each CTU-level filter from the encoded bitstream in CTU order.

[0387] 5. For a 4*4 luminance block in any CTU with the CTU-level filter turned on, based on the category to which the 4*4 luminance block belongs, obtain the filter coefficient of the category from the coded bitstream.

[0388] 6. Based on the gradients of the 4*4 brightness block in the horizontal, vertical and diagonal directions, a geometric transformation direction of the 4*4 brightness block is determined.

[0389] 7. Based on the geometric transformation direction of the 4*4 luminance block, perform an inverse geometric transformation on the filter coefficients of the classification to which the obtained 4*4 luminance block belongs.

[0390] 8. Filter the 4*4 luminance block based on the filter coefficients after the geometric inverse transform.

[0391] Embodiment 7

[0392] Encoding equipment:

[0393] 1. Determine the optimal block division method for performing Wiener filtering on the brightness component of the current image in the manner described in the first embodiment.

[0394] 2. Based on the optimal block division method, the brightness component of the current image is divided into blocks to obtain multiple brightness blocks.

[0395] 3. Determine the classification of the multiple brightness blocks according to the method described in the second embodiment.

[0396] 4. For any luminance block, determine the geometric transformation direction of the luminance block based on the gradients in four directions: horizontal, vertical and diagonal.

[0397] 5. For any category, rotate each luminance block in the category according to the geometric transformation direction determined in step 3 to obtain the filter coefficient of the category.

[0398] 6. Determine the on / off of the CTU level filter and the on / off of the frame level filter.

[0399] 7. The optimal block division method, the filter coefficients of each category, the on / off of the CTU-level filter, and the on / off of the frame-level filter are notified to the decoding end device through the encoded bit stream.

[0400] Decoding device:

[0401] 1. Get the on / off status of the frame-level filter from the encoded bitstream of the current image; if it is on, go to step 2; otherwise, end the current process.

[0402] 2. Obtain the optimal block division method for performing Wiener filtering on the brightness component of the current image from the encoded bit stream of the current image.

[0403] 3. Based on the optimal block division method, the brightness component of the current image is divided into blocks to obtain multiple brightness blocks.

[0404] 4. Determine the classification of the multiple brightness blocks according to the method described in the second embodiment.

[0405] 5. Obtain the on / off status of each CTU-level filter from the encoded bitstream in sequence according to the CTU order.

[0406] 6. For any luma block in a CTU with a CTU-level filter turned on, based on the category to which the luma block belongs, obtain the filter coefficient of the category from the coded bitstream.

[0407] 7. Based on the gradients of the brightness block in four directions, namely horizontal, vertical and diagonal, determine the geometric transformation direction of the brightness block.

[0408] 8. Based on the geometric transformation direction of the luminance block, perform an inverse geometric transformation on the filter coefficients of the classification to which the luminance block belongs.

[0409] 9. Filter the luminance block based on the filter coefficients after the geometric inverse transform.

[0410] Embodiment 8

[0411] Encoding equipment:

[0412] 1. Determine the optimal block division method for performing Wiener filtering on the brightness component of the current image in the manner described in the first embodiment.

[0413] 2. Based on the optimal block division method, the brightness component of the current image is divided into blocks to obtain multiple brightness blocks.

[0414] 3. Classify the multiple brightness blocks based on the gradients in four directions: horizontal, vertical and diagonal.

[0415] 4. For any luminance block, determine the geometric transformation direction of the luminance block based on the gradients in four directions: horizontal, vertical and diagonal.

[0416] 5. For any category, rotate each luminance block in the category according to the geometric transformation direction determined in step 3 to obtain the filter coefficient of the category.

[0417] 6. Adaptively adjust the filter coefficients of the classification in the manner described in the third embodiment.

[0418] 7. Determine the on / off of the CTU level filter and the on / off of the frame level filter.

[0419] 8. The optimal block division method, the adaptively adjusted filter coefficients of each classification, the on / off of the CTU-level filter, and the on / off of the frame-level filter are notified to the decoding end device through the encoded bit stream.

[0420] Decoding equipment:

[0421] 1. Get the on / off status of the frame-level filter from the encoded bitstream of the current image; if it is on, go to step 2; otherwise, end the current process.

[0422] 2. Obtain the optimal block division method for performing Wiener filtering on the brightness component of the current image from the encoded bit stream of the current image.

[0423] 3. Based on the optimal block division method, the brightness component of the current image is divided into blocks to obtain multiple brightness blocks.

[0424] 4. Classify each brightness block based on its gradient in four directions: horizontal, vertical and diagonal.

[0425] 5. Obtain the on / off status of each CTU-level filter from the encoded bitstream in sequence according to the CTU order.

[0426] 6. For any luma block in a CTU with a CTU-level filter turned on, based on the category to which the luma block belongs, obtain the filter coefficient of the category from the coded bitstream.

[0427] 7. Based on the gradients of the brightness block in four directions, namely horizontal, vertical and diagonal, determine the geometric transformation direction of the brightness block.

[0428] 8. Based on the geometric transformation direction of the luminance block, perform an inverse geometric transformation on the filter coefficients of the classification to which the luminance block belongs.

[0429] 9. Filter the luminance block based on the filter coefficients after the geometric inverse transform.

[0430] Embodiment 9

[0431] Encoding equipment:

[0432] 1. Divide the brightness component of the current image into blocks to obtain multiple 4*4 brightness blocks.

[0433] 2. Determine the classification of the multiple 4*4 brightness blocks according to the method described in the second embodiment.

[0434] 3. For any 4*4 brightness block, determine the geometric transformation direction of the 4*4 brightness block based on the gradients in the horizontal, vertical and diagonal directions of the 4*4 brightness block;

[0435] 4. For any category, rotate each luminance block in the category according to the geometric transformation direction determined in step 3 to obtain the filter coefficient of the category.

[0436] 5. Adaptively adjust the filter coefficients of the classification in the manner described in the third embodiment.

[0437] 6. Determine the on / off of the CTU level filter and the on / off of the frame level filter.

[0438] 7. The adaptively adjusted filter coefficients of each classification, the on / off of the CTU-level filter, and the on / off of the frame-level filter are notified to the decoding end device through the encoded bit stream.

[0439] Decoding device:

[0440] 1. Get the on / off status of the frame-level filter from the encoded bitstream of the current image; if it is on, go to step 2; otherwise, end the current process.

[0441] 2. Divide the brightness component of the current image into blocks to obtain multiple 4*4 brightness blocks.

[0442] 3. Determine the classification of the multiple 4*4 brightness blocks according to the method described in the second embodiment.

[0443] 4. Obtain the on / off status of each CTU-level filter from the encoded bitstream in CTU order.

[0444] 5. For a 4*4 luminance block in any CTU with the CTU-level filter turned on, based on the category to which the 4*4 luminance block belongs, obtain the filter coefficient of the category from the coded bitstream.

[0445] 6. Based on the gradients of the 4*4 brightness block in the horizontal, vertical and diagonal directions, a geometric transformation direction of the 4*4 brightness block is determined.

[0446] 7. Based on the geometric transformation direction of the 4*4 luminance block, perform an inverse geometric transformation on the filter coefficients of the classification to which the obtained 4*4 luminance block belongs.

[0447] 8. Filter the 4*4 luminance block based on the filter coefficients after the geometric inverse transform.

[0448] Embodiment 10

[0449] Encoding equipment:

[0450] 1. Determine the optimal block division method for performing Wiener filtering on the brightness component of the current image in the manner described in the first embodiment.

[0451] 2. Based on the optimal block division method, the brightness component of the current image is divided into blocks to obtain multiple brightness blocks.

[0452] 3. Determine the classification of the multiple brightness blocks according to the method described in the second embodiment.

[0453] 4. For any luminance block, determine the geometric transformation direction of the luminance block based on the gradients in the horizontal, vertical and diagonal directions of the luminance block;

[0454] 5. For any category, rotate each luminance block in the category according to the geometric transformation direction determined in step 3 to obtain the filter coefficient of the category.

[0455] 6. Adaptively adjust the filter coefficients of the classification in the manner described in the third embodiment.

[0456] 7. Determine the on / off of the CTU level filter and the on / off of the frame level filter.

[0457] 8. The optimal block division method, the adaptively adjusted filter coefficients of each classification, the on / off of the CTU-level filter, and the on / off of the frame-level filter are notified to the decoding end device through the encoded bit stream.

[0458] Decoding equipment:

[0459] 1. Get the on / off status of the frame-level filter from the encoded bitstream of the current image; if it is on, go to step 2; otherwise, end the current process.

[0460] 2. Obtain the optimal block division method for performing Wiener filtering on the brightness component of the current image from the encoded bit stream of the current image.

[0461] 3. Based on the optimal block division method, the brightness component of the current image is divided into blocks to obtain multiple brightness blocks.

[0462] 4. Determine the classification of the multiple brightness blocks according to the method described in the second embodiment.

[0463] 5. Obtain the on / off status of each CTU-level filter from the encoded bitstream in sequence according to the CTU order.

[0464] 6. For any luma block in a CTU with a CTU-level filter turned on, based on the category to which the luma block belongs, obtain the filter coefficient of the category from the coded bitstream.

[0465] 7. Based on the gradients of the brightness block in four directions, namely horizontal, vertical and diagonal, determine the geometric transformation direction of the brightness block.

[0466] 8. Based on the geometric transformation direction of the luminance block, perform an inverse geometric transformation on the filter coefficients of the classification to which the luminance block belongs.

[0467] 9. Filter the luminance block based on the filter coefficients after the geometric inverse transform.

[0468] Embodiment 11: Geometric transformation of chromaticity blocks

[0469] Since all chrominance pixels belong to the same class and the filter shape is Figure 1C (b) As shown. Therefore, for the filtering parameter chrominance component, only 6 coefficients need to be encoded at most. In order to make the directions of all data points of the chrominance component of the current image consistent, the gradients of the horizontal, vertical and diagonal directions of each 4*4 chrominance block before Wiener filtering can be calculated, and each 4*4 chrominance block can be rotated to make the directions of all pixel blocks consistent.

[0470] Encoding equipment:

[0471] 1. Divide the chrominance components of the current image into blocks to obtain multiple 4*4 chrominance blocks.

[0472] 2. For the gradients in the horizontal, vertical and diagonal directions of any 4*4 chromaticity block, determine the geometric transformation direction of the 4*4 chromaticity block.

[0473] 3. Rotate each 4*4 chrominance block of the current image according to the geometric transformation direction determined in step 2 to obtain the filter coefficient.

[0474] 4. Determine the on / off of the CTU level filter and the on / off of the frame level filter.

[0475] 5. The filter coefficients, the on / off of the CTU-level filter, and the on / off of the frame-level filter are notified to the decoding end device through the encoded bitstream.

[0476] Decoding device:

[0477] 1. Get the on / off status of the frame-level filter from the encoded bitstream of the current image; if it is on, go to step 2; otherwise, end the current process.

[0478] 2. Divide the chrominance components of the current image into blocks to obtain multiple 4*4 chrominance blocks.

[0479] 3. Obtain the on / off status of each CTU-level filter from the encoded bitstream in CTU order.

[0480] 4. Get the filter coefficients from the encoded bitstream,

[0481] 5. For a 4*4 chroma block in any CTU with the CTU-level filter turned on, determine the geometric transformation direction of the 4*4 chroma block based on the gradients of the 4*4 chroma block in four directions: horizontal, vertical, and diagonal.

[0482] 6. Perform geometric inverse transformation on the filter coefficients based on the geometric transformation direction of the 4*4 chrominance block.

[0483] 7. Filter the 4*4 chrominance block based on the filter coefficients after the geometric inverse transform.

[0484] It should be noted that in the embodiments of the present application, for the chrominance component, Wiener filtering processing can also be performed in the manner described in any one of Embodiments 4 to 10, and its specific implementation will not be described in detail here.

[0485] Embodiment 12

[0486] For the brightness component of the current image, Wiener filtering is performed in the manner described in any one of Embodiments 4 to 10.

[0487] For the chrominance component of the current image, Wiener filtering is performed in the manner described in the eleventh embodiment.

[0488] It should be recognized that the above-mentioned embodiments are specific examples of the technical solutions provided by the embodiments of the present application, and are not limitations on the protection scope of the present application. New embodiments obtained by simple modifications of the above-mentioned embodiments or combinations of any embodiments by those skilled in the art without paying any creative work should all fall within the protection scope of the present application.

[0489] The method provided by the present application is described above. The device provided by the present application is described below:

[0490] See also Fig.10 , Fig.10 is a structural schematic diagram of a filtering device provided in an embodiment of the present application, the device may include: a determining unit 1010 and a filtering unit 1020;

[0491] The determining unit 1010 is used to determine the optimal block division method for performing Wiener filtering on the current image when it is determined that Wiener filtering needs to be performed on the current image;

[0492] The filtering unit 1020 is used to perform Wiener filtering on the current image based on the optimal block division method.

[0493] In a possible implementation manner, the determining unit 1010 determines an optimal block division method for performing Wiener filtering on the current image, including:

[0494] An optimal block division method for performing Wiener filtering on the current image is obtained from the coded code stream of the current image.

[0495] In a possible implementation manner, the determining unit 1010 determines an optimal block division method for performing Wiener filtering on the brightness component of the current image, including:

[0496] Based on the acquired decoding information, an optimal block division method for performing Wiener filtering on the current image is determined.

[0497] In a possible implementation manner, the filtering unit 1020 performs Wiener filtering on the current image based on the optimal block division method, including:

[0498] Dividing the current image into blocks based on the optimal block division method to obtain a plurality of image blocks;

[0499] Classifying the multiple image blocks to determine classification results of the multiple image blocks;

[0500] For any image block in the CTU that needs to be Wiener filtered, determine the geometric transformation direction of the image block based on the gradients of the image block in N1 directions;

[0501] Based on the category to which the image block belongs, obtaining filter coefficients of the category from the coded bitstream of the current image;

[0502] The classified filter coefficients are geometrically inversely transformed based on the geometric transformation direction of the image block, and the image block is filtered based on the filter coefficients after the geometric inverse transformation.

[0503] In a possible implementation manner, the filtering unit 1020 classifies the multiple image blocks to determine the classification results of the multiple image blocks, including:

[0504] Classifying the multiple image blocks based on the gradients in N2 directions of each image block to determine preliminary classification results of the multiple image blocks;

[0505] Based on the preliminary classification results, determining the number of image blocks included in each classification;

[0506] When the number of image blocks in the first category with the most image blocks is greater than a first threshold, and the number of image blocks in the second category with the least image blocks is less than a second threshold, the image blocks in the first category are split into two parts, and the classification of one part of the image blocks is modified to the second classification, and the classification of the other part of the image blocks is maintained as the first classification, so as to determine a final classification result of the multiple image blocks; wherein the first threshold is greater than the second threshold.

[0507] In a possible implementation manner, the filtering unit 1020 splits the image blocks in the first classification into two parts, and changes the classification of one part of the image blocks to the second classification, and keeps the classification of the other part of the image blocks as the first classification, including:

[0508] Dividing the current image into a first sub-image and a second sub-image;

[0509] Keeping the classification of the image blocks belonging to the first classification and located in the first sub-image as the first classification;

[0510] The classification of the image block belonging to the first classification and located in the second sub-image is changed to the second classification.

[0511] In a possible implementation manner, the image block is a brightness block.

[0512] In one possible implementation,

[0513] The determining unit 1010 is further configured to, when it is determined that the chrominance component of the current image needs to be Wiener filtered, determine, for any chrominance block in the CTU for which the chrominance component needs to be Wiener filtered, a geometric transformation direction of the chrominance block based on the gradients of the chrominance block in N3 directions;

[0514] The filtering unit 1020 is also used to obtain filter coefficients of the chroma block from the encoded code stream of the current image; perform geometric inverse transformation on the obtained filter coefficients based on the geometric transformation direction of the chroma block, and filter the chroma block based on the filter coefficients after geometric inverse transformation.

[0515] See also Fig.11 , is a schematic diagram of the hardware structure of a decoding end device provided in an embodiment of the present application. The decoding end device may include a processor 1101 and a machine-readable storage medium 1102 storing machine-executable instructions. The processor 1101 and the machine-readable storage medium 1102 may communicate via a system bus 1103. In addition, by reading and executing the machine-executable instructions corresponding to the filtering control logic in the machine-readable storage medium 1102, the processor 1101 may execute the filtering method of the decoding end device described above.

[0516] The machine-readable storage medium 1102 mentioned herein may be any electronic, magnetic, optical or other physical storage device that may contain or store information, such as executable instructions, data, etc. For example, the machine-readable storage medium may be: RAM (Radom Access Memory), volatile memory, non-volatile memory, flash memory, storage drive (such as hard disk drive), solid state drive, any type of storage disk (such as optical disk, DVD, etc.), or similar storage medium, or a combination thereof.

[0517] In some embodiments, a machine-readable storage medium is also provided, in which machine-executable instructions are stored, and when the machine-executable instructions are executed by a processor, the filtering method of the decoding end device described above is implemented. For example, the machine-readable storage medium can be a ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.

[0518] See also Fig.12 , Fig.12 is a structural diagram of a filtering device provided in an embodiment of the present application, the device may include: a determining unit 1210 and a notifying unit 1220;

[0519] The determining unit 1210 is used to determine the optimal block division method for performing Wiener filtering on the current image;

[0520] The notification unit 1220 is used to notify the decoding end device of the filter coefficients of the classification to which each image block of the current image belongs under the optimal block division method through the encoding code stream of the current image when it is determined that the current image needs to be Wiener filtered.

[0521] In a possible implementation manner, the determining unit 1210 determines an optimal block division method for performing Wiener filtering on the current image, including:

[0522] For any candidate block partitioning method among the plurality of candidate block partitioning methods, determining a rate-distortion cost of performing Wiener filtering on the current image under the candidate block partitioning method;

[0523] Based on the rate-distortion cost of performing Wiener filtering on the current image under each candidate block division mode, the candidate block division mode with the smallest rate-distortion cost is determined as the optimal block division mode.

[0524] In a possible implementation manner, the determining unit 1210 determines the rate-distortion cost of performing Wiener filtering on the current image in the candidate block division mode, including:

[0525] Classifying the image blocks obtained by block division according to the candidate block division method to determine a preliminary classification result corresponding to the candidate block division method;

[0526] Based on the preliminary classification results corresponding to the candidate block division method, determine the number of image blocks included in each classification;

[0527] When the number of image blocks in the first category with the most image blocks is greater than a first threshold, and the number of image blocks in the second category with the least image blocks is less than a second threshold, the image blocks in the first category are split into two parts, and the classification of one part of the image blocks is modified to the second classification, and the classification of the other part of the image blocks is maintained as the first classification, so as to determine the final classification result corresponding to the candidate block division method; wherein the first threshold is greater than the second threshold;

[0528] Based on a final classification result corresponding to the candidate block division method, a rate-distortion cost of performing Wiener filtering on the current image under the candidate block division method is determined.

[0529] In a possible implementation manner, the determining unit 1210 divides the image blocks in the first classification into two parts, and changes the classification of one part of the image blocks to the second classification, and keeps the classification of the other part of the image blocks as the first classification, including:

[0530] Dividing the current image into a first sub-image and a second sub-image;

[0531] Keeping the classification of the image blocks belonging to the first classification and located in the first sub-image as the first classification;

[0532] The classification of the image block belonging to the first classification and located in the second sub-image is changed to the second classification.

[0533] In a possible implementation manner, the filter coefficients of the categories to which each image block of the current image belongs under the optimal block division method are determined in the following manner:

[0534] Determine the filter coefficients of the categories to which each image block of the current image belongs under the optimal block division method;

[0535] For any filter coefficient of the classification, based on the rate-distortion cost of Wiener filtering of the current image in the optimal block partitioning mode, adaptively adjusting the filter coefficient;

[0536] The filter coefficients after the adaptive adjustment are determined as the filter coefficients of the classification.

[0537] In a possible implementation manner, the determining unit 1210 adaptively adjusts the filter coefficient based on the rate-distortion cost of performing Wiener filtering on the current image in the optimal block division mode, including:

[0538] According to the preset order, try to delete the coefficients at the specified position in the filter coefficients in turn;

[0539] For any attempted deletion operation, if the rate-distortion cost of performing Wiener filtering on the current image before deleting the coefficients is greater than or equal to the rate-distortion cost of performing Wiener filtering on the current image after deleting the coefficients, then it is determined that the deletion operation is effective, and attempts are made to delete the coefficients at the specified positions in the filter coefficients that have not been deleted in sequence in accordance with the preset order; otherwise, it is determined that the deletion operation is ineffective, and attempts are made to delete the coefficients at the specified positions in the filter coefficients that have not been deleted in sequence in accordance with the preset order.

[0540] In a possible implementation manner, the image block is a brightness block.

[0541] In a possible implementation, the determination unit 1210 is further used to determine, for any chroma block of a preset size in the current image, a geometric transformation direction of the chroma block based on the gradients of the chroma block in N3 directions; determine a filter coefficient of the chroma block based on the geometric transformation direction of the chroma block; and determine on / off of a CTU-level filter and on / off of a frame-level filter of the current image based on the filter coefficients of each chroma block in the current image.

[0542] See also Fig.13 , is a schematic diagram of the hardware structure of an encoding end device provided in an embodiment of the present application. The encoding end device may include a processor 1301 and a machine-readable storage medium 1302 storing machine-executable instructions. The processor 1301 and the machine-readable storage medium 1302 may communicate via a system bus 1303. In addition, by reading and executing the machine-executable instructions corresponding to the filtering control logic in the machine-readable storage medium 1302, the processor 1301 may execute the filtering method of the encoding end device described above.

[0543] The machine-readable storage medium 1302 mentioned herein may be any electronic, magnetic, optical or other physical storage device that may contain or store information, such as executable instructions, data, etc. For example, the machine-readable storage medium may be: RAM (Radom Access Memory), volatile memory, non-volatile memory, flash memory, storage drive (such as hard disk drive), solid state drive, any type of storage disk (such as optical disk, DVD, etc.), or similar storage medium, or a combination thereof.

[0544] In some embodiments, a machine-readable storage medium is further provided, wherein the machine-readable storage medium stores machine-executable instructions, and when the machine-executable instructions are executed by a processor, the filtering method of the encoding end device described above is implemented. For example, the machine-readable storage medium may be a ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.

[0545] In some embodiments, a camera device is also provided, comprising the filtering device in any of the above embodiments.

[0546] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0547] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A filtering method, applied to a decoding end device, characterized in that: The method comprises: When it is determined that the current image needs to be subjected to Wiener filtering, determining an optimal block division method for performing Wiener filtering on the current image; Performing Wiener filtering on the current image based on the optimal block division method; Wherein, determining whether to perform Wiener filtering on the current image includes: Determine whether Wiener filtering is required for the current image based on the on / off of the frame-level filter carried in the bitstream; when the frame-level filter is on, determine that Wiener filtering is required for the current image; Wherein, the luminance component and the chrominance component are marked as needing Wiener filtering by turning on / off different frame-level filters; or, The luminance component and the chrominance component are marked as needing Wiener filtering by turning on / off the filter at the same frame level; The step of determining the optimal block division method for performing Wiener filtering on the current image includes: Obtaining an optimal block division method for performing Wiener filtering on the current image from the coded bitstream of the current image; or, Based on the acquired decoding information, an optimal block division method for performing Wiener filtering on the current image is determined.

2. The method according to claim 1, characterized in that In the optimal block division method, each image block of the current image is a luminance block.

3. A filtering method, applied to an encoding end device, characterized in that: The method comprises: Determine the optimal block partitioning method for Wiener filtering of the current image; When it is determined that the current image needs to be subjected to Wiener filtering, the filter coefficients of the classification to which each image block of the current image belongs under the optimal block division method are notified to a decoding end device through the encoding code stream of the current image; Wherein, the method further comprises: According to whether the current image needs Wiener filtering, determining whether the encoding bitstream of the current image carries a frame-level filter on / off; when the frame-level filter is on, determining that Wiener filtering is required for the current image; Wherein, the luminance component and the chrominance component are marked as needing Wiener filtering by turning on / off different frame-level filters; or, The luminance component and the chrominance component are marked as needing Wiener filtering by turning on / off the filter at the same frame level; The step of determining the optimal block division method for performing Wiener filtering on the current image includes: For any candidate block partitioning method among the plurality of candidate block partitioning methods, determining a rate-distortion cost of performing Wiener filtering on the current image under the candidate block partitioning method; Based on the rate-distortion cost of performing Wiener filtering on the current image under each candidate block division mode, the candidate block division mode with the smallest rate-distortion cost is determined as the optimal block division mode.

4. The method according to claim 3, characterized in that The image block is a brightness block.

5. A filtering device, applied to a decoding end device, characterized in that: The device comprises: a determination unit and a filtering unit; The determining unit is used to determine the optimal block division method for performing Wiener filtering on the current image when it is determined that Wiener filtering needs to be performed on the current image; The filtering unit is used to perform Wiener filtering on the current image based on the optimal block division method; The determining unit is further used to determine whether Wiener filtering is required for the current image based on the on / off of the frame-level filter carried in the bitstream; when the frame-level filter is on, it is determined that Wiener filtering is required for the current image; Wherein, the luminance component and the chrominance component are marked as needing Wiener filtering by turning on / off different frame-level filters; or, The luminance component and the chrominance component are marked as needing Wiener filtering by turning on / off the filter at the same frame level; The determining unit determines the optimal block division method for performing Wiener filtering on the current image, including: Obtaining an optimal block division method for performing Wiener filtering on the current image from the coded bitstream of the current image; or, Based on the acquired decoding information, an optimal block division method for performing Wiener filtering on the current image is determined.

6. A filtering device, applied to an encoding end device, characterized in that: The device comprises: a determination unit and a notification unit; The determining unit is used to determine the optimal block division method for performing Wiener filtering on the current image; The notification unit is used to notify the decoding end device of the filter coefficients of the classification to which each image block of the current image belongs under the optimal block division mode through the encoding code stream of the current image when it is determined that the current image needs to be subjected to Wiener filtering; The determining unit is further configured to determine whether the frame-level filter is on or off in the coded bitstream of the current image according to whether the current image needs Wiener filtering; when the frame-level filter is on, determining that Wiener filtering is required for the current image; Wherein, the luminance component and the chrominance component are marked as needing Wiener filtering by turning on / off different frame-level filters; or, The luminance component and the chrominance component are marked as needing Wiener filtering by turning on / off the filter at the same frame level; The determining unit determines an optimal block division method for performing Wiener filtering on the current image, including: For any candidate block partitioning method among the plurality of candidate block partitioning methods, determining a rate-distortion cost of performing Wiener filtering on the current image under the candidate block partitioning method; Based on the rate-distortion cost of performing Wiener filtering on the current image under each candidate block division mode, the candidate block division mode with the smallest rate-distortion cost is determined as the optimal block division mode.

7. A decoding end device, characterized in that: The method comprises a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor, and the processor is used to execute the machine-executable instructions to implement the method according to claim 1 or 2.

8. A coding end device, characterized in that: The method comprises a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor, and the processor is used to execute the machine-executable instructions to implement the method according to claim 3 or 4.

Images