A method and apparatus for post-processing of video decoding

Abstract

The application discloses a video decoding post-processing method and device, and belongs to the technical field of video decoding post-processing. The method comprises the following steps: analyzing the macroblock layer coding parameters of a current decoding frame, and extracting a quantization step and a transform coefficient scanning mode; calculating a frequency band compensation value for each pixel according to the quantization step and the scanning mode, and determining the compensation value based on a transform coefficient truncation error; superimposing the compensation value to generate a first modified frame; extracting the local texture direction of each pixel in the first modified frame, and determining the local texture direction by comparing the gray difference; performing directional low-pass filtering according to the local texture direction, and changing the filtering strength along with the included angle between the texture direction and a preset edge direction; and weighting and fusing the filtered frame and the current decoding frame to generate a final output frame, wherein the weight is dynamically adjusted according to the pixel value difference. The scheme can accurately compensate the quantization truncation error and adaptively preserve the edge filtering, and the decoding image quality is improved.

Description

Technical Field

[0001] This invention relates to the field of video decoding post-processing technology, specifically to a video decoding post-processing method and apparatus. Background Technology

[0002] Video compression coding reduces data volume through transform, quantization, and coefficient truncation. These lossy operations cause distortions in the decoded image, such as blockiness, ringing, and loss of texture details. To improve the subjective effect, post-processing techniques such as deblocking filtering and adaptive sample compensation are commonly used at the decoding end. Adaptive sample compensation divides pixels into multiple band or edge categories and applies a fixed offset value to each category. This type of method does not utilize the differences in quantization step size during encoding, nor does it incorporate the frequency distribution information implicit in the transform coefficient scanning mode, resulting in the compensation amount not matching the actual frequency truncation error distribution. In areas with coarse quantization, the fixed offset is insufficient to recover high-frequency details, and the texture remains blurry; in areas with fine quantization, the fixed offset may overcompensate, introducing false contours or stripe artifacts in flat areas. In terms of directional filtering denoising, existing adaptive filters mostly rely on gradient magnitude or single-direction determination to adjust the intensity, without considering the angle relationship between the local texture direction and the preset edge direction. When the texture direction deviates significantly from the filter's assumed edge direction, the filtering intensity is inappropriate, causing oblique textures or weak edges to be unnecessarily smoothed. Some solutions directly output the filtering results, discarding highly reliable structural information from the original decoded frame, causing detail degradation or residual noise in strong edges or high-noise areas. Post-decoding processing urgently needs to address the problem of accurately recovering truncation errors by combining quantization step size and transform coefficient scanning mode in frequency band compensation, and the problem of dynamically controlling the filtering intensity based on the angle between texture direction and edge direction when performing directional smoothing of the image and fusing it with the original decoded frame to avoid loss of detail. Summary of the Invention

[0003] This invention provides a video decoding post-processing method and apparatus, aiming to solve the problems of inaccurate error recovery caused by the lack of frequency band compensation by combining quantization step size and scanning mode in existing post-processing, as well as the loss of detail and edge blurring caused by the inability of directional filtering intensity to adaptively adjust with the angle between local texture and edge direction and the failure to fuse with the original decoded frame, thereby improving the quality of decoded images.

[0004] To achieve the above objectives, the present invention provides the following technical solution: The present invention provides a video decoding post-processing method and apparatus for post-processing decoded video frames to improve image quality. The method includes: parsing the macroblock layer coding parameters of the current decoded frame, extracting the quantization step size and transform coefficient scanning mode of each macroblock; calculating a frequency band compensation value for each pixel position of the current decoded frame according to the quantization step size and transform coefficient scanning mode, wherein the frequency band compensation value is determined based on the transform coefficient truncation error associated with the current pixel position; superimposing the frequency band compensation value onto the corresponding pixel value of the current decoded frame to generate a first corrected frame; extracting the local texture direction of each pixel in the first corrected frame, wherein the local texture direction is determined by comparing the grayscale difference of pixels in a preset angle direction; performing directional low-pass filtering on the first corrected frame according to the local texture direction, wherein the filtering intensity of the directional low-pass filtering varies with the angle between the local texture direction and the preset edge direction; and performing weighted fusion of the filtered first corrected frame and the current decoded frame, wherein the weight coefficient of the weighted fusion is dynamically adjusted according to the pixel value difference between the filtered first corrected frame and the current decoded frame to generate a final output frame.

[0005] The above scheme reconstructs the frequency error information lost due to transform coefficient truncation by obtaining the quantization step size and scanning mode from the decoding parameters, and uses this error as a frequency band compensation value to compensate for the decoded frame, effectively restoring the high-frequency details lost during quantization. The first corrected frame obtained after superimposing the compensation values ​​has a preliminary enhanced texture structure, but may be accompanied by local artifacts or grainy noise. By extracting the local texture direction of each pixel in the first corrected frame and performing directional low-pass filtering accordingly, the filtering operation can be performed along the texture edge direction, smoothing noise while protecting edge sharpness. The filtering intensity is dynamically adjusted according to the angle between the local texture direction and the preset edge direction, making the filtering lighter for major edge areas and stronger for flat areas, thus balancing the need to preserve details and remove artifacts. Dynamically weighted fusion of the filtered corrected frame and the original decoded frame based on pixel value differences can achieve a smooth transition between the compensated area and the original decoded frame, avoiding visual unnaturalness caused by over- or under-compensation. The final output frame is improved in both detail and overall smoothness.

[0006] In a preferred implementation, when calculating the band compensation value, the current decoded frame is divided into multiple band compensation calculation units, each corresponding to a pixel region of the smallest transform block size. For each band compensation calculation unit, all pixel positions within the pixel region are traversed, and a compensation mapping table is established for each pixel position. This compensation mapping table stores the correspondence between quantization step size indices and compensation coefficients. Then, the corresponding compensation coefficient is read according to the quantization step size index of the macroblock to which the current pixel position belongs, and this coefficient is multiplied by the truncation flag of the transform coefficient to which the current pixel position belongs. The product result is used as the band compensation value. The truncation flag is set according to the comparison result of the absolute value of the transform coefficient and the quantization step size threshold. By establishing a compensation mapping table pixel by pixel and combining it with the truncation flag, the compensation value can accurately reflect the truncation of each coefficient during the quantization process, which can significantly improve the texture distortion of the decoded image under low bit rate encoding.

[0007] As another technical solution of the present invention, when establishing the compensation mapping table, the scanning order of the transform coefficients corresponding to all pixel positions within the current frequency band compensation calculation unit is obtained, and the frequency index of each pixel position in the transform domain is determined according to the scanning order. The product of the frequency index and the quantization step size is used as the lookup key, and the corresponding compensation coefficient is retrieved in the preset compensation database. Then, the compensation coefficient is associated with and stored with the frequency index. This method utilizes the correlation between the frequency index and the quantization parameter for table lookup, reducing real-time computation overhead. At the same time, the value of the compensation coefficient can adaptively change with the frequency position, which conforms to the characteristic of the human eye's sensitivity to different frequency components.

[0008] Preferably, when extracting local texture directions, a local window of a preset size is constructed centered on the current pixel. The sum of gray-level gradients of all pixels within the window in the horizontal and vertical directions is calculated. The initial texture direction angle is calculated based on the ratio of the sums of gradients in these two directions. Then, the initial texture direction angle is quantized and mapped to a preset set of discrete angles. The mapping result serves as the local texture direction. Angle quantization simplifies the computational complexity of subsequent filter selection and rotation, enhances the robustness of direction estimation, and avoids frequent changes in direction parameters due to minor noise.

[0009] When performing directional low-pass filtering, a corresponding directional filter is selected from a preset filter set based on the local texture direction of the current pixel. This set contains multiple filter kernels with different filtering axes. The filtering axis of the selected directional filter is rotated to align with the local texture direction, generating a rotated filter kernel. This kernel is then used to perform a weighted average of the neighboring pixel values ​​of the current pixel. By rotating the filter kernel to align with the texture direction, filtering is performed along the texture direction, preventing blurring caused by crossing edges when smoothing quantization noise in non-edge regions.

[0010] More preferably, the filtering intensity of the directional low-pass filter is determined by the angle between the local texture direction and the preset edge direction. Multiple angle values ​​are calculated by subtracting each direction from the set of preset edge directions, and the minimum value is selected as the final angle. An intensity mapping table, which stores the mapping relationship between angle intervals and filter intensity coefficients, is consulted based on the final angle. Each element in the rotated filter kernel is multiplied by the read filter intensity coefficient to obtain the adjusted filter kernel. Thus, when the local texture direction is highly consistent with a preset edge direction, the pixel is determined to be on a distinct edge and given a lower filtering intensity to maintain edge sharpness; otherwise, it is considered a non-edge region, and the filtering is enhanced to fully remove residual noise introduced during the compensation process.

[0011] In a preferred embodiment of the present invention, during the weighted fusion process, the absolute value of the difference between the pixel values ​​at the same pixel position in the filtered first corrected frame and the current decoded frame is calculated and compared with a preset difference threshold. If the absolute value of the difference is less than the threshold, a first fusion weight is assigned to the current pixel; if it is greater than or equal to the threshold, a second fusion weight is assigned. The assigned weights are used to perform a weighted summation of the pixel values ​​from the two sources to obtain the final output pixel value. This adaptive weighting strategy can identify areas with large compensation and filtering modifications and appropriately reduce the weight of the corrected frame to avoid pseudo-contours or ringing effects caused by over-correction.

[0012] As a further improvement of the present invention, after generating the first corrected frame, the motion vector field of the current decoded frame is extracted. The motion vector field contains the motion vector of each macroblock. Motion compensation correction is performed on the macroblocks in the first corrected frame based on the confidence value of each motion vector in the motion vector field. The confidence value is determined based on the median difference between the motion vector and the neighboring motion vectors. The motion compensation correction replaces the pixel value of the current macroblock with a combination of the corresponding pixel value in the reference frame and the frequency band compensation value. This process can utilize temporal information to further correct compensation deviations caused by inaccurate motion estimation or scene occlusion.

[0013] The motion compensation correction described above specifically includes: calculating the difference between the median of the motion vector of the current macroblock and the motion vectors of surrounding macroblocks; normalizing the reciprocal of this difference and using it as the confidence value; if the confidence value is lower than a preset confidence threshold, the current macroblock is marked as a low-confidence macroblock. For each low-confidence macroblock, a matching block matching the texture features of the current macroblock is searched from the reference frame. The texture features are obtained through local binary mode descriptor quantization. The pixel values ​​of the matching block are used as the base values, and a frequency band compensation value is superimposed to obtain the corrected macroblock pixel values. This scheme utilizes texture matching to find more reliable reference information in the temporal domain when motion vectors are unreliable, and combines it with frequency band compensation values ​​to recover details, thereby avoiding inter-frame flicker or block effects caused by motion mismatch.

[0014] The present invention also provides a video decoding post-processing apparatus for implementing the above method. The apparatus includes: a parameter parsing module for parsing the macroblock layer coding parameters of the current decoded frame and extracting the quantization step size and transform coefficient scanning mode of each macroblock; a frequency band compensation calculation module for calculating a frequency band compensation value for each pixel position of the current decoded frame based on the quantization step size and transform coefficient scanning mode, wherein the frequency band compensation value is determined based on the transform coefficient truncation error associated with the current pixel position; a first corrected frame generation module for superimposing the frequency band compensation value onto the corresponding pixel value of the current decoded frame to generate a first corrected frame; and a texture direction extraction module for extracting the texture direction from the first corrected frame. The device employs a local texture direction for each pixel, determined by comparing grayscale differences between pixels at a preset angle. A directional filtering module performs directional low-pass filtering on the first corrected frame based on the local texture direction, with the filtering intensity varying with the angle between the local texture direction and a preset edge direction. A weighted fusion module performs weighted fusion of the filtered first corrected frame and the current decoded frame, with the weighting coefficients dynamically adjusted based on the pixel value differences between the filtered first corrected frame and the current decoded frame to generate the final output frame. The modules of this device work collaboratively, enabling compensation and adaptive filtering with low computational complexity at the decoding end, making it suitable for real-time video decoding post-processing applications.

[0015] The technical effects and advantages provided by the present invention in the above technical solution are as follows: The bandwidth compensation value based on the truncation error of the transform coefficients is calculated for each pixel position according to the quantization step size and the transform coefficient scanning mode. The decoded frame is divided into bandwidth compensation calculation units corresponding to the smallest transform block size. All pixels within the unit are traversed, and the frequency index in the transform domain is determined for each pixel according to the scanning order. The product of the frequency index and the quantization step size is used as the lookup key to retrieve the corresponding compensation coefficient from the preset compensation database and establish a compensation mapping table. The compensation coefficient is read according to the quantization step size index of the macroblock in which the pixel is located, and then multiplied by the truncation flag of the transform coefficient to obtain the bandwidth compensation value. The truncation flag is set by the comparison result of the absolute value of the transform coefficient and the quantization step size threshold. This process is directly related to the truncation operation at the encoding end, and the compensation amount adapts to the quantization coarsness and frequency position. Larger compensation is applied to the high-frequency texture coefficients that are severely truncated by coarse quantization to recover the lost detail signals; corresponding small compensation is applied to the low-frequency components that are finely quantized to avoid overshoot. The compensation value is linked to the current pixel truncation state, avoiding overcompensation artifacts in flat areas or residual blurring caused by insufficient compensation in textured areas, which is a problem with traditional fixed classification offset compensation. This reduces block artifacts in the reconstructed frame and significantly restores edge sharpness and texture fidelity. The local texture direction of the first corrected frame is extracted and directional low-pass filtered. The filter intensity varies with the angle between the texture direction and a preset edge direction. The filtering result is then dynamically weighted and fused with the current decoded frame based on pixel value differences. When extracting the texture direction, a local window is constructed centered on the current pixel. The ratio of the sum of the horizontal and vertical grayscale gradients within the window is calculated to obtain the initial direction angle, which is then quantized to a preset discrete angle set to obtain the local texture direction. During filtering, a corresponding directional filter is selected based on the texture direction. The filter's principal axis is rotated to align with the texture direction, and the angle between the texture direction and each preset edge direction is calculated. The smallest angle is selected as the final direction angle. The intensity mapping table is consulted based on the final direction angle to obtain the filter intensity coefficient, and the rotated filter kernel is scaled. Pixels aligned along the edge direction with small angles are filtered with high intensity to smooth noise along the artifact direction while maintaining steep edges. Pixels with large angles, such as flat areas with inconsistent texture directions or cross-texture areas, have reduced filtering intensity to protect fine structures. After directional filtering, the absolute value of the difference between pixel values ​​at the same position in the filtered frame and the original decoded frame is calculated. The difference is compared with a preset threshold. If the difference is less than the threshold, the first fusion weight is assigned to highlight the noise reduction effect of the filtered frame; if the difference is greater than or equal to the threshold, the second fusion weight is assigned to increase the proportion of the original decoded information and prevent edge damage. The adaptive intensity mechanism based on texture and edge angle, combined with the difference-driven fusion strategy, effectively eliminates and compresses high-frequency noise and mosquito noise while fully preserving the sharpness of object boundaries and linear textures. This avoids edge blurring and texture smearing caused by conventional directional filtering due to fixed intensity or failure to fuse with the original frame. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0017] Figure 1 This is a flowchart of the video decoding post-processing method; Figure 2 This is a flowchart of the frequency band compensation value calculation based on the quantization step size and the transform coefficient scanning mode; Figure 3 It is a local texture direction statistical histogram; Figure 4 This is a schematic diagram of the confidence distribution of macroblock motion vectors; Figure 5 This is a time distribution diagram of each module in the video decoding post-processing. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] See Figure 1 This invention provides a video decoding post-processing method, comprising: parsing the macroblock layer coding parameters of the current decoding frame, extracting the quantization step size and transform coefficient scanning mode of each macroblock; calculating a frequency band compensation value for each pixel position of the current decoding frame according to the quantization step size and transform coefficient scanning mode, wherein the frequency band compensation value is determined based on the transform coefficient truncation error associated with the current pixel position; superimposing the frequency band compensation value onto the corresponding pixel value of the current decoding frame to generate a first corrected frame; extracting the local texture direction of each pixel in the first corrected frame, wherein the local texture direction is determined by comparing the grayscale difference of pixels in a preset angle direction; performing directional low-pass filtering on the first corrected frame according to the local texture direction, wherein the filtering intensity of the directional low-pass filtering varies with the angle between the local texture direction and the preset edge direction; and performing weighted fusion of the filtered first corrected frame and the current decoding frame, wherein the weight coefficient of the weighted fusion is dynamically adjusted according to the pixel value difference between the filtered first corrected frame and the current decoding frame to generate a final output frame.

[0020] Example 1: In specific implementation, please refer to Figure 2 The process calculates the frequency band compensation value for each pixel position in the current decoded frame based on the quantization step size and transform coefficient scanning mode. This includes dividing the current decoded frame into multiple frequency band compensation calculation units, each corresponding to a pixel region with a minimum transform block size. The minimum transform block size is determined by the video coding standard and is set to 4×4 pixels. For each frequency band compensation calculation unit, the process iterates through all pixel positions within the pixel region covered by the unit, establishing a compensation mapping table for each pixel position. This compensation mapping table stores the correspondence between the quantization step size index and the compensation coefficient.

[0021] The process of establishing a compensation mapping table for pixel positions is implemented as follows: First, the scan order of transform coefficients corresponding to all pixel positions within the current frequency band compensation calculation unit is obtained. This scan order is determined by the transform coefficient scan mode in the macroblock layer coding parameters of the current decoded frame. Second, based on the transform coefficient scan order, the frequency index of each pixel position in the transform domain is determined. The frequency index represents the sequence number of the corresponding transform coefficient in the scan order. Third, the frequency index is multiplied by the quantization step size, and the product is used as the lookup key. Fourth, the lookup key is used to retrieve the corresponding compensation coefficient in the preset compensation database. The preset compensation database contains multiple preset keys and compensation coefficients associated with each preset key. The correspondence between preset keys and compensation coefficients is established offline based on the variation characteristics of transform coefficient truncation error with quantization step size and frequency. The retrieval process is as follows: The similarity between the lookup key and each preset key in the preset compensation database is calculated. The preset key with the highest similarity is selected, and the compensation coefficient associated with that preset key is extracted. The extracted compensation coefficient is stored in association with the frequency index, and a mapping relationship from the quantization step size index to the compensation coefficient is established to generate the compensation mapping table. Repeat the above retrieval and storage operations for all possible quantization step size indices to complete the construction of the compensation mapping table for a pixel position.

[0022] After traversing all pixel positions within the frequency band compensation calculation unit, a compensation mapping table for each pixel position is obtained. When calculating the frequency band compensation value, the corresponding compensation coefficient is read from the compensation mapping table for the current pixel position based on the quantization step size index of the macroblock to which the current pixel position belongs. Simultaneously, the truncation flag of the transform coefficient to which the current pixel position belongs is obtained. The truncation flag is set based on the comparison result between the absolute value of the transform coefficient and the quantization step size threshold. The quantization step size threshold is set to be equal to the quantization step size, and the value of the truncation flag is defined by the following formula:

[0023] in, Indicates pixel position The corresponding truncation flag, Indicates pixel position The corresponding transformation coefficients, Indicates pixel position The quantization step size of the macroblock. The above setting is based on the fact that the quantization step size directly determines the dead zone boundary during transform coefficient quantization. When the absolute value of the transform coefficient exceeds the quantization step size, it means that the coefficient will produce a truncation error after dequantization, requiring compensation. The compensation coefficient is multiplied by the truncation flag, and the product is used as the pixel position. The frequency band compensation value is generated by processing all pixel positions of the current decoded frame in the manner described above.

[0024] Example 2: In a specific implementation, the local texture direction of each pixel in the first corrected frame is extracted, including constructing a local window of a preset size centered on the current pixel. The preset size is set to a 5×5 pixel area, based on the premise that a 5×5 area can cover sufficient texture change information while avoiding crossing texture boundaries that would cause ambiguity in the direction estimation. The sum of gray-level gradients in the horizontal direction is calculated within the local window. The sum of gray-level gradients in the vertical direction The sum of gray-level gradients in the horizontal direction. The method of obtaining it is: to level Operator Convolve with the pixel values ​​within the local window. The definition of The absolute values ​​of the convolution results are then summed to obtain... The sum of gray-level gradients in the vertical direction. The method of obtaining it is: vertical Operator Convolve with the pixel values ​​within the local window. The definition of The absolute values ​​of the convolution results are then summed to obtain... .

[0025] Based on the accumulation of gray-level gradients in the horizontal direction The sum of gray-level gradients in the vertical direction The ratio of the two values ​​is used to calculate the initial texture orientation angle. The calculation formula is as follows:

[0026] in, This represents the initial texture orientation angle, with a value range of [value missing]. , Represents the arctangent function. This represents the sum of gray-level gradients in the vertical direction. This represents the sum of gray-level gradients in the horizontal direction. When... zero and When greater than zero, Pick .when zero and When equal to zero, Pick .

[0027] For the initial texture orientation angle Angle quantization is performed, mapping the initial texture orientation angle to a preset set of discrete angles. The mapping result serves as the local texture orientation. The preset set of discrete angles is set as follows: There are a total of 8 discrete directions. The basis for this 8-direction division is that it can fully describe the direction of the natural texture while reducing the computational complexity of subsequent directional filtering. Angle quantization processing is performed by initial texture direction angles... Calculate the absolute value of the difference between the discrete angle and each angle value in the discrete angle set, and select the discrete angle with the smallest absolute value of the difference as the local texture direction.

[0028] Directional low-pass filtering is performed on the first corrected frame based on the local texture direction. This includes selecting a corresponding directional filter from a preset filter set based on the local texture direction of the current pixel. The preset filter set contains a basic directional filter kernel, which is a 5×5 Gaussian kernel with its filtering principal axis along the horizontal direction. The kernel coefficients are obtained by discrete sampling of a two-dimensional Gaussian function with a principal axis scale of 1.0 and an orthogonal direction scale of 0.3. The filtering principal axis of the basic directional filter kernel is rotated to align with the local texture direction. The rotation operation is implemented using a coordinate rotation transformation matrix, and the rotation angle is taken as the angle value corresponding to the local texture direction. After rotation, a rotated filter kernel is generated, maintaining a size of 5×5. The rotated filter kernel is used to perform a weighted average of the neighboring pixel values ​​of the current pixel in the first corrected frame. The weighted average method is to multiply the coefficients of the rotated filter kernel by the pixel values ​​at each position within the 5×5 neighborhood of the current pixel and then sum the results. The sum is used as the filtered pixel value of the current pixel. After completing the filtering process for one pixel, move to the next pixel and repeat the entire process from extracting the local texture direction to weighted averaging until all pixels in the first corrected frame have obtained the filtered pixel value.

[0029] The filtering intensity of a directional low-pass filter varies with the angle between the local texture direction and a preset edge direction. This is achieved by calculating the difference between the local texture direction of the current pixel and each preset edge direction in the preset edge direction set, resulting in multiple angle values. The preset edge direction set is set as follows: The settings are based on the most common target edge directions in the video frames corresponding to these four directions. The difference is calculated as the absolute value of the difference between the local texture direction angle value and the preset edge direction angle value. If the absolute value of the difference exceeds... Then use The absolute value of the difference is subtracted to obtain the direction angle value, and the range of the direction angle value is limited to [value missing]. The minimum value among the four angle values ​​calculated for four preset edge directions is selected as the final angle. The final angle is then used to look up the intensity mapping table, which stores the mapping relationship between angle intervals and filter intensity coefficients. The intensity mapping table contains angle intervals... The corresponding filter strength coefficient is 0.2, within the included angle range. The corresponding filter strength coefficient is 0.5, within the included angle range. The corresponding filter strength coefficient is 0.8, within the included angle range. The corresponding filter strength coefficient is 1.0. The mapping relationship is set based on the following: the smaller the final direction angle, the closer the local texture direction is to the edge direction, and the filter strength should be reduced to protect the edge structure from being smoothed; the larger the final direction angle, the farther the local texture direction deviates from the edge direction, and the region is more likely to have flat or messy textures, so a stronger filter can be applied to suppress noise. Each element in the rotated filter kernel is multiplied by the filter strength coefficient read from the strength mapping table to obtain the adjusted filter kernel. The adjusted filter kernel is used to replace the original rotated filter kernel to perform a weighted average, and the resulting filtered pixel value is the result after considering the change in filter strength with the angle.

[0030] See Figure 3 The bar chart in the figure shows the distribution of pixel counts corresponding to different local texture directions in the first corrected frame. The horizontal axis represents the angle value of the local texture direction, including eight discrete directions: 0°, 22.5°, 45°, 67.5°, 90°, 112.5°, 135°, and 157.5°, corresponding to the discrete angle set to which the initial texture direction angle quantization is mapped by the texture direction extraction module in Embodiment 2. The vertical axis represents the number of pixels corresponding to each direction, in pixels.

[0031] As shown in the image, the highest number of pixels (approximately 118) are located at 0°, indicating that horizontal texture is relatively concentrated in the first corrected frame. The number of pixels decreases as the texture angle increases, reaching a minimum of approximately 33 pixels at 90°, indicating relatively less texture in the vertical direction. Beyond 90°, the number of pixels gradually increases, reaching nearly 96 pixels at 157.5°, indicating a richer texture distribution in this direction.

[0032] Example 3: In practice, the filtered first corrected frame and the current decoded frame are weighted and fused, including calculating the absolute value of the pixel value difference between the filtered first corrected frame and the current decoded frame at the same pixel position. For position coordinates... The pixel values ​​of the first corrected frame after filtering are denoted as . The pixel value of the current decoded frame is denoted as absolute value of pixel value difference The calculation method is as follows:

[0033] in, This indicates the operation of taking the absolute value.

[0034] absolute value of pixel value difference Difference threshold with preset value Compare the values. Preset difference threshold. The value is dynamically set based on the quantization step size in the video encoding. The value is the product of the quantization step size of the macroblock to which the current pixel belongs and a fixed coefficient. The fixed coefficient is set to 2. This is because the quantization step size directly reflects the quantization distortion level of the decoded frame. An absolute difference exceeding twice the quantization step size means that the pixel value change before and after filtering significantly exceeds the distortion range introduced by quantization, and this location may belong to an area where details need to be preserved. If the absolute value of the pixel value difference... Less than the preset difference threshold , for position Pixel allocation first fusion weight If the absolute value of the pixel value difference Greater than or equal to the preset difference threshold , for position Pixel allocation of second fusion weights First fusion weight Set to 0.85, second fusion weight The value is set to 0.35. The rationale for this setting is as follows: when the difference between pixel values ​​before and after filtering is small, the reliability of the first corrected frame after filtering is high, and it should be given a larger weight to achieve the denoising effect; when the difference between pixel values ​​before and after filtering is large, the filtering process may destroy the edge or texture information of the original image, and the weight of the filtering result should be reduced to preserve the original information in the current decoded frame.

[0035] Using the first fusion weight Or the second fusion weight Pixel values ​​of the first corrected frame after filtering and the pixel values ​​of the current decoded frame Perform a weighted summation to obtain the merged pixel values. The weighted summation formula is: Among them, weight The first fusion weight is determined based on the comparison results. Or the second fusion weight After traversing and processing in the above manner, the pixel values ​​of all pixel positions in the final output frame are generated.

[0036] Example 4: In practice, after generating the first corrected frame, the motion vector field of the current decoded frame is extracted. This motion vector field contains the motion vectors of each macroblock. Based on the confidence value of each motion vector in the motion vector field, motion compensation correction is performed on the macroblocks in the first corrected frame. The confidence value is determined based on the median difference between the motion vectors and their neighboring motion vectors. Motion compensation correction is achieved by replacing the pixel values ​​of the current macroblock with a combination of the corresponding pixel values ​​from the reference frame and a frequency band compensation value.

[0037] The process of motion compensation correction for macroblocks in the first correction frame based on the confidence value of each motion vector in the motion vector field is as follows: Calculate the difference between the median of the motion vector of the current macroblock and the motion vectors of surrounding macroblocks, and normalize the reciprocal of this difference to obtain the confidence value. Surrounding macroblocks are defined as the macroblocks spatially adjacent to the current macroblock: the macroblock above, below, to the left, and to the right. Extract the motion vector of the current macroblock, denoted as... Extract the motion vectors from the four surrounding macroblocks to form a motion vector set. Calculate the median vector of the motion vector set. The median vector is obtained as follows: The horizontal components of the four motion vectors in the motion vector set are sorted by value, and the median value is taken as the horizontal component of the median vector; the vertical components of the four motion vectors are sorted by value, and the median value is taken as the vertical component of the median vector. These components are then combined to obtain the median vector. The magnitude of the difference between the motion vector and the median vector of the current macroblock. The calculation method is as follows:

[0038] in, This represents the horizontal component of the current macroblock motion vector. This represents the vertical component of the current macroblock motion vector. Represents the horizontal component of the median vector. This represents the vertical component of the median vector. The confidence score is obtained by normalizing the reciprocal of the difference of medians. Determined by the following formula:

[0039] in, This represents the confidence value of the current macroblock, with a value range of [value missing]. The magnitude of the difference between the motion vector of the current macroblock and the median motion vector of the surrounding macroblocks; It is a positive integer less than 1, used to avoid division by zero errors during calculations. Values The setting is based on typical values ​​that are much smaller than the difference in motion vectors, so as to ensure numerical stability without affecting the relative discriminative power of the confidence value; Indicates the first [number]th [frame] in the current decoded frame. The magnitude of the difference between the motion vector of a macroblock and the median of the motion vectors of its surrounding macroblocks; The denominator represents the total number of macroblocks in the current decoded frame; the denominator is the sum of the values ​​of all macroblocks in the current decoded frame. Summation. This normalization method makes the sum of the confidence values ​​of all macroblocks in the current frame equal to 1.

[0040] In practice, if the confidence value is lower than a preset confidence threshold, the current macroblock is marked as a low-confidence macroblock. The preset confidence threshold is set to... The setting is based on the fact that the normalized confidence value reflects the relative consistency between the macroblock motion vector and the neighborhood motion vector, and is lower than the set value. The macroblocks exhibit significant deviations from the neighborhood motion trend in the motion vector field, resulting in insufficient reliability of motion estimation, which requires compensation and correction.

[0041] For each low-confidence macroblock, a matching block matching the texture features of the current macroblock is searched in the reference frame. The texture features are obtained by quantization using local binary mode descriptors. The process of extracting the local binary mode descriptor of the current macroblock is as follows: taking each pixel in the current macroblock as the center, a 3×3 neighborhood is taken. The values ​​of the 8 neighboring pixels in the neighborhood are compared one by one with the value of the center pixel. If the neighboring pixel value is greater than or equal to the value of the center pixel, it is encoded as 1; if it is less, it is encoded as 0. The 8-bit binary codes are concatenated in a fixed order and converted into decimal values, which are used as the local binary mode feature value of that pixel. The frequency of the local binary mode feature values ​​of all pixels in the current macroblock is counted to generate a 256-dimensional frequency histogram, which is used as the local binary mode descriptor of the current macroblock. Within the preset search window of the reference frame, blocks of the same size as the current macroblock are selected as candidate blocks, and the local binary mode descriptor of the same dimension is extracted for each candidate block. Calculate the chi-square distance between the local binary mode descriptor of the current macroblock and the local binary mode descriptors of candidate blocks, and select the candidate block with the smallest chi-square distance as the matching block. Use the pixel values ​​of the matching block as the base values, and superimpose the frequency band compensation values ​​calculated for each pixel position of the current macroblock during the generation of the first correction frame. The superposition method is to add the pixel values ​​of the matching block to the frequency band compensation values ​​at the corresponding pixel positions pixel by pixel to obtain the corrected macroblock pixel values. Replace the pixel values ​​of the current low-confidence macroblocks in the first correction frame with the corrected macroblock pixel values ​​to complete the motion compensation correction for that macroblock. Perform the above operation on all regions marked as low-confidence macroblocks in the first correction frame, while keeping the original pixel values ​​of the remaining macroblocks unchanged.

[0042] See Figure 4The horizontal axis of the graph represents the macroblock confidence value, ranging from 0 to approximately 0.42, while the vertical axis represents the number of macroblocks within the corresponding confidence range. The graph uses purple bars to show the distribution of macroblock numbers with varying confidence values, and a red dashed line indicates the preset confidence threshold of 0.1. It can be seen that the confidence values ​​of most macroblocks are concentrated between 0 and 0.2, with a large number of macroblocks having confidence values ​​less than 0.1. The peak occurs in the 0.05 to 0.1 range, where the number of macroblocks reaches a significant high. After the confidence value exceeds 0.1, the number of macroblocks gradually decreases, and the number of macroblocks corresponding to high confidence values ​​above 0.3 decreases significantly.

[0043] Example 5: In a specific implementation, a video decoding post-processing device includes a parameter parsing module, a frequency band compensation calculation module, a first correction frame generation module, a texture direction extraction module, a directional filtering module, and a weighted fusion module.

[0044] The parameter parsing module is used to parse the macroblock layer coding parameters of the current decoded frame, extracting the quantization step size and transform coefficient scan mode for each macroblock. The module receives the input encoded bitstream and reads the quantization step size field and transform coefficient scan mode flag from the macroblock layer syntax elements of the bitstream. The quantization step size field indicates the step size value used during the quantization process of the current macroblock, and the transform coefficient scan mode flag indicates the scan order type used by the transform coefficients of the current macroblock during decoding.

[0045] The frequency band compensation calculation module calculates the frequency band compensation value for each pixel position in the current decoded frame based on the quantization step size and transform coefficient scanning mode. The frequency band compensation value is determined based on the transform coefficient truncation error associated with the current pixel position. The module receives the quantization step size and transform coefficient scanning mode of each macroblock output by the parameter parsing module, and also receives the transform coefficient data of the current decoded frame. The module divides the current decoded frame into multiple frequency band compensation calculation units, each corresponding to a pixel region of the smallest transform block size. The module iterates through all pixel positions within each frequency band compensation calculation unit, establishing a compensation mapping table for each pixel position. The compensation mapping table stores the correspondence between the quantization step size index and the compensation coefficient. When establishing the compensation mapping table, the module obtains the transform coefficient scanning order corresponding to all pixel positions within the frequency band compensation calculation unit, determines the frequency index of each pixel position in the transform domain based on the transform coefficient scanning order, uses the product of the frequency index and the quantization step size as the lookup key, retrieves the corresponding compensation coefficient in the preset compensation database using the lookup key, and stores the compensation coefficient and frequency index together to generate the compensation mapping table. When calculating the bandwidth compensation value, the bandwidth compensation calculation module reads the corresponding compensation coefficient from the compensation mapping table based on the quantization step size index of the macroblock to which the current pixel position belongs. It then obtains the truncation flag of the transform coefficient to which the current pixel position belongs. The truncation flag is set based on the comparison result of the absolute value of the transform coefficient and the quantization step size threshold. The compensation coefficient is multiplied by the truncation flag, and the product is used as the bandwidth compensation value. The bandwidth compensation calculation module outputs the calculated bandwidth compensation value to the first correction frame generation module.

[0046] The first correction frame generation module is used to superimpose the frequency band compensation value onto the corresponding pixel value of the current decoded frame to generate the first correction frame. The first correction frame generation module receives the frequency band compensation value output by the frequency band compensation calculation module and the pixel value of the current decoded frame, performs the addition operation of the frequency band compensation value and the pixel value at each pixel position, generates the pixel value of the corresponding pixel position in the first correction frame, and outputs the complete first correction frame after all pixel positions have been processed.

[0047] The texture direction extraction module extracts the local texture direction of each pixel in the first corrected frame. The local texture direction is determined by comparing the grayscale differences of pixels in a preset angle direction. The texture direction extraction module receives the first corrected frame output by the first corrected frame generation module and performs direction extraction on each pixel in the first corrected frame. The texture direction extraction module constructs a local window of a preset size, centered on the current pixel, with the preset size set to a 5×5 pixel area. The texture direction extraction module calculates the sum of grayscale gradients of all pixels within the local window in the horizontal direction and the sum of grayscale gradients in the vertical direction. It calculates the initial texture direction angle based on the ratio of the sums of grayscale gradients in the two directions, performs angle quantization processing on the initial texture direction angle, and maps the initial texture direction angle to a preset discrete angle set. The mapping result serves as the local texture direction. The preset discrete angle set contains 8 discrete directions. The texture direction extraction module outputs the extracted local texture direction to the directional filtering module.

[0048] The directional filtering module performs directional low-pass filtering on the first corrected frame based on the local texture direction. The filtering intensity of the directional low-pass filter varies with the angle between the local texture direction and the preset edge direction. The directional filtering module receives the local texture direction output by the texture direction extraction module and the first corrected frame. The directional filtering module selects a corresponding directional filter from a preset filter set based on the local texture direction of the current pixel. The preset filter set contains multiple filter kernels with different filtering axes. The directional filtering module rotates the filtering axis of the directional filter to be consistent with the local texture direction, generating a rotated filter kernel. The directional filtering module calculates the difference between the local texture direction and each preset edge direction in the preset edge direction set to obtain multiple directional angle values. It selects the minimum value from these multiple directional angle values ​​as the final directional angle, queries the intensity mapping table based on the final directional angle to read the filtering intensity coefficient, and multiplies each element in the rotated filter kernel by the filtering intensity coefficient to obtain the adjusted filter kernel. The directional filtering module uses the adjusted filter core to perform a weighted average of the neighboring pixel values ​​of the current pixel in the first corrected frame, generates the filtered pixel value, and outputs the filtered first corrected frame after traversing all pixels in the first corrected frame.

[0049] The weighted fusion module performs a weighted fusion of the filtered first corrected frame and the current decoded frame. The weighting coefficients are dynamically adjusted based on the pixel value differences between the filtered first corrected frame and the current decoded frame to generate the final output frame. The weighted fusion module receives the filtered first corrected frame and the current decoded frame from the directional filtering module. It calculates the absolute value of the pixel value difference at the same pixel position between the filtered first corrected frame and the current decoded frame, and compares this absolute value with a preset difference threshold. If the absolute value of the pixel value difference is less than the preset difference threshold, the weighted fusion module assigns a first fusion weight to the current pixel; if the absolute value of the pixel value difference is greater than or equal to the preset difference threshold, the weighted fusion module assigns a second fusion weight to the current pixel. The weighted fusion module uses the assigned first or second fusion weight to perform a weighted summation of the pixel values ​​of the filtered first corrected frame and the current decoded frame to obtain the fused pixel value. The weighted fusion module then iterates through all pixel positions to generate and output the final output frame.

[0050] In specific implementations, the frequency band compensation calculation module and the directional filtering module in the video decoding post-processing device can also perform the motion compensation correction processing and directional angle filtering intensity processing related in the other embodiments described above, respectively, to enhance the correction effect.

[0051] See Figure 5 The figure shows a box plot of the processing time distribution of each module in Embodiment 5 of the present invention. The vertical axis represents the processing time in milliseconds (ms); the horizontal axis represents, in order, the parameter parsing module, the frequency band compensation calculation module, the first corrected frame generation module, the texture direction extraction module, the directional filtering module, and the weighted fusion module. Each box in the figure represents the statistical distribution of the processing time of each module. The middle line is the median, the upper and lower edges of the box represent the upper and lower quartiles, respectively. The dashed lines represent the maximum and minimum values ​​within the non-outlier range, and the circles represent outliers in the statistics.

[0052] As shown in the figure, the median processing time of the frequency band compensation calculation module is the highest, exceeding 4ms, with a maximum processing time exceeding 8ms. This indicates that the module has high computational complexity, mainly because it needs to traverse all pixel positions within each frequency band compensation calculation unit and establish a compensation mapping table, as well as retrieve the preset compensation database to calculate the frequency band compensation value. Secondly, the median processing time of the directional filtering module is approximately 3.3ms, with a maximum exceeding 4ms. This reflects that the directional filtering module needs to perform filter rotation and weighted averaging operations when performing low-pass filtering based on local texture direction on the first corrected frame, resulting in a large computational load.

[0053] The median processing time of the texture direction extraction module is approximately 1.8ms, indicating a low overall processing time and high efficiency in calculating grayscale gradient accumulation and angle quantization within the local window. The median processing time of the first correction frame generation module is approximately 1.7ms, with a short processing latency, consistent with its function of only performing frequency band compensation and superposition on pixel values.

[0054] The median processing time of the parameter parsing module is approximately 2.5ms, exhibiting stable and low processing latency, reflecting its high efficiency in reading and parsing macroblock layer coding parameters. The weighted fusion module has the lowest median processing time, approximately 1.5ms, and fewer outliers, indicating that this module has a lighter computational burden in the weighted fusion operation of the filtered frame and the current decoded frame pixels.

[0055] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A video decoding post-processing method, characterized in that, The method includes: Analyze the macroblock layer coding parameters of the current decoded frame and extract the quantization step size and transform coefficient scanning mode of each macroblock; Based on the quantization step size and transform coefficient scanning mode, a frequency band compensation value is calculated for each pixel position of the current decoded frame. The frequency band compensation value is determined based on the transform coefficient truncation error associated with the current pixel position. The frequency band compensation value is superimposed on the corresponding pixel value of the current decoded frame to generate a first corrected frame; The local texture direction of each pixel in the first corrected frame is extracted, and the local texture direction is determined by comparing the grayscale difference of the pixels in a preset angle direction; The first corrected frame is subjected to directional low-pass filtering based on the local texture direction, and the filtering intensity of the directional low-pass filtering varies with the angle between the local texture direction and the preset edge direction. The filtered first corrected frame is weighted and fused with the current decoded frame. The weight coefficients of the weighted fusion are dynamically adjusted according to the pixel value difference between the filtered first corrected frame and the current decoded frame to generate the final output frame.

2. The video decoding post-processing method according to claim 1, characterized in that, The step of calculating the frequency band compensation value for each pixel position of the current decoded frame based on the quantization step size and the transform coefficient scanning mode specifically includes: The current decoded frame is divided into multiple frequency band compensation calculation units, and each frequency band compensation calculation unit corresponds to a pixel region with a minimum transform block size; For each frequency band compensation calculation unit, all pixel positions within the pixel region are traversed, and a compensation mapping table is established for each pixel position. The compensation mapping table stores the correspondence between the quantization step size index and the compensation coefficient. Based on the quantization step size index of the macroblock to which the current pixel position belongs, the corresponding compensation coefficient is read from the compensation mapping table; The compensation coefficient is multiplied by the truncation flag of the transform coefficient to which the current pixel position belongs. The truncation flag is set according to the comparison result of the absolute value of the transform coefficient and the quantization step size threshold. The product result is used as the frequency band compensation value.

3. The video decoding post-processing method according to claim 2, characterized in that, The step of establishing a compensation mapping table for each pixel location specifically includes: Obtain the scan order of the transform coefficients corresponding to all pixel positions within the current frequency band compensation calculation unit; Based on the scanning order of the transform coefficients, determine the frequency index of each pixel position in the transform domain; The product of the frequency index and the quantization step size is used as the lookup key, and the corresponding compensation coefficient is retrieved from the preset compensation database using the lookup key. The compensation coefficients are associated with and stored with the frequency index to generate the compensation mapping table.

4. The video decoding post-processing method according to claim 1, characterized in that, The step of extracting the local texture direction of each pixel in the first corrected frame specifically includes: Construct a local window of a preset size centered on the current pixel; Calculate the sum of gray-level gradients of all pixels within the local window in the horizontal direction and the sum of gray-level gradients in the vertical direction; The initial texture orientation angle is calculated based on the ratio of the sum of gray-level gradients in the horizontal direction to the sum of gray-level gradients in the vertical direction. The initial texture direction angle is subjected to angle quantization processing, which maps the initial texture direction angle to a preset discrete angle set, and the mapping result is used as the local texture direction.

5. The video decoding post-processing method according to claim 4, characterized in that, The step of performing directional low-pass filtering on the first corrected frame based on the local texture direction specifically includes: Based on the local texture direction of the current pixel, a corresponding directional filter is selected from a preset filter set, which contains multiple filter kernels with different filtering axes; The filtering axis of the directional filter is rotated to be consistent with the direction of the local texture to generate a rotated filter kernel; The rotated filter kernel is used to perform a weighted average of the neighboring pixel values ​​of the current pixel in the first corrected frame to generate the filtered pixel value. Repeat the above steps until the filtering process for all pixels in the first corrected frame is completed.

6. The video decoding post-processing method according to claim 5, characterized in that, The filtering intensity of the directional low-pass filter varies with the angle between the local texture direction and the preset edge direction, specifically including: The difference between the local texture direction and each preset edge direction in the preset edge direction set is calculated to obtain multiple directional angle values. Select the minimum value from the plurality of directional angle values ​​as the final directional angle; The intensity mapping table is queried according to the final direction angle. The intensity mapping table stores the mapping relationship between the angle interval and the filter intensity coefficient. Each element in the rotated filter kernel is multiplied by a filter intensity coefficient read from the intensity mapping table to obtain the adjusted filter kernel.

7. The video decoding post-processing method according to claim 1, characterized in that, The step of weighted fusion of the filtered first corrected frame and the current decoded frame specifically includes: Calculate the absolute value of the pixel value difference between the filtered first corrected frame and the current decoded frame at the same pixel position; The absolute value of the pixel value difference is compared with a preset difference threshold. If the absolute value of the pixel value difference is less than the preset difference threshold, a first fusion weight is assigned to the current pixel. If the absolute value of the pixel value difference is greater than or equal to the preset difference threshold, a second fusion weight is assigned to the current pixel. Using the first fusion weight or the second fusion weight, the pixel values ​​of the filtered first corrected frame and the pixel values ​​of the current decoded frame are weighted and summed to obtain the fused pixel values.

8. The video decoding post-processing method according to claim 1, characterized in that, After generating the first corrected frame, the method further includes: Extract the motion vector field of the current decoded frame, the motion vector field containing the motion vector of each macroblock; Motion compensation correction is performed on macroblocks in the first correction frame based on the confidence value of each motion vector in the motion vector field. The confidence value is determined based on the median difference between the motion vector and the neighboring motion vectors. The motion compensation correction is performed by replacing the pixel value of the current macroblock with a combination of the pixel value at the corresponding position in the reference frame and the frequency band compensation value.

9. The video decoding post-processing method according to claim 8, characterized in that, The step of performing motion compensation correction on macroblocks in the first corrected frame based on the confidence value of each motion vector in the motion vector field specifically includes: Calculate the difference between the motion vector of the current macroblock and the median of the motion vectors of the surrounding macroblocks, and normalize the reciprocal of the difference of medians as the confidence value; If the confidence value is lower than the preset confidence threshold, the current macroblock is marked as a low-confidence macroblock; For each low-confidence macroblock, a matching block matching the texture features of the current macroblock is searched from the reference frame. The texture features are obtained by local binary mode descriptor quantization. The pixel values ​​of the matching block are used as the base values, and the frequency band compensation values ​​are superimposed to obtain the corrected macroblock pixel values.

10. A video decoding post-processing apparatus for implementing the video decoding post-processing method as described in any one of claims 1 to 9, characterized in that, The device includes: The parameter parsing module is used to parse the macroblock layer coding parameters of the current decoding frame and extract the quantization step size and transform coefficient scanning mode of each macroblock. The frequency band compensation calculation module is used to calculate the frequency band compensation value for each pixel position of the current decoded frame according to the quantization step size and the transform coefficient scanning mode. The frequency band compensation value is determined based on the transform coefficient truncation error associated with the current pixel position. The first correction frame generation module is used to superimpose the frequency band compensation value onto the corresponding pixel value of the current decoded frame to generate the first correction frame; The texture direction extraction module is used to extract the local texture direction of each pixel in the first corrected frame. The local texture direction is determined by comparing the grayscale difference of the pixels in a preset angle direction. A directional filtering module is used to perform directional low-pass filtering on the first modified frame according to the local texture direction, wherein the filtering intensity of the directional low-pass filtering varies with the angle between the local texture direction and the preset edge direction. The weighted fusion module is used to perform weighted fusion of the filtered first corrected frame and the current decoded frame. The weight coefficients of the weighted fusion are dynamically adjusted according to the pixel value difference between the filtered first corrected frame and the current decoded frame to generate the final output frame.

Images