A method for calculating adaptive macroblock-level quantization parameters

Abstract

This invention provides a method for calculating adaptive macroblock-level quantization parameters. The method includes the following steps: S1, dividing the source image into 16x16 macroblocks and analyzing the characteristics of each macroblock; scaling the macroblock data to reduce noise interference with real detail information; S2, using the Sobel operator to calculate the gradient values ​​of the macroblock in four directions, using the gradient information as a basis for measuring the importance of the macroblock data, and dividing the image into three categories of macroblocks: flat, edge, and texture; S3, calculating the image complexity mad of each macroblock, using mad to represent the information content of the macroblock, and comparing mad with multiple thresholds to classify the information content level; S4, based on the characteristics of human visual perception, adaptively adjusting the macroblock-level qp of images at different frame levels, adjusting the qp according to category to improve the information compression rate of texture blocks and reduce image distortion of flat and edge blocks.

Description

Technical Field

[0001] This invention relates to the field of video image processing technology, and in particular to an adaptive macroblock-level quantization parameter calculation method based on the characteristics of human visual sensitivity. Background Technology

[0002] With the development of computer technology and the widespread application of computer vision principles, computer image processing technology has become increasingly popular. Utilizing the visual sensitivity characteristics of the human eye, macroblock-level quantization parameters (qp) are adjusted based on the image complexity of macroblocks. For macroblocks with high complexity, the qp is increased based on the frame-level qp, while for macroblocks with low complexity, the qp is decreased. This rational bitstream allocation aims to improve video compression rate and image quality.

[0003] Commonly used macroblock-level QP adjustment methods based on image complexity only take into account the different sensitivities of the human eye to blocks with different image complexities. They do not consider video compression from the perspective of the importance of image information and ignore the fact that non-important information blocks can also play a role in improving the video compression rate.

[0004] In addition, the terminology used in the prior art:

[0005] The Sobel operator is an important processing method in computer vision. It is primarily used to obtain the first-order gradient of digital images, and a common application is edge detection. The Sobel operator weights the gray values ​​of the four neighboring pixels (top, bottom, left, and right) of each pixel in an image, maximizing the gradient at edges to detect them.

[0006] mad: Mean Absolute Difference, represents the degree of matching between a data block and a template block containing data of the mean. The larger the mad, the worse the matching between the data block and the template, and the more new information it carries. In video coding, it is often used to describe image complexity.

[0007] Quantization parameter qp is the process of mapping continuous signal values ​​to multiple discrete amplitude values, achieving a many-to-one mapping of signal values. Quantization can effectively reduce the range of signal values, thus achieving better compression results; however, quantization is also the root cause of distortion.

[0008] A macroblock is a 16x16 pixel data block and is the basic unit in the H.264 encoding protocol. An image is first divided into multiple macroblocks, and the encoder encodes each macroblock sequentially. In the H.265 protocol, the encoding block size supports multiple sizes, but a 16x16 data block is still generally referred to as a macroblock.

[0009] Frame-level QP: generally refers to QP applied to an entire frame.

[0010] Macroblock-level QP: refers to QP applied to macroblocks.

[0011] Macroblock qp: refers specifically to the qp of a particular macroblock. Summary of the Invention

[0012] To address the problems in the existing technology, the purpose of this application is to add the characteristic of image data importance to the commonly used macroblock-level QP algorithm based on image complexity adjustment. By calculating the image importance feature value, macroblocks in the image are classified, and non-important macroblocks are compressed in a targeted manner to further improve the compression ratio.

[0013] Specifically, the present invention provides a method for calculating adaptive macroblock-level quantization parameters, the method comprising the following steps:

[0014] S1, Divide the source image into macroblocks of size 16x16, and analyze the characteristics of each macroblock in turn;

[0015] Scaling macroblock data reduces the interference of noise on true detail information;

[0016] S2 uses the Sobel operator to calculate the gradient values ​​in the four directions of the macroblock, and uses the gradient information as the basis for measuring the importance of macroblock data to divide the image into three types of macroblocks: flat, edge, and texture.

[0017] S3, calculate the image complexity mad for each macroblock, use mad to represent the information content of the macroblock, and compare mad with multiple thresholds to classify the information content level;

[0018] S4. Based on the characteristics of human visual perception, macroblock-level qp is adaptively adjusted for images at different frame levels. The qp is adjusted according to category to improve the information compression rate of texture blocks and reduce image distortion of flat and edge blocks.

[0019] Step S1 further includes: preprocessing:

[0020] Divide a source image to be encoded into To reduce the impact of noise on information content and importance judgment, macroblocks of varying sizes need to be scaled down, retaining only detailed information. Then, macroblocks in the image are classified based on the amount of detailed information and their importance. The scaling method is as follows:

[0021]

[0022]

[0023] Where avg represents the average pixel value of the macroblock; pixel represents the pixel value of the macroblock; and p represents the scaled data of the macroblock.

[0024] Step S2 further includes: macroblock importance classification labeling:

[0025] Compute using the Sobel operator on the scaled macroblock , , and The gradient values ​​in the four directions are calculated using the following formula:

[0026]

[0027]

[0028]

[0029]

[0030] The gradient values ​​of the scaled data blocks obtained in step S1 are calculated using the gradient calculation formula described above. The gradient values ​​of the three data blocks are as follows:

[0031] Gradient values ​​in four directions of the flat block: , , ,

[0032] Gradient values ​​of the edge block in four directions: , , ,

[0033] Gradient values ​​of the texture patch in four directions: , , ,

[0034] Analyzing the gradient values ​​of the three types of scaled data blocks reveals that flat blocks have small gradient values ​​in all directions; edge blocks have large gradient values ​​in one direction but small gradient values ​​in their vertical direction; and textured blocks have large gradient values ​​in all directions with little difference. Therefore, by giving high and low gradient thresholds... and The ratio of vertical gradients to vertical gradients and To classify the data blocks; the classification method is as follows:

[0035]

[0036] The flag value of 0, 1, and 2 represents flat blocks, edge blocks, and texture blocks, respectively.

[0037] Step S3 further includes: macroblock information content:

[0038] The commonly used image complexity MAD is used to represent the information content of a macroblock. Unscaled source data is used here to meet the need for richer information content level division, so as to be suitable for adjusting different amplitudes and levels of macroblock-level QP. Its calculation method is as follows:

[0039] .

[0040] Step S4 further includes: an adaptive macroblock-level qp calculation method.

[0041] Based on the macroblock importance flag obtained in step S2 and the information size flag mad obtained in step S3, different frame-level quantization parameters are used. Under different compression requirements for macroblocks, the macroblock-level qp is adaptively adjusted to increase the compression rate of the video image at texture blocks that carry non-critical information, while not increasing the compression rate at edge blocks and flat blocks that carry critical information, so as to preserve better subjective viewing quality.

[0042] The method of increasing the compression rate of video images at texture blocks carrying non-essential information, while not increasing the compression rate at edge blocks and flat blocks carrying essential information to preserve better subjective viewing quality, further includes: based on data analysis, when... The smaller the value, the lower the image compression ratio, the less image distortion, and the lower the sensitivity of the human eye to distortion at edges and texture blocks. Increasing the QP value at edge and texture blocks improves the compression ratio, and the magnitude of the increase is proportional to the information content identifier (MAD) of the macroblock; according to The size of the qp value is controlled by the maximum increase in the value to keep the distortion of edge blocks within the range perceptible to the human eye, while flat blocks are not adjusted in qp or are adjusted in a small range to slightly reduce image distortion. The magnitude of the adjustment is inversely proportional to the information content indicator (mad) of the macroblock; conversely, when... The larger the QP value, the higher the image compression ratio, but also the greater the image distortion. The human eye is more sensitive to distortion at edges and texture blocks. In this case, no QP adjustment is made, or the QP value is increased slightly within a small range to slightly improve the image compression ratio. Conversely, the human eye is more sensitive to distortion at flat blocks. Decreasing the QP value at flat blocks reduces image distortion; the magnitude of the decrease is inversely proportional to the macroblock's information content identifier (MAD). The size control qp is adjusted to reduce the maximum amount so that the compression ratio of the flat block is within an acceptable range.

[0043] The method for calculating the adaptive macroblock-level qp is as follows:

[0044] S4.1, Set the threshold:

[0045] Based on the analysis of macroblock MAD distribution in multiple scenarios, the following thresholds are given:

[0046] ,

[0047] ,

[0048] ;

[0049] ,

[0050] ,

[0051] ;

[0052] Among them, array The frame-level QP thresholds for flat blocks, edge blocks, and texture blocks are used to determine the current frame. The level at which it is located; array This represents three sets of image complexity thresholds: flat blocks, edge blocks, and complex blocks, used to calculate macroblock qp;

[0053] S4.2, will With a given threshold array Compare in ascending order, when When the importance of a macroblock falls between two thresholds, the QP level of the three types of macroblocks is recorded respectively. The calculation method is as follows:

[0054] ;

[0055] S4.3, according to Get mad and Subscript index movement step during comparison and maximum qp offset The purpose is to dynamically determine the magnitude of macroblock QP adjustment based on frame-level QP.

[0056] Threshold index movement step size:

[0057]

[0058] Maximum qp offset:

[0059]

[0060] S4.4, combine the macroblock's mad with In comparison, when the MAD is between two thresholds, the macroblock QP is calculated relative to the following method. offset value ;

[0061] When the macroblock is a flat block:

[0062] Where k is The index value, k takes values ​​from 9 to And the step size of k is ;

[0063] When the macroblock is a texture or edge block:

[0064]

[0065] Where k is The index value, k takes values ​​from The step size of k is up to 9. ;

[0066] By adjusting the increment or decrement of k, the threshold range and the adjustment magnitude of qp can be controlled; macroblock Macroblocks with the same flag will have their qp values ​​change accordingly. The adjustment range varies depending on the differences in the model and the level of the MAD (mad system), in order to achieve adaptive behavior.

[0067] The method can also meet the requirements for different compression rates by adjusting the threshold corresponding to the texture block.

[0068] The method is based on the characteristics of human visual sensitivity, namely that the human eye is more sensitive to distortion in flat areas and less sensitive to distortion in complex areas, and that the information carried by textured areas in complex areas is not easily recognized by the human eye.

[0069] Therefore, the advantage of this application lies in the following: The solution uses a simple method to improve efficiency. By adding the characteristic of image data importance, macroblocks are classified, and non-important macroblocks are compressed, thereby improving the compression ratio. The degree of improvement is related to the number of non-important blocks and their information content identifier (DMA). Attached Figure Description

[0070] The accompanying drawings, which are provided to further illustrate the invention and form part of this application, are not intended to limit the scope of the invention.

[0071] Figure 1 This is a flowchart of the method of the present invention.

[0072] Figure 2 This is a schematic diagram of the flat block data involved in the present invention.

[0073] Figure 3 yes Figure 2A schematic diagram of scaled flat block data.

[0074] Figure 4 This is a schematic diagram of edge block data involved in the present invention.

[0075] Figure 5 yes Figure 4 A schematic diagram of scaled edge block data.

[0076] Figure 6 This is a schematic diagram of the texture block data involved in the present invention.

[0077] Figure 7 yes Figure 6 A schematic diagram of the scaled texture block data. Detailed Implementation

[0078] To better understand the technical content and advantages of the present invention, the present invention will now be described in further detail with reference to the accompanying drawings.

[0079] This invention relates to an adaptive macroblock-level quantization parameter calculation method. Based on the characteristics of human visual sensitivity, it adaptively adjusts the macroblock-level quantization parameter (QP) of images at different frame levels by calculating the importance markers and information content of macroblock-level data in the image. This achieves targeted control of data compression and improved image quality. Based on the characteristics of human visual sensitivity, the human eye is more sensitive to distortion in flat areas, less sensitive to distortion in complex areas, and indifferent to texture information. Therefore, as much bitstream as possible can be used to encode important information blocks in the image, while compressing non-important information blocks.

[0080] like Figure 1 As shown, the basic idea of ​​the method of the present invention is:

[0081] S1, First, sort the source image by The macroblocks are divided into large and small blocks, and the characteristics of each macroblock are analyzed in turn. Then, the macroblock data is scaled to reduce the interference of noise on the real details.

[0082] S2 uses the Sobel operator to calculate the gradient values ​​in the four directions of the macroblock, and uses the gradient information as the basis for measuring the importance of macroblock data to divide the image into three types of macroblocks: flat, edge, and texture.

[0083] S3, calculate the image complexity mad for each macroblock, use mad to represent the information content of the macroblock, and compare mad with multiple thresholds to classify the information content level;

[0084] S4 adaptively adjusts the macroblock-level qp of images at different frame levels based on the characteristics of human vision, so as to achieve the best image compression rate and subjective quality.

[0085] The specific calculation method is as follows:

[0086] S1. Preprocessing:

[0087] A source image to be encoded is divided into 16x16 macroblocks. To reduce the impact of noise on information content and importance judgment, the macroblocks need to be scaled to retain only detailed information. Then, the macroblocks in the image are classified based on the amount of detailed information and their importance. The scaling method is as follows:

[0088]

[0089]

[0090] Where avg represents the average pixel value of the macroblock; pixel represents the pixel value of the macroblock; and p represents the scaled data of the macroblock.

[0091] Following the above method, such as Figures 2 to 7 As shown, the source data and scaled data of three types of macroblocks are presented, which can intuitively compare the distribution characteristics of the three types of macroblock data.

[0092] S2. Macroblock Importance Classification Markers:

[0093] Compute using the Sobel operator on the scaled macroblock , , and The gradient values ​​in the four directions are calculated using the following formula:

[0094]

[0095]

[0096]

[0097]

[0098] The gradient values ​​for the scaled data blocks obtained in the previous step are calculated using the gradient calculation formula described above. The gradient values ​​for the three data blocks are as follows:

[0099] Gradient values ​​in four directions of the flat block: , , ,

[0100] Gradient values ​​of the edge block in four directions: , , ,

[0101] Gradient values ​​of the texture patch in four directions: , , ,

[0102] Analyzing the gradient values ​​of the three types of scaled data blocks reveals that flat blocks have small gradient values ​​in all directions; edge blocks have large gradient values ​​in one direction but small gradient values ​​in their vertical direction; and textured blocks have relatively large gradient values ​​in all directions with little difference. Therefore, a high / low gradient threshold can be given. and The ratio of vertical gradients to vertical gradients and To classify the data blocks, the classification method is as follows:

[0103]

[0104] The flag value of 0, 1, and 2 represents flat blocks, edge blocks, and texture blocks, respectively.

[0105] S3. Macroblock information content:

[0106] The most commonly used image complexity MAD is used to represent the information content of a macroblock. Unscaled source data is used here to meet the need for richer information content level division, so as to be suitable for adjusting different amplitudes and levels of macroblock-level QP. Its calculation method is as follows:

[0107]

[0108] S4. Adaptive macroblock-level qp calculation method:

[0109] Based on the macroblock importance flag and information size flag obtained in the above steps, different frame-level quantization parameters are used. For different types of macroblocks, the macroblock-level qp is adaptively adjusted to achieve the overall optimal video image compression rate and subjective quality.

[0110] Data analysis revealed that when The smaller the value, the less sensitive the human eye is to distortion from edges and texture blocks, and distortion from flat blocks. In this case, the QP of edges and texture blocks can be increased, while the QP of flat blocks can be decreased. Conversely, when... The larger the value, the more sensitive the human eye is to distortion in edges and texture blocks and distortion in flat blocks. Therefore, the qp of edge and texture blocks should be increased less, while the qp of flat blocks can be decreased more. The adaptive macroblock-level qp calculation method is as follows:

[0111] S4.1, Set thresholds. Based on the analysis of the distribution of macroblock mads in multiple scenarios, the following thresholds are given.

[0112] ,

[0113] ,

[0114] ;

[0115] ,

[0116] ,

[0117] ;

[0118] Among them, array The frame-level QP thresholds for flat blocks, edge blocks, and texture blocks are used to determine the current frame. The level at which it is located. (Array) This represents three sets of image complexity thresholds: flat blocks, edge blocks, and complex blocks, used to calculate macroblock qp.

[0119] S4.2, will With a given threshold array Compare in ascending order, when When the importance of a macroblock falls between two thresholds, the QP level of the three types of macroblocks is recorded respectively. The calculation method is as follows:

[0120] ;

[0121] S4.3, according to Get mad and Subscript index movement step during comparison and maximum qp offset The goal is to dynamically determine the magnitude of macroblock QP adjustment based on frame-level QP.

[0122] Threshold index movement step size:

[0123]

[0124] Maximum qp offset:

[0125] ;

[0126] S4.4, combine the macroblock's mad with In comparison, when the MAD is between two thresholds, the macroblock QP is calculated relative to the following method. offset value .

[0127] When the macroblock is a flat block:

[0128]

[0129] Where k is The index value, k takes values ​​from 9 to And the step size of k is .

[0130] When the macroblock is a texture or edge block:

[0131]

[0132] Where k is The index value, k takes values ​​from The step size of k is up to 9. .

[0133] By adjusting the increment or decrement of k, the threshold range and the adjustment magnitude of qp can be controlled. (Macroblock) Macroblocks with the same flag will have their qp values ​​change accordingly. The adjustment range varies depending on the type and level of the mad file, achieving an adaptive effect. Additionally, the threshold corresponding to the texture block can be adjusted to meet different compression ratio requirements.

[0134] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations can be made to the embodiments of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for calculating adaptive macroblock-level quantization parameters, characterized in that, The method includes the following steps: S1, sort the source image by The macroblocks are divided into large and small blocks, and the characteristics of each macroblock are analyzed in turn. Scaling the macroblock data reduces the interference of noise on the true details; step S1 further includes: preprocessing: Divide a source image to be encoded into To reduce the impact of noise on information content and importance judgment, macroblocks of varying sizes need to be scaled down, retaining only detailed information. Then, macroblocks in the image are classified based on the amount of detailed information and their importance. The scaling method is as follows: ； ； Where avg represents the average pixel value of the macroblock; pixel represents the pixel value of the macroblock; and p represents the scaled data of the macroblock. S2, using the Sobel operator to calculate the gradient values ​​in the four directions of the macroblock, and using the gradient information as a basis for measuring the importance of macroblock data, the image is divided into three categories of macroblocks: flat, edge, and texture; step S2 further includes: macroblock importance classification label: Compute using the Sobel operator on the scaled macroblock , , and The gradient values ​​in these four directions are calculated using the following formulas: ； ； ； ； The gradient values ​​of the scaled data blocks obtained in step S1 are calculated using the gradient calculation formula described above. The gradient values ​​of the three data blocks are as follows: Gradient values ​​in four directions of the flat block: , , , ; Gradient values ​​of the edge block in four directions: , , , ; Gradient values ​​of the texture patch in four directions: , , , ; Analyzing the gradient values ​​of the three types of scaled data blocks reveals that flat blocks have small gradient values ​​in all directions; edge blocks have large gradient values ​​in one direction but small gradient values ​​in their vertical direction; and textured blocks have large gradient values ​​in all directions with little difference. Therefore, by giving high and low gradient thresholds... and The ratio of vertical gradients to vertical gradients and To classify the data blocks; the classification method is as follows: ； The flag value 0, 1, and 2 represent flat blocks, edge blocks, and texture blocks, respectively. S3, calculate the image complexity mad for each macroblock, using mad to represent the information content of the macroblock, and compare mad with multiple thresholds to classify the information content levels; step S3 further includes: macroblock information content: The commonly used image complexity MAD is used to represent the information content of a macroblock. Unscaled source data is used here to meet the need for richer information content level division, so as to be suitable for adjusting different amplitudes and levels of macroblock-level QP. Its calculation method is as follows: ； S4, based on the characteristics of human visual perception, adaptively adjust the macroblock-level qp of images at different frame levels, adjusting the qp according to category to improve the information compression rate of texture blocks and reduce image distortion of flat and edge blocks; step S4 further includes: an adaptive macroblock-level qp calculation method: Based on the macroblock importance flag obtained in step S2 and the information size flag mad obtained in step S3, the macroblock-level qp is adaptively adjusted under the compression requirements of different types of macroblocks by different frame-level quantization parameters frm_qp. This increases the compression rate of the video image at texture blocks carrying non-important information, while not increasing the compression rate at edge blocks and flat blocks carrying important information, so as to preserve better subjective viewing quality.

2. The method for calculating adaptive macroblock-level quantization parameters according to claim 1, characterized in that, The method of increasing the compression rate of video images at texture blocks carrying non-essential information, while not increasing the compression rate at edge blocks and flat blocks carrying essential information to preserve better subjective viewing quality, further includes: based on data analysis, when... The smaller the value, the lower the image compression ratio, the less image distortion, and the lower the sensitivity of the human eye to distortion at edges and texture blocks. Increasing the QP value at edge and texture blocks improves the compression ratio, and the magnitude of the increase is proportional to the information content identifier (MAD) of the macroblock; according to The size of `frm_qp` controls the maximum increase in value to keep the distortion of edge blocks within the perceptible range for the human eye. For flat blocks, `frm_qp` is not adjusted or is reduced within a small range to minimize image distortion. The reduction is inversely proportional to the macroblock's information content identifier (`mad`). Conversely, the larger `frm_qp` is, the higher the image compression ratio and the greater the image distortion. The human eye is more sensitive to distortion at edges and texture blocks, so in this case, `frm_qp` is not adjusted or is increased within a small range to slightly improve the image compression ratio. The human eye is even more sensitive to distortion at flat blocks, so `frm_qp` is reduced at flat blocks to minimize image distortion. The reduction is inversely proportional to the macroblock's information content identifier (`mad`). The size control qp is adjusted to reduce the maximum amount so that the compression ratio of the flat block is within an acceptable range.

3. The method for calculating adaptive macroblock-level quantization parameters according to claim 1, characterized in that, The method for calculating the adaptive macroblock-level qp is as follows: S4.1, Set the threshold: Based on the analysis of macroblock MAD distribution in multiple scenarios, the following thresholds are given: , , ; , , ; Among them, array The frame-level QP thresholds for flat blocks, edge blocks, and texture blocks are used to determine the current frame. The level at which it is located; array This represents three sets of image complexity thresholds: flat blocks, edge blocks, and complex blocks, used to calculate macroblock qp; S4.2, will With a given threshold array Compare in ascending order, when When the importance of a macroblock falls between two thresholds, the QP level of the three types of macroblocks is recorded respectively. The calculation method is as follows: ； S4.3, according to Get mad and The index movement step size ofst[flag] and the maximum qp offset during comparison. The purpose is to dynamically determine the magnitude of macroblock QP adjustment based on frame-level QP. Threshold index movement step size: ； Maximum qp offset: ； S4.4, combine the macroblock's mad with In comparison, when the MAD is between two thresholds, the macroblock QP is calculated relative to the following method. offset value ; When the macroblock is a flat block: ； Where k is The index value, k takes values ​​from 9 to And the step size of k is ; When the macroblock is a texture or edge block: ； Where k is The index value, k takes values ​​from The step size of k is up to 9. ; By adjusting the increment or decrement of k, the threshold range and the adjustment magnitude of qp can be controlled; macroblock Macroblocks with the same flag will have their qp values ​​change accordingly. The adjustment range varies depending on the differences in the model and the level of the MAD (mad system), in order to achieve adaptive behavior.

4. The method for calculating adaptive macroblock-level quantization parameters according to claim 3, characterized in that, The method can also meet the requirements for different compression rates by adjusting the threshold corresponding to the texture block.

5. The method for calculating adaptive macroblock-level quantization parameters according to claim 1, characterized in that, The method is based on the characteristics of human visual sensitivity, namely that the human eye is more sensitive to distortion in flat areas and less sensitive to distortion in complex areas, and that the information carried by textured areas in complex areas is not easily recognized by the human eye.

Images