An optimization method and an image quality evaluation method based on JPEG image compression standard

Abstract

The application discloses an optimization method and an image quality evaluation method based on a JPEG image compression standard, wherein the whole code rate interval is equally divided into 100 code rate points to correspond to 100 quality index values (QF) of JPEG; for each QF, a set of optimal quantization tables is generated by using a back propagation mechanism; for other code rate points, the optimal quantization tables are obtained based on curve fitting; in the iteration, the loss function calculation considers edge weighting; in addition, based on the "just noticeable difference" of the human eye, a new picture quality evaluation index "just perceptible distortion" (JPD) is proposed, and the method only accumulates those pixel distortions greater than a preset distortion threshold; and further, the block effect often presented by JPEG in low code rate compression is considered in the distortion accumulation of JPD, and the JPD-B is further expanded.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image compression, and in particular to an optimization method based on JPEG image compression standard and an image quality evaluation method. BACKGROUND

[0002] JPEG is an international standard for natural color image compression, whose purpose is to reduce the file size generated by digital cameras when taking pictures while maintaining good image quality as much as possible. JPEG takes 8x8 small blocks as a unit and uses Discrete Cosine Transformation (DCT) to convert the image from the spatial domain (pixel value) to the frequency domain (DCT coefficient), and then uses the JPEG quantization table for quantization (compression). Specifically, each number (quantization parameter) in the JPEG quantization table is used as the divisor of the DCT coefficient at the corresponding position in the small block, and after division, the integer is rounded off, which results in a reduction in the dynamic range of the DCT coefficient and many high-frequency DCT coefficients becoming zero, thereby achieving the effect of data compression.

[0003] Although JPEG specifies 100 quality factor (QF) values from 1 (lowest quality) to 100 (highest quality), but JPEG only provides a set (two) of default quantization tables for QF = 50, one for luminance compression and the other for chrominance compression. For the quantization tables corresponding to other QF values, JPEG only provides a simple linear conversion formula designed based on the QF = 50 quantization table set. Obviously, for natural color images with diverse content, the image quality produced by the image compressed by this extremely simplified quantization method is difficult to satisfy, and the code rate is also difficult to control, so there are a large number of research works trying to propose self-defined "optimal" quantization tables to improve the quality and efficiency of image compression.

[0004] The known optimization methods can be roughly divided into two categories: traditional methods and deep learning-based methods. The main traditional methods include heuristic algorithms, dynamic programming, greedy search, and gradient descent, etc. However, these methods have the following disadvantages: high complexity of iterative optimization process, limited improvement of compressed image quality, and difficulty in controlling the code rate. Deep learning methods generate a quantization table based on image content through convolutional neural network (CNN) operations. Compared with traditional methods, these methods can provide better compressed image quality. However, the CNN used in these methods needs to limit the size of the input image and may require a long image compression time. In addition, how to select images to establish an effective image training dataset to ensure that the code rate generated by all 100 quality indicators QF covers the entire image code rate interval and forms an equidistant distribution as much as possible is also a challenge.

[0005] Finally, the most widely used indicator for evaluating the quality of compressed images is the peak signal-to-noise ratio (PSNR). However, PSNR measures the mean square error (MSE) between compressed images and original images pixel by pixel as a quality evaluation indicator for compressed images. However, the human eye's perception of image distortion is often based on a certain area, object, or even certain points, rather than simply relying on the average error value of the entire photo to judge the quality of the image. Therefore, PSNR often does not conform to the subjective perception of the human eye when evaluating image quality, that is, an image may be judged as a poor quality image in the human eye's subjective perception even if it has a high PSNR.

[0006] To address the inconsistency between human perception and objective measurement, structural similarity (SSIM) is another widely used quality evaluation indicator that is more consistent with human perception than PSNR. However, the SSIM indicator value still produces inconsistencies with human perception. The problem lies in that SSIM calculates all pixels in the entire image in the same way, that is, the errors produced by pixels that the human eye cannot perceive are also included in the calculation of the SSIM value, leading to misjudgment. SUMMARY

[0007] In order to solve the foregoing technical problems, the application provides an optimization method based on a JPEG image compression standard and an image quality evaluation method, which can quickly provide a set of optimal quantization tables for any code rate, and a new picture quality evaluation formula based on the Just Noticeable Difference (JND) characteristics of the human eye and the Peak Absolute Error (PAE) presented by the image content.

[0008] In order to achieve the above technical purposes, the technical scheme adopted by the application is as follows:

[0009] In the first aspect, the application discloses an optimization method based on a JPEG image compression standard, which comprises the following steps:

[0010] S1, the entire code rate range of a natural image is equally divided into 100 code rate points, which correspond to 100 quality index values from low to high;

[0011] S2, a total loss function is set, and the total loss function value is the sum of the image quality loss and the weighted code rate term; the code rate term weight is adjusted manually, so that the error between the code rate of the optimized learnable quantization table and the specified code rate of the quality index value is less than a preset error threshold;

[0012] S3, for each quality index value, the back propagation mathematical mechanism in an artificial intelligence convolutional neural network is used to generate a set of optimal quantization tables for it through a synchronous iteration operation, and each set of optimal quantization tables comprises a luminance quantization table and a chroma quantization table; in the back propagation iteration optimization process, all quantization parameters in the optimal quantization table are updated synchronously in each iteration until the optimization effect reaches saturation; the foregoing iteration process is repeated until the optimal quantization tables corresponding to all quality index values are output;

[0013] S4, the code rates corresponding to all quality index values are used as independent variables, and the quantization parameters at each position in the corresponding optimal quantization table are used as dependent variables, and curve fitting is performed to obtain the fitting curve of each quantization parameter;

[0014] S5, for any code rate, the quantization parameter value corresponding to the code rate is calculated through the fitting curve of each quantization parameter in step S4, the corresponding luminance quantization table and chroma quantization table are generated, and the optimal quantization table corresponding to the code rate is obtained.

[0015] Further, in step S2, the calculation process of the total loss function value comprises:

[0016] The original picture is converted from RGB color space to YCbCr color space, and the chroma components are down-sampled to obtain the original luminance component and the down-sampled chroma components;

[0017] After DCT transformation of the original luminance component and the down-sampled chroma components, DCT coefficients are obtained, and the DCT coefficients are quantized using a learnable quantization table;

[0018] After differentiable approximate rounding of the quantized DCT coefficients, the inverse quantization is performed through the learnable quantization table, and the inverse DCT transformation is performed on the inverse quantization result to obtain the decoded luminance component and chroma component;

[0019] The image quality loss is calculated by combining the original luminance component and the down-sampled chroma component, and the decoded luminance component and chroma component; the reciprocal of all quantization parameters in the learnable quantization table is taken, and the sum is obtained to obtain the rate item, which is multiplied by the rate item weight to obtain the weighted rate item; the image quality loss and the weighted rate item are summed to obtain the total loss function value.

[0020] Further, the process of calculating the image quality loss by combining the original luminance component and the down-sampled chroma component, and the decoded luminance component and chroma component includes the following steps:

[0021] The original luminance component is processed by the Sobel edge detection algorithm to calculate the gradient value of each pixel in the horizontal and vertical directions, and the gradient value is normalized and then added by 1 to obtain the image edge guided weight;

[0022] The first absolute value error between the original luminance component and the decoded luminance component, and the second absolute value error between the down-sampled chroma component and the decoded chroma component are calculated, the first absolute value error of the luminance component is multiplied by the image edge guided weight, and then added to the second absolute value error of the chroma component, and finally averaged to obtain the image quality loss.

[0023] Further, in step S3, the training data set used by the learnable quantization table includes six basic perceptual element images: noise images with random probability distribution, noise images generated by high ISO, texture images, edge images, blurred images and smooth images.

[0024] Further, the generation process of the training data set includes:

[0025] Noise images with various random probability distributions including Gaussian, Poisson, uniform and salt and pepper are generated by computer code;

[0026] Noise images generated by high ISO, blurred images and smooth images are obtained by manual cutting;

[0027] The initial texture images are obtained by cropping, for each initial texture image, the co-occurrence matrix in four directions of 0 degrees, 45 degrees, 90 degrees and 135 degrees is counted respectively, the average entropy value of the four co-occurrence matrices is calculated, and the initial texture image with the average entropy value greater than the set entropy threshold value is screened out as the texture image finally included in the training data set;

[0028] The initial edge images are obtained by cropping, and the initial edge image with the proportion of all edge pixel numbers greater than the set pixel number threshold value is screened out as the edge image finally included in the training data set by using the Canny edge detection algorithm;

[0029] The number and type of the six basic perceptual element images in the training data set are adjusted, so that the number of the six basic perceptual element images tends to be consistent, and the data set tends to cover the entire code rate interval of a natural image, that is, the pixel depth , the pixel depth of the luminance component , and the pixel depth of the two chrominance components .

[0030] In the second aspect, the application discloses an image quality evaluation method, and the method comprises the following steps:

[0031] The absolute value error that can be perceived by the human eye on the distorted image is calculated:

[0032] An matrix with the same size as the original picture is initialized , the absolute value error of each pixel is compared with the corresponding preset distortion threshold value, if the absolute value error of any pixel is greater than the corresponding preset distortion threshold value, the value of the corresponding position in the matrix is set to 1, otherwise, the value of the corresponding position in the matrix is set to 0;

[0033] The absolute value error of each pixel on the distorted image and the original image is point multiplied with the matrix , the pixel distortion with the absolute value error less than the preset distortion threshold value is removed, and is obtained; the peak absolute value error that can be perceived on the distorted image is calculated by summing the following formula :

[0034] ;

[0035] Wherein is the peak error value that can be reached by a single pixel on the distorted image;

[0036] The average error of on the whole image is calculated by taking the number of pixels in the whole image as the divisor, and the image quality evaluation index JPD is obtained.

[0037] Further, the calculation process of the preset distortion threshold includes the following steps:

[0038] calculating the background uniform brightness of each pixel of the original image in the local area , the maximum contrast difference in the four directions of 0 degrees, 45 degrees, 90 degrees and 135 degrees in the local area , and ;The specific calculation formula is as follows:

[0039] ;

[0040] ;

[0041] wherein i = 1, 2, 3, 4, 5, j = 1, 2, 3, 4, 5, and are the operator matrices for calculating the background uniform brightness and the maximum contrast difference, and max is the maximum value function;

[0042] The preset distortion threshold of each pixel is calculated by the following formula :

[0043] ;

[0044] wherein the formula is specifically:

[0045] ;

[0046] wherein the formula is specifically:

[0047] ;

[0048] wherein the parameter value in the function , .

[0049] Further, the method is used for quality evaluation of a JPEG compressed image, including the following steps:

[0050] calculating the absolute value error of the blocky artifacts that can be perceived by the human eye on the distorted image:

[0051] initializing a matrix with the same size as the original image , comparing the absolute value error of each pixel with the preset distortion threshold corresponding to the pixel, and if the absolute value error of any pixel is greater than the preset distortion threshold corresponding to the pixel, then the matrix The value at the position corresponding to that pixel is set to 1, otherwise the matrix is ​​set to... The value at the corresponding position is set to 0;

[0052] Initialize a matrix with the same size as the distorted image. For distorted images For pixels at the edge of a small block, calculate the change in value between its left, right, top, and bottom adjacent pixels. If this change value exceeds a preset distortion threshold, then this pixel is considered to have a perceptible blocky effect. The matrix is ​​then... The corresponding position logic value is set to 1; otherwise, the matrix is ​​set to 0. The corresponding logical value is set to 0;

[0053] Initialize a matrix with the same size as the original image. For the original image For pixels at the edge of a small block, calculate the change in value between its left, right, top, and bottom adjacent pixels. If this change value exceeds a preset distortion threshold, then this pixel is considered to have a perceptible blocky effect. The matrix is ​​then... The corresponding position logic value is set to 1; otherwise, the matrix is ​​set to 0. The corresponding logical value is set to 0;

[0054] matrix With matrix Perform an XOR operation if the matrix With matrix If the logical values ​​at corresponding positions are different, set the logical values ​​at corresponding positions in the two matrices to 1; otherwise, set the logical values ​​at corresponding positions in the two matrices to 0, resulting in a matrix... The perceptible block artifact pixels caused by compression were filtered out.

[0055] matrix With matrix Taking the union of the sets yields the matrix. Then the matrix The result is obtained by multiplying the absolute error of the distorted image and the original image at each pixel. The cumulative peak absolute value error of the block artifacts perceived by the human eye is calculated using the following formula. :

[0056] ;

[0057] in This represents the peak error value that a single pixel can achieve in a distorted image;

[0058] Divide by the total number of pixels in the image and calculate The average error across the entire image is used to obtain the image quality evaluation index JPD-B.

[0059] Compared with the prior art, the present application has the following advantages:

[0060] First, the optimization method based on the JPEG image compression standard of the present application generates a set of optimal quantization tables for each of the 100 Quality Factors (QF) specified by JPEG, including a luminance quantization table and a chrominance quantization table, each with 64 quantization parameters. These 128 parameters are optimized simultaneously in each iteration. The resulting quantization tables are used to quantize (i.e., compress) the Discrete Cosine Transform (DCT) coefficients obtained after the frequency domain conversion of the image, thereby achieving image compression and obtaining the code rate at the specified QF. Since the code rate point can be any value within the entire code rate range, the parameter values at each position in the corresponding quantization table for the code rate points distributed among the 100 code rate points can be directly obtained from the nonlinear relationship curve fitted using the code rate values (independent variables) and the corresponding 100 sets of quantization tables (dependent variables) for the 100 QFs. In other words, the optimal quantization tables can be directly read from the corresponding dependent variables in the 64 luminance curves and 64 chrominance curves without any additional calculations.

[0061] Second, the optimization method based on the JPEG image compression standard of the present application uses a specially designed loss function, which is obtained by adding a distortion term and a weighted code rate term. The distortion is calculated as the absolute difference between the compressed image and the original image for each pixel, and further weighted more heavily for the edge pixel positions detected in the image. The loss function value obtained above is optimized simultaneously with the 128 parameters of the quantization table using the gradient descent of BackPropagation (BP).

[0062] Third, the optimization method based on the JPEG image compression standard of the present application proposes a set of data set establishment methods. The established data set contains six types of "Visual Perception Element Maps" (Visualets). This data set not only conforms to the perception and judgment of the human eye when evaluating natural images, but also effectively helps the code rate points generated by the 100 QFs to be evenly distributed in the entire code rate range of natural color images, thereby achieving better image compression efficiency and quality.

[0063] Fourth, the image quality evaluation method of the present application, based on the "Just Noticeable Difference" (JND) characteristics of the human eye and the peak absolute error (PAE) presented on the image content, proposes a new image quality evaluation method based on the image quality evaluation index "Just Perceptible Distortion" (JPD). The image quality evaluation index JPD only accumulates the distortion generated by the pixels whose distortion is greater than the set distortion threshold, so that the quality evaluation value obtained is more accurate to reflect the true feeling of the human eye when evaluating the distorted image (compared with the original image). The image quality evaluation index JPD is suitable for quality evaluation after all image processing (including compression and non-compression processing, such as super-resolution, interpolation, denoising, etc.). In addition, JPEG often presents a block effect at low code rate compression. To reflect the evaluation of this part of distortion, the obvious block edge pixels are also included in the cumulative distortion calculation of the image quality evaluation index JPD, and the image quality evaluation index "Just Perceptible Distortion and Block Artifact" JPD-B is obtained. JPD-B is used to evaluate the image quality of the low code rate compression image, and the character JPD-B is used to represent it to distinguish from JPD (where "B" comes from the first letter of Block, representing "block"). BRIEF DESCRIPTION OF DRAWINGS

[0064] Figure 1 The flowchart of the optimization method based on the JPEG image compression standard of the present application;

[0065] Figure 2 The flowchart of the differentiable JPEG quantization table optimization algorithm based on back propagation in the embodiment;

[0066] Figure 3 The flowchart of the quality loss calculation in the embodiment;

[0067] Figure 4 The flowchart of the code rate value calculation in the embodiment;

[0068] Figure 5 The flowchart of the target code rate point optimal quantization table generation in the embodiment;

[0069] Figure 6 The flowchart of the image quality evaluation method based on the image quality evaluation index "Just Perceptible Distortion" (JPD);

[0070] Figure 7 The flowchart of the image quality evaluation method based on the image quality evaluation index "Just Perceptible Distortion and Block Artifact" (JPD-B);

[0071] Figure 8 is a schematic diagram of the operator matrix and is a schematic diagram of the operator matrix DETAILED DESCRIPTION

[0072] Embodiments of the present application will be further described below in conjunction with the accompanying drawings. Embodiment 1

[0073] The present embodiment discloses an optimization method based on the JPEG image compression standard, which comprises the following steps:

[0074] S1, equally divide the entire code rate interval of a natural image into 100 code rate points, which correspond to 100 quality factor (QF) values from low to high respectively;

[0075] S2, set a total loss function, and the total loss function value is the sum of the image quality loss and the weighted code rate term; wherein the code rate term weight is adjusted manually, so that the error between the code rate of the optimized learnable quantization table and the code rate specified by the quality factor value is less than a preset error threshold;

[0076] S3, for each QF, use the backpropagation mathematical mechanism in the artificial intelligence convolutional neural network to generate a set of optimal quantization tables for it through synchronous iterative operation, and each set of optimal quantization tables includes a luminance quantization table and a chroma quantization table; during the backpropagation iterative optimization process, all quantization parameters in the optimal quantization table are updated synchronously at each iteration until the optimization effect reaches saturation. Repeat the above iteration process until all the optimal quantization tables corresponding to the quality factor values are output;

[0077] S4, take the code rate corresponding to all QFs as the independent variable, and the quantization parameter at each position in the corresponding optimal quantization table as the dependent variable, and perform curve fitting to obtain the fitting curve of each quantization parameter;

[0078] S5, for any code rate, calculate the quantization parameter value corresponding to the code rate through the fitting curve of each quantization parameter in step S4, generate the corresponding luminance quantization table and chroma quantization table, and obtain the optimal quantization table corresponding to the code rate.

[0079] Referring to Figure 1 , the optimization method based on the JPEG image compression standard of the present embodiment specifically comprises the following steps:

[0080] Step 1, establish a special training data set called “Visualet”.

[0081] The embodiment proposes a set of image training dataset establishment method, the dataset contains six kinds of human eye survey natural image when "visual meta image" (Visualet): random probability distribution (gaussian, poisson, uniform, pepper and salt) noise image, noise image generated by high ISO, texture image, edge image, virtual blur image and smooth image. Among them, the noise image is generated by computer code, the high ISO, virtual blur and smooth image are obtained by manual cutting, and the texture and edge image are screened by algorithm after cutting. Specifically, for each cut texture image, the gray-level co-occurrence matrix (GLCM) in 0 degree, 45 degree, 90 degree and 135 degree is counted respectively, and then the average entropy value of the four matrices is calculated, when the average entropy value is greater than the set threshold value, the image is identified as a texture image and is included in the dataset. The edge image is screened by using the Canny edge detection algorithm, that is, for each candidate edge image, when the proportion of the edge pixel number detected by the Canny algorithm is greater than the set threshold value, the image is identified as an edge image and is included in the dataset.

[0082] Step 2, select a corresponding code rate point of QF.

[0083] The present application equally divides the entire code rate interval of the natural image into 100 code rate points; from low to high, respectively corresponding to QF=1 to QF=100. For each QF, the code rate generated by the optimal quantization table is as close as possible to the code rate specified by the QF (error less than 0.02 bpp).

[0084] Step 3, set the weight of the code rate term in the loss function, so that the code rate generated by the optimized quantization table is as close as possible to the code rate point corresponding to the current QF.

[0085] In the optimization process of the quantization table, the 128 quantization parameters of the quantization table are iteratively updated by back propagation through the loss function value. The optimization goal is to gradually reduce the loss function value in the iteration process until it reaches the saturation point (that is, the minimum value). Specifically, the loss function value is affected by the "code rate term" and the "distortion term": when the quantization parameter in the quantization table increases, the "code rate term" in the loss function decreases, and the "distortion term" increases; on the contrary, the "code rate term" increases, and the "distortion term" decreases.

[0086] Therefore, the change trend of the loss function value is determined by one of the main items: when the weight of the "code rate term" is larger, even if the value of the "distortion term" increases, the value of the "code rate term" decreases more, and the total loss value still decreases, at this time the optimization of the code rate in the optimization goal is dominant, and finally the quantization table of the low code rate point is optimized. On the contrary, when the weight of the "code rate term" is smaller, the quantization table of the low code rate point will be finally optimized.

[0087] From the above, setting different weights will get different quantization tables, corresponding to different code rates. The weight is artificially set, and before setting each time, the existing weight and code rate corresponding relationship is used as a priori to guide the setting of more accurate weight, so as to make the code rate of the optimized quantization table as close as possible to the target code rate.

[0088] Step 4, the input picture (Visualet) is compressed by differentiable JPEG encoding and decoding.

[0089] As shown in Figure 2 , the differentiable JPEG encoding includes: color space conversion (RGB to YCbCr) 101, chroma downsampling 102, DCT transform 103, quantization 104, differentiable approximate rounding 105, and learnable quantization table 106. The specific process is: the original picture (Visualet) is converted from the RGB color space to the YCbCr color space, and then the chroma components (Cb, Cr) are downsampled (JPEG 4:2:0) to obtain "original luminance component and downsampled chroma component". After DCT transform on the above components, DCT coefficients are obtained, and the DCT coefficients are quantized using a learnable quantization table. Then, in order to preserve the gradient required in the back propagation process (require partial derivative), the quantized DCT coefficients are subjected to differentiable approximate rounding: the quantized DCT coefficients are subjected to differentiable approximate rounding according to the following formula to obtain , wherein represents rounding operation on :

[0090] ;

[0091] The differentiable JPEG decoding includes: inverse quantization 201, and inverse DCT transform 202. The specific process is: the quantized DCT coefficients are subjected to inverse quantization through the learnable quantization table, and then subjected to inverse DCT transform to obtain "decoded luminance component and chroma component".

[0092] Step 5, calculate the total loss function value.

[0093] Figure 2 The dashed line connection part in the middle is the process of total loss function calculation. Specifically, the "original luminance component and downsampled chroma component" and the "decoded luminance component and chroma component" in step 4 are used to calculate the image quality loss, and the code rate term is calculated on the learnable quantization table. Then, the weight set in step 3 is multiplied by the code rate term, and then added to the quality loss to obtain the total loss function value.

[0094] The specific process of quality loss calculation 301 is as follows: Figure 3As shown: First, the rate of change (gradient value) around each pixel in the horizontal and vertical directions is calculated using the Sobel edge detection algorithm on the "original luminance component". The gradient values ​​are normalized and then incremented by 1 to obtain the weights for image edge guidance. The absolute error between the "original luminance component and downsampled chrominance component" and the "decoded luminance component and chrominance component" is calculated and multiplied by the aforementioned weights. Finally, the average is calculated to obtain the quality loss. The bitrate calculation in the total loss function is as follows: 302. Figure 4 As shown: The bitrate value is obtained by taking the reciprocal of all 128 quantization parameters in the learnable quantization table and then summing them.

[0095] Step 6: Has the total loss function value decreased to its minimum?

[0096] Determine if the total loss function value has decreased to a minimum; if it continues to decrease (corresponding to...). Figure 1 If the 'N' branch is selected, the quantization parameters in the quantization table are updated through backpropagation based on the total loss function value, and steps 4 and 5 are repeated before returning to step 6 for judgment. This process is repeated until the total loss function value remains constant or fluctuates slightly (corresponding to...). Figure 1 (In the 'Y' branch), the quantization table has been optimized to saturation, and we proceed to step 7.

[0097] Step 7: Obtain the optimal quantization table for the current QF.

[0098] At this point, the quantization table optimization is complete, and the default JPEG quantization table can be replaced to compress the image.

[0099] See Figure 5 The process for generating the optimal quantization table for the target bit rate is as follows:

[0100] After obtaining the optimal quantization tables for 100 QFs using the above process, the bitrates corresponding to the 100 QFs are used as the 'independent variables', and the quantization parameters at each position in the 100 optimal quantization tables corresponding to the 100 QFs are used as the 'dependent variables', and curve fitting is performed. Therefore, 64 fitting curves are obtained for both the luminance and chrominance quantization tables. For other bitrates between these 100 QFs, their luminance and chrominance quantization tables can be obtained from the curves fitted for luminance and chrominance respectively, meaning the quantization parameter values ​​(taking positive integers) in the quantization tables can be directly obtained from the dependent variables of these curves.

[0101] This invention proposes a quality assessment method that conforms to human visual perception, using a novel image quality assessment metric, "Just Perceptible Distortion" (JPD), to evaluate the image quality of compressed images. See also... Figure 6 The method includes the following steps:

[0102] S1, preset distortion threshold (JND threshold) calculation. Specifically includes:

[0103] S11, calculate the original image each pixel In The background uniformity of the local area , and The maximum contrast difference in four directions (0 degrees, 45 degrees, 90 degrees, 135 degrees) in the local area . The specific calculation formula is as follows:

[0104] ;

[0105] ;

[0106] Where, i=1,2,3,4,5, j=1,2,3,4,5, max is the maximum value function, the operator matrix And As Figure 8 Indicated.

[0107] S12, calculate the preset distortion threshold of each pixel using the JND theory formula :

[0108] ;

[0109] Where the formula Specifically:

[0110] ;

[0111] Where the formula Specifically:

[0112] ;

[0113] Where The function value , .

[0114] S2, absolute value error calculation, that is, calculate the absolute value error (Absolute Error, AE) of each pixel between the distorted picture and the original picture.

[0115] S3, JND-based mask (Mask) matrix calculation. Specifically includes:

[0116] S31, initialize a Matrix with the same size as the original picture.

[0117] ​S32, comparing the AE of each pixel with its preset distortion threshold value: if the AE of the pixel is greater than its preset distortion threshold value, then The corresponding position is set to 1 (True); otherwise, The value of the corresponding position is set to 0 (False).

[0118] S4, calculating the image quality evaluation index JPD. Specifically, it includes:

[0119] S41, multiplying the AE obtained in step S2 and the AE obtained in step S3 Point multiplication, so that the pixel distortion of those pixels less than the preset distortion threshold value is removed, and the image quality evaluation index JPD is obtained The Peak Absolute Error (PAE) that can be perceived is calculated by the following formula:

[0120] ;

[0121] Wherein is the peak error value that can be reached by a single pixel on the image.

[0122] S42, calculating the average error of the PAE obtained in step S41 on the whole image, that is, taking the number of pixels of the whole image as the divisor, to obtain the image quality evaluation index JPD.

[0123] The present application provides a quality evaluation method for JPEG compressed image processing, which conforms to human eye perception. The method is based on a new image quality evaluation index: "just perceptible distortion and block artifacts" (JPD-B) for low code rate compressed image quality evaluation. Referring to Figure 7 , the method specifically includes the following steps:

[0124] B1, preset distortion threshold value (JND threshold value) calculation;

[0125] B2, absolute value error (AE) calculation;

[0126] B3, JND-based mask ( ) matrix calculation.

[0127] The above steps B1, B2, B3 are the same as steps S1, S2, S3 in the image quality evaluation method based on JPD.

[0128] B4, screening of perceptible block artifact pixels caused by compression. Specifically, it includes:

[0129] B41, calculating the perceptible block effect mask matrix ( ) of the distorted image:

[0130] The distorted image For pixels at the edge of a small block, calculate the change in value between its left, right, top, and bottom adjacent pixels. If this change value is greater than a preset distortion threshold, the pixel is considered to have a blocky effect perceptible to the human eye. In other words, if the change value at this pixel position is greater than the threshold, The corresponding logical value is set to 1 (True), otherwise, The corresponding logical value is set to 0 (False).

[0131] B42, the block structure mask matrix in the original image ( )calculate:

[0132] Perform the same operation as B41 on the original image to obtain the matrix. The result is the block structure that was already present in the original image.

[0133] B43, Perceptible blocky artifact mask matrix caused by compression ( )calculate:

[0134] The matrix obtained in step B41 It contains both blocky artifact pixels caused by compression and the blocky structure of the image itself; therefore, only pixels with blocky artifacts caused by compression need to be filtered. Specifically, B41... With B42 Performing an "XOR" operation: This means that the logical values ​​at corresponding positions in the two matrices are different. The corresponding logical value is set to 1 (True); otherwise, The corresponding logical value is set to 0 (False). This step completes the filtering of perceptible blocky artifact pixels caused by compression.

[0135] B5, calculate JPD-B. Specifically, this includes:

[0136] B51, the "just noticeable difference" obtained in step B3. Compared to the "perceptible blocky artifacts caused by compression" obtained in step B4 Perform a merge (take the union) to obtain , matrix Perform a dot product with the AE from step 2, so that all AE values ​​with just noticeable differences and block artifacts are accumulated (i.e., Finally, the peak absolute value error is calculated using the following formula:

[0137] ;

[0138] in This represents the peak error value that a single pixel in the image can achieve.

[0139] B52, find the result of B51 The average error on the whole image, i.e. divided by the number of pixels of the whole image, obtains the image quality evaluation index JPD-B.

[0140] Embodiment 2

[0141] This embodiment constructs a JPEG optimization method for medical images, which is different from embodiment 1 in that different data sets are used to optimize JPEG, that is, when constructing the Visualet data set, medical images are used instead of natural images. The medical images that can be used include but are not limited to X-ray images, CT (Computed Tomography) images, MRI (Magnetic Resonance Imaging) and Ultrasound (Ultrasound), and specific image data can be selected according to requirements in application.

[0142] Embodiment 3

[0143] This embodiment constructs a JPEG optimization method for remote sensing images, which is different from embodiment 1 in that different data sets are used to optimize JPEG, that is, when constructing the Visualet data set, remote sensing images are used instead of natural images. The remote sensing images that can be used include but are not limited to unmanned aerial vehicle remote sensing images, aerial remote sensing images, satellite remote sensing images, and specific image data can be selected according to requirements in application.

[0144] Embodiment 4

[0145] This embodiment constructs an optimization algorithm for MPEG (Moving Picture Experts Group) video encoder, which is a further extension based on JPEG: MPEG not only supports the compression of single frame images (JPEG), but also supports the compression of video sequences (multiple consecutive images). In order to ensure high compression efficiency on video sequences, MPEG introduces the inter-frame motion prediction and motion compensation algorithm mechanism based on JPEG, so as to obtain the DCT coefficient residual of adjacent two frames, and then compress the DCT coefficient residual using a new "residual quantization table". Therefore, this embodiment optimizes this "residual quantization table".

[0146] Specifically, first, the DCT residual between two frames is obtained using the MPEG encoder on a multi-frame video sequence, and this residual is used to construct a training data set; subsequently, the same optimization steps as in embodiment 1 are used to iteratively optimize the "residual quantization table" on the new data set; and finally, the optimized quantization table is specific to MPEG and is used to quantize the DCT residual coefficients generated between frames, thereby improving the MPEG compression performance (rate-distortion performance).

[0147] While the preferred embodiments of the application have been described, additional variations and modifications can be made to these embodiments by those skilled in the art once they have the benefit of the present disclosure without departing from the spirit and scope of the application. Accordingly, it is intended that such additions and modifications be included within the scope of the application. It is the following claims, including any amendments thereto, which define the scope of the application.

[0148] Obviously, numerous modifications and variations of the present application are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. An optimization method based on the JPEG image compression standard, characterized in that, The method comprises the following steps: S1, equally dividing the entire code rate interval of a natural image into 100 code rate points, which correspond to 100 quality index values from low to high respectively; S2, setting a total loss function, the total loss function value being the sum of an image quality loss and a weighted code rate item; wherein the code rate item weight is adjusted manually, so that the error between the code rate of the optimized learnable quantization table and the code rate specified by the quality index value is less than a preset error threshold; S3, for each quality index value, generating a set of optimal quantization tables on a JPEG framework through a synchronous iterative operation mode by using the back propagation mathematical mechanism in an artificial intelligence convolutional neural network, each set of optimal quantization tables comprising a luminance quantization table and a chroma quantization table; in the back propagation iterative optimization process, all quantization parameters in the optimal quantization table are updated synchronously each time until the optimization effect reaches saturation; repeating the foregoing iterative process until the optimal quantization tables corresponding to all quality index values are output; S4, performing curve fitting to obtain a fitting curve of each quantization parameter, taking the code rate corresponding to all quality index values as the independent variable and the quantization parameter at each position in the corresponding optimal quantization table as the dependent variable; S5, for any code rate, calculating the quantization parameter value corresponding to the code rate through the fitting curve of each quantization parameter in step S4, generating the corresponding luminance quantization table and chroma quantization table, and obtaining the optimal quantization table corresponding to the code rate.

2. The method for optimizing based on JPEG image compression standard according to claim 1, characterized in that, In step S2, the calculation process of the total loss function value comprises: Converting the original picture from the RGB color space to the YCbCr color space, and then down-sampling the chroma component to obtain the original luminance component and the down-sampled chroma component; After DCT transformation of the original luminance component and the down-sampled chroma component, obtaining DCT coefficients, and quantizing the DCT coefficients using a learnable quantization table; After differentiable approximate rounding of the quantized DCT coefficients, performing inverse quantization through the learnable quantization table, and performing inverse DCT transformation on the inverse quantization result to obtain decoded luminance and chroma components; Calculating the image quality loss by combining the original luminance component and the down-sampled chroma component, and the decoded luminance and chroma components; taking the reciprocal of all quantization parameters in the learnable quantization table, summing them up to obtain the code rate item, multiplying it by the code rate item weight to obtain the weighted code rate item; summing the image quality loss and the weighted code rate item to obtain the total loss function value.

3. The method for optimizing based on JPEG image compression standard according to claim 2, characterized in that, The process of calculating the image quality loss by combining the original luminance component and the down-sampled chroma component, and the decoded luminance and chroma components comprises the following steps: Processing the original luminance component by using the Sobel edge detection algorithm to calculate the gradient value of each pixel in the horizontal and vertical directions, normalizing the gradient value, and then adding 1 to obtain the weight guided by the image edge; The first absolute value error between the original luminance component and the decoded luminance component and the second absolute value error between the down-sampled chroma component and the decoded chroma component are calculated, the first absolute value error of the luminance component is multiplied by the weight guided by the image edge, and then added to the second absolute value error of the chroma component, and finally averaged to obtain the image quality loss.

4. The method for optimizing based on JPEG image compression standard according to claim 1, characterized in that, In step S3, the training data set used by the learnable quantization table includes six basic element images: noise images with random probability distribution, noise images generated by high ISO, texture images, edge images, blurred images and smooth images.

5. The method for optimizing based on JPEG image compression standard according to claim 4, characterized in that, The construction process of the training data set includes: Noise images with various random probability distributions including Gaussian, Poisson, uniform and salt and pepper are generated by computer code; Noise images generated by high ISO, blurred images and smooth images are obtained by manual cutting; Initial texture images are obtained by cutting, for each initial texture image, the co-occurrence matrix in four directions of 0 degrees, 45 degrees, 90 degrees and 135 degrees is respectively calculated, and the average entropy value of the four co-occurrence matrices is calculated, and the initial texture image with an average entropy value greater than a set entropy threshold value is selected as the final texture image included in the training data set; Initial edge images are obtained by cutting, and the initial edge image with a proportion of all edge pixels greater than a set pixel threshold value is selected as the final edge image included in the training data set by using the Canny edge detection algorithm; The number and type of the six basic perception element images in the training data set are adjusted so that the number of the six basic perception element images tends to be consistent, and the data set tends to cover the entire code rate interval of natural images, that is, the pixel depth bpp is in [0, 12], wherein the pixel depth bpp of the luminance component is in [0, 8], and the pixel depth bpp of the two chroma components is in [0, 2].

Images