Optimization method based on JPEG (Joint Photographic Experts Group) image compression standard and image quality evaluation method

Abstract

The invention discloses an optimization method based on a JPEG (Joint Photographic Experts Group) image compression standard and an image quality evaluation method. The method comprises the following steps: dividing a whole code rate interval into 100 code rate points at equal intervals so as to correspond to 100 quality index values (QF) of the JPEG; for each QF, generating a group of optimal quantization tables by adopting a back propagation mechanism; for other code rate points, an optimal quantization table is obtained based on curve fitting; edge weighting is considered in loss function calculation in iteration; in addition, a new picture quality evaluation index, namely JPD (just noticeable distortion), is provided based on just noticeable difference of human eyes, and accumulation is carried out only for pixels with the pixel distortion larger than a preset distortion degree threshold value; and further aiming at a unique block effect which is commonly presented by the JPEG during low-bit-rate compression, the block effect is considered into distortion accumulation of the JPD, and the block effect is further expanded into JPD-B.

Description

Technical Field

[0001] The present invention relates to the technical field of image compression, and in particular to an optimization method and an image quality evaluation method based on the JPEG image compression standard. Background Art

[0002] JPEG is an international standard for natural color image compression. Its purpose is to reduce the file size generated by digital cameras while maintaining good image quality. JPEG uses discrete cosine transform (DCT) to convert images from the spatial domain (pixel values) to the frequency domain (DCT coefficients) in 8×8 blocks. JPEG then uses the JPEG quantization table for quantization (compression). Specifically, each number (quantization parameter) in the JPEG quantization table is used as a divisor for the DCT coefficient at the corresponding position in the block, rounded to the nearest integer. This reduces the dynamic range of the DCT coefficients and reduces many high-frequency DCT coefficients to zero, thus achieving data compression.

[0003] Although JPEG specifies 100 Quality Factor (QF) values, ranging from 1 (lowest quality) to 100 (highest quality), it only provides two default quantization tables for QF = 50: one for luma compression and one for chroma compression. For quantization tables corresponding to other QF values, JPEG only provides a simple linear conversion formula designed based on the QF = 50 quantization table set. Clearly, for the ever-changing natural color images, this extremely simplified quantization method produces unsatisfactory image quality and difficult to control bitrate. Consequently, a significant amount of research has attempted to develop custom "optimal" quantization tables to improve image compression quality and efficiency.

[0004] Currently known optimization methods can be broadly categorized into traditional methods and deep learning-based methods. Key traditional methods include heuristic algorithms, dynamic programming, greedy search, and gradient descent. However, these methods suffer from the high complexity of the iterative optimization process, limited improvement in compressed image quality, and difficulty controlling the bitrate. Deep learning methods generate quantization tables based on image content using a convolutional neural network (CNN). Compared to traditional methods, these methods can provide better compressed image quality. However, the CNNs used in these methods require limited input image size and can potentially require longer image compression times. Furthermore, selecting images to construct an effective image training dataset to ensure that the bitrates generated by all 100 quality indicators (QFs) cover the entire image bitrate range and are distributed as evenly as possible is a challenge.

[0005] Finally, the most widely used metric for evaluating compressed image quality is the Peak Signal-to-Noise Ratio (PSNR). PSNR is measured by calculating the pixel-by-pixel mean square error (MSE) between the compressed and original images, serving as a quality metric for compressed images. However, the human eye's perception of image distortion is often specific to specific areas, objects, or even points, rather than simply judging image quality based on the average error across the entire photo. Therefore, PSNR often does not reflect the human eye's subjective perception of image quality. This means that even an image with a high PSNR may be perceived as poor quality by the human eye.

[0006] To address the discrepancy between human perception and objective measurements, Structural Similarity (SSIM) is another widely used quality assessment metric. It's more consistent with human perception than PSNR. However, SSIM values ​​can still be inconsistent with human perception. This is because SSIM performs the same calculation on all pixels in the entire image. This means that errors caused by pixels whose distortion is imperceptible to the human eye are also included in the SSIM calculation, leading to misjudgments. Summary of the Invention

[0007] In response to the aforementioned technical problems, the present invention proposes an optimization method and an image quality assessment method based on the JPEG image compression standard, which can quickly propose a set of optimal quantization tables for any bit rate, and proposes a new image quality assessment formula based on the "Just Noticeable Difference" (JND) characteristics of the human eye and the Peak Absolute Error (PAE) presented in the image connotation.

[0008] In order to achieve the above technical objectives, the technical solution adopted by the present invention is: In a first aspect, the present invention discloses an optimization method based on the JPEG image compression standard, the method comprising the following steps:

[0009] S1, the entire bitrate range of natural images is divided into 100 bitrate points at equal intervals, corresponding to 100 quality index values ​​from low to high;

[0010] S2, setting a total loss function, where the total loss function value is the sum of the image quality loss and the weighted bit rate term; wherein the bit rate term weight is manually adjusted so that the error between the bit rate of the optimized learnable quantization table and the bit rate specified by the quality index value is less than a preset error threshold;

[0011] S3: For each quality index value, a set of optimal quantization tables is generated through synchronous iterative operations using the back-propagation mathematical mechanism in the artificial intelligence convolutional neural network. Each set of optimal quantization tables includes a luminance quantization table and a chrominance quantization table. During the back-propagation iterative optimization process, all quantization parameters in the optimal quantization table are synchronously updated at each iteration until the optimization effect reaches saturation. The aforementioned iterative process is repeated until the optimal quantization tables corresponding to all quality index values ​​are output.

[0012] S4, using the bit rates corresponding to all quality index values ​​as independent variables and the quantization parameters at each position in the corresponding optimal quantization table as dependent variables, and performing curve fitting to obtain a fitting curve for each quantization parameter;

[0013] S5, for any bit rate, calculate the quantization parameter value corresponding to the bit rate through the fitting curve of each quantization parameter in step S4, generate the corresponding luminance quantization table and chrominance quantization table, and obtain the optimal quantization table corresponding to the bit rate.

[0014] Furthermore, in step S2, the calculation process of the total loss function value includes:

[0015] Convert the original image from RGB to YCbCr, and then downsample the chrominance components to obtain the original luminance component and the downsampled chrominance component.

[0016] After DCT transformation of the original luminance component and the downsampled chrominance component, the DCT coefficients are obtained and quantized using a learnable quantization table;

[0017] After performing differentiable approximate rounding on the quantized DCT coefficients, dequantization is performed using a learnable quantization table, and the dequantized results are subjected to inverse DCT transformation to obtain the decoded luminance and chrominance components.

[0018] The image quality loss is calculated by combining the original luminance component and the downsampled chrominance component, as well as the decoded luminance component and chrominance component. The reciprocal of all quantization parameters in the learnable quantization table is taken and summed to obtain the bit rate term, which is then multiplied by the bit rate term weight to obtain the weighted bit rate term. The image quality loss and the weighted bit rate term are summed to obtain the total loss function value.

[0019] Furthermore, the process of calculating the image quality loss by combining the original luminance component and the downsampled chrominance component, and the decoded luminance component and chrominance component comprises the following steps:

[0020] The original brightness component is processed by the Sobel edge detection algorithm to calculate the gradient value in the horizontal and vertical directions around each pixel. The gradient value is normalized and then added with the number 1 to obtain the weight of the image edge guidance.

[0021] Calculate the first absolute value error between the original luminance component and the decoded luminance component, and the second absolute value error between the downsampled chrominance component and the decoded chrominance component. Multiply the first absolute value error of the luminance component by the image edge-guided weight, add it to the second absolute value error of the chrominance component, and finally average it to obtain the image quality loss.

[0022] Furthermore, in step S3, the training data set used by the learnable quantization table includes six basic perceptual element images: noise images with random probability distribution, noise images generated by high ISO, texture images, edge images, blurred images, and smooth images.

[0023] Furthermore, the process of generating the training data set includes:

[0024] Generate noise images with various random probability distributions including Gaussian, Poisson, uniform, and salt and pepper through computer code;

[0025] The noisy image, blurred image and smooth image caused by high ISO were obtained by manual cropping;

[0026] The initial texture image is obtained by cropping. For each initial texture image, its co-occurrence matrix in the four directions of 0 degrees, 45 degrees, 90 degrees and 135 degrees is counted respectively. Then, the average entropy value of the four co-occurrence matrices is calculated. The initial texture image with an average entropy value greater than the set entropy value threshold is selected as the final texture image included in the training dataset;

[0027] The initial edge image is obtained by cropping, and the Canny edge detection algorithm is used to filter out all the initial edge images whose edge pixel ratio is greater than the set pixel number threshold, which are used as the edge images finally included in the training data set;

[0028] Adjust the number and type of the six basic perceptual element images in the training data set so that the number of the six basic perceptual element images is close to the same, and the dataset tends to cover the entire bitrate range of natural images, that is, pixel depth , where the pixel depth of the luminance component is , the pixel depth of the two chroma components .

[0029] In a second aspect, the present invention discloses a method for image quality assessment, comprising the following steps:

[0030] Calculate the absolute value error of the distorted image that can be perceived by the human eye:

[0031] Initialize a matrix with the same size as the original image , compare the absolute value error of each pixel with its corresponding preset distortion threshold. If the absolute value error of any pixel is greater than its corresponding preset distortion threshold, the matrix The value of the position corresponding to the pixel in the matrix is ​​set to 1; otherwise, the ... The value of the corresponding position is set to 0;

[0032] The absolute value error between the distorted image and the original image at each pixel is compared with the matrix Perform dot multiplication to remove pixel distortion whose absolute value error is less than the preset distortion threshold, and obtain ; Then calculate the perceptible peak absolute value error of the distorted image by summing the following formula :

[0033] ;

[0034] in is the peak error value that a single pixel on the distorted image can reach;

[0035] Using the number of pixels in the entire image as the divisor, calculate The average error over the entire image is used to obtain the image quality evaluation index JPD.

[0036] Furthermore, the calculation process of the preset distortion threshold includes the following steps:

[0037] Calculate each pixel of the original image exist Uniform background brightness in a local area ,as well as The maximum contrast difference in the local area at 0 degrees, 45 degrees, 90 degrees, and 135 degrees ; The specific calculation formula is as follows:

[0038] ;

[0039] ;

[0040] Among them, i=1,2,3,4,5, j=1,2,3,4,5, and is the operator matrix for calculating the difference between the background uniform brightness and the maximum contrast, and max is the maximum value function;

[0041] The preset distortion threshold for each pixel is calculated using the following formula: :

[0042] ;

[0043] The formula Specifically:

[0044] ;

[0045] The formula Specifically:

[0046] ;

[0047] Among them, the parameter value in the function , .

[0048] Furthermore, the method is used for quality assessment of JPEG compressed images, comprising the following steps:

[0049] Calculate the absolute value error of the distorted image that can be perceived by the human eye and the block artifacts:

[0050] Initialize a matrix with the same size as the original image , compare the absolute value error of each pixel with its corresponding preset distortion threshold. If the absolute value error of any pixel is greater than its corresponding preset distortion threshold, the matrix The value of the position corresponding to the pixel in the matrix is ​​set to 1, otherwise, The value of the corresponding position is set to 0;

[0051] Initialize a matrix with the same size as the distorted image , for distorted images The edge position pixel of the small block is calculated for the change value of the adjacent pixels on the left or above and below. If the change value is greater than the preset distortion threshold, the pixel is judged to have a block effect that can be perceived by the human eye, and the matrix The corresponding position logic value is set to 1, otherwise, the matrix The logical value of the corresponding position is set to 0;

[0052] Initialize a matrix with the same size as the original image , for the original image The edge position pixel of the small block is calculated for the change value of the adjacent pixels on the left or above and below. If the change value is greater than the preset distortion threshold, the pixel is judged to have a block effect that can be perceived by the human eye, and the matrix The corresponding position logic value is set to 1, otherwise, the matrix The logical value of the corresponding position is set to 0;

[0053] The matrix With the matrix Perform XOR operation, if the matrix With the matrix If the logical values ​​at the corresponding positions are different, the logical values ​​at the corresponding positions of the two matrices are set to 1; otherwise, the logical values ​​at the corresponding positions of the two matrices are set to 0, and the matrix is ​​obtained. , filtering out pixels with perceptible block artifacts caused by compression;

[0054] The matrix With the matrix Take the union and get the matrix ; Then the matrix The absolute value error of each pixel between the distorted image and the original image is multiplied by The cumulative peak absolute value error of the human eye and the block artifacts is calculated by the following formula: :

[0055] ;

[0056] in is the peak error value that a single pixel on the distorted image can reach;

[0057] Using the number of pixels in the entire image as the divisor, calculate The average error over the entire image is used to obtain the image quality evaluation index JPD-B.

[0058] Compared with the prior art, the application has the following advantages:

[0059] First, the optimization method based on the JPEG image compression standard generates a set of optimal quantization tables for each of the 100 quality factors (QF) specified by JPEG, including a luminance quantization table and a chrominance quantization table, each with 64 quantization parameters. These 128 parameters are optimized simultaneously in each iteration. The resulting quantization tables are used to quantize (i.e., compress) the discrete cosine transform (DCT) coefficients obtained after the frequency domain conversion of the image, achieving the goal of image compression and obtaining the code rate at the specified QF. Since the code rate point can be any value within the entire code rate range, the parameter values at each position in the corresponding quantization table for the code rate points distributed between the 100 code rate points can be directly obtained from the nonlinear relationship curve fitted using the code rate values (independent variables) and the corresponding 100 sets of quantization tables (dependent variables) for the 100 QFs. In other words, the optimal quantization tables can be directly read from the corresponding dependent variables in the 64 luminance curves and 64 chrominance curves without any additional calculations.

[0060] Second, the optimization method based on the JPEG image compression standard uses a specially designed loss function, which is the sum of a distortion term and a weighted code rate term. The distortion term calculates the absolute value difference between each pixel in the compressed image and the original image, and further assigns higher weights to the distortion at the edge pixel positions detected in the image. The loss function value obtained above is optimized simultaneously with the 128 parameters of the quantization table using back propagation (BP) gradient descent.

[0061] Third, the optimization method based on the JPEG image compression standard proposes a set of data set establishment methods. The established data set contains six types of "visual perception element maps" (Visualet). This data set not only conforms to the perception and judgment of the human eye when evaluating natural images, but also effectively helps the code rate points generated by the 100 QFs to be evenly distributed in the entire code rate range of natural color images, thereby achieving better image compression efficiency and quality.

[0062] Fourth, the present invention proposes an image quality assessment method based on the human eye's Just Noticeable Difference (JND) characteristic and the Peak Absolute Error (PAE) characteristic of image quality. This method utilizes a novel image quality assessment metric, Just Perceptible Distortion (JPD), which is based on the human eye's Just Noticeable Difference (JND) characteristic and the image's inherent peak absolute error (PAE). This JPD metric only accumulates the distortion generated by pixels with distortion greater than a set threshold. This resulting quality assessment more accurately reflects the human eye's true perception of distorted images (compared to the original image). The JPD metric is applicable to quality assessment after all image processing, including both compression and non-compression processes, such as super-resolution, interpolation, and denoising. In addition, JPEG often exhibits a unique blocking effect when compressed at low bit rates. To reflect this distortion, obvious block edge pixels are also included in the cumulative distortion calculation of the image quality assessment indicator JPD. The image quality assessment indicator is just perceptible distortion and blocking artifacts JPD-B. JPD-B is used to evaluate the image quality of low-bitrate compressed images and is represented by the characters JPD-B to distinguish it from JPD (the "B" comes from the first letter of Block, representing "blocky"). BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 This is a flow chart of the optimization method based on the JPEG image compression standard of the present invention;

[0064] Figure 2 This is a flow chart of a differentiable JPEG quantization table optimization algorithm based on back propagation in an embodiment;

[0065] Figure 3 This is a flow chart of mass loss calculation in the embodiment;

[0066] Figure 4 This is a flow chart of bit rate calculation in the embodiment;

[0067] Figure 5 A flow chart for generating an optimal quantization table for a target bit rate point in an embodiment;

[0068] Figure 6 A flowchart for calculating the image quality assessment method based on the image quality assessment metric "Just Perceptible Distortion" (JPD);

[0069] Figure 7 This is a calculation flow chart for the image quality assessment method based on the image quality assessment metric "Just Perceptible Distortion and Blockiness" (JPD-B);

[0070] Figure 8 is the operator matrix and Schematic diagram of . DETAILED DESCRIPTION

[0071] The embodiments of the present invention are described in further detail below with reference to the accompanying drawings. Example 1

[0072] This embodiment discloses an optimization method based on the JPEG image compression standard, the optimization method comprising the following steps:

[0073] S1, divide the entire bitrate range of natural images into 100 bitrate points at equal intervals, corresponding to 100 quality factor values ​​(QF) from low to high;

[0074] S2, setting a total loss function, where the total loss function value is the sum of the image quality loss and the weighted bit rate term; wherein the bit rate term weight is manually adjusted so that the error between the bit rate of the optimized learnable quantization table and the bit rate specified by the quality index value is less than a preset error threshold;

[0075] S3: For each QF, we use the back-propagation mathematical mechanism in artificial intelligence convolutional neural networks to generate a set of optimal quantization tables through synchronous iterative operations. Each set of optimal quantization tables includes a luminance quantization table and a chrominance quantization table. During the back-propagation iterative optimization process, all quantization parameters in the optimal quantization table are synchronously updated at each iteration until the optimization effect reaches saturation. This iterative process is repeated until the optimal quantization tables corresponding to all quality index values ​​are output.

[0076] S4, taking the bit rates corresponding to all QFs as independent variables and the quantization parameters at each position in the corresponding optimal quantization table as dependent variables, and performing curve fitting to obtain a fitting curve for each quantization parameter;

[0077] S5, for any bit rate, calculate the quantization parameter value corresponding to the bit rate through the fitting curve of each quantization parameter in step S4, generate the corresponding luminance quantization table and chrominance quantization table, and obtain the optimal quantization table corresponding to the bit rate.

[0078] See also Figure 1 The optimization method based on the JPEG image compression standard of this embodiment specifically includes the following steps:

[0079] Step 1: Create a special training dataset called "Visual Meta-Image" (Visualet).

[0080] This embodiment proposes a method for establishing an image training dataset. The dataset contains six types of "visual meta-images" (Visualet) when the human eye examines natural images: noise images with random probability distribution (Gaussian, Poisson, uniform, and salt and pepper), noise images generated by high ISO, texture images, edge images, blurred images, and smooth images. The noise images are generated by computer code, the high ISO, blurred, and smooth images are obtained by manual cropping, and the texture and edge images are screened by an algorithm after cropping. Specifically, for each cropped texture image, we calculate its gray-level co-occurrence matrix (GLCM) in the four directions of 0 degrees, 45 degrees, 90 degrees, and 135 degrees, and then calculate the average entropy of these four matrices. When the average entropy value is greater than the set threshold, the image is identified as a texture image and included in the dataset. Edge images are screened using the Canny edge detection algorithm. That is, for each candidate edge image, when the proportion of edge pixels detected by the Canny algorithm is greater than the set threshold, the image is identified as an edge image and included in the dataset.

[0081] Step 2: Select a corresponding bit rate point of QF.

[0082] This paper divides the entire bitrate range of natural images into 100 equally spaced bitrate points, corresponding to QF = 1 to QF = 100, from low to high. For each QF, the optimal quantization table generates a bitrate as close as possible to the bitrate specified by that QF (with an error of less than 0.02 bpp).

[0083] Step 3: Set the weight of the bit rate term in the loss function so that the bit rate generated by the optimized quantization table is as close as possible to the bit rate point corresponding to the current QF.

[0084] During quantization table optimization, backpropagation iteratively updates the table's 128 quantization parameters using the loss function. The optimization goal is to gradually reduce the loss function value during the iteration process until it reaches saturation (i.e., its minimum value). Specifically, the loss function value is influenced by both the "bitrate term" and the "distortion term": as the quantization parameters in the quantization table increase, the "bitrate term" in the loss function decreases while the "distortion term" increases; conversely, the "bitrate term" increases while the "distortion term" decreases.

[0085] Therefore, the trend of the loss function value is determined by the dominant term: when the weight of the "bitrate term" is large, even if the value of the "distortion term" increases, the "bitrate term" decreases more, and the total loss value still decreases. In this case, the bitrate optimization in the optimization objective is dominant, and the quantization table of the low bitrate point is ultimately optimized. Conversely, when the weight of the "bitrate term" is small, the quantization table of the low bitrate point is ultimately optimized.

[0086] As mentioned above, setting different weights will result in different quantization tables, corresponding to different bitrates. These weights are manually set, and the existing relationship between weights and bitrates is used as a priori to guide the setting of more accurate weights, thereby ensuring that the bitrate of the optimized quantization table is as close to the target bitrate as possible.

[0087] Step 4: The input image (Visualet) is compressed by differentiable JPEG encoding and decoding.

[0088] like Figure 2 As shown, the differentiable JPEG encoding includes: color space conversion (RGB to YCbCr) 101, chroma downsampling 102, DCT transformation 103, quantization 104, differentiable approximate rounding 105, and learnable quantization table 106. Specific process: The original image (Visualet) is converted from the RGB color space to the YCbCr color space, and then the chroma components (Cb, Cr) are downsampled (JPEG 4:2:0) to obtain "original luminance components and downsampled chroma components". After performing DCT transformation on the above components, DCT coefficients are obtained, and the DCT coefficients are quantized using a learnable quantization table. Subsequently, in order to retain the gradient required in the back-propagation process (partial derivatives are required to obtain), the quantized DCT coefficients are differentiable approximate rounding: the quantized DCT coefficients According to the following formula, the approximate rounding of the differentiable ,in Express Perform rounding operations:

[0089] ;

[0090] Differentiable JPEG decoding includes: inverse quantization 201 and inverse DCT transformation 202. The specific process is: the quantized DCT coefficients are inverse quantized using a learnable quantization table, and then inverse DCT transformation is performed to obtain "decoded luminance components and chrominance components."

[0091] Step 5: Calculate the total loss function value.

[0092] Figure 2 The dotted line in the middle represents the total loss function calculation process. Specifically, the image quality loss is calculated using the "original luminance component and downsampled chrominance components" and the "decoded luminance component and chrominance components" from step 4. The bitrate term is then calculated using the learnable quantization table. The bitrate term is then multiplied by the weight set in step 3 and added to the quality loss to obtain the total loss function value.

[0093] The specific process of quality loss calculation 301 is as follows Figure 3As shown below: First, the Sobel edge detection algorithm is used on the original luminance component to calculate the rate of change (gradient value) in the horizontal and vertical directions around each pixel. The gradient value is normalized and then added with 1 to obtain the weight for image edge guidance. After calculating the absolute value error between the original luminance component and the downsampled chrominance component and the decoded luminance component and chrominance component, it is multiplied by the above weights and finally the average is calculated to obtain the quality loss. The bit rate calculation 302 in the total loss function is as follows. Figure 4 As shown in the figure: take the reciprocal of all 128 quantization parameters in the learnable quantization table and sum them up to get the bit rate value.

[0094] Step 6: Whether the total loss function value drops to the minimum.

[0095] Determine whether the total loss function value has dropped to the minimum. If it continues to decrease (corresponding to Figure 1 The 'N' branch in the quantization table is updated by back propagation based on the total loss function value, and steps 4 and 5 are repeated to return to step 6 for judgment. The above process is repeated until the total loss function value remains unchanged or fluctuates slightly (corresponding to Figure 1 At this time, the quantization table has been optimized to saturation, and the process goes to step 7.

[0096] Step 7: Get the optimal quantization table of the current QF.

[0097] At this point, the quantization table optimization is complete, and the JPEG default quantization table can be replaced to compress the image.

[0098] See also Figure 5 , the process of generating the optimal quantization table for the target bit rate point is as follows:

[0099] After obtaining the optimal quantization tables for 100 QFs using the above process, curve fitting is performed using the bitrates corresponding to the 100 QFs as the "independent variable" and the quantization parameters at each position in the 100 optimal quantization tables corresponding to the 100 QFs as the "dependent variable." This results in 64 fitted curves for each luma and chroma quantization table. For other bitrates between these 100 QFs, their luma and chroma quantization tables can be derived using the previously fitted curves for luma and chroma, respectively. In other words, the quantization parameter values ​​(positive integers) in the quantization tables can be directly derived from the dependent variables of these curves.

[0100] This paper proposes a quality assessment method that is consistent with human perception and uses a new image quality assessment metric, "Just Perceptible Distortion" (JPD), to evaluate the image quality of compressed images. Figure 6 , the method comprises the following steps:

[0101] S1, calculate the preset distortion threshold (JND threshold). Specifically including:

[0102] S11, calculate each pixel of the original image exist Uniform background brightness in a local area ,as well as The maximum contrast difference in the local area in four directions (0 degrees, 45 degrees, 90 degrees, 135 degrees) The specific calculation formula is as follows:

[0103] ;

[0104] ;

[0105] Among them, i=1,2,3,4,5, j=1,2,3,4,5, max is the maximum value function, and the operator matrix required in the above calculation process is and like Figure 8 shown.

[0106] S12, calculate the preset distortion threshold of each pixel using the JND theoretical formula :

[0107] ;

[0108] The formula Specifically:

[0109] ;

[0110] The formula Specifically:

[0111] ;

[0112] in The function value in , .

[0113] S2, absolute error calculation, that is, calculating the absolute error (AE) of each pixel between the distorted image and the original image.

[0114] S3, JND-based mask ( ) Matrix calculations. Specifically including:

[0115] S31, initialize a picture with the same size as the original picture matrix.

[0116] S32, compare the AE of each pixel with its preset distortion threshold: if the AE of the pixel is greater than its preset distortion threshold, then The corresponding position is set to 1 (True); otherwise, The value of the corresponding position is set to 0 (False).

[0117] S4, calculate the image quality evaluation index JPD. Specifically including:

[0118] S41, the AE obtained in step S2 and the Therefore, the pixel distortions whose AE is less than the preset distortion threshold will be removed, and we get , and then calculate the perceived Peak Absolute Error (PAE) using the following formula:

[0119] ;

[0120] in is the peak error value that a single pixel in the image can reach.

[0121] S42 , calculating the average error of the PAE obtained in step S41 over the entire image, that is, using the number of pixels in the entire image as a divisor to obtain the image quality evaluation index JPD.

[0122] This paper proposes a quality assessment method for JPEG compressed images that is consistent with human perception. This method uses a new image quality evaluation metric: "Just Perceptible Distortion and Blocking Artifacts" (JPD-B) to evaluate the image quality of low-bitrate compressed images. Figure 7 , the method specifically comprises the following steps:

[0123] B1, calculation of preset distortion threshold (JND threshold);

[0124] B2, absolute error (AE) calculation;

[0125] B3, JND-based mask ( ) matrix calculations.

[0126] The above steps B1, B2, and B3 are the same as steps S1, S2, and S3 in the JPD-based image quality assessment method.

[0127] B4, filtering of perceptible blocky artifact pixels caused by compression. Specifically includes:

[0128] B41, mask matrix of perceptible blockiness in distorted images ( )calculate:

[0129] For distorted images For the edge pixel of a small block, calculate the change value of the adjacent pixels on the left or above and below it. If the change value is greater than the preset distortion threshold, the pixel is judged to have a block effect that is perceptible to the human eye. In other words, if the change value of this pixel position is greater than the threshold, The corresponding position logic value is set to 1 (True), otherwise, The logical value of the corresponding position is set to 0 (False).

[0130] B42, the block structure mask matrix in the original image ( )calculate:

[0131] Perform the same operation as B41 on the original image to obtain the matrix , the result is the block structure contained in the original image.

[0132] B43, mask matrix for perceptual blocking artifacts caused by compression ( )calculate:

[0133] Step B41: Matrix It also includes block artifact pixels caused by compression and the block structure of the image itself, so only the block artifact pixels caused by compression need to be filtered. With B42 Perform "XOR" operation: the logical values ​​of the corresponding positions of the two matrices are different. The corresponding position logic value is set to 1 (True); otherwise, The logical value of the corresponding position is set to 0 (False). After this step, the pixels with perceptible blocky artifacts caused by compression are filtered out.

[0134] B5, calculate JPD-B. Specifically include:

[0135] B51, the “just noticeable difference” obtained in step B3 Compared with the "perceptible blockiness caused by compression" obtained in step B4 Merge (take the union) and get , the matrix Perform a dot product with the AE of step 2 so that all AE values ​​that are just noticeable differences and block artifacts are accumulated (i.e. ); Finally, the peak absolute value error is calculated using the following formula:

[0136] ;

[0137] in is the peak error value that a single pixel in the image can reach.

[0138] B52, find B51 The average error on the entire image, that is, the number of pixels in the entire image is used as the divisor to obtain the image quality evaluation index JPD-B.

[0139] Example 2

[0140] This example constructs a JPEG optimization method for medical images. This method differs from Example 1 in that it uses a different dataset for JPEG optimization. Specifically, the Visualet dataset is constructed using medical images rather than natural images. Applicable medical images include, but are not limited to, X-rays, CT (computed tomography) images, MRI (magnetic resonance imaging), and ultrasound. The specific image data can be selected based on application needs.

[0141] Example 3

[0142] This example constructs a JPEG optimization method for remote sensing images. This method differs from Example 1 in that it uses a different dataset for JPEG optimization. Specifically, when constructing the Visualet dataset, remote sensing images are used instead of natural images. The remote sensing images that can be used include, but are not limited to, drone remote sensing images, aerial remote sensing images, and satellite remote sensing images. The specific image data can be selected based on application needs.

[0143] Example 4

[0144] This embodiment constructs an optimization algorithm for MPEG (Moving Picture Experts Group) video encoders. MPEG is a further extension of JPEG: MPEG supports not only the compression of single-frame images (JPEG), but also the compression of video sequences (multiple consecutive frames). To ensure efficient compression of video sequences, MPEG introduces inter-frame motion prediction and motion compensation algorithms based on JPEG. This algorithm obtains the DCT coefficient residuals of two adjacent frames and then compresses them using a new "residual quantization table." Therefore, this embodiment optimizes this "residual quantization table."

[0145] Specifically, an MPEG encoder is first used on a multi-frame video sequence to obtain the DCT residual between two frames, and this residual is used to construct a training data set; then, the same optimization steps as in Example 1 are adopted to iteratively optimize the "residual quantization table" on the new data set; the final optimized quantization table is specifically used in MPEG and quantizes the DCT residual coefficients generated between frames, thereby improving MPEG compression performance (rate-distortion performance).

[0146] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0147] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. An optimization method based on the JPEG image compression standard, characterized in that: The method comprises the following steps: S1, the entire bitrate range of natural images is divided into 100 bitrate points at equal intervals, corresponding to 100 quality index values ​​from low to high; S2, setting a total loss function, where the total loss function value is the sum of the image quality loss and the weighted bit rate term; wherein the bit rate term weight is manually adjusted so that the error between the bit rate of the optimized learnable quantization table and the bit rate specified by the quality index value is less than a preset error threshold; S3: For each quality index value, the back-propagation mathematical mechanism in the artificial intelligence convolutional neural network is used to generate a set of optimal quantization tables on the JPEG frame through synchronous iterative operations. Each set of optimal quantization tables includes a luminance quantization table and a chrominance quantization table. During the back-propagation iterative optimization process, all quantization parameters in the optimal quantization table are synchronously updated at each iteration until the optimization effect reaches saturation. The aforementioned iterative process is repeated until the optimal quantization tables corresponding to all quality index values ​​are output. S4, using the bit rates corresponding to all quality index values ​​as independent variables and the quantization parameters at each position in the corresponding optimal quantization table as dependent variables, and performing curve fitting to obtain a fitting curve for each quantization parameter; S5, for any bit rate, calculate the quantization parameter value corresponding to the bit rate through the fitting curve of each quantization parameter in step S4, generate the corresponding luminance quantization table and chrominance quantization table, and obtain the optimal quantization table corresponding to the bit rate.

2. The optimization method based on the JPEG image compression standard according to claim 1, characterized in that: In step S2, the calculation process of the total loss function value includes: Convert the original image from RGB to YCbCr, and then downsample the chrominance components to obtain the original luminance component and the downsampled chrominance component. After DCT transformation of the original luminance component and the downsampled chrominance component, the DCT coefficients are obtained and quantized using a learnable quantization table; After performing differentiable approximate rounding on the quantized DCT coefficients, dequantization is performed using a learnable quantization table, and the dequantized results are subjected to inverse DCT transformation to obtain the decoded luminance and chrominance components. The image quality loss is calculated by combining the original luminance component and the downsampled chrominance component, as well as the decoded luminance component and chrominance component. The reciprocal of all quantization parameters in the learnable quantization table is taken and summed to obtain the bit rate term, which is then multiplied by the bit rate term weight to obtain the weighted bit rate term. The image quality loss and the weighted bit rate term are summed to obtain the total loss function value.

3. The optimization method based on the JPEG image compression standard according to claim 2, characterized in that: The process of calculating the image quality loss by combining the original luma component and the downsampled chroma components, as well as the decoded luma component and chroma components, includes the following steps: The original brightness component is processed by the Sobel edge detection algorithm to calculate the gradient value in the horizontal and vertical directions around each pixel. The gradient value is normalized and then added with the number 1 to obtain the weight of the image edge guidance. Calculate the first absolute value error between the original luminance component and the decoded luminance component, and the second absolute value error between the downsampled chrominance component and the decoded chrominance component. Multiply the first absolute value error of the luminance component by the image edge-guided weight, add it to the second absolute value error of the chrominance component, and finally average it to obtain the image quality loss.

4. The optimization method based on the JPEG image compression standard according to claim 1, characterized in that: In step S3, the training data set used by the learnable quantization table includes six basic meta-images: noise images with random probability distribution, noise images generated by high ISO, texture images, edge images, blurred images, and smooth images.

5. The optimization method based on the JPEG image compression standard according to claim 4, characterized in that: The process of constructing the training dataset includes: Generate noise images with various random probability distributions including Gaussian, Poisson, uniform, and salt and pepper through computer code; The noisy image, blurred image and smooth image caused by high ISO were obtained by manual cropping; The initial texture image is obtained by cropping. For each initial texture image, its co-occurrence matrix in the four directions of 0 degrees, 45 degrees, 90 degrees and 135 degrees is counted respectively. Then, the average entropy value of the four co-occurrence matrices is calculated. The initial texture image with an average entropy value greater than the set entropy value threshold is selected as the final texture image included in the training dataset; The initial edge image is obtained by cropping, and the Canny edge detection algorithm is used to filter out all the initial edge images whose edge pixel ratio is greater than the set pixel number threshold, which are used as the edge images finally included in the training data set; Adjust the number and type of the six basic perceptual element images in the training data set so that the number of the six basic perceptual element images is close to the same, and the dataset tends to cover the entire bitrate range of natural images, that is, pixel depth , where the pixel depth of the luminance component is , the pixel depth of the two chroma components .

6. A method for image quality assessment, characterized in that: The method comprises the following steps: Calculate the absolute value error of the distorted image that can be perceived by the human eye: Initialize a matrix with the same size as the original image , compare the absolute value error of each pixel with its corresponding preset distortion threshold. If the absolute value error of any pixel is greater than its corresponding preset distortion threshold, the matrix The value of the position corresponding to the pixel in the matrix is ​​set to 1; otherwise, the ... The value of the corresponding position is set to 0; The absolute value error between the distorted image and the original image at each pixel is compared with the matrix Perform dot multiplication to remove pixel distortion whose absolute value error is less than the preset distortion threshold, and obtain ; Then calculate the perceptible peak absolute value error of the distorted image by summing the following formula : ； in is the peak error value that a single pixel on the distorted image can reach; Using the number of pixels in the entire image as the divisor, calculate The average error over the entire image is used to obtain the image quality evaluation index JPD.

7. The image quality assessment method according to claim 6, wherein: The calculation process of the preset distortion threshold comprises the following steps: Calculate each pixel of the original image exist Uniform background brightness in a local area ,as well as The maximum contrast difference in the local area at 0 degrees, 45 degrees, 90 degrees, and 135 degrees ; The specific calculation formula is as follows: ； ； Among them, i=1,2,3,4,5, j=1,2,3,4,5, and is the operator matrix for calculating the difference between the background uniform brightness and the maximum contrast, and max is the maximum value function; The preset distortion threshold for each pixel is calculated using the following formula: : ； The formula Specifically: ； The formula Specifically: ； Among them, the parameter value in the function , .

8. The image quality assessment method according to claim 6, wherein: The method is used for quality assessment of JPEG compressed images and comprises the following steps: Calculate the absolute value error of the distorted image that can be perceived by the human eye and the block artifacts: Initialize a matrix with the same size as the original image , compare the absolute value error of each pixel with its corresponding preset distortion threshold. If the absolute value error of any pixel is greater than its corresponding preset distortion threshold, the matrix The value of the position corresponding to the pixel in the matrix is ​​set to 1, otherwise, The value of the corresponding position is set to 0; Initialize a matrix with the same size as the distorted image , for distorted images The edge position pixel of the small block is calculated for the change value of the adjacent pixels on the left or above and below. If the change value is greater than the preset distortion threshold, the pixel is judged to have a block effect that can be perceived by the human eye, and the matrix The corresponding position logic value is set to 1, otherwise, the matrix The logical value of the corresponding position is set to 0; Initialize a matrix with the same size as the original image , for the original image The edge position pixel of the small block is calculated for the change value of the adjacent pixels on the left or above and below. If the change value is greater than the preset distortion threshold, the pixel is judged to have a block effect that can be perceived by the human eye, and the matrix The corresponding position logic value is set to 1, otherwise, the matrix The logical value of the corresponding position is set to 0; The matrix With the matrix Perform XOR operation, if the matrix With the matrix If the logical values ​​at the corresponding positions are different, the logical values ​​at the corresponding positions of the two matrices are set to 1; otherwise, the logical values ​​at the corresponding positions of the two matrices are set to 0, and the matrix is ​​obtained. , filtering out pixels with perceptible block artifacts caused by compression; The matrix With the matrix Merge to get the matrix ; Then the matrix The absolute value error of each pixel between the distorted image and the original image is multiplied by The cumulative peak absolute value error of the human eye and the block artifacts is calculated by the following formula: : ； in is the peak error value that a single pixel on the distorted image can reach; Using the number of pixels in the entire image as the divisor, calculate The average error over the entire image is used to obtain the image quality evaluation index JPD-B.

Images