A method, model, device, and medium for constructing and applying an image reconstruction model

Abstract

The present application relates to the field of image processing. The present application proposes a method, model, device, and medium for constructing and applying an image reconstruction model. The scheme uses the frequency domain coefficients and quantization table of a JPEG image as a data set to train an initial image reconstruction model. By inputting prior knowledge of the compression process (i.e., the quantization table used when compressing the image), the initial image reconstruction model is guided to gradually reduce the distortion caused by image compression to obtain an image reconstruction model. The image reconstruction model obtained based on the scheme of the present application can better enhance the visual quality of JPEG images.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a method, model, device, and medium for constructing and applying an image reconstruction model. Background Art

[0002] In modern digital image processing, the rapid growth of image data poses significant challenges to storage, transmission, and processing. To address this issue, image compression technology has emerged. Its core purpose is to reduce data redundancy while preserving as much visual information as possible, thereby reducing storage space and transmission bandwidth requirements. JPEG, a widely used lossy compression standard, significantly reduces file size by quantizing and encoding high-frequency details that are insensitive to the human eye, while maintaining high subjective visual quality. This efficient compression method has made JPEG a mainstream format for image storage and transmission, especially in resource-constrained scenarios (such as network transmission and mobile devices).

[0003] Furthermore, during image compression, especially with lossy compression methods like JPEG, the compression ratio is typically expressed by the quality factor (QF). Low QFs often introduce significant artifacts, resulting in a decrease in visual quality, impacting both the visual quality of the image and the performance of subsequent processing tasks. However, many practical applications (such as medical image analysis, security monitoring, and satellite remote sensing) require high image quality, and directly using compressed images may not meet these requirements.

[0004] Therefore, there is an urgent need for a solution that can effectively remove artifacts that appear in compressed images. This solution can restore lost details in compressed images, suppress distortion, and improve the overall quality of the image, thereby achieving a better visual experience and higher application value based on efficient compression. Summary of the Invention

[0005] The present application provides a method, model, device, and medium for constructing an image reconstruction model to solve the problem of artifacts appearing in existing compressed JPEG images.

[0006] A first aspect of the present application provides a method for constructing an image reconstruction model, where the image reconstruction model is used to reconstruct a JPEG image. The method comprises:

[0007] Obtaining a JPEG image and decoding it to obtain frequency domain coefficients and a quantization table of the JPEG image;

[0008] Build an initial image reconstruction model based on deep learning methods;

[0009] The frequency domain coefficients and the quantization table constitute a data set, and the data set is used to perform multiple iterative training on the initial image reconstruction model until convergence, so as to obtain the image reconstruction model.

[0010] In some embodiments of the present application, the initial image reconstruction model takes frequency domain coefficients and a quantization table as input, and outputs a reconstructed image corresponding to the JPEG image, and the initial image reconstruction model includes:

[0011] A learnable offset guidance module is used to calculate a corrected first feature of the frequency domain coefficient according to the frequency domain coefficient and the quantization table;

[0012] The domain transformation module is used to transform the corrected first feature of the frequency domain coefficient into the pixel domain to obtain the first pixel feature of the JPEG image, and calculate the reconstructed image corresponding to the JPEG image based on the first pixel feature.

[0013] In some embodiments of the present application, the learnable offset guidance module includes:

[0014] The quantization offset submodule is used to calculate the quantization offset feature based on the frequency domain coefficient and the quantization table;

[0015] The frequency domain correction submodule is used to calculate the corrected first feature of the frequency domain coefficient according to the quantization offset feature and the frequency domain coefficient.

[0016] In some embodiments of the present application, the learnable offset guidance module further includes a dequantization submodule, and:

[0017] The dequantization submodule is used to calculate the dequantized frequency domain coefficient feature according to the corrected first frequency domain coefficient feature and the quantization table;

[0018] The domain transformation module is further used to transform the dequantized frequency domain coefficient features into the pixel domain to obtain the second pixel features of the JPEG image, and calculate the reconstructed image corresponding to the JPEG image based on the second pixel features.

[0019] In some embodiments of the present application, the image reconstruction model further includes:

[0020] The codec module is used to calculate the quantization guide feature of the JPEG image according to the quantization table, the first pixel feature or the second pixel feature, and calculate the reconstructed image corresponding to the JPEG image based on the quantization guide feature.

[0021] In some embodiments of the present application, the codec module includes:

[0022] An encoder submodule, configured to calculate a target intermediate feature of the JPEG image based on the quantization table, the first pixel feature, or the second pixel feature;

[0023] The decoder submodule is used to calculate the quantization guide features of the JPEG image according to the target intermediate features and the quantization table, and calculate the reconstructed image corresponding to the JPEG image based on the quantization guide features.

[0024] A second aspect of the present application provides an image reconstruction model, the image reconstruction model comprising:

[0025] A learnable offset guidance module is used to calculate a corrected first feature of the frequency domain coefficient according to the acquired frequency domain coefficient and quantization table of the JPEG image;

[0026] The domain transformation module is used to transform the corrected first feature of the frequency domain coefficient into the pixel domain to obtain the first pixel feature of the JPEG image, and calculate the reconstructed image corresponding to the JPEG image based on the first pixel feature.

[0027] A third aspect of the present application provides an application method of an image reconstruction model, the method comprising:

[0028] Obtaining a JPEG image and decoding it to obtain frequency domain coefficients and a quantization table of the JPEG image;

[0029] The image reconstruction model described in the above embodiment is used to calculate and obtain a reconstructed image corresponding to the JPEG image based on the frequency domain coefficients and the quantization table.

[0030] The fourth aspect of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the method described in any one of the first and third aspects of the above embodiments are implemented.

[0031] The fifth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method described in any one of the first and third aspects of the above embodiments is implemented.

[0032] This application has the following beneficial effects:

[0033] This application uses the frequency domain coefficients and quantization table of the PEG image as a data set to train the initial image reconstruction model. By inputting prior knowledge of the compression process (i.e., the quantization table used when compressing the image), the initial image reconstruction model is guided to gradually reduce the distortion caused by image compression to obtain an image reconstruction model. The image reconstruction model obtained based on the scheme of this application can better enhance the visual quality of JPEG images. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The drawings herein are incorporated into and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and, together with the specification, are used to illustrate the technical solutions of the present application.

[0035] Figure 1 This is a schematic diagram of an example process of compressing an image into JPEG format provided by this application;

[0036] Figure 2 This is a flow chart of an embodiment of a method for constructing an image reconstruction model provided by the present application;

[0037] Figure 3 This is a schematic diagram of the framework of the first embodiment of the image reconstruction model provided by this application;

[0038] Figure 4 This is a schematic diagram of the framework of the second embodiment of the image reconstruction model provided by this application;

[0039] Figure 5 This is a schematic diagram of the framework of an embodiment of a learnable offset guidance module provided by the present application;

[0040] Figure 6 This is a schematic diagram of the framework of an embodiment of a coding quantization table guiding unit provided by the present application;

[0041] Figure 7 This is a schematic diagram of the framework of an embodiment of a decoding quantization table guiding unit provided by the present application;

[0042] Figure 8 This is a schematic diagram of the framework of the third embodiment of the image reconstruction model provided by this application;

[0043] Figure 9 This is a schematic diagram of the framework of an embodiment of an electronic device provided by the present application;

[0044] Figure 10 It is a schematic diagram of a framework of an embodiment of a computer-readable storage medium provided by this application. DETAILED DESCRIPTION

[0045] The following describes the embodiments of the present application in detail with reference to the accompanying drawings.

[0046] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0047] The term "and / or" in this article is simply a description of the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent three situations: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects are in an "or" relationship. In addition, "many" in this article means two or more than two. In addition, the term "at least one" in this article means any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0048] The inventors have discovered that, in order to effectively remove artifacts from JPEG images, relevant researchers have proposed a model-based method, which is mainly based on filter design and only solves limited blocking and ringing artifacts. In recent years, inspired by the success of deep neural networks in image classification, researchers have begun to explore the use of deep neural networks to remove artifacts from JPEG images. Some methods for removing JPEG artifacts guide the network model to eliminate artifacts by dynamically predicting quality factors or quality factor graphs. However, these methods fail to fully utilize the prior knowledge of JPEG compression, that is, the JPEG image is directly input into the grid model, and the model only passively fits the input data and learns the mapping from the JPEG image to the reconstructed image. This method also does not consider the root cause of image distortion (that is, the compression method of the JPEG format will cause quantization error and chromatic distortion), resulting in the image restoring incorrect details and unnatural colors.

[0049] To this end, the inventors provide a new scheme for removing JPEG artifacts, also known as a blind JPEG artifact removal scheme based on inverse JPEG compression. In the scheme of this application, prior knowledge of the input compression process (i.e., the quantization table used when compressing the image) is used to guide the initial image reconstruction model to gradually reduce the image distortion phenomenon (i.e., this application takes into account the root cause of image distortion), thereby better enhancing the details and color consistency of the JPEG image, that is, better enhancing the visual quality of JPEG.

[0050] In order to better understand the solution for removing JPEG artifacts proposed in this application, the root cause of image distortion is considered. Figure 1 Explain the specific process of compressing images into JPEG format.

[0051] like Figure 1 As shown in FIG, it is a process of compressing an image into a JPEG format image, and the process is:

[0052] (1) Color space conversion: The real image is first converted from the RGB color space to the YCbCr color space. Among them, the Y component represents the luminance component (Luminance), and Cb and Cr represent the blue component and the red component respectively; (2) Downsampling: In the YCbCr color space, the chrominance components (Cb and Cr) are usually downsampled to reduce the amount of data. This step introduces chrominance distortion. (3) Discrete Cosine Transform (DCT): Discrete Cosine Transform (DCT) is applied to each 8x8 pixel block. DCT converts the pixel value into a frequency domain coefficient (i.e., the frequency domain coefficient mentioned in this application), so that the energy is concentrated in the low-frequency part and the high-frequency part contains less energy. (4) Quantization: The quantization table is used to quantize the frequency domain coefficients. High-frequency coefficients are usually discarded or greatly reduced, while low-frequency coefficients retain more details. This step introduces quantization error, but the human eye is not very sensitive to these high-frequency details. (5) Nearest neighbor rounding: The quantized frequency domain coefficients are further simplified by nearest neighbor rounding to reduce the amount of data. (6) Bitstream encoding: Use entropy coding (such as Huffman coding or arithmetic coding) to encode the quantized frequency domain coefficients to generate the final JPEG file.

[0053] Therefore, the technical solution of this application is Figure 1 The inverse compression process of the image compressed in JPEG format is performed, that is, the present application decodes the JPEG compressed image to obtain its corresponding frequency domain coefficients and quantization table, and inputs the compression prior knowledge such as the quantization table into the initial image reconstruction model to simulate the JPEG inverse compression process to compensate for the quantization error and / or chromatic distortion, so that the model can reconstruct a high-quality image.

[0054] The present application is described in detail below with reference to the accompanying drawings and specific embodiments.

[0055] According to one embodiment of the present application, Figure 2 As shown, the present application provides a method for constructing an image reconstruction model. The method obtains the image reconstruction model by executing the following steps S1-S3: S1, obtaining a JPEG image and decoding it to obtain the frequency domain coefficients and quantization table of the JPEG image; S2, constructing an initial image reconstruction model based on a deep learning method; S3, constructing a data set with the frequency domain coefficients and quantization table, and using the data set to iteratively train the initial image reconstruction model multiple times until convergence, thereby obtaining the image reconstruction model. Each step is explained in detail below.

[0056] 1. Step S1

[0057] The inventors found that relevant researchers trained a specific grid model for each quality factor, lacking the flexibility to learn a single grid model for different JPEG quality factors. Moreover, since the quality factor is usually unknown in practical applications, this greatly limits the practicality of the network model.

[0058] To this end, according to one embodiment of the present application, in step S1, the acquired JPEG image is compressed using different quality factors, and the JPEG images corresponding to different quality factors are decoded respectively to obtain frequency domain coefficients and quantization tables of the JPEG images corresponding to different quality factors.

[0059] It can be seen from the above embodiments that the present application uses JPEG images corresponding to different quality factors to construct a data set to guide the initial image enhancement model to gradually reduce the distortion caused by image compression, so that the obtained image reconstruction model can perform image enhancement on images compressed with any quality factor, thereby improving the flexibility and practicality of the trained image reconstruction model; at the same time, it also improves the generalization of the image reconstruction model, that is, when facing JPEG image training corresponding to quality factors that have not appeared during training, the model can also perform high-quality reconstruction of JPEG images compressed based on the quality factor.

[0060] 2. Step S2

[0061] According to one embodiment of the present application, Figure 3 As shown, the initial image reconstruction model takes the frequency domain coefficients and the quantization table as input, and outputs the reconstructed image corresponding to the JPEG image, and the initial image reconstruction model includes: a learnable offset guidance module, which is used to calculate the corrected first feature of the frequency domain coefficients based on the frequency domain coefficients and the quantization table; a domain transformation module, which is used to convert the corrected first feature of the frequency domain coefficients into the pixel domain to obtain the first pixel feature of the JPEG image, and calculate the reconstructed image corresponding to the JPEG image based on the first pixel feature.

[0062] From the above description, it can be seen that the learnable offset guidance module of the above embodiment of the present application can correct the quantization error of the frequency domain coefficients introduced during image compression based on the quantization prior (quantization table), effectively restore the frequency domain information lost during the image compression process, and generate a high-quality reconstructed image.

[0063] To better understand how the learnable offset guidance module corrects the frequency domain coefficients of a compressed JPEG image and how the domain transformation module reconstructs the final image, the following describes the structure and functions of the learnable offset guidance module and the domain transformation module.

[0064] (1) Learnable Offset Guidance Module

[0065] According to one embodiment of the present application, the frequency domain coefficients are divided into brightness coefficients and chrominance coefficients, and still refer to Figure 3 As shown, the initial image reconstruction model includes two learnable offset guidance modules, one for calculating the corrected first feature of the brightness coefficient based on the brightness coefficient and the quantization table, and the other for calculating the corrected first feature of the chroma coefficient based on the chroma coefficient and the quantization table.

[0066] As can be seen from the above description, the above embodiment of the present application uses two independent learnable offset guidance modules to take the brightness coefficient and the chrominance coefficient as input respectively, and uses the quantization table prior to correct the quantization error of these coefficients (brightness coefficient and chrominance coefficient), effectively restoring the brightness information and chrominance information lost during the image compression process to generate a high-quality reconstructed image.

[0067] Among them, according to one embodiment of the present application, the learnable offset guidance module includes: a quantization offset sub-module, which is used to calculate the quantization offset feature based on the frequency domain coefficient and the quantization table; and a frequency domain correction sub-module, which is used to calculate the corrected frequency domain coefficient first feature based on the quantization offset feature and the frequency domain coefficient.

[0068] From the above description, it can be seen that the frequency domain correction submodule of the above embodiment of the present application calculates the first feature of the corrected frequency domain coefficient by combining the quantization offset feature and the frequency domain coefficient. This method can effectively compensate for the quantization error introduced in the compression process, thereby restoring the lost detail information of the image and improving the image reconstruction quality (that is, the learnable offset guidance module aims to learn the quantization offset to reduce the quantization error caused by rounding operations, and correct the frequency domain coefficient based on the quantization offset to generate a high-quality reconstructed image.).

[0069] The following combination Figure 3 、 Figure 4 and Figure 5 Describe the submodules of the learnable offset guidance module and the functions performed.

[0070] 1. Rearrange submodules

[0071] According to one embodiment of the present application, Figure 4 As shown, the learnable offset guidance module also includes: a first rearrangement submodule, which is used to rearrange the input frequency domain coefficients and transmit the rearranged frequency domain coefficients to the quantization offset submodule for subsequent processing; a second rearrangement submodule, which is used to rearrange the first features of the corrected frequency domain coefficients output by the frequency domain correction submodule, or the dequantized frequency domain coefficient features output by the dequantization submodule, and transmit the rearranged results to the domain transformation module for processing.

[0072] For example, rearrange the frequency domain coefficients X to obtain , about to The frequency domain coefficients are rearranged into , so that the rearranged frequency domain coefficients have inherent frequency correlation in the channel dimension while maintaining the spatial positions between different 8×8 blocks.

[0073] As can be seen from the above description, the above embodiment of the present application can maintain the spatial position between different 8×8 blocks by rearranging the frequency domain coefficients, while arranging them in frequency order in the channel dimension. This arrangement allows the model to more easily capture the correlation in the frequency dimension, thereby better adapting to the characteristics of the image reconstruction model and significantly improving the image quality of the model's reconstruction of compressed images.

[0074] 2. Quantization offset submodule

[0075] According to one embodiment of the present application, the quantization offset submodule includes: a parameter pair calculation unit, which is used to calculate the first parameter and the second parameter according to the quantization table; and a quantization offset calculation unit, which is used to calculate the quantization offset feature according to the frequency domain coefficient, the first parameter and the second parameter.

[0076] Among them, such as Figure 5 As shown, the parameter pair calculation unit consists of a multi-layer perceptron (MLP), a Tanh activation function, and a Sigmiod activation function. In this embodiment of the application, the quantization table is flattened into a 128-dimensional vector and a four-layer multi-layer perceptron (MLP) is used to learn the mapping from this vector to the first parameter and the second parameter. The quantization offset calculation unit consists of a convolution block (the convolution block consists of two convolution layers (1×1 convolution layers) with a PReLU activation function and a Tanh activation function).

[0077] According to one embodiment of the present application, reference is still made to Figure 5 As shown, the parameter pair calculation unit is configured to obtain the quantized offset feature in the following manner:

[0078]

[0079] in, represents the quantization offset feature, is the first parameter, represents the intermediate features of the frequency domain coefficients, is the second parameter.

[0080]

[0081] in, represents a convolutional layer, represents another convolutional layer, is the PReLU activation function, Represents the frequency domain coefficients after rearrangement.

[0082] From the above description, we can see that the first parameter and the second parameter are used for the intermediate feature The affine transformation of enables the image reconstruction model to adaptively refine the feature map to generate high-quality reconstructed images.

[0083] 3. Frequency domain correction submodule

[0084] According to one embodiment of the present application, still referring to Figure 5 As shown, the frequency domain correction submodule is configured to obtain the first characteristic of the corrected frequency domain coefficients in the following manner:

[0085]

[0086] in, represents the first characteristic of the corrected frequency domain coefficients, is the scaling factor, is the Tanh activation function. Set to 0.5 to constrain the range of quantization offsets.

[0087] From the above description, it can be seen that the above embodiment of the present application introduces a scaling factor to constrain the range of the quantization offset (also known as the quantization range), and can correct the quantization error of the frequency domain coefficients introduced during image compression based on the rounding prior (scaling factor) and the quantization prior (quantization table), effectively restoring the frequency domain information lost during the image compression process to generate a high-quality reconstructed image.

[0088] It should be noted that since the range of the quantization error in the image compression process is between [-0.5, 0.5), the present application presets a scaling factor to constrain the range of the quantization offset. 0.5 is only one of the embodiments in the experimental process of the present application. The setting of the specific scaling factor can be adjusted according to the specific value of the quantization offset obtained, and no specific limitation is made here.

[0089] As can be seen from the above description, the above embodiment of the present application can effectively control the quantization error caused by the compression process by setting a scaling factor to constrain the range of the quantization offset, ensuring that the first feature of the corrected frequency domain coefficient is more accurate and stable. By appropriately adjusting the scaling factor, the quantization offset can be limited to a reasonable range, thereby reducing unnecessary noise introduction and information loss, and improving the quality of image reconstruction or restoration. This method not only helps to optimize the performance of the model, but also improves processing efficiency, making the frequency domain correction submodule more adaptable and more accurate in different application scenarios.

[0090] 4. Dequantization submodule

[0091] According to one embodiment of the present application, still referring to Figure 5As shown, the learnable offset guidance module also includes a dequantization submodule, and: the dequantization submodule is used to calculate the dequantized frequency domain coefficient feature based on the corrected first feature of the frequency domain coefficient and the quantization table; the domain transformation module is also used to convert the dequantized frequency domain coefficient feature into the pixel domain to obtain the second pixel feature of the JPEG image, and calculate the reconstructed image corresponding to the JPEG image based on the second pixel feature.

[0092] According to one embodiment of the present application, reference is still made to Figure 5 As shown, the dequantization submodule is configured to obtain dequantized frequency domain coefficient features in the following manner:

[0093]

[0094] in, Represents the dequantized frequency domain coefficient characteristics, Represents a quantization table.

[0095] From the above description, it can be seen that the dequantization submodule of the above embodiment of the present application calculates the dequantized frequency domain coefficient characteristics based on the corrected first characteristics of the frequency domain coefficients and the quantization table. This process effectively reverses the quantization operation applied during the compression process and restores more accurate frequency domain information, thereby significantly reducing the errors and distortions caused by quantization and improving the quality of the reconstructed image.

[0096] (2) Domain Transformation Module

[0097] According to one embodiment of the present application, the domain transformation module is configured to: use an inverse discrete cosine transform to convert the corrected first feature of the frequency domain coefficient or the dequantized frequency domain coefficient feature to the pixel domain, obtain the first pixel feature or the second pixel feature of the JPEG image, and calculate the reconstructed image corresponding to the JPEG image based on the first pixel feature or the second pixel feature.

[0098] For example, the inverse discrete cosine transform (IDCT) is used to reconstruct these corrected or dequantized frequency domain coefficients (luminance coefficients and chrominance coefficients) into a YCbCr image. Since the chrominance image CbCr has been downsampled, bilinear interpolation is used to upsample the chrominance image to the same size as the luminance image Y, and then the YCbCr image is converted to an RGB image.

[0099] As can be seen from the above description, the above embodiment of the present application uses the inverse discrete cosine transform (IDCT) through the domain transformation module to convert the corrected or dequantized frequency domain coefficients (including luminance coefficients and chrominance coefficients) to the pixel domain, generate the pixel features of the JPEG image, and further calculate the reconstructed image of the JPEG image. Among them, in view of the downsampling characteristics of the chrominance image CbCr, bilinear interpolation is used for upsampling to make it consistent with the size of the luminance image Y, and then YCbCr is converted to an RGB image. This design not only efficiently restores the frequency domain information of the image, but also effectively preserves the chrominance details through interpolation operations, significantly improving the quality and visual expression of the reconstructed image. In particular, it can still maintain a high level of image fidelity in high compression rate scenarios, and has a wide range of application value.

[0100] (3) Codec module

[0101] Since the JPEG format uses an 8×8 block strategy during compression, the chroma distortion varies greatly between blocks.

[0102] According to one embodiment of the present application, still referring to Figure 3 and Figure 4 As shown, the image reconstruction model also includes: a codec module, which is used to calculate the quantization guide feature of the JPEG image according to the quantization table, the first pixel feature or the second pixel feature, and calculate the reconstructed image corresponding to the JPEG image based on the quantization guide feature.

[0103] As can be seen from the above description, the above embodiment of the present application combines the quantization table to compensate for the error of the chrominance channel, restores more chrominance details, and thus reduces the chrominance distortion problem caused by the downsampling operation during the image compression process.

[0104] Among them, according to one embodiment of the present application, the codec module includes: an encoder sub-module, which is used to calculate the target intermediate feature of the JPEG image based on the quantization table, the first pixel feature or the second pixel feature; a decoder sub-module, which is used to calculate the quantization guide feature of the JPEG image based on the target intermediate feature and the quantization table, and calculate the reconstructed image corresponding to the JPEG image based on the quantization guide feature.

[0105] According to one embodiment of the present application, the codec module includes encoders and decoders of different scales, and the scales of the encoders and decoders correspond one to one.

[0106] As can be seen from the above description, the multi-scale codec structure of the above embodiment of the present application provides stronger contextual learning capabilities with the same number of parameters, can more accurately restore color details, reduce chromaticity distortion, and make the color representation of the reconstructed image more accurate and natural, thereby significantly improving the overall visual quality. At the same time, by processing information at multiple resolution levels, this architecture can flexibly handle images of various compression levels, achieving efficient and high-quality chromaticity compensation.

[0107] The following combination Figure 3 、 Figure 6 、 Figure 7 and Figure 8 The structures of the encoder submodule and the decoder submodule are described in detail.

[0108] 1. Encoder submodule

[0109] According to one embodiment of the present application, still referring to Figure 3 As shown, the encoder submodule includes three scale encoders, wherein the first scale encoder is used to calculate the first coding feature of the JPEG image based on the quantization table, the first pixel feature or the second pixel feature; the second scale encoder is used to calculate the second coding feature of the JPEG image based on the quantization table and the first coding feature; the third scale encoder is used to calculate the target intermediate feature of the JPEG image based on the quantization table and the second coding feature.

[0110] According to one embodiment of the present application, reference is still made to Figure 3 As shown, the encoders of the three scales all include a quantization table guide unit and a residual unit.

[0111] According to one embodiment of the present application, an encoder of a first scale includes: a first coding quantization table guiding unit, used to calculate a first intermediate feature of a JPEG image based on a quantization table, a first pixel feature, or a second pixel feature; a first coding residual unit, used to calculate a first coding feature of a JPEG image based on the first intermediate feature; an encoder of a second scale includes: a second coding quantization table guiding unit, used to calculate a second intermediate feature of a JPEG image based on a quantization table and the first coding feature; a second coding residual unit, used to calculate a second coding feature of a JPEG image based on the second intermediate feature; an encoder of a third scale includes: a third coding quantization table guiding unit, used to calculate a third intermediate feature of a JPEG image based on the quantization table and the second coding feature; and a third coding residual unit, used to calculate a target intermediate feature of the JPEG image based on the third intermediate feature.

[0112] like Figure 6As shown in FIG, it is the structure of the coding quantization table guidance unit in multiple scale encoders, wherein the coding quantization table guidance unit is composed of a PReLU activation function, two convolutional layers, a multi-layer perceptron (MLP), a Tanh activation function, and a Sigmiod activation function.

[0113] According to one embodiment of the present application, still referring to Figure 6 As shown, the encoding quantization table guide unit in the encoder submodule is configured to calculate its corresponding output features in the following manner:

[0114]

[0115]

[0116] in, Indicates the The output characteristics of the coding quantization table guide unit, Indicates the The input features of the coding quantization table guide unit, Indicates the The second convolutional layer in the quantization table-guided unit, is the PReLU activation function, Indicates the The first convolutional layer in the quantization table-guided unit, Indicates the The first parameter of the coding quantization table guide unit calculation, Indicates the The second parameter calculated by the coding quantization table guide unit. It should be noted that, Corresponding to the above-mentioned first coding quantization table guiding unit, second coding quantization table guiding unit, and third coding quantization table guiding unit.

[0117] As can be seen from the above description, the above embodiment of the present application introduces a multi-scale encoder and a quantization table guidance unit to gradually encode the pixel features of the JPEG image at different scales, thereby achieving refined feature extraction and optimization. Specifically, the first quantization table guidance unit, the second quantization table guidance unit, and the third quantization table guidance unit each combine the quantization table and the previous layer features to dynamically adjust the image features, effectively reducing the impact of quantization error on coding performance. At the same time, the residual unit further enhances the expressiveness of features and improves the accuracy and robustness of coding by introducing a residual learning mechanism. This solution can significantly improve the coding efficiency and quality of features, enabling the model to generate high-quality reconstructed images.

[0118] 2. Decoder submodule

[0119] According to one embodiment of the present application, still referring to Figure 3As shown, the decoder submodule includes three scale decoders, wherein the first scale decoder is used to calculate the first decoding feature of the JPEG image based on the target intermediate feature and the quantization table; the second scale decoder is used to calculate the second decoding feature of the JPEG image based on the first decoding feature and the quantization table; the third scale decoder is used to calculate the quantization guide feature of the JPEG image based on the second decoding feature and the quantization table, and calculate the reconstructed image corresponding to the JPEG image based on the quantization guide feature.

[0120] According to one embodiment of the present application, reference is still made to Figure 3 As shown, the decoders of the three scales also include a quantization table guide unit and a residual unit.

[0121] According to one embodiment of the present application, a decoder of a first scale includes: a first decoding residual unit, which calculates a first residual feature of a JPEG image based on a target intermediate feature; a first decoding quantization table guidance unit, which is used to calculate a first decoding feature of the JPEG image based on the first residual feature and a quantization table; a decoder of a second scale includes: a second decoding residual unit, which is used to calculate a second residual feature of the JPEG image based on the first decoding feature; a second decoding quantization table guidance unit, which is used to calculate a second decoding feature of the JPEG image based on the second residual feature and the quantization table; a decoder of a third scale includes: a third decoding residual unit, which calculates a third residual feature of the JPEG image based on the second decoding feature; a third decoding quantization table guidance unit, which is used to calculate a quantization guidance feature of the JPEG image based on the third residual feature and the quantization table, and calculate a reconstructed image corresponding to the JPEG image based on the quantization guidance feature.

[0122] As can be seen from the above embodiments, the above embodiments of the present application introduce a multi-scale decoder and a decoding quantization table guidance unit to gradually decode and reconstruct the features output by the encoder at different scales, significantly improving the quality and accuracy of image reconstruction. Specifically, the decoding residual unit effectively enhances the expressive power of features by introducing a residual learning mechanism; the decoding quantization table guidance unit dynamically adjusts image features by combining the quantization table and input features, reducing the impact of chromatic distortion and quantization error caused by the image compression process on the reconstructed image.

[0123] like Figure 7 As shown in Figure 1, it is the structure of the quantization table guidance unit in multiple scale decoders, where the decoding quantization table guidance unit is composed of a PReLU activation function, two convolutional layers, a multi-layer perceptron (MLP), a Tanh activation function, and a Sigmiod activation function.

[0124] According to one embodiment of the present application, reference is still made to Figure 7As shown, the decoding quantization table guide unit in the decoder submodule is configured to calculate its corresponding output feature in the following manner:

[0125]

[0126]

[0127] in, Indicates the The output characteristics of the decoding quantization table guide unit, Indicates the The input features of the decoding quantization table guidance unit, Indicates the The second convolutional layer in the decoded quantization table guide unit, is the PReLU activation function, Indicates the The first convolutional layer in the decoded quantization table guide unit, Indicates the The first parameter calculated by the decoding quantization table guide unit, Indicates the The second parameter calculated by the decoding quantization table guide unit. It should be noted that j Corresponding to the above-mentioned first decoding quantization table guiding unit, second decoding quantization table guiding unit, and third decoding quantization table guiding unit.

[0128] As can be seen from the above description, the decoding quantization table guidance unit in the above embodiment of the present application uses a structure such as the PReLU activation function, convolutional layers, a multi-layer perceptron (MLP), the Tanh activation function, and the Sigmoid activation function to further optimize the feature extraction and parameter calculation process, ensuring the detailed restoration and overall quality of the reconstructed image. This solution can significantly improve the reconstruction effect of JPEG images.

[0129] According to one embodiment of the present application, a four-layer multilayer perceptron (MLP) is designed to learn transformation parameter pairs (i.e., first and second parameters) at three scales. The first three layers of the MLP are responsible for generating a shared embedding representation of the quantization table, while the last layer focuses on learning the specific parameter pairs for each scale.

[0130] As can be seen from the above embodiments, the above embodiments of the present application achieve efficient learning of transformation parameter pairs (first and second parameters) at three scales by designing a four-layer multi-layer perceptron (MLP). The first three layers of the MLP effectively capture the global characteristics of the quantization table by generating a shared embedding representation of the quantization table, reducing computational complexity. The final layer, on the other hand, focuses on learning specific parameter pairs for each scale, ensuring targeted and adaptable feature transformations at different scales. This layered design not only improves the efficiency and accuracy of parameter learning but also enhances the multi-scale decoder's ability to decode image features, significantly improving the quality and detail restoration of image reconstruction.

[0131] According to one embodiment of the present application, each scale encoder includes four encoding residual units, and each scale decoder includes four decoding residual units. They are used to process the input features in sequence. It should be noted that the input and output of each residual unit are processed according to the conventional feature extraction method in the deep learning method, and the input and output of each residual unit will not be described here.

[0132] According to one embodiment of the present application, the residual units in the encoder and decoder are both composed of two PReLU activation functions with 3×3 convolutional layers.

[0133] As can be seen from the above description, the above embodiment of the present application achieves deep processing and optimization of input features by introducing 4 residual units in the encoder and decoder at each scale. The residual unit effectively alleviates the gradient vanishing problem during image reconstruction model training through residual connection and nonlinear transformation, enhances the expressiveness of features, and can capture richer detail information. The introduction of this multi-level residual structure significantly improves the feature extraction and reconstruction capabilities of the encoder and decoder, ensuring accurate encoding of image features and high-quality reconstruction of images.

[0134] Based on the above analysis, the encoder submodule of the above embodiment of the present application includes an encoder at three scales, and the decoder submodule includes a decoder at three scales. Given the same number of parameters, shallow-scale (1 scale, 2 scales) encoder-decoders have poor context learning capabilities and high computational overhead. In contrast, deep-scale (3 scales) encoder-decoders not only reduce computational overhead but also enhance the model's ability to remove artifacts from images with varying degrees of compression. At the same time, in order to effectively reduce the chroma distortion phenomenon in the image compression process in the above embodiment, the present application uses the prior knowledge of the quantization table (chroma and luminance quantization table) to perform channel-level affine transformation on the features of different scales of the encoder-decoder network, wherein the quantization table guidance unit is used to guide the encoder-decoder network to reduce the chroma distortion in the preliminary reconstructed RGB image, which is caused by the downsampling operation of the chroma during the image compression process. The quantization table guidance unit is embedded in both ends of each scale of the encoder and decoder networks, aiming to use the prior information of the quantization table (such as the luminance quantization table and the chroma quantization table) to guide the networks of different scales to emphasize features that are critical to chroma restoration; and introduces a channel attention mechanism, so that both the encoder and the decoder can efficiently learn the common embedded representation of the quantization table, and use these representations to guide the image reconstruction model to adaptively enhance beneficial features, thereby improving the generalization performance of the image reconstruction model under different degrees of chroma distortion.

[0135] In addition, in order to make the feature information flow better between different scales, pixel shuffling and inverse pixel shuffling are applied in the model for upsampling and downsampling respectively to reduce the loss of spatial details.

[0136] To this end, according to one embodiment of the present application, the encoders of the three scales and the decoders of the three scales include a first sampling unit and a second sampling unit, respectively, wherein the first sampling unit is used to upsample the input features of the encoder and / or decoder, and transmit the processing results to the encoding quantization table guide unit and / or the decoding residual unit for processing; the second sampling unit is used to downsample the output features of the encoder and / or decoder, and transmit the processing results to the encoder and / or decoder of the next scale.

[0137] According to an embodiment of the present application, the first sampling unit and the second sampling unit are respectively configured to perform upsampling and downsampling operations using a pixel shuffle method and an inverse pixel shuffle method, respectively.

[0138] As can be seen from the above description, the pixel shuffling in the above embodiment of the present application increases the resolution of the feature map by rearranging the pixel positions, while the inverse pixel shuffling reduces the resolution by performing the opposite operation. This method can effectively reduce the loss of spatial detail in the feature map during scaling, thereby preserving more image detail information, making the image reconstruction model more efficient and accurate when processing multi-scale features, and improving overall performance.

[0139] 3. Residual Submodule

[0140] According to one embodiment of the present application, Figure 8 As shown, the encoding and decoding module also includes a residual submodule, which is used to perform residual processing on the output of the encoder submodule and output the processing result to the decoder submodule for processing.

[0141] According to one embodiment of the present application, the residual submodule is composed of two PReLU activation functions with a 3×3 convolutional layer.

[0142] As can be seen from the above description, the above embodiment of the present application can perform residual processing on the output of the encoder submodule by introducing a residual submodule in the codec module, thereby effectively capturing and retaining high-frequency detail information in the input features. By passing the processing results to the decoder submodule, the residual submodule further enhances the expressiveness of features, reduces information loss, and alleviates the gradient vanishing problem in deep networks. This design significantly improves the feature extraction and reconstruction capabilities of the codec module, ensures that the decoder can more accurately restore image details, and ultimately improves the overall performance and quality of image processing tasks.

[0143] Step S3

[0144] According to one embodiment of the present application, step S3 includes: during each iterative training process, using a preset total loss function to update the parameters of the image reconstruction model.

[0145] According to one embodiment of the present application, the total loss function is:

[0146]

[0147] in, represents the total loss function, represents the mean absolute error loss function, represents the fast Fourier transform loss function, represents the hyperparameter. =0.1. It should be noted that this value is an example value taken during the experiment of this application. The specific value can be adjusted according to the specific experimental results and is not specifically limited here.

[0148] According to one embodiment of the present application,

[0149]

[0150]

[0151] in, is the number of JPEG images, Indicates the The real image before JPEG image compression, Indicates the The reconstructed image after the JPEG image passes through the image reconstruction model, is the fast Fourier transform function.

[0152] As can be seen from the above description, the above-mentioned embodiment of the present application can effectively improve the quality of the reconstructed image and the ability to restore details by training the image reconstruction model by combining the mean absolute error loss and the fast Fourier transform loss to form a total loss function. Among them, the mean absolute error loss directly measures the distance between the reconstructed image and the real image in the pixel domain, ensuring the accuracy of the reconstructed image in terms of overall structure and low-frequency information; while the fast Fourier transform loss converts the image to the frequency domain and calculates the frequency domain distance between the two, specifically optimizing the high-frequency information lost during the image compression process (i.e., using the frequency domain loss to guide the model to restore high-frequency details), thereby guiding the image reconstruction model to restore the details and texture of the image. This dual-loss design not only takes into account the overall fidelity of the image, but also significantly enhances the reconstruction effect of high-frequency details, allowing the model to still generate more visually realistic and delicate images in complex compression scenarios.

[0153] Furthermore, based on the method for constructing an image reconstruction model of the above embodiment, according to one embodiment of the present application, the present application proposes an image reconstruction model, which includes: a learnable offset guidance module for calculating a corrected first feature of the frequency domain coefficients based on the acquired frequency domain coefficients and quantization table of the JPEG image; a domain transformation module for converting the corrected first feature of the frequency domain coefficients to the pixel domain to obtain the first pixel feature of the JPEG image, and calculating a reconstructed image corresponding to the JPEG image based on the first pixel feature. For further details on the functions of the constructed image reconstruction model, please refer to the description in the above construction method embodiment, and will not be repeated here.

[0154] In addition, according to one embodiment of the present application, the present application proposes a JPEG decoder, in which the image reconstruction model of the above embodiment is configured.

[0155] As can be seen from the above description, the JPEG decoder proposed in the above embodiments of this application integrates an image reconstruction model. This decoder can effectively restore image details and textures, generating reconstructed images that are more visually realistic and detailed. This design not only improves the limitations of traditional JPEG decoders in compressed image processing, but also provides users with a higher-fidelity image decoding experience, with broad application value and practicality.

[0156] In summary, the above embodiments of the present application gradually guide the image reconstruction model to reduce quantization error and chromatic distortion by compressing prior knowledge, thereby enhancing the visual quality of the image.

[0157] In order to verify the effectiveness of the image reconstruction model construction scheme proposed in the above embodiment of the present application, the inventors conducted the following experiments:

[0158] 1. Dataset Description

[0159] This experiment uses the LIVE1 and BSDS500 test sets, as well as the ICB dataset, to evaluate the performance of the proposed IJCN (the image reconstruction model) on color JPEG image restoration. For grayscale JPEG image restoration, the IJCN is trained using the Y channel as the grayscale image, using only the luminance quantization table to guide the network. The performance of the IJCN is evaluated on the Classic5 and LIVE1 datasets.

[0160] II. Description of Evaluation Indicators

[0161] The evaluation used three metrics: PSNR (dB), SSIM, and PSNR-B (dB). The higher their values, the better the image restoration effect. PSNR-B is a metric specifically used to evaluate the blocking effect in the image.

[0162] (1) PSNR (Peak Signal-to-Noise Ratio): PSNR is based on the mean square error (MSE) and measures the pixel-level error between the reconstructed image and the true image, expressed in decibels (dB). Advantages: Simple calculation and clear physical meaning (directly reflects pixel error). Disadvantages: Weak correlation with human visual perception (HVS), sensitive to global brightness changes, but insensitive to local structures (such as textures and edges).

[0163] (2) SSIM (Structural Similarity Index): SSIM evaluates image similarity based on three dimensions: brightness, contrast, and structure, with a range of [-1, 1] (1 indicating perfect similarity). Advantages: More consistent with human visual perception and sensitive to the preservation of structural information (such as edges and textures). It comprehensively evaluates differences in brightness, contrast, and structure. Disadvantages: The computational complexity is higher than PSNR and the sensitivity to local distortion (such as blocking artifacts) is limited.

[0164] (3) PSNR-B (Peak Signal-to-Noise Ratio for Block Effect): PSNR-B is an improved version of PSNR, specifically designed to evaluate blocking effects. Block effects are common in images compressed with JPEG or after block processing, and appear as discontinuous boundaries between adjacent blocks. Advantages: It directly quantifies the severity of blocking effects, making up for the shortcomings of PSNR and SSIM, which are insensitive to blocking effects. It provides targeted guidance for the optimization of compression algorithms (such as JPEG). Disadvantages: It is only applicable to scenarios with blocking effects (such as compressed images) and ignores distortion in non-boundary areas.

[0165] 3. Experimental Results

[0166]

[0167] Table 1 above is a comparison of experimental results for color JPEG image artifact removal. Among them, the corresponding PSNR (dB), SSIM and PSNR-B (dB) values ​​of existing methods such as JPEG, QGAC, FBCNN, CRL, and DAGN when the quality factors are set to 10, 20, 30, and 40 respectively are not as high as those obtained by the solution proposed in this application. The higher the values ​​of these indicators, the better the image restoration effect.

[0168]

[0169] Table 2 above is a comparison of experimental results for grayscale JPEG image artifact removal. Among them, the corresponding PSNR (dB), SSIM and PSNR-B (dB) values ​​of existing methods such as JPEG, QGAC, FBCNN, CRL, and DAGN when the quality factors are set to 10, 20, 30, and 40 respectively (the higher the quality factor, the better the compressed image quality) are not as high as those obtained by the solution proposed in this application. The higher the values ​​of these indicators, the better the image restoration effect.

[0170] As can be seen from the above description, the inventors compared IJCN (the image reconstruction model proposed in this application) with several state-of-the-art methods, including QGAC, FBCNN, CRL, and DAGN. The evaluation focused on quality factors of 10, 20, 30, and 40. As shown in Tables 1 and 2, IJCN outperformed existing methods in color JPEG image restoration tasks and achieved comparable performance to state-of-the-art methods in grayscale JPEG image restoration tasks, demonstrating its robustness and generalization capabilities for enhancing JPEG images.

[0171] To sum up, compared with the existing JPEG artifact removal method, the above-mentioned embodiment of the present application takes into account the root cause of image distortion during the image compression process, that is, inputs the prior knowledge of compression into the image reconstruction model, so that the image reconstruction model gradually reduces the distortion caused by irreversible operations during the compression process, thereby obtaining a high-quality reconstructed image.

[0172] Based on the inventive concept of the above embodiments, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in the above embodiments when executing the computer program. Figure 9 Provide detailed explanation.

[0173] like Figure 9 As shown in FIG, the electronic device 100 of the present application is shown, which may specifically include a processor 110 and a memory 120. The memory 120 is coupled to the processor 110.

[0174] Processor 110 is used to control the operation of the electronic device. Processor 110 may also be referred to as a CPU (Central Processing Unit). Processor 110 may be an integrated circuit chip with signal processing capabilities. Processor 110 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component. A general-purpose processor may be a microprocessor, or processor 110 may be any conventional processor.

[0175] The memory 120 is used to store computer programs and can be RAM, ROM, or other types of storage terminals. Specifically, the memory 120 may include one or more computer-readable storage media, which may be non-transitory or transient. The memory 120 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage terminals and flash memory storage terminals. In some embodiments, the non-transitory computer-readable storage medium in the memory 120 is used to store at least one program code.

[0176] The processor 110 is configured to execute the computer program stored in the memory 120 to implement the methods described in the various method embodiments of the present application.

[0177] In some embodiments, the electronic device may further include a peripheral terminal interface 130 and at least one peripheral terminal. The processor 110, memory 120, and peripheral terminal interface 130 may be connected via a bus or signal lines. Each peripheral terminal may be connected to the peripheral terminal interface 130 via a bus, signal lines, or circuit boards. Specifically, the peripheral terminal includes at least one of a radio frequency circuit 140, a display screen 150, an audio circuit 160, and a power supply 170.

[0178] The peripheral terminal interface 130 can be used to connect at least one peripheral terminal related to I / O (Input / Output) to the processor 110 and the memory 120. In some embodiments, the processor 110, the memory 120, and the peripheral terminal interface 130 are integrated on the same chip or circuit board; in some other implementations, any one or two of the processor 110, the memory 120, and the peripheral terminal interface 130 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0179] The RF circuit 140 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 140 communicates with communication networks and other IoT devices via electromagnetic signals, and is therefore the communication circuitry of the electronic device. The RF circuit 140 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. Optionally, the RF circuit 140 includes an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, an operator identification module card, and the like. The RF circuit 140 can communicate with other terminals via at least one wireless communication protocol. Such wireless communication protocols include, but are not limited to, the World Wide Web, metropolitan area networks, intranets, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and / or WiFi (Wireless Fidelity) networks. In some embodiments, the RF circuit 140 may also include circuitry related to Near Field Communication (NFC), although this application does not limit this.

[0180] The display screen 150 is used to display a user interface (UI). The UI may include graphics, text, icons, videos, or any combination thereof. When the display screen 150 is a touch screen display, the display screen 150 is also capable of collecting touch signals on or above the surface of the display screen 150. The touch signals can be input as control signals to the processor 110 for processing. In this case, the display screen 150 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, there can be one display screen 150, disposed on the front panel of the electronic device; in other embodiments, there can be at least two display screens 150, disposed on different surfaces of the electronic device or in a foldable design; in still other embodiments, the display screen 150 can be a flexible display, disposed on a curved or foldable surface of the electronic device. Furthermore, the display screen 150 can be configured as a non-rectangular irregular shape, i.e., a special-shaped screen. The display screen 150 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0181] The audio circuit 160 may include a microphone and a speaker. The microphone is used to collect sound waves from the operator and the environment, and convert the sound waves into electrical signals and input them into the processor 110 for processing, or input them into the radio frequency circuit 140 to achieve voice communication. For the purpose of stereo acquisition or noise reduction, there can be multiple microphones, which are respectively set in different parts of the electronic device. The microphone can also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 110 or the radio frequency circuit 140 into sound waves. The speaker can be a traditional thin film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 160 may also include a headphone jack.

[0182] Power supply 170 is used to power various components in the electronic device. Power supply 170 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 170 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged via a wired line, while a wireless rechargeable battery is a battery that is charged via a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0183] For a detailed description of the functions and execution processes of the functional modules or components in the electronic device embodiments of the present application, please refer to the description in the above-mentioned method embodiments of the present application, which will not be repeated here.

[0184] In the several embodiments provided in this application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the various embodiments of the electronic devices described above are merely illustrative. For example, the division of modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some data can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0185] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0186] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0187] Based on the inventive concept of the above embodiments, the present application further provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the steps of the method described in any of the above embodiments are described. Figure 10 The execution process of the above embodiment in a computer-readable storage medium is described.

[0188] like Figure 10As shown, it shows the computer-readable storage medium of the present application. If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in the computer-readable storage medium 200. Based on this understanding, the technical solution of the present application, or the part that contributes to the existing technology, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions / computer programs for enabling an IoT device (which can be a personal computer, server, or network terminal, etc.) or a processor to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes various media such as USB flash drives, mobile hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks or optical disks, as well as electronic terminals such as computers, mobile phones, laptops, tablet computers, cameras, etc. that have the above-mentioned storage media.

[0189] The description of the execution process of the program data in the computer-readable storage medium can refer to the description in the above-mentioned method embodiments of the present application, and will not be repeated here.

[0190] The above description is merely an embodiment of the present application and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

[0191] Those skilled in the art will understand that in the above-mentioned method of the specific implementation method, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

Claims

1. A method for constructing an image reconstruction model, wherein the image reconstruction model is used to reconstruct a JPEG image, characterized in that: The method comprises: Acquire a JPEG image and decode it to obtain frequency domain coefficients and a quantization table of the JPEG image, wherein the JPEG image comprises an image obtained by compressing an uncompressed real image of the JPEG image using different quality factors, the frequency domain coefficients comprise a luminance coefficient and a chrominance coefficient, and the quantization table comprises a luminance quantization table and a chrominance quantization table; Constructing an initial image reconstruction model based on a deep learning method, wherein the initial image reconstruction model takes the frequency domain coefficients and the quantization table as input and outputs a reconstructed image corresponding to the JPEG image; The frequency domain coefficients and the quantization table constitute a data set, and the initial image reconstruction model is iteratively trained multiple times using the data set until convergence to obtain the image reconstruction model, wherein, during each iterative training process, the parameters of the image reconstruction model are updated according to the total loss function determined by the real image and the reconstructed image.

2. The method for constructing an image reconstruction model according to claim 1, wherein: The initial image reconstruction model includes: A learnable offset guidance module, configured to calculate a corrected first feature of the frequency domain coefficient based on the frequency domain coefficient and the quantization table; A domain transformation module is used to convert the corrected first feature of the frequency domain coefficient into the pixel domain to obtain the first pixel feature of the JPEG image, and calculate the reconstructed image corresponding to the JPEG image based on the first pixel feature.

3. The method for constructing an image reconstruction model according to claim 2, wherein: The learnable offset guidance module includes: A quantization offset submodule, configured to calculate a quantization offset feature based on the frequency domain coefficients and the quantization table; The frequency domain correction submodule is used to calculate the first feature of the corrected frequency domain coefficient according to the quantization offset feature and the frequency domain coefficient.

4. The method for constructing an image reconstruction model according to claim 2, wherein: The learnable offset guidance module further includes a dequantization submodule, and: The dequantization submodule is configured to calculate a dequantized frequency domain coefficient feature based on the corrected first frequency domain coefficient feature and the quantization table; The domain transformation module is further used to convert the dequantized frequency domain coefficient features into the pixel domain to obtain a second pixel feature of the JPEG image, and calculate a reconstructed image corresponding to the JPEG image based on the second pixel feature.

5. The method for constructing an image reconstruction model according to claim 4, wherein: The image reconstruction model also includes: A codec module is configured to calculate a quantization guide feature of the JPEG image according to the quantization table, the first pixel feature, or the second pixel feature, and calculate a reconstructed image corresponding to the JPEG image based on the quantization guide feature.

6. The method for constructing an image reconstruction model according to claim 5, wherein: The codec module includes: an encoder submodule, configured to calculate a target intermediate feature of the JPEG image based on the quantization table, the first pixel feature, or the second pixel feature; The decoder submodule is used to calculate the quantization guide feature of the JPEG image according to the target intermediate feature and the quantization table, and calculate the reconstructed image corresponding to the JPEG image based on the quantization guide feature.

7. An image reconstruction model obtained based on the construction method according to any one of claims 1 to 6, characterized in that: The image reconstruction model includes: A learnable offset guidance module is used to calculate a corrected first feature of the frequency domain coefficient according to the acquired frequency domain coefficient and quantization table of the JPEG image; A domain transformation module is used to convert the corrected first feature of the frequency domain coefficient into the pixel domain to obtain the first pixel feature of the JPEG image, and calculate the reconstructed image corresponding to the JPEG image based on the first pixel feature.

8. An application method of an image reconstruction model, characterized in that: The method comprises: Acquire a JPEG image and decode it to obtain frequency domain coefficients and a quantization table of the JPEG image; The image reconstruction model according to claim 7 is used to calculate a reconstructed image corresponding to the JPEG image based on the frequency domain coefficients and the quantization table.

9. An electronic device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method according to any one of claims 1 to 6 and 8 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 and 8 are implemented.

Images