A construction and application method, system, device, and medium of an image reconstruction system

Abstract

The present application relates to the field of image processing. The present application proposes a construction and application method, model, device, and medium for an image reconstruction system. The method constructs a two-stage task decoupling architecture (i.e., constructs an initial image reconstruction system consisting of a first-stage quality-aware prior learning model and a second-stage hierarchical guided feedback model) to respectively realize the extraction of compression prior features and image reconstruction (i.e., artifact removal) of JPEG images. The first-stage architecture (i.e., the quality-aware prior learning model) is used to perform multi-level iterative residual feature extraction operations on JPEG images to obtain fine-grained hierarchical compression prior residual features to capture quality changes between regions; the second-stage architecture (i.e., the hierarchical guided feedback model) is used to perform alignment and fusion processing based on the compression features of JPEG images and the hierarchical compression prior residual features, which can capture the distribution characteristics of local artifacts and improve the quality of the reconstructed image.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a method, model, device, and medium for constructing and applying an image reconstruction system. Background Art

[0002] In modern digital image processing, the rapid growth of image data poses significant challenges to storage, transmission, and processing. To address this issue, image compression technology has emerged. Its core goal is to reduce data redundancy while preserving visual information as much as possible, thereby reducing storage space and transmission bandwidth requirements. JPEG, a widely used lossy compression standard, significantly reduces file size while maintaining high subjective visual quality by removing high-frequency details that are insensitive to the human eye through quantization and encoding. This efficient compression method has made JPEG a mainstream format for image storage and transmission, particularly in resource-constrained scenarios such as network transmission and mobile devices. However, in the image compression process, especially for lossy compression methods like JPEG, the compression ratio is typically expressed by the quality factor (QF). Low QFs often introduce significant artifacts, resulting in a decrease in visual quality, affecting both the visual quality of the image and the performance of subsequent processing tasks. However, in many practical applications (such as medical image analysis, security monitoring, and satellite remote sensing), high image quality is required, and directly using compressed images may not meet practical requirements.

[0003] Therefore, JPEG artifact removal technology has emerged, aiming to mitigate compression-induced distortion and restore the visual quality of images. Clear JPEG images are crucial for downstream low-level vision tasks that use them as input, including contrast enhancement, super-resolution, and edge detection. Traditional JPEG artifact removal methods primarily use manually designed filters to filter artifacts, or formulate artifact removal as a maximum a posteriori problem and introduce prior-based methods. However, these traditional methods rely on manually designed prior knowledge, resulting in limited restoration effectiveness. Therefore, they often struggle to adapt to complex and changing image content and compression scenarios.

[0004] In summary, there is an urgent need for a solution that can effectively remove artifacts that appear in compressed images. This solution can restore lost details in compressed images, suppress distortion, and improve the overall quality of the image. Summary of the Invention

[0005] The present application provides a method, system, device, and medium for constructing and applying an image reconstruction system to address the problem that existing JPEG artifact removal solutions are difficult to adapt to complex and changeable image content and compression scenarios, resulting in the inability to restore lost details in compressed JPEG images and suppress distortion, thereby resulting in low quality of reconstructed JPEG images.

[0006] A first aspect of the present application provides a method for constructing an image reconstruction system, wherein the image reconstruction system is used to reconstruct a JPEG image. The method comprises:

[0007] Get JPEG images and build a dataset based on them;

[0008] Construct an initial image reconstruction system, wherein the initial image reconstruction system includes: a quality-aware prior learning model and a hierarchical guided feedback model, and:

[0009] The quality-aware prior learning model is configured to: perform a multi-level iterative residual feature extraction operation on the JPEG image to obtain a hierarchical compression prior residual feature of the JPEG image;

[0010] The hierarchical guided feedback model is configured to: perform a feature extraction operation on the JPEG image to obtain a compressed feature of the JPEG image, perform an alignment and a fusion operation on the hierarchical compressed prior residual feature and the compressed feature, and calculate a reconstructed image of the JPEG image based on the result of the operation;

[0011] The initial image reconstruction system is trained multiple times iteratively using the data set until convergence to obtain the image reconstruction system.

[0012] In some embodiments of the present application, the hierarchical compressed prior residual features include a last layer compressed prior residual feature, and the step of constructing the initial image reconstruction system includes:

[0013] The quality-aware prior learning model is further configured to: perform a mapping operation on the last layer compressed prior residual features to obtain a quality factor prediction value of the JPEG image;

[0014] The hierarchical guided feedback model is configured to calculate a reconstructed image of the JPEG image based on the quality factor prediction value and the local calibration features, where the local calibration features are the result of the fusion operation.

[0015] In some embodiments of the present application, the step of constructing the initial image reconstruction system further includes:

[0016] The quality-aware prior learning model is further configured to perform a residual feature extraction operation on the last layer compressed prior residual feature, and perform a mapping operation on the result of the residual feature extraction operation to obtain a quality factor prediction value of the JPEG image.

[0017] In some embodiments of the present application, the hierarchical guided feedback model includes: a feature extraction submodule, an encoder submodule, and a decoder submodule, and the step of constructing the initial image reconstruction system further includes:

[0018] The feature extraction submodule is configured to: perform a feature extraction operation on the JPEG image to obtain compression features of the JPEG image;

[0019] The encoder submodule is configured to: align and fuse the hierarchical compressed prior residual features and the compressed features to obtain local calibration features of the JPEG image, and calculate the global calibration coding features of the JPEG image based on the quality factor prediction value and the local calibration features;

[0020] The decoder submodule is configured to calculate a reconstructed image of the JPEG image based on the global calibration coding features.

[0021] In some embodiments of the present application, the step of constructing the initial image reconstruction system further includes:

[0022] The decoder submodule is further configured to calculate a global calibration decoding feature of the JPEG image based on the local calibration feature or the global calibration encoding feature and the quality factor prediction value, and calculate a reconstructed image of the JPEG image based on the global calibration decoding feature.

[0023] In some embodiments of the present application, the step of performing multiple iterative training on the initial image reconstruction system using the data set until convergence includes:

[0024] The quality perception prior learning model is trained iteratively multiple times until convergence using a preset first loss function;

[0025] The converged quality-aware prior learning model and the hierarchical guided feedback model constitute the initial image reconstruction system, and then the preset second loss function is used to perform multiple iterative training on the initial image reconstruction system until convergence. In each iterative training process, the parameters of the quality-aware prior learning model are frozen and then participate in the training.

[0026] A second aspect of the present application provides an image reconstruction system constructed using the method for constructing an image reconstruction system according to any of the above embodiments, the image reconstruction system comprising:

[0027] A quality-aware prior learning model is used to perform multi-level iterative residual feature extraction operations on the acquired JPEG image to obtain hierarchical compression prior residual features of the JPEG image;

[0028] The hierarchical guided feedback model is used to perform feature extraction operations on JPEG images to obtain compressed features of the JPEG images, and to align and fuse the hierarchical compressed prior residual features and compressed features, and to calculate the reconstructed image of the JPEG image based on the results of the operations.

[0029] A third aspect of the present application provides an application method of an image reconstruction system, and the application method of the present application includes:

[0030] Get a JPEG image;

[0031] Using the quality perception prior learning model in the image reconstruction system described in the above embodiment, a multi-level iterative residual feature extraction operation is performed on the JPEG image to obtain a hierarchical compression prior residual feature of the JPEG image;

[0032] Using the hierarchical guided feedback model in the image reconstruction system as described in the above embodiment, a feature extraction operation is performed on the JPEG image to obtain the compression features of the JPEG image, and the hierarchical compression prior residual features and the compression features are aligned and fused, and a reconstructed image of the JPEG image is calculated based on the results of the operations.

[0033] The fourth aspect of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the method described in any one of the first and third aspects of the above embodiments are implemented.

[0034] The fifth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method described in any one of the first and third aspects of the above embodiments is implemented.

[0035] This application has the following beneficial effects:

[0036] The above-mentioned embodiment of the present application constructs a two-stage task decoupling architecture (i.e., constructs an initial image reconstruction system consisting of a first-stage quality-aware prior learning model and a second-stage hierarchical guided feedback model) to respectively realize the extraction of compression prior features and image reconstruction (i.e., artifact removal) of JPEG images, wherein the first-stage architecture (i.e., the quality-aware prior learning model) is used to perform multi-level iterative residual feature extraction operations on JPEG images to obtain fine-grained hierarchical compression prior residual features to capture quality changes between regions; the second-stage architecture (i.e., the hierarchical guided feedback model) is used to perform alignment and fusion processing based on the compression features of the JPEG image and the hierarchical compression prior residual features, which can capture the distribution characteristics of local artifacts to suppress image artifacts and restore image details, thereby improving the quality of the reconstructed image. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The drawings herein are incorporated into and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and, together with the specification, are used to illustrate the technical solutions of the present application.

[0038] Figure 1 This is a flow chart of an embodiment of a method for constructing an image reconstruction system provided by the present application;

[0039] Figure 2 It is a schematic diagram of the framework of multiple embodiments of the image reconstruction system provided by this application;

[0040] Figure 3 This is a schematic diagram of the framework of an embodiment of the quality perception prior learning model provided by this application;

[0041] Figure 4 This is a schematic diagram of the framework of an embodiment of a local alignment calibration unit provided by the present application;

[0042] Figure 5 This is a schematic diagram of the framework of an embodiment of a global guided calibration encoding unit provided by the present application;

[0043] Figure 6 This is a schematic diagram of the framework of an embodiment of a global guided calibration decoding unit provided by the present application;

[0044] Figure 7 This is a schematic diagram of the framework of an embodiment of an electronic device provided by the present application;

[0045] Figure 8 It is a schematic diagram of a framework of an embodiment of a computer-readable storage medium provided by this application. DETAILED DESCRIPTION

[0046] The following describes the embodiments of the present application in detail with reference to the accompanying drawings.

[0047] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0048] The term "and / or" in this article is simply a description of the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent three situations: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects are in an "or" relationship. In addition, "many" in this article means two or more than two. In addition, the term "at least one" in this article means any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0049] The inventors have found through research that the advancement of deep learning has enabled JPEG artifact removal to evolve from methods based on manually designed priors to methods based on deep learning. Relevant researchers have made significant improvements in artifact removal by constructing deep neural networks to learn the mapping relationship between compressed images and original images from large-scale paired data sets. However, this learning of the mapping relationship between compressed images and original images mainly relies on the data fitting ability of neural networks, and fails to use the inherent prior knowledge of JPEG to guide the reconstruction process, which still hinders the development of artifact removal models. The latest JPEG artifact removal method mainly uses the quality factor as a priori information to guide the restoration of the reconstructed image. However, as a global indicator, the quality factor can only roughly reflect the overall compression strength of the image and cannot accurately describe the distribution characteristics of local artifacts. Due to the significant heterogeneity of image content, there are obvious differences in the distribution of artifacts in texture-rich areas and smooth areas. It is difficult to achieve local adaptive artifact removal by relying solely on the global quality factor, which restricts the overall restoration quality of the reconstructed image. In addition, most existing methods adopt a single-stage network architecture, coupling prior extraction and artifact removal in the same network model for image reconstruction, ignoring the inherent conflict between the two. That is, accurate estimation of the degradation prior requires extracting compression distortion information, while artifact removal requires suppressing this distortion information to restore image details. This contradiction in learning objectives makes it difficult for a single-stage network architecture to optimize both tasks simultaneously, limiting further improvements in artifact removal and image quality.

[0050] In order to solve the above problems, this application proposes a new JPEG artifact removal scheme, which constructs a two-stage task decoupling architecture to respectively realize the extraction of compression prior features and image reconstruction (i.e., artifact removal) of JPEG images. Among them, the first-stage architecture (i.e., the quality-aware prior learning model) is used to perform multi-level iterative residual feature extraction operations on JPEG images to obtain fine-grained hierarchical compression prior residual features to capture the changes in quality between regions; the second-stage architecture (i.e., the hierarchical guided feedback model) is used to align and fuse the compression features of JPEG images and the hierarchical compression prior residual features, which can capture the distribution characteristics of local artifacts to suppress image artifacts and restore image details, thereby improving the quality of the reconstructed image.

[0051] The present application is described below with reference to the accompanying drawings and specific embodiments.

[0052] According to one embodiment of the present application, Figure 1 As shown, the present application proposes a method for constructing an image reconstruction system, the method comprising: S1, obtaining a JPEG image and constructing a data set based thereon; S2, constructing an initial image reconstruction system, wherein, as Figure 2As shown, the initial image reconstruction system includes: a quality-aware prior learning model and a hierarchical guided feedback model, wherein: the quality-aware prior learning model is configured to: perform a multi-level iterative residual feature extraction operation on the JPEG image to obtain a hierarchical compressed prior residual feature of the JPEG image; the hierarchical guided feedback model is configured to: perform a feature extraction operation on the JPEG image to obtain a compressed feature of the JPEG image, align and fuse the hierarchical compressed prior residual feature and the compressed feature, and calculate a reconstructed image of the JPEG image based on the results of the operation; S3, using the dataset, iteratively train the initial image reconstruction system multiple times until convergence to obtain an image reconstruction system. Each model is described in detail below.

[0053] 1. Quality-aware prior learning model

[0054] (1) Hierarchical compression of the prior residual features includes the output of all residual layers

[0055] According to one embodiment of the present application, the hierarchical compression prior residual features include the last layer compression prior residual features, and the steps of constructing the initial image reconstruction system include: the quality-aware prior learning model is also configured to: perform a mapping operation on the last layer compression prior residual features to obtain a quality factor prediction value of the JPEG image; the hierarchical guided feedback model is configured to: calculate the reconstructed image of the JPEG image based on the quality factor prediction value and the first local calibration feature, wherein the first local calibration feature is the result after the fusion operation.

[0056] As can be seen from the above description, the quality-aware prior learning model in the above embodiment of the present application learns the global quality factor prior (i.e., the quality factor prediction value) and the local hierarchical compression feature prior (i.e., the hierarchical compression prior residual feature) from the input JPEG image. This prior information can not only reflect the global quality changes, but also accurately describe the distribution characteristics of local compression artifacts, providing important guidance for subsequent image reconstruction, thereby improving the quality of the reconstructed image.

[0057] (2) Hierarchical compression of prior residual features including the output of some residual layers

[0058] According to one embodiment of the present application, the step of constructing the initial image reconstruction system also includes: the quality-aware prior learning model is also configured to: perform a residual feature extraction operation on the last layer compression prior residual feature, and perform a mapping operation on the result after the residual feature extraction operation to obtain a quality factor prediction value of the JPEG image.

[0059] From the above description, it can be seen that the above embodiment of the present application can further enhance the expressive ability of the features by performing a residual feature extraction operation on the last layer compression prior residual features, thereby obtaining a more accurate quality factor prediction value of the JPEG image.

[0060] According to one embodiment of the present application, the quality-aware prior learning model is an improved ResNet34, wherein Figure 2 and 3 As shown in Figure 1, the improved ResNet34 consists of a convolutional layer, four residual layers, a pooling layer, and a multi-layer perceptron (MLP). The process for obtaining hierarchical compressed prior residual features for JPEG images is as follows: First, the input JPEG image is preliminarily processed by a 1×1 convolutional layer, expanding the number of channels to 64. Subsequently, four residual layers are introduced. Each residual layer is composed of multiple stacked residual blocks. Each residual block contains two 3×3 convolutional layers with a ReLU activation function in between to extract and deepen image features layer by layer. The output residual features of the first three residual layers serve as the hierarchical compressed prior residual features (i.e., the output of the third residual layer serves as the compressed prior residual features of the last layer), guiding feature calibration in the second stage. To effectively capture residual features at different scales, the number of output channels of each residual layer is set to 64, 128, 256, and 512, respectively, gradually enhancing feature representation capabilities. Furthermore, after each residual layer, a convolution with a stride of 2 is applied for downsampling, reducing the resolution of the residual feature map and increasing the receptive field, thereby capturing a wider range of contextual information. Still refer to Figure 3 As shown, the number of residual blocks in each residual layer is 3, 4, 6, and 3, respectively, to balance the model complexity and performance. After completing the feature extraction of all residual layers, the residual feature map size is compressed to 1×1 through adaptive average pooling for subsequent quality factor prediction. It should be noted that this application is based on ResNet34 to construct a quality-aware prior learning model, which is only an embodiment of this application. With the development of deep network models, this application can also use other residual networks with better performance to construct quality-aware prior learning models, which is not specifically limited here.

[0061] The inventors have found that using MLP to map hierarchical compression prior residual features to QF labels (i.e., the true value of the quality factor) can effectively handle local and complex artifacts in JPEG images. To this end, according to one embodiment of the present application, the present application introduces a multi-layer perceptron (MLP), still referring to Figure 2 and 3 As shown in the figure, the MLP consists of three fully connected layers, of which the first two layers are followed by ReLU activation functions to enhance the nonlinear expression ability of the model. The MLP takes a 512-dimensional feature vector as input and uses a hidden layer with 512 neurons, which can effectively capture local artifacts while preserving global features. Finally, a Sigmoid activation function is applied at the output of the MLP to generate the quality factor prediction value of the JPEG image. , its value is limited to the range of [0,1], and the specific formula is as follows:

[0062]

[0063] in, Represents the Sigmoid activation function, M represents a multi-layer perceptron, N refers to the network layers before the multi-layer perceptron (i.e., convolutional layer, four residual layers, pooling layer), and is the input JPEG image (can be a three-channel color image or a single grayscale channel). To obtain a QF prediction value that better meets the needs of actual applications, the output value is multiplied by 100 and rounded down to an integer value.

[0064] From the above description, it can be seen that the above embodiment of the present application constructs a quality-aware prior learning model based on a deep residual network to extract a composite prior consisting of a global quality factor and a local hierarchical compression feature. These prior information can not only reflect the global quality changes, but also accurately describe the distribution characteristics of local compression artifacts, providing important guidance for subsequent image reconstruction to improve the quality of the reconstructed image.

[0065] 2. Hierarchical Guidance Feedback Model

[0066] (1) Introducing quality factor prediction value at the encoder end

[0067] According to one embodiment of the present application, still referring to Figure 2 As shown, the hierarchical guided feedback model includes: a feature extraction submodule, an encoder submodule and a decoder submodule, and the step of constructing the initial image reconstruction system also includes: the feature extraction submodule is configured to: perform feature extraction operations on the JPEG image to obtain compression features of the JPEG image; the encoder submodule is configured to: perform alignment and fusion operations on the hierarchical compression prior residual features and the compression features to obtain local calibration features of the JPEG image, and calculate the global calibration coding features of the JPEG image based on the quality factor prediction value and the local calibration features; the decoder submodule is configured to: calculate the reconstructed image of the JPEG image based on the global calibration coding features.

[0068] It can be seen from the above embodiments that the above embodiments of the present application utilize the hierarchical compression prior residual features and quality factor prediction values ​​extracted by the quality-aware prior learning model in the first stage to guide the hierarchical guided feedback model in the second stage to perform feature calibration and image reconstruction. Among them, the hierarchical compression prior residual features are used to align and refine local features, suppress artifacts and restore details; and the quality factor prediction value is used to guide the integration of global information to ensure color consistency and structural integrity. Through this hierarchical guidance strategy, the present application maintains the global consistency of the image while restoring details, significantly improving the artifact removal effect. Among them, the introduction of the quality factor prediction value in the encoder submodule can effectively guide the integration of global information and ensure color consistency and structural integrity. Through the quality factor prediction value, the encoder submodule can better capture the overall characteristics of the JPEG image, suppress artifacts and enhance detail recovery, thereby providing higher quality global calibration coding features for the subsequent decoder submodule.

[0069] According to one embodiment of the present application, the local calibration feature includes a first local calibration feature, the global calibration coding feature includes a first global calibration coding feature, and still referring to Figure 2 As shown, the encoder submodule includes a first-scale local alignment calibration unit (i.e., LCM1), a first-scale global guided calibration encoding unit (i.e., GCM-EN1), and a first-scale downsampling unit (i.e., DS1), wherein the first-scale local alignment calibration unit is configured to align and fuse the hierarchical compressed prior residual features and the compressed features to obtain the first local calibration features of the JPEG image; the first-scale global guided calibration encoding unit is configured to calculate the first global calibration coding features of the JPEG image based on the quality factor prediction value and the first local calibration features; the first-scale downsampling unit is configured to downsample the first global calibration coding features and transmit the processing results to the decoder submodule for processing. Still referring to Figure 2 As shown, the decoder submodule includes a first-scale upsampling unit (i.e., US1), which is configured to upsample the first globally calibrated coded features output by the first-scale downsampling unit after the downsampling operation and calculate a reconstructed image of the JPEG image based on the result of the upsampling operation. The encoder submodule of this embodiment includes LCM1, GCM-EN1, and DS1, and the decoder submodule includes US1.

[0070] As can be seen from the above description, the above embodiment of the present application can effectively extract and compress high-level features of the image by introducing a downsampling operation in the encoder submodule, reducing computational complexity while retaining key information to guide subsequent reconstruction; introducing an upsampling operation in the decoder submodule can gradually restore the detailed information of the image. This combination of downsampling and upsampling in the encoder-decoder structure not only optimizes the efficiency of feature extraction and reconstruction, but also enhances the model's ability to recover local details and global consistency, thereby significantly improving the quality of image reconstruction.

[0071] According to one embodiment of the present application, the local calibration feature further includes a second local calibration feature, the global calibration coding feature further includes a second global calibration coding feature, and still refers to Figure 2 As shown, the encoder submodule also includes: a second-scale local alignment calibration unit (i.e., LCM2), a second-scale global guided calibration encoding unit (i.e., GCM-EN2), and a second-scale downsampling unit (i.e., DS2), wherein the second-scale local alignment calibration unit is configured to perform alignment and fusion operations on the first global calibration coding features after the downsampling operation output by the first-scale downsampling unit to obtain the second local calibration features of the JPEG image; the second-scale global guided calibration encoding unit is configured to calculate the second global calibration coding features of the JPEG image based on the quality factor prediction value and the second local calibration features; the second-scale downsampling unit is configured to perform downsampling operations on the second global calibration coding features and transmit the processing results to the decoder submodule for processing. Still referring to Figure 2 As shown, the decoder submodule also includes a second-scale upsampling unit (i.e., US2). The second-scale upsampling unit is configured to perform an upsampling operation on the second globally calibrated coded features after the downsampling operation output by the second-scale downsampling unit, and transmit the result to the first-scale upsampling unit for processing to calculate a reconstructed image of the JPEG image. The encoder submodule of this embodiment includes: LCM1, GCM-EN1, DS1, LCM2, GCM-EN2, and DS2, and the decoder submodule includes: US2 and US1.

[0072] According to one embodiment of the present application, the local calibration feature further includes a third local calibration feature, the global calibration coding feature further includes a third global calibration coding feature, and still refers to Figure 2As shown, the encoder submodule also includes: a third-scale local alignment calibration unit (i.e., LCM3) and a third-scale global guided calibration encoding unit (i.e., GCM-EN3), wherein the third-scale local alignment calibration unit is configured to align and fuse the second global calibration coding features after the downsampling operation output by the second-scale downsampling unit to obtain the third local calibration features of the JPEG image; the third-scale global guided calibration encoding unit is configured to calculate the third global calibration coding features of the JPEG image based on the quality factor prediction value and the third local calibration features and transmit them to the second-scale upsampling unit for processing. The encoder submodule of this embodiment includes: LCM1, GCM-EN1, DS1, LCM2, GCM-EN2, DS2, LCM3, GCM-EN3, and the decoder submodule includes: US2 and US1.

[0073] According to one embodiment of the present application, the number of GCM-EN1, GCM-EN2, and GCM-EN3 is 4. The specific number is not limited by the present embodiment and can be dynamically adjusted according to the experimental environment, experimental results, etc. during the experiment.

[0074] It should be noted that the number of scale settings of the local alignment calibration unit, global guided calibration encoding unit, and downsampling unit in the encoder submodule is not affected by Figure 2 The specific scale and number settings can be dynamically adjusted based on experimental results. The scale number and scale size of the downsampling unit and the upsampling unit correspond one to one.

[0075] As can be seen from the above description, the above embodiments of the present application introduce local alignment calibration units, global guided calibration encoding units, and downsampling units of different scales in the encoder submodule, and introduce upsampling units of different scales in the decoder submodule, which can perform fine alignment and global guidance of features at multiple resolution levels, effectively capture and fuse local details and global structural information of the image, and significantly improve the effects of artifact removal and detail recovery; at the same time, the introduction of multi-scale downsampling units and upsampling units realizes multi-level extraction and reconstruction of features, enhances the model's ability to express multi-scale information of the image, and further optimizes the quality of the reconstructed image, making it perform well in local details, global consistency and visual naturalness.

[0076] (2) Introducing quality factor prediction value at the decoder end

[0077] According to one embodiment of the present application, the step of constructing an initial image reconstruction system also includes: the decoder submodule is also configured to: calculate the global calibration decoding features of the JPEG image based on the local calibration features and the quality factor prediction value, and calculate the reconstructed image of the JPEG image based on the global calibration decoding features.

[0078] As can be seen from the above description, the above-mentioned embodiments of the present application introduce a quality factor prediction value into the decoder submodule, which can directly guide the reconstruction process and ensure that the color and structure of the final output image meet expectations. The decoder submodule uses the quality factor prediction value to more flexibly adjust the reconstruction strategy and optimize the balance between local detail and global consistency. This design enables the decoder submodule to more accurately restore image details while simplifying the complexity of the encoder submodule.

[0079] According to one embodiment of the present application, the global calibration decoding feature includes a first global calibration decoding feature, and still refers to Figure 2 As shown, the decoder submodule further includes a first-scale globally guided calibration decoding unit (i.e., GCM-DE1), and the first-scale upsampling unit is further configured to perform an upsampling operation on the first local calibration features after the downsampling operation output by the first-scale downsampling unit. The first-scale globally guided calibration decoding unit is configured to calculate a first global calibration decoding feature of the JPEG image based on the first local calibration features after the upsampling operation output by the first-scale upsampling unit and the quality factor prediction value, and calculate a reconstructed image of the JPEG image based on the first global calibration decoding feature. The encoder submodule of this embodiment includes LCM1 and DS1, and the decoder submodule includes US1 and GCM-DE1.

[0080] According to one embodiment of the present application, the global calibration decoding feature further includes a second global calibration decoding feature, and still refers to Figure 2 As shown, the decoder submodule also includes a second-scale globally guided calibration decoding unit (i.e., GCM-DE2), and the second-scale upsampling unit is further configured to perform an upsampling operation on the second local calibration features after the downsampling operation output by the second-scale downsampling unit. The second-scale globally guided calibration decoding unit is configured to calculate the second global calibration decoding features of the JPEG image based on the second local calibration features after the upsampling operation and the quality factor prediction value output by the second-scale upsampling unit, and transmit them to the first-scale upsampling unit for processing. The encoder submodule of this embodiment includes LCM1, DS1, LCM2, and DS2, and the decoder submodule includes US2, GCM-DE2, US1, and GCM-DE1.

[0081] According to one embodiment of the present application, the global calibration decoding feature further includes a third global calibration decoding feature, and still refers to Figure 2As shown, the decoder submodule also includes a third-scale globally guided calibration decoding unit (GCM-DE3), which is configured to calculate a third globally calibrated decoding feature for the JPEG image based on the third local calibration feature and the quality factor prediction value, and transmit the feature to the second-scale upsampling unit for processing. The encoder submodule of this embodiment includes LCM1, DS1, LCM2, DS2, and LCM3, and the decoder submodule includes GCM-DE3, US2, GCM-DE3, US1, and GCM-DE1.

[0082] According to one embodiment of the present application, the number of GCM-DE3, GCM-DE2, and GCM-DE1 is 4. The specific number is not limited by this embodiment and can be dynamically adjusted according to the experimental environment, experimental results, etc. during the experiment.

[0083] It should be noted that the number of scale settings for the local alignment calibration unit and the global guided calibration decoding unit in the decoder submodule is not affected by Figure 2 The specific scale and quantity settings can be dynamically adjusted based on experimental results. The encoder submodules in the above embodiments do not introduce a global guided calibration encoding unit, but only local alignment calibration units and downsampling units of different scales. The number and size of the scales of the local alignment calibration units correspond one-to-one to those of the global guided calibration decoding unit.

[0084] From the above description, it can be seen that the above embodiment of the present application can fully utilize multi-scale feature information, restore the global structural consistency of the image layer by layer, and dynamically calibrate the features of different scales in combination with the quality factor prediction value, thereby effectively improving the detail restoration and visual naturalness of the reconstructed image.

[0085] (3) Introducing quality factor prediction values ​​at both the encoder and decoder ends

[0086] According to one embodiment of the present application, the step of constructing an initial image reconstruction system also includes: the decoder submodule is also configured to calculate the global calibration decoding features of the JPEG image based on the global calibration encoding features and the quality factor prediction value, and calculate the reconstructed image of the JPEG image based on the global calibration decoding features.

[0087] As can be seen from the above embodiments, the above embodiments of the present application can achieve multi-level residual feature calibration and reconstruction optimization by introducing quality factor prediction values ​​in both the encoder and decoder submodules. The encoder submodule uses the quality factor prediction value to integrate global information to ensure the high quality of the initial features; the decoder submodule further uses the quality factor prediction value to refine the reconstruction process, restore details and maintain global consistency. This quality factor prior introduction strategy significantly improves the artifact removal effect, while taking into account the restoration of local details and global consistency, thereby obtaining a higher quality reconstructed image.

[0088] According to one embodiment of the present application, the third-scale globally guided calibration decoding unit is configured to: calculate the third globally guided calibration decoding feature of the JPEG image based on the third globally guided calibration encoding feature and the quality factor prediction value output by the third-scale globally guided calibration encoding unit, transmit the third globally guided calibration decoding feature to the second-scale upsampling unit for upsampling, and transmit the upsampling result to the second-scale globally guided calibration decoding unit for processing. The encoder submodule of this embodiment includes: LCM1, GCM-EN1, DS1, LCM2, GCM-EN2, DS2, LCM3, GCM-EN3, and the decoder submodule includes: GCM-DE3, US2, GCM-DE3, US1, GCM-DE1. The hierarchical guided feedback model can also be combined in such a way that the encoder submodule includes LCM1, GCM-EN1, and DS1, and the decoder submodule includes GCM-DE3, US1, and GCM-DE1, or the encoder submodule includes LCM1, GCM-EN1, DS1, LCM2, GCM-EN2, and DS2, and the decoder submodule includes US2, GCM-DE3, US1, and GCM-DE1. The specific combination is not limited to this embodiment and can be dynamically adjusted according to the experimental environment, experimental results, and the like during the experiment.

[0089] It should be noted that the number of scale settings of the local alignment calibration unit, global guided calibration encoding unit, downsampling unit and upsampling unit in the encoder submodule is not affected by Figure 2 The specific scale and quantity settings can be dynamically adjusted according to the experimental results. The scale settings and quantity of the local alignment calibration unit and the global guided calibration decoding unit in the decoder submodule are not limited by Figure 2 The specific scale and quantity settings can be dynamically adjusted based on experimental results. At the same time, if the experimental hardware and experimental computing efficiency support it, DS1 and DS2 in the encoder submodule and US2 and US1 in the decoder submodule of the above embodiment can be omitted.

[0090] It can be seen from the above description that the above embodiment of the present application introduces global guided calibration encoding units of different scales in the encoder submodule, which can dynamically calibrate the features in combination with the quality factor prediction value, effectively extract and retain the global structural information of the image at different resolutions, and provide high-quality encoding features for the subsequent decoding process; at the same time, global guided calibration decoding units of different scales are introduced in the decoder submodule, and the quality factor prediction value is further used to guide and optimize the features layer by layer, realizing the gradual reconstruction of features from coarse to fine, significantly improving the global consistency and detail restoration of the reconstructed image, and ultimately achieving better results in artifact removal, detail retention and overall visual quality.

[0091] Furthermore, according to one embodiment of the present application, the second-stage hierarchical guided feedback model is designed based on an asymmetric codec architecture similar to UNet. It utilizes the coarse-grained quality factor predictions (i.e., QF priors) extracted by the quality-aware prior learning model in the first stage and the fine-grained hierarchical compression prior residual features (i.e., hierarchical compression feature priors) to guide artifact removal from JPEG images at different compression levels from coarse to fine scales. The coarse-grained QF prior guides global restoration, while the fine-grained prior refines local details and corrects region-specific artifacts, thereby achieving more accurate image reconstruction. The encoder submodule integrates a local alignment unit at different scales, a globally guided alignment encoding unit, and a downsampling unit to process and enhance features at different scales. The downsampling unit includes a convolutional layer with a kernel size of 2 and a stride of 2 for downsampling. The decoder submodule upsamples the features processed by the globally guided alignment decoding unit through transposed convolutions, gradually restoring the details and structure of the compressed image. The encoder submodule and the decoder main module are connected through jump connections to preserve spatial detail information at different scales and ensure the integrity of feature transfer.

[0092] Among them, according to one embodiment of the present application, local alignment calibration units of different scales are configured to: use the attention mechanism to perform attention processing on their corresponding inputs to obtain a spatial attention weight map, and then use the spatial attention weight map to adjust the compressed features processed by convolution, and align and fuse the compressed features and the adjusted compressed features to obtain corresponding local calibration features.

[0093] From the above description, it can be seen that the above embodiment of the present application integrates the local alignment calibration unit into various scales of the encoder, enhances the feature extraction and context understanding capabilities through the attention mechanism, so that features can be fused and aligned within the local alignment calibration unit, thereby suppressing fine-grained artifacts and accurately restoring local details.

[0094] The following describes the structures of the different-scale local alignment calibration unit and the global-guided calibration encoding unit of the encoder submodule and the different-scale global-guided calibration decoding unit of the decoder submodule.

[0095] (1) Local alignment calibration unit of the encoder submodule

[0096] According to one embodiment of the present application, the local alignment calibration units of different scales each include two independent 3×3 convolutional layers and a 1×1 convolutional layer, and as shown in FIG. Figure 4 As shown, the local alignment calibration unit is configured to obtain the local calibration features in the following manner:

[0097]

[0098]

[0099] in, Indicates the The spatial attention weight map output by the scale-local alignment calibration unit, is the Sigmoid activation function, is a 1×1 convolutional layer, and Both are 3×3 convolutional layers, Indicates input Compressed features of JPEG images of scale-locally aligned calibration units, Indicates input The characteristics of the scale local alignment calibration unit, where if it is the first scale local alignment calibration unit, then is the hierarchical compressed prior residual feature of the input first-scale local alignment calibration unit; if it is the second-scale local alignment calibration unit, then is the feature output by the first scale downsampling unit; if it is the third scale local alignment calibration unit, then is the feature output by the second scale downsampling unit. and The processed results are fused element by element to obtain the feature map, which is further refined using a 1×1 convolutional layer; then, the Sigmoid activation function is applied to generate , used to adjust the The compressed features of the output are still referenced Figure 2 As shown, the input and output features in each LCM have the same dimensions. For example, in the first LCM, the dimensions of the input and output features are , the feature dimension in subsequent LCM is reduced by half layer by layer. By adjusting the compressed features of JPEG images, LCM can enhance the feature representation of important areas, effectively suppressing local artifacts and recovering details. Finally, a residual connection is introduced to align the adjusted compressed features with the compressed features of the input LCM to prevent information loss and ensure feature integrity. This design enables LCM to capture the distribution characteristics of local artifacts at different scales, providing important support for subsequent image restoration.

[0100] (2) Global guided calibration encoding unit of the encoder submodule

[0101] The quality factor prior is integrated into the encoder submodule to dynamically adjust the feature extraction strategy according to the image quality. Figure 5 As shown in Figure 2, the structures of the global guided calibration coding units of different scales all include a coding residual block, a coding initializer, and a coding generator. The coding residual block consists of two convolutional layers and a ReLU activation function to capture high-frequency details and residual information, thereby refining the features obtained from the previous processing; the coding initializer includes a three-layer multi-layer perceptron to map the QF prediction value to a high-dimensional feature space. , which enables the model to adjust the feature extraction process according to the image quality; the code generator includes a fully connected layer and a Sigmoid or Tanh activation function to generate modulation parameter pairs ( ), the code generator is configured to obtain :

[0102]

[0103]

[0104]

[0105] in, is the Sigmoid activation function, is the fully connected layer, represents the quality factor prediction value of the high-dimensional feature space, is the hyperbolic tangent function. Then, the modulation parameter is applied to ( ) Transform the features after the coding residual block processing, the transformation formula is:

[0106]

[0107] in, For the The scale global guidance calibrates the output of the coded residual block in the coding unit, Indicates the The scale global guide calibrates the first modulation parameter output by the code generator in the coding unit, Indicates the The scale-global guidance calibrates the second modulation parameters output by the code generator in the coding unit. Finally, a residual connection is introduced to prevent information loss and ensure feature integrity. The input features of the coded residual block in each scale-global guidance calibration coding unit are the outputs of the local alignment calibration unit at the corresponding scale.

[0108] It can be seen from the above description that the modulation parameters of the above embodiment of the present application ( ) is used to scale and offset features, enabling the model to adapt to images of varying compression levels, thereby optimizing output quality. This design enables the globally guided calibration encoding unit of the encoder submodule to dynamically adjust feature extraction strategies based on image quality, thereby globally enhancing color consistency and structural integrity, further improving artifact removal.

[0109] (3) Global guided calibration decoding unit of the decoder submodule

[0110] The quality factor prior is integrated into the decoder submodule to dynamically adjust the feature extraction strategy according to the image quality. Figure 6 As shown in Figure 2, the structure of the global guided calibration decoding unit includes a decoding residual block, a decoding initializer, and a decoding generator. The decoding residual block consists of two convolutional layers and a ReLU activation function to capture high-frequency details and residual information, thereby refining the features obtained from the previous processing; the decoding initializer includes a three-layer multi-layer perceptron to map the QF prediction value to a high-dimensional feature space. , which enables the model to adjust the feature extraction process according to the image quality; the decoding generator includes a fully connected layer and a Sigmoid or Tanh activation function to generate modulation parameter pairs ( ), the decoding generator is configured to obtain :

[0111]

[0112]

[0113]

[0114] in, is the Sigmoid activation function, is the fully connected layer, represents the quality factor prediction value of the high-dimensional feature space, is the hyperbolic tangent function. Then, the modulation parameter is applied to ( ) Transform the features after decoding the residual block. The transformation formula is:

[0115]

[0116] in, For the The scale global guide calibrates the output of the decoded residual block in the decoding unit, Indicates the The first modulation parameter output by the decoding generator in the scale global guide calibration decoding unit, Indicates the The scale global guidance calibrates the second modulation parameters output by the decoding generator in the decoding unit. Finally, the residual connection is introduced to prevent information loss and ensure the integrity of the features.

[0117] It can be seen from the above description that the modulation parameters of the above embodiment of the present application ( ) is used to scale and offset features, enabling the model to adapt to images of varying compression levels, thereby optimizing output quality. This design enables the decoder submodule's globally guided calibration decoding unit to dynamically adjust feature extraction strategies based on image quality, thereby globally enhancing color consistency and structural integrity, further improving artifact removal.

[0118] In addition, according to one embodiment of the present application, step S3 includes: using a preset first loss function to perform multiple iterative training on the quality perception prior learning model until convergence; the converged quality perception prior learning model and the hierarchical guided feedback model constitute an initial image reconstruction system, and then using a preset second loss function to perform multiple iterative training on the initial image reconstruction system until convergence, wherein, in each iterative training process, the parameters of the quality perception prior learning model are frozen and then participate in the training.

[0119] As can be seen from the above description, the above embodiment of the present application iteratively trains the quality-aware prior learning model until convergence using a preset first loss function, ensuring that the model can effectively capture image quality features. The converged model is then combined with a hierarchical guided feedback model to form an initial image reconstruction system, and iterative training is performed using a preset second loss function. During the training process, the parameters of the quality-aware prior learning model are frozen (frozen parameters refer to the fact that during the training of the second loss function, the parameters of the quality-aware prior learning model are always the parameters after convergence using the first loss function training), preventing parameter updates from interfering with the stability of the overall system. This method can significantly improve the performance of the image reconstruction system, ensuring high quality and high fidelity of the reconstructed image, while also improving training efficiency and model robustness.

[0120] According to one embodiment of the present application, the first loss function is:

[0121]

[0122] Where N is the number of JPEG images in the dataset, represents the quality factor prediction value of the m-th JPEG image, Indicates the true value of the quality factor of the m-th JPEG image.

[0123] As can be seen from the above description, the above embodiment of the present application adopts a first loss function based on the difference between the quality factor prediction value and the true value, which can effectively optimize the training process of the quality perception prior learning model and ensure that the model can accurately predict the quality factor of the JPEG image. The design of this loss function enables the model to better learn image quality characteristics during the training process, thereby improving the accuracy and robustness of the model's image quality perception. In addition, by minimizing the difference between the predicted value and the true value, this embodiment can significantly improve the performance of the model in image reconstruction tasks and provide more reliable quality factor prior information for subsequent image processing tasks.

[0124] According to one embodiment of the present application, the second loss function is:

[0125]

[0126] in, represents the mth JPEG image, Represents the reconstructed image of the m-th JPEG image.

[0127] As can be seen from the above description, the above-described embodiment of the present application, by employing a second loss function based on the difference between the original JPEG image and the reconstructed image, can effectively optimize the training process of the initial image reconstruction system, ensuring that the reconstructed image is highly consistent with the original image in both detail and overall quality. The design of this loss function enables the system to better learn the fine features of image reconstruction during training, thereby significantly improving the visual quality and fidelity of the reconstructed image.

[0128] In order to verify the effectiveness of the construction method of the above embodiment, the inventors conducted the following experiments:

[0129] Firstly, the DIV2K and Flickr2K datasets are compressed to different degrees with the help of the OpenCV library to obtain JPEG image datasets with different quality factors. Then, the JPEG image datasets are used to fine-tune the ResNet34 model pre-trained on the large-scale ImageNet dataset to extract hierarchical compression prior residual features and predict the quality factor.

[0130] To demonstrate the advantages of the technical solution proposed in this application, three evaluation metrics were used to measure the performance of different JPEG artifact removal models. The peak signal-to-noise ratio (PSNR) assesses image quality by calculating the pixel-level difference between the reconstructed image and the original image. The structural similarity (SSIM) is an image quality assessment method based on structural information. It considers similarities in brightness, contrast, and structure, and better reflects the perceptual characteristics of the human visual system than PSNR. The blocking-sensitive peak signal-to-noise ratio (PSNR-B) is an improved version of the traditional PSNR, incorporating block-level calculations to better reflect local image distortion. For all three metrics, larger values ​​indicate better model performance. This application fully validates the proposed solution in terms of reconstruction quality and computational complexity.

[0131] (1) Reconstruction quality: In order to demonstrate the superiority of the present application in reconstruction quality, the present application was compared with JPEG artifact removal methods such as QGAC, FBCNN, EARN and DAGN on the color LIVE1 and grayscale Classic5 datasets, where the DAGN method is only for color JPEG image artifact removal. It can be observed from Table 1 (Table 1 is the comparative experimental results on the color LIVE1 dataset) and Table 2 (Table 2 is the comparative experimental results on the Classic5 dataset) that the performance of the present application on both datasets is significantly better than the comparative methods, confirming the effectiveness and superiority of the image reconstruction system constructed by the present application in removing artifacts and restoring image quality. In addition, the present application further analyzes the influence of the embedding position of the QF prediction value in the codec submodule in the global guided calibration unit (corresponding to the global guided calibration encoding unit and the global guided calibration decoding unit) on the final image reconstruction result. As can be seen from Table 3 (Table 3 shows the ablation study of QF embedding position on the grayscale Classic5 dataset (PSNR / SSIM / PSNR-B)), the collaborative embedding of QF prior information at both ends of the codec submodule can more fully guide network learning, thereby improving the quality and consistency of image reconstruction.

[0132] Table 1

[0133]

[0134] Table 2

[0135]

[0136] Table 3

[0137]

[0138] (2) Computational Complexity: To evaluate computational efficiency, this application was compared with other advanced methods in terms of parameter count, floating-point operations, and runtime (comparison of parameter count, floating-point operations, and runtime on a 512×512 color image). The experimental results are shown in Table 4. The experimental results show that this application achieves superior performance at a lower computational cost, demonstrating an effective balance between quality and efficiency.

[0139] Table 4

[0140]

[0141] From the above experiments, it can be seen that compared with the conflicting nature of the single-stage framework of the prior art, this application proposes a blind JPEG artifact removal method based on two-stage hierarchical quality-aware guidance (i.e., building an image reconstruction system with a two-stage architecture), which effectively alleviates the inherent conflicts in the task objectives through the separate processing of "prior learning-guided reconstruction". In order to accurately reconstruct the image, this application introduces a coarse-to-fine guidance strategy, combining fine hierarchical compression prior residual features with coarse QF priors to guide image reconstruction. In addition, in order to better guide image reconstruction, a local alignment calibration unit and a global guided calibration codec unit are proposed to effectively utilize the QF prior information calibration features to achieve prior guidance, thereby solving the defect that the existing JPEG artifact removal method cannot capture local quality changes, thereby significantly improving the quality of JPEG images, enhancing local details while maintaining global consistency. In addition, compared with other advanced technologies, the method proposed in this application can achieve competitive or even superior performance using only a small amount of parameters and floating-point operations, demonstrating an effective balance between quality and efficiency; at the same time, this application can achieve leading reconstruction performance for various JPEG compressed data sets, whether grayscale images or color images, which illustrates the effectiveness and superiority of the image reconstruction system constructed by the construction method proposed in this application in accurately restoring image quality. The method also exhibits excellent reconstruction capabilities in visual effects, and can better retain image details and edge information while removing artifacts, making the restored image closer to the original image, which shows that the method has broad application prospects and practical value.

[0142] Based on the construction method of the above-mentioned embodiment of the present application, according to one embodiment of the present application, the present application proposes an image reconstruction system, which includes: a quality-aware prior learning model, which is used to perform a multi-level iterative residual feature extraction operation on the acquired JPEG image to obtain a hierarchical compression prior residual feature of the JPEG image; a hierarchical guided feedback model, which is used to perform a feature extraction operation on the JPEG image to obtain the compression feature of the JPEG image, and to align and fuse the hierarchical compression prior residual feature and the compression feature, and calculate the reconstructed image of the JPEG image based on the result of the operation. For further details about the functions of the constructed image reconstruction model, please refer to the description in the above-mentioned construction method embodiment, which will not be repeated here.

[0143] In addition, according to one embodiment of the present application, the present application proposes an application method of an image reconstruction system, and the application method of the present application includes: obtaining a JPEG image; using the quality-aware prior learning model in the image reconstruction system as described in the above embodiment to perform a multi-level iterative residual feature extraction operation on the JPEG image to obtain a hierarchical compression prior residual feature of the JPEG image; using the hierarchical guided feedback model in the image reconstruction system as described in the above embodiment to perform a feature extraction operation on the JPEG image to obtain a compression feature of the JPEG image, and aligning and fusing the hierarchical compression prior residual feature and the compression feature, and calculating a reconstructed image of the JPEG image based on the result of the operation.

[0144] At the same time, according to one embodiment of the present application, the present application proposes a JPEG decoder, in which the image reconstruction system of the above embodiment is configured.

[0145] As can be seen from the above description, the JPEG decoder proposed in the above embodiments of this application integrates an image reconstruction system. This decoder can effectively restore image details and textures, generating reconstructed images that are more visually realistic and detailed. This design not only improves the limitations of traditional JPEG decoders in compressed image processing, but also provides users with a higher-fidelity image decoding experience, with broad application value and practicality.

[0146] In summary, compared with the existing artifact removal scheme using a single-stage architecture, the present application constructs a two-stage task decoupling architecture (i.e., constructs an initial image reconstruction system consisting of a first-stage quality-aware prior learning model and a second-stage hierarchical guided feedback model) to respectively realize the extraction of compression prior features and image reconstruction (i.e., artifact removal) of JPEG images. Among them, the first-stage architecture (i.e., the quality-aware prior learning model) is used to perform multi-level iterative residual feature extraction operations on JPEG images to obtain fine-grained hierarchical compression prior residual features to capture the changes in quality between regions; the second-stage architecture (i.e., the hierarchical guided feedback model) is used to perform alignment and fusion processing based on the compression features of JPEG images and the hierarchical compression prior residual features, which can capture the distribution characteristics of local artifacts to suppress image artifacts and restore image details, thereby improving the quality of the reconstructed image.

[0147] Based on the inventive concept of the above embodiments, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in the above embodiments when executing the computer program. Figure 7 Provide detailed explanation.

[0148] like Figure 7 As shown in FIG, the electronic device 100 of the present application is shown, which may specifically include a processor 110 and a memory 120. The memory 120 is coupled to the processor 110.

[0149] Processor 110 is used to control the operation of the electronic device. Processor 110 may also be referred to as a CPU (Central Processing Unit). Processor 110 may be an integrated circuit chip with signal processing capabilities. Processor 110 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component. A general-purpose processor may be a microprocessor, or processor 110 may be any conventional processor.

[0150] The memory 120 is used to store computer programs and can be RAM, ROM, or other types of storage terminals. Specifically, the memory 120 may include one or more computer-readable storage media, which may be non-transitory or transient. The memory 120 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage terminals and flash memory storage terminals. In some embodiments, the non-transitory computer-readable storage medium in the memory 120 is used to store at least one program code.

[0151] The processor 110 is configured to execute the computer program stored in the memory 120 to implement the methods described in the various method embodiments of the present application.

[0152] In some embodiments, the electronic device may further include a peripheral terminal interface 130 and at least one peripheral terminal. The processor 110, memory 120, and peripheral terminal interface 130 may be connected via a bus or signal lines. Each peripheral terminal may be connected to the peripheral terminal interface 130 via a bus, signal lines, or circuit boards. Specifically, the peripheral terminal includes at least one of a radio frequency circuit 140, a display screen 150, an audio circuit 160, and a power supply 170.

[0153] The peripheral terminal interface 130 can be used to connect at least one peripheral terminal related to I / O (Input / Output) to the processor 110 and the memory 120. In some embodiments, the processor 110, the memory 120, and the peripheral terminal interface 130 are integrated on the same chip or circuit board; in some other implementations, any one or two of the processor 110, the memory 120, and the peripheral terminal interface 130 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0154] The RF circuit 140 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 140 communicates with communication networks and other IoT devices via electromagnetic signals, and is therefore the communication circuitry of the electronic device. The RF circuit 140 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. Optionally, the RF circuit 140 includes an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, an operator identification module card, and the like. The RF circuit 140 can communicate with other terminals via at least one wireless communication protocol. Such wireless communication protocols include, but are not limited to, the World Wide Web, metropolitan area networks, intranets, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and / or WiFi (Wireless Fidelity) networks. In some embodiments, the RF circuit 140 may also include circuitry related to Near Field Communication (NFC), although this application does not limit this.

[0155] The display screen 150 is used to display a user interface (UI). The UI may include graphics, text, icons, videos, or any combination thereof. When the display screen 150 is a touch screen display, the display screen 150 is also capable of collecting touch signals on or above the surface of the display screen 150. The touch signals can be input as control signals to the processor 110 for processing. In this case, the display screen 150 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, there can be one display screen 150, disposed on the front panel of the electronic device; in other embodiments, there can be at least two display screens 150, disposed on different surfaces of the electronic device or in a foldable design; in still other embodiments, the display screen 150 can be a flexible display, disposed on a curved or foldable surface of the electronic device. Furthermore, the display screen 150 can be configured as a non-rectangular irregular shape, i.e., a special-shaped screen. The display screen 150 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0156] The audio circuit 160 may include a microphone and a speaker. The microphone is used to collect sound waves from the operator and the environment, and convert the sound waves into electrical signals and input them into the processor 110 for processing, or input them into the radio frequency circuit 140 to achieve voice communication. For the purpose of stereo acquisition or noise reduction, there can be multiple microphones, which are respectively set in different parts of the electronic device. The microphone can also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 110 or the radio frequency circuit 140 into sound waves. The speaker can be a traditional thin film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 160 may also include a headphone jack.

[0157] Power supply 170 is used to power various components in the electronic device. Power supply 170 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 170 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged via a wired line, while a wireless rechargeable battery is a battery that is charged via a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0158] For a detailed description of the functions and execution processes of the various functional modules or components in the electronic device embodiments of the present application, please refer to the description in the above-mentioned method embodiments of the present application, which will not be repeated here.

[0159] In the several embodiments provided in this application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the various embodiments of the electronic devices described above are merely illustrative. For example, the division of modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some data can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0160] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0161] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0162] Based on the inventive concept of the above embodiments, the present application further provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the steps of the method described in any of the above embodiments are described. Figure 8 The execution process of the above embodiment in a computer-readable storage medium is described.

[0163] like Figure 8As shown, it shows the computer-readable storage medium of the present application. If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in the computer-readable storage medium 200. Based on this understanding, the technical solution of the present application, or the part that contributes to the existing technology, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions / computer programs for enabling an IoT device (which can be a personal computer, server, or network terminal, etc.) or a processor to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes various media such as USB flash drives, mobile hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks or optical disks, as well as electronic terminals such as computers, mobile phones, laptops, tablet computers, cameras, etc. that have the above-mentioned storage media.

[0164] The description of the execution process of the program data in the computer-readable storage medium can refer to the description in the above-mentioned method embodiments of the present application, and will not be repeated here.

[0165] The above description is merely an embodiment of the present application and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

[0166] Those skilled in the art will understand that in the above-mentioned method of the specific implementation method, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

Claims

1. A method for constructing an image reconstruction system, wherein the image reconstruction system is used to reconstruct a JPEG image, characterized in that: The method comprises: Get JPEG images and build a dataset based on them; An initial image reconstruction system is constructed, wherein the initial image reconstruction system includes: a quality-aware prior learning model and a hierarchical guided feedback model, and: The quality-aware prior learning model is configured to: perform a multi-level iterative residual feature extraction operation on the JPEG image to obtain a hierarchical compressed prior residual feature of the JPEG image, wherein the hierarchical compressed prior residual feature includes a last-layer compressed prior residual feature; and perform a mapping operation on the last-layer compressed prior residual feature to obtain a quality factor prediction value of the JPEG image; The hierarchical guided feedback model includes: a feature extraction submodule, an encoder submodule and a decoder submodule, wherein: The feature extraction submodule is configured to: perform a feature extraction operation on the JPEG image to obtain compression features of the JPEG image; The encoder submodule is configured to: align and fuse the hierarchical compressed prior residual features and the compressed features to obtain local calibration features of the JPEG image, and calculate a global calibration coding feature of the JPEG image based on the quality factor prediction value and the local calibration features; The decoder submodule is configured to: calculate and obtain a reconstructed image of the JPEG image based on the global calibration coding feature; The initial image reconstruction system is trained multiple times iteratively using the data set until convergence, so as to obtain the image reconstruction system.

2. The method for constructing an image reconstruction system according to claim 1, wherein: The step of constructing the initial image reconstruction system also includes: The quality-aware prior learning model is further configured to: perform a residual feature extraction operation on the last layer compressed prior residual feature, and perform a mapping operation on the result after the residual feature extraction operation to obtain a quality factor prediction value of the JPEG image.

3. The method for constructing an image reconstruction system according to claim 1, wherein: The step of constructing the initial image reconstruction system also includes: The decoder submodule is further configured to calculate a global calibration decoding feature of the JPEG image based on the local calibration feature or the global calibration encoding feature and the quality factor prediction value, and calculate a reconstructed image of the JPEG image based on the global calibration decoding feature.

4. The method for constructing an image reconstruction system according to claim 1, wherein: The step of performing multiple iterative training on the initial image reconstruction system using the data set until convergence comprises: Performing multiple iterative training on the quality perception prior learning model using a preset first loss function until convergence; The converged quality-aware prior learning model and the hierarchical guided feedback model constitute the initial image reconstruction system, and then the initial image reconstruction system is iteratively trained multiple times using a preset second loss function until convergence. During each iterative training process, the parameters of the quality-aware prior learning model are frozen and then participate in the training.

5. An image reconstruction system constructed using the method for constructing an image reconstruction system according to claim 1, characterized in that: The image reconstruction system comprises: A quality-aware prior learning model is used to perform a multi-level iterative residual feature extraction operation on the acquired JPEG image to obtain a hierarchical compressed prior residual feature of the JPEG image; A hierarchical guided feedback model is used to perform a feature extraction operation on the JPEG image to obtain the compression features of the JPEG image, and to align and fuse the hierarchical compression prior residual features and the compression features, and to calculate a reconstructed image of the JPEG image based on the results of the operation.

6. An application method of an image reconstruction system, characterized in that: The method comprises: Get a JPEG image; Using the quality perception prior learning model in the image reconstruction system according to claim 5, performing a multi-level iterative residual feature extraction operation on the JPEG image to obtain a hierarchical compression prior residual feature of the JPEG image; Using the hierarchical guided feedback model in the image reconstruction system as described in claim 5, a feature extraction operation is performed on the JPEG image to obtain the compression features of the JPEG image, and the hierarchical compression prior residual features and the compression features are aligned and fused, and a reconstructed image of the JPEG image is calculated based on the results of the operations.

7. An electronic device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method according to any one of claims 1 to 4 and 6 when executing the computer program.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 and 6 are implemented.

Images