A method and apparatus for compressing image restoration

Abstract

This invention provides a method for restoring compressed images, comprising the following steps: acquiring a compressed image; inputting the compressed image into a dual dense network, and processing it through the dual dense network to obtain a restored image corresponding to the compressed image; wherein the dual dense network comprises a Low-Level Feature Representation Network (LFR), a Hierarchical Feature Extraction Network (HFE), and an Image Reconstruction Network (RNet) connected in sequence, and the processing includes: S1, using the LFR to represent the input compressed image as a set of low-level feature maps; S2, inputting the obtained low-level feature maps into the HFE to obtain hierarchical features at different levels; S3, inputting the hierarchical features at different levels into the image reconstruction network to obtain the corresponding restored image. This invention can not only greatly remove artifacts from compressed images, but also improve visual quality, making the details of the restored image clearer and more complete.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and apparatus for recovering compressed images. Background Technology

[0002] Image compression provides an effective technique for the efficient transmission and rapid storage of large amounts of data, but it also leads to a loss of visual quality. Image compression often causes severe distortion and introduces various compression artifacts, such as blurring, ringing, and occlusion artifacts. These not only affect the quality of the human visual experience but also impair the performance of various image processing technologies and image-driven applications, such as object recognition and image segmentation. Therefore, removing artifacts from compressed images and improving visual quality is a crucial and urgent task in computer vision.

[0003] Currently, based on traditional models and neural network training, domestic and international scholars in the field of computer vision have proposed various algorithms for removing artifacts from compressed images and image restoration. Existing methods for removing compressed image artifacts can be divided into filter-based methods, prior knowledge-based methods, and deep learning-based methods. Filter-based methods reduce artifacts in compressed images through explicit filtering operations. Prior knowledge-based methods are implemented based on the maximum a posteriori probability estimation framework. Currently, various image prior knowledge is used to remove compressed image artifacts, including sparsity, low rank, and self-similarity. In recent years, many deep learning networks have been extensively studied for removing compressed image artifacts. These methods significantly outperform existing filter-based and prior knowledge-based methods. Current deep learning-based methods directly learn the mapping relationship between compressed images and original high-quality images. Classic deep learning network structures include shallow structures, deep structures, and multi-scale U-shaped networks.

[0004] While the aforementioned deep learning-based algorithms have achieved considerable success in eliminating artifacts in compressed images, they still suffer from the following shortcomings: First, most methods are limited by their receptive field, typically capturing perceptual features locally through convolutional operations without capturing long-range dependencies between pixels and channels, thus failing to completely remove artifacts from compressed images. Second, most methods treat spatial and channel features equally, lacking flexibility in handling different types of information. This is detrimental to reconstructing artifact-free images with high-frequency details and clearer structure. Third, these methods neglect the crucial role of hierarchical features in artifact removal and image reconstruction, resulting in insufficient detail in the recovered image.

[0005] It should be noted that the information disclosed in the background section above is only for understanding the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the above-mentioned background technology and provide a compressed image restoration method and apparatus to remove artifacts from compressed images and improve visual quality.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a compressed image restoration method, comprising the following steps: acquiring a compressed image; inputting the compressed image into a dual dense network; processing the compressed image in the dual dense network to obtain a restored image corresponding to the compressed image; wherein the dual dense network comprises a low-level feature representation network (LFR), a hierarchical feature extraction network (HFE), and an image reconstruction network (RNet) connected in sequence; the processing comprises: S1, using the low-level feature representation network (LFR) to represent the input compressed image as a set of low-level feature maps; S2, inputting the obtained low-level feature maps into the hierarchical feature extraction network (HFE) to obtain hierarchical features at different levels; S3, inputting the hierarchical features at different levels into the image reconstruction network (RNet) to obtain the corresponding restored image.

[0008] Secondly, the present invention provides a compressed image restoration apparatus, comprising: a compressed image acquisition unit for acquiring a compressed image; and a compressed image restoration unit for inputting the compressed image into a dual dense network to obtain a restored image corresponding to the compressed image output by the dual dense network; wherein the dual dense network comprises a low-level feature representation network (LFR), a hierarchical feature extraction network (HFE), and an image reconstruction network (RNet) connected in sequence, and the processing comprises: using the low-level feature representation network (LFR) to represent the input compressed image as a set of low-level feature maps; inputting the obtained low-level feature maps into the hierarchical feature extraction network (HFE) to obtain hierarchical features at different levels; and inputting the hierarchical features at different levels into the image reconstruction network (RNet) to obtain the corresponding restored image.

[0009] Thirdly, the present invention provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the compressed image restoration method as described in the first aspect.

[0010] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the compressed image restoration method as described in the first aspect.

[0011] This invention offers the following advantages: It utilizes the densely connected structure of deep learning networks to remove artifacts from compressed images and enhance visual quality. This dense connection structure allows the network to more fully utilize information from previous states, preventing the loss of important information. This invention not only significantly removes artifacts from compressed images but also improves visual quality, resulting in clearer and more complete details in the recovered image. The proposed method can handle images with varying degrees of compression and complex application scenarios using different compression methods, exhibiting high robustness. This invention can be applied as image processing software to other image processing tasks, such as image segmentation, object recognition, and detection.

[0012] Furthermore, this invention takes into account the characteristics of different scales and different attention mechanisms, enabling the network to learn richer and more important hierarchical information. At the same time, this invention makes full use of hierarchical features throughout the process and adaptively selects and fuses features through hierarchical attention mechanisms, guiding the network to continuously acquire important information and suppress useless information. Attached Figure Description

[0013] Figure 1 This is a flowchart of the compressed image restoration method according to a specific embodiment of the present invention; Figure 2 This is the overall structure of the dual dense network DIDNet in a specific embodiment of the present invention; Figure 3 yes Figure 2 A dual-scale and attention-group DSAG network structure in a hierarchical feature extraction network (HFE); Figure 4 yes Figure 3 The network structure of DSAU, a dual-scale and attention mechanism unit in the two-level hierarchical feature extraction network SHFE; Figure 5 yes Figure 4 A schematic diagram illustrating the specific operation process of the Dual Attention Fusion Module (DAFM) in the diagram; Figure 6 yes Figure 2 and Figure 3 The network structure of the hierarchical attention fusion module HAF in the middle; Figure 7 The images from left to right are the original image, the compressed image, and the restored image; Figure 8 This describes the training process of the dual dense network DIDNet in a specific embodiment of the present invention. Detailed Implementation

[0014] The embodiments of the present invention will be described in detail below. It should be emphasized that the following description is merely exemplary and not intended to limit the scope and application of the present invention.

[0015] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of the present invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0016] This invention, in its specific implementation, uses a densely connected network as its basic framework. It extracts rich hierarchical features using multi-scale and multi-attention mechanisms, and then explores the importance of these hierarchical features in the entire network learning process using a hierarchical feature attention mechanism. This guides artifact removal and image reconstruction of compressed images. A dual-dense network, DIDNet (Dense in Dense Network), is proposed. DIDNet comprises three sequentially connected parts: a Low-Level Feature Representation (LFR) network, a Hierarchical Feature Extraction (HFE) network, and a Reconstruction Network (RNet). Using this dual-dense network DIDNet, this invention provides a compressed image restoration method that removes artifacts and improves visual quality, such as… Figure 1 and Figure 2 As shown, the steps are as follows:

[0017] S1, Input compressed image (e.g., grayscale image), based on a pre-defined low-level feature representation network (LFR) (e.g., through a convolutional kernel of 3...). 3 Convolutional layer operations 3) Obtain a set of low-level feature maps of the compressed image. :

[0018] S2, transfer the low-level feature map Input a pre-defined hierarchical feature extraction network (HFE) to obtain hierarchical features at different levels. .

[0019] Among them, such as Figure 2As shown, the pre-defined hierarchical feature extraction network (HFE) is a densely connected network, containing... One (in this example) The network consists of densely connected dual-scale and dual-attention groups (DSAGs), each followed by a hierarchical attention fusion module (HAF). The DSAGs extract rich hierarchical features, while the HAF adaptively fuses these features and selects the most important ones for image reconstruction. Therefore, the specific steps of the hierarchical feature extraction network (HFE) are as follows:

[0020] (1) Input low-level feature map The first intermediate hierarchical features are obtained by using the first dual-scale and attention mechanism to form DSAG. Then the low-level feature map and the first intermediate layer feature The features are stacked together and then adaptively selected and fused through the first hierarchical attention fusion module (HAF) to obtain the fused hierarchical features. :

[0021]

[0022] in, and These represent the first dual-scale and attention mechanism group DSAG and the first hierarchical attention fusion module HAF in the hierarchical feature extraction network HFE, respectively. () indicates the stacking operation (also known as stacking operation) operate).

[0023] (2) And so on, the first The value of m ranges from [2, ..., ]) Dual-scale and attention-based DSAG with the first The output of the hierarchical attention fusion module HAF (i.e., the fused hierarchical features) Then output the corresponding intermediate layer features. Then through the first A hierarchical attention fusion module (HAF) processes low-level feature maps. And all the previous ones The outputs of the dual-scale and attention-group DSAG are fused to obtain the corresponding fused hierarchical features. :

[0024]

[0025]

[0026] in, and These represent the first and second layers of the hierarchical feature extraction network HFE. The dual-scale and attention mechanism group DSAG and the first A hierarchical attention fusion module (HAF), ( ) indicates a stacking operation.

[0027] The final The outputs of the dual-scale and attention mechanism group DSAG and the hierarchical attention fusion module HAF are as follows:

[0028]

[0029] S3, Combine all the merged layered features (All layered features after fusion) Represented as hierarchical features at different levels Using this as input, the hierarchical features from different levels are processed by a pre-defined image reconstruction network, RNet (Reconstruction Network). Converted into an artifact-free restored image .

[0030]

[0031] Among them, such as Figure 2 As shown, image reconstruction network Contains a convolution kernel of 3 3 Convolutional layer operations A long-range residual connection (implemented through element-wise addition) and a convolutional kernel of 1. 1 Convolutional layer operation 1.

[0032] As a preferred embodiment, such as Figure 3As shown, in S2, each dual-scale and attention-group DSAG includes a Local Feature Extraction (LFE) network, a Second Hierarchical Feature Extraction (SHFE) network, and a Feature Refinement (FER) network. First, the LFE network transforms the input into a local feature through a convolutional layer. Then, the SHFE network generates multiple second-level hierarchical features. Finally, the FER network refines these second-level hierarchical features to obtain the final, corrected second-level hierarchical features. (The text then repeats the first example.) Taking the dual-scale and attention-group DSAG as an example, its input and output are respectively after the first... The first layered attention fusion module HAF fusion The output of each DSAG and The specific steps are as follows:

[0033] S2-1, The fused hierarchical features of the input Local features are obtained through the Local Feature Extraction Network (LFE). :

[0034] Among them, the local feature extraction network Contains a convolution kernel of 3 3 convolutional layers 3.

[0035] S2-2, Based on the two-level hierarchical feature extraction network SHFE, local features are extracted. Transformed into two-level hierarchical features .

[0036] As a preferred embodiment, continue to refer to Figure 3 As shown, the pre-defined two-level hierarchical feature extraction network SHFE is a densely connected network, containing... In this example, The network consists of densely connected Dual-Scale and Dual-Attention Units (DSAUs), each followed by a Hierarchical Attention Fusion Module (HAF). The DSAUs extract rich secondary hierarchical features, while the HAF adaptively fuses these features and selects the most important ones for image reconstruction. Specifically, the steps of the Secondary Hierarchical Feature Extraction Network (SHFE) are as follows:

[0037] (1) Input local features The first second-level intermediate hierarchical features are obtained through the first dual-scale and attention mechanism unit DSAU. Then local features and secondary intermediate hierarchical features These features are stacked together and adaptively selected and fused using the Hierarchical Attention Fusion (HAF) module to obtain the fused second-level hierarchical features. :

[0038]

[0039] in, and This represents the first dual-scale and attention unit DSAU and the first hierarchical attention fusion module HAF in the two-level hierarchical feature extraction network SHFE. () indicates a stacking operation.

[0040] (2) And so on, the first indivual( The range of values ​​for is [2, ])) Dual-scale and attention unit DSAU with the first The output of the hierarchical attention fusion module HAF As input, the corresponding second-level intermediate hierarchical features are then output. Then through the first A hierarchical attention fusion module (HAF) focuses on local features. And all the previous ones The outputs of the dual-scale and attention unit DSAU are fused to obtain the corresponding fused second-level hierarchical features. :

[0041]

[0042]

[0043] in, and This represents the second-level hierarchical feature extraction network SHFE. The nth DSAU and the nth HAF. The final nth The outputs of DSAU and HAF are:

[0044]

[0045]

[0046] S2-3, Through the fused secondary hierarchical features (All secondary hierarchical features after fusion) Represented as two-level hierarchical features at different levels The refined second-level hierarchical features are obtained based on the Feature Extraction Network (FER). :

[0047] The feature extraction network FER contains a convolutional kernel with a size of 3. 3 Convolutional layer operations 3. A short-range residual connection (implemented through element-wise addition) and a convolutional kernel of 1. 1 Convolutional layer operation 1.

[0048] As a preferred embodiment, such as Figure 4 As shown, each dual-scale and attention unit (DSAU) in the second-level hierarchical feature extraction network SHFE in step S2-2 contains two parallel branches: an upper branch and a lower branch. These two branches extract perceptual features at different sizes using convolutional kernels of different sizes, and then use a second dual-attention fusion module (DAFM) to select and fuse features from both spatial and channel dimensions. Specifically, the network structure of both the upper and lower branches includes: two sets of composite convolutional operations and a first dual-attention fusion module (DAFM). The composite convolutional operations in the upper branch are performed by a kernel of size 3... A convolutional layer with a 3x3 convolutional kernel, an LReLU activation function, and a kernel size of 3. The convolutional layer consists of a 3-kernel convolutional layer, and the compound convolutional operation in the lower branch is composed of a 5-kernel convolutional layer. A convolutional layer with a 5-kernel diameter, an LReLU activation function, and a kernel size of 5. The convolutional layer consists of a 5-kernel convolutional layer, which obtains the perceptual features of the upper and lower branches respectively. and Then, the perceived features are processed through the Concat operation. and The data is then fused, and then passed through the second dual-attention fusion module (DAFM) to obtain the final output of the dual-scale and attention unit (DSAU). Taking a dual-scale attention unit (DSAU) as an example, its input and output are respectively and The specific operating steps are as follows:

[0049] (1) Upper branch: The upper branch uses a convolution kernel size of 3 3 convolutional layers 3. Extracting perceptual features, mainly including two groups 3-LReLU- The third operation (i.e., compound convolution operation) is then performed, and the perceptual features of the upper branch are obtained by fusion selection through the first dual attention fusion module DAFM. ; (2) Lower branch: The lower branch has the same structure as the upper branch, the difference being that the kernel size is 5. 5 convolutional layers 5, meaning the lower branch contains two groups. 5-LReLU- The process involves 5 steps, followed by fusion selection via the first dual attention fusion module (DAFM) to obtain the perceptual features for the next branch. ; (3) Perform a concat operation on the perceptual features of these two branches, and then pass them through the second dual attention fusion module DAFM to obtain the final secondary intermediate layer features. .

[0050] As a preferred embodiment, such as Figure 4 As shown, each Dual Attention Fusion Module (DAFM) includes a spatial saliency mechanism and a channel-level saliency mechanism. The spatial saliency mechanism obtains a spatial attention weight map by averaging the channel information of the input feature map, while the channel-level saliency mechanism obtains a channel attention weight map by averaging the spatial information of the input feature map. These two weight maps are then element-wise multiplied with the input feature map and summed to obtain the final feature map represented by both spatial and channel attention. The Dual Attention Fusion Module (DAFM) can effectively highlight important information and suppress useless information. Specifically, Figure 4All three dual attention fusion modules (DAFM) in DSAU perform the following operations (i.e., the operation processes of the three dual attention fusion modules DAFM in DSAU are exactly the same): Dual Scale and Attention Unit. The dual attention fusion module DAFM in DSAU selects the most important perceptual features from both spatial and channel dimensions, such as... Figure 5 As shown, assume its input is a feature map. , , and These represent the height, width, and number of channels of the feature map, respectively. If we represent the real number field, then the specific operation steps of the Dual Attention Fusion Module (DAFM) are as follows:

[0051] (1) Spatial saliency mechanism: The feature map is calculated by averaging the channel information of the input feature map. Spatial weight feature map :

[0052] The values ​​of u, v, and c are [1, U], [1, V], and [1, C], respectively.

[0053] (2) Channel-level saliency mechanism: Calculate the feature map by averaging the spatial information of the input feature map. Channel weight feature map :

[0054] (3) and Perform softmax operation respectively to obtain and , then calculate Get the final output .

[0055] In a preferred embodiment, the hierarchical attention fusion module (HAF) included in the hierarchical feature extraction network (HFE) and the secondary hierarchical feature extraction network (SHFE) is dedicated to adaptively selecting and fusing hierarchical features and secondary hierarchical features, respectively, to obtain the most important information for reconstructing an artifact-free image. The hierarchical attention fusion module (HAF) introduces a hierarchical attention mechanism to calculate the corresponding weight of each hierarchical feature (or secondary hierarchical feature), emphasizing important hierarchical features and suppressing redundant information. Therefore, it can utilize hierarchical features more fully and effectively for image restoration. Specifically, each hierarchical attention fusion module (HAF) in S2 and S2-2 uses a hierarchical attention mechanism to adaptively select and fuse different hierarchical features (or secondary hierarchical features), thereby enabling more full utilization of hierarchical features for image reconstruction, such as... Figure 6As shown, assume that the input to the hierarchical attention fusion module HAF is a set of hierarchical features. The specific operating steps are as follows:

[0056] (1) Using an average pooling operation and a convolution kernel size of 1 1 convolutional layer 1 For each hierarchical feature Processing to obtain :

[0057] in .

[0058] (2) Then They were piled together and obtained After a The function obtains the hierarchical attention weights. :

[0059] (3) All hierarchical features Perform a stacking operation to obtain the layered features after stacking. Then, the stacked layered features and hierarchical attention weights The final fused layered features are obtained by using element-wise multiplication operations. .

[0060] like Figure 7 As shown, the left image is the original image, the middle image is the compressed image, and the right image is the restored image after being restored by the compressed image restoration method according to the specific embodiment of the present invention. The three smaller images below each image represent the corresponding magnified areas of the respective graphic. Inputting the compressed image shown in the middle image, after processing by the compressed image restoration method according to the above embodiment of the present invention, yields the restored image shown in the right image. Specifically, a preset low-level feature representation network represents the input compressed image as a set of low-level feature maps. The obtained low-level feature maps are input into a hierarchical feature extraction network to obtain hierarchical features at different levels. The hierarchical feature extraction network extracts multi-scale features in parallel using two different convolutional kernels, and then adaptively selects and fuses the multi-scale features using spatial saliency and channel-level saliency mechanisms to form the final hierarchical features. The hierarchical feature extraction network continuously utilizes the hierarchical features extracted by all previous modules, and then flexibly fuses and selects the most important information using a hierarchical feature attention mechanism. Finally, all important hierarchical features are input into a preset image reconstruction network to obtain the corresponding restored image. This restored image greatly removes artifacts from the compressed image and makes the details of the restored image clearer and more complete.

[0061] In a preferred embodiment, after constructing the dual-dense network DIDNet, it is preferable to first train it using a dataset to obtain a trained dual-dense network DIDNet, and then use the trained dual-dense network DIDNet to reconstruct the actual compressed image, such as... Figure 8 As shown, the specific training process mainly includes the following two steps: (1) Dataset preparation: Obtain a set of pairs of original images and compressed images as training dataset, which contains pairs of original images and compressed images, and is the number of datasets; (2) According to the network (DIDNet) preset by the present invention, the mapping relationship between the compressed image and the original image is learned, and a trained model is obtained (i.e., a trained DIDNet is obtained).

[0062] A specific embodiment of the present invention also provides a compressed image restoration device, comprising: a compressed image acquisition unit for acquiring a compressed image; and a compressed image restoration unit for inputting the compressed image into a dual dense network to obtain a restored image output by the dual dense network corresponding to the compressed image.

[0063] A further embodiment of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the compressed image restoration method.

[0064] A further embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the compressed image restoration method as described in the first aspect.

[0065] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0066] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0067] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0068] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0069] The above description provides a further detailed explanation of the present invention in conjunction with specific / preferred embodiments, and it should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various substitutions or modifications can be made to these described embodiments without departing from the concept of the present invention, and all such substitutions or modifications should be considered within the scope of protection of the present invention. In the description of this specification, the reference to terms such as "an embodiment," "some embodiments," "preferred embodiment," "example," "specific example," or "some examples," etc., indicates that the specific features, structures, materials, or characteristics described in connection with that embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples. Without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification and the features of different embodiments or examples. Although the embodiments of the present invention and their advantages have been described in detail, it should be understood that various changes, substitutions, and modifications can be made herein without departing from the scope of protection of the patent application.

Claims

1. A method for restoring compressed images, characterized in that, Includes the following steps: Obtain a compressed image; The compressed image is input into a dual dense network, and the recovered image corresponding to the compressed image is obtained through the dual dense network. The dual dense network comprises a low-level feature representation network (LFR), a hierarchical feature extraction network (HFE), and an image reconstruction network (RNet) connected in sequence. The hierarchical feature extraction network (HFE) includes... Each densely connected dual-scale and attention group (DSAG) is followed by a first hierarchical attention fusion module (HAF). The DSAG is used to extract hierarchical features, and the HAF adaptively fuses the densely connected hierarchical features to obtain hierarchical features at different levels. The processing includes: S1. The input compressed image is represented as a set of low-level feature maps using the Low-Level Feature Representation Network (LFR); S2. Input the obtained low-level feature map into the hierarchical feature extraction network (HFE) to obtain hierarchical features at different levels; S3. Input the hierarchical features of different levels into the image reconstruction network (RNet) to obtain the corresponding restored image.

2. The compressed image restoration method as described in claim 1, characterized in that, In step S1, the Low-Level Feature Representation Network (LFR) represents the input compressed image as a set of low-level feature maps through a convolutional layer.

3. The compressed image restoration method as described in claim 2, characterized in that, The low-level feature representation network (LFR) uses a convolutional kernel with a 3x3 convolutional value. 3 convolutional layers Operation 3 represents the input compressed image as a set of low-level feature maps.

4. The compressed image restoration method as described in claim 1, characterized in that, Step S2 specifically includes the following steps: (1) Input low-level feature map The first intermediate hierarchical feature is obtained after the first dual-scale and attention group (DSAG). Then the low-level feature map and the first intermediate layer feature These features are stacked together and adaptively selected and fused through the first hierarchical attention fusion module (HAF) to obtain the fused hierarchical features. ; (2) No. The dual-scale and attention group (DSAG) with the first The output of the first layer attention fusion module (HAF) Then output the corresponding intermediate layer features. Then through the first The first-level attention fusion module HAF processes the low-level feature map. And all the previous ones The outputs of dual-scale and attention group (DSAG) are fused to obtain the corresponding fused hierarchical features. until the final number is obtained The output of the first hierarchical attention fusion module HAF, that is, the output is the fused features of all hierarchical layers. This is used as input to the image reconstruction network (RNet) in step S3.

5. The compressed image restoration method as described in claim 1 or 4, characterized in that, In step S2, each dual-scale and attention group (DSAG) contains a Local Feature Extraction Network (LFE), a Hierarchical Secondary Feature Extraction Network (SHFE), and a Feature Refinement Network (FER) connected in sequence. The LFE transforms the input into a local feature, which is then transformed into a hierarchical secondary feature by the SHFE. Finally, the FER refines the hierarchical secondary feature to obtain the refined hierarchical secondary feature.

6. The compressed image restoration method as described in claim 5, characterized in that, The Local Feature Extraction Network (LFE) contains a convolutional kernel with a size of 3. 3 convolutional layers 3. Operation: The Feature Extraction Network (FER) contains a convolutional kernel with 3... 3 Convolutional layer operations 3. A short-range residual connection and a convolution kernel of 1 1 Convolutional layer operation 1.

7. The compressed image restoration method as described in claim 5, characterized in that, The two-level hierarchical feature extraction network (SHFE) includes Each DSAU is a densely connected dual-scale and attention mechanism unit, and each DSAU is followed by a second hierarchical attention fusion module (HAF). The DSAU is used to extract second-level hierarchical features, and the HAF is used to adaptively fuse the densely connected second-level hierarchical features to obtain hierarchical features of different levels.

8. The compressed image restoration method as described in claim 7, characterized in that, The Hierarchical Secondary Feature Extraction Network (SHFE) transforms the local features into hierarchical secondary features, specifically including the following steps: (1) Local features input from the Local Feature Extraction Network (LFE) The first secondary intermediate hierarchical feature is obtained after passing through the first dual-scale and attention unit (DSAU). Then the local features and the secondary intermediate layer features These features are stacked together and adaptively selected and fused through the first second-level attention fusion module (HAF) to obtain the fused second-level hierarchical features. ; (2) No. The second dual-scale and attention unit (DSAU) with the first The output of the second-layer attention fusion module (HAF) Then output the corresponding second-level intermediate hierarchical features. Then through the first A second-layer attention fusion module (HAF) for the local features And all the previous ones The outputs of dual-scale and attention units (DSAU) are fused to obtain the corresponding fused second-level hierarchical features. until the final number is obtained The output of the second-level attention fusion module HAF is the fused second-level hierarchical feature. This is used as the input to the Feature Extraction Network (FER).

9. The compressed image restoration method as described in claim 7, characterized in that, Each Dual Scale and Attention Unit (DSAU) comprises: a parallel upper branch and a lower branch, and a second Dual Attention Fusion Module (DAFM); the upper branch and the lower branch respectively extract perceptual features of the image using convolutional kernels of different sizes, wherein each upper branch and the lower branch includes: two sets of compound convolutional operations and a first Dual Attention Fusion Module (DAFM), the compound convolutional operations of the upper branch being of size 3 A convolutional layer with a 3x3 kernel, an activation function, and a size of 3. The convolutional layer consists of a 3-kernel convolutional layer, and the compound convolutional operation of the lower branch is composed of a 5-kernel convolutional layer. A convolutional layer with a kernel size of 5, an activation function, and a kernel size of 5. The convolutional layer consists of a 5-kernel convolutional layer; when the perceptual features of the upper branch and the lower branch are obtained respectively. and Then, the perceived features are processed through the Concat operation. and The components are fused and then passed through the second dual attention fusion module (DAFM) to obtain the final output of the dual scale and attention unit (DSAU).

10. The compressed image restoration method as described in claim 9, characterized in that, Both the second dual attention fusion module (DAFM) and the first dual attention fusion module (DAFM) include a spatial saliency mechanism and a channel-level saliency mechanism. The spatial saliency mechanism obtains a spatial weight feature map by averaging the channel information of the input feature map. The channel-level saliency mechanism obtains a channel weight feature map by averaging the spatial information of the input feature map. Then, the spatial weight feature map and the channel weight feature map are multiplied element-wise with the input feature map respectively, and then added together to obtain the final feature map represented by spatial and channel attention.

11. The compressed image restoration method as described in claim 1, characterized in that, The image reconstruction network (RNet) includes a convolutional kernel with a size of 3. 3 Convolutional layer operations A long-range residual connection and a convolution kernel of 1 1 Convolutional layer operation 1.

12. A compressed image restoration device, characterized in that, include: A compressed image acquisition unit is used to acquire compressed images; The compressed image restoration unit is used to input the compressed image into a dual dense network for processing, and obtain the restored image corresponding to the compressed image output by the dual dense network; The dual dense network comprises a low-level feature representation network (LFR), a hierarchical feature extraction network (HFE), and an image reconstruction network (RNet) connected in sequence. The hierarchical feature extraction network (HFE) includes... Each densely connected dual-scale and attention group (DSAG) is followed by a first hierarchical attention fusion module (HAF). The DSAG is used to extract hierarchical features, and the HAF adaptively fuses the densely connected hierarchical features to obtain hierarchical features at different levels. The processing includes: The low-level feature representation network (LFR) is used to represent the input compressed image as a set of low-level feature maps; The obtained low-level feature maps are input into the hierarchical feature extraction network (HFE) to obtain hierarchical features at different levels; The hierarchical features of different levels are input into the image reconstruction network (RNet) to obtain the corresponding restored image.

13. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the compressed image restoration method as described in any one of claims 1 to 11.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the compressed image restoration method as described in any one of claims 1 to 11.

Images