An image restoration method based on hierarchical multi-head attention driving

Abstract

This invention relates to the field of image restoration technology, and provides an image restoration method based on hierarchical multi-head attention. The invention includes the following steps: inputting a degraded image into a convolutional layer to extract shallow features; inputting the shallow features into a restoration image module to obtain deep features; generating a residual image from the deep features using a convolutional layer; and adding the residual image and the shallow features element-wise to obtain the restored image. This invention improves the accuracy and effectiveness of image restoration by utilizing a hierarchical multi-head attention mechanism equipped with query-key cache updates, enabling the restoration of complex images containing rain, snow, and fog.

Description

Technical Field

[0001] This invention relates to the field of image restoration technology, and provides an image restoration method based on hierarchical multi-head attention-driven approach. Background Technology

[0002] Image restoration is a challenging task aimed at recovering a clean image from a degraded input image. In recent years, convolutional neural network (CNN)-based methods have become the mainstream solution due to their superior performance. However, the convolution operation, the fundamental module of CNNs, limits the model's ability to estimate global dependencies due to its limited receptive field and its independence from the input content. To overcome these limitations, Transformer-based models have been introduced into image restoration tasks with encouraging results. The success of Transformers is mainly attributed to the self-attention (SA) mechanism, which models non-local relationships between pixels, crucial for restoring the global structure of an image. Many studies have focused on developing efficient variants of self-attention mechanisms to achieve high-quality output. Multi-head attention (MHA), by using multiple heads in parallel computation to operate from a uniformly partitioned subspace, has become a key component for improving computational efficiency and enhancing feature diversity.

[0003] Despite the significant progress made by Transformer-based methods in image restoration tasks, their core mechanism—multi-head attention (MHA)—still suffers from the following problems: In standard MHA, all heads are assigned to subspaces of the same dimension for independent computation. Experiments show that different heads tend to focus on the same regions while ignoring certain degraded areas. This redundancy not only wastes computational resources but can also lead to unsatisfactory restoration results. The lack of an effective collaboration mechanism between heads further exacerbates the redundancy problem. The feature information captured by different heads fails to fully interact and fuse, limiting the model's ability to learn diverse contextual relationships. Summary of the Invention

[0004] This invention aims to at least solve one of the technical problems existing in related technologies. To this end, this invention provides an image restoration method based on hierarchical multi-head attention-driven approaches, capable of handling various image restoration tasks, including low-light enhancement, desnow removal, and dehazing.

[0005] This invention provides an image restoration method based on hierarchical multi-head attention, comprising the following steps:

[0006] S1: Input the degraded image into the convolutional layer to extract shallow features;

[0007] S2: Input the shallow features into the image restoration module to obtain deep features;

[0008] S3: Generate a residual image from the deep features using a convolutional layer;

[0009] S4: Add the residual image and the shallow features element by element to obtain the restored image.

[0010] According to the present invention, an image restoration method based on hierarchical multi-head attention-driven method is provided, wherein the image restoration module includes the following steps:

[0011] S10: Input the shallow features into the encoder to obtain the encoder output;

[0012] S20: Input the encoder output into the decoder to obtain the decoder output;

[0013] S30: Input the decoder output into the refinement layer to obtain deep features.

[0014] According to the present invention, an image restoration method based on hierarchical multi-head attention-driven encoding is provided, wherein the encoder includes a feedforward network module and a downsampling convolutional layer.

[0015] According to the present invention, an image restoration method based on hierarchical multi-head attention-driven decoding is provided, wherein the decoder includes the following steps:

[0016] S100: Input the encoder output into the hierarchical multi-head attention layer and update it using the query-key cache (QK-Cache) mechanism to obtain the first decoder result;

[0017] S200: Input the first decoder result into the feed-forward network (FFN) module to obtain the second decoder result;

[0018] S300: Upsample and convolve the result of the second decoder to obtain the result of the third decoder;

[0019] S400: The third decoder result and the encoder output are connected in a skip connection, and then convolution is performed to obtain the decoder output.

[0020] According to the present invention, an image restoration method based on hierarchical multi-head attention (HMHA) is provided, wherein the hierarchical multi-head attention layer includes the following steps:

[0021] S101: Divide the encoder output The channels are reordered in ascending order of Pearson similarity to obtain the channel space. , ,in, For the first The channel subspace of the layer, The channel ordinal number. , Total number of channels;

[0022] S102: Calculated based on the scaling point attention mechanism, the first decoder result is obtained:

[0023]

[0024]

[0025]

[0026]

[0027] in, Let be the attention function. For querying the matrix, The key matrix, For value matrices, As the first activation function, Let be the dimension of the key matrix. For matrix transpose, For the input tensor, To query the weight matrix of the matrix, The weight matrix is ​​the key matrix. The weight matrix is ​​the value matrix. For the first The weight matrix of the query matrix for the channel. For the first The weight matrix of the channel's key matrix. For the first The weight matrix of the channel's value matrix. For the first Attention value of the channel, To output the projection matrix, For HMHA output, This is a concatenation function.

[0028] According to the image restoration method based on hierarchical multi-head attention provided by the present invention, the query-key cache update mechanism includes the following methods: intra-layer caching and inter-layer caching.

[0029] The in-layer cache includes the following steps:

[0030] S111: Calculate the modulation components within the layer :

[0031]

[0032] in, This is an in-layer cache value;

[0033] S112: Calculate key information for gating mechanism selection :

[0034]

[0035] in, For the second activation function, It is a convolution function. For element-wise multiplication;

[0036] S113: Reconstruct the key information selected by the gating mechanism to obtain... :

[0037]

[0038] in, Forward convolution, This is a backward convolution;

[0039] S114: Update in-layer cache:

[0040]

[0041] in, For the first Intra-layer matrix of the channel, For the first The query matrix of the channel. No. Channel key matrix;

[0042] The inter-layer caching includes the following steps:

[0043] S121: Calculate modulation components :

[0044]

[0045]

[0046]

[0047]

[0048] in, As a modulating component, To get attention, To adjust the inter-layer cache value, For offset components, The offset component weight matrix, For scale components, This is the scale component weight matrix;

[0049] S122: Calculation :

[0050]

[0051] S123: Update inter-layer cache values:

[0052]

[0053]

[0054] in, To adjust the inter-layer cache value, To adjust the size function, For hyperparameters, To update the inter-layer cache value, To update the inter-layer cache value.

[0055] According to the present invention, an image restoration method based on hierarchical multi-head attention-driven processing is provided, wherein the refining layer includes the following steps:

[0056] S1000: Input the decoder output into the convolutional layer and update it using a query-key cache mechanism to obtain the first refined layer result;

[0057] S2000: Input the results of the first refined layer into the feedforward network module to obtain deep features.

[0058] The present invention also provides an image restoration system based on hierarchical multi-head attention-driven methods, comprising:

[0059] Shallow feature extraction module: Inputs the degraded image into the convolutional layer to extract shallow features;

[0060] Deep feature extraction module: Inputs the shallow features into the image restoration module to obtain deep features;

[0061] Image restoration module: Generates a residual image from the deep features using a convolutional layer; adds the residual image and the shallow features element by element to obtain the restored image.

[0062] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of an image restoration system based on hierarchical multi-head attention driving as described above.

[0063] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of an image restoration system based on hierarchical multi-head attention driving as described above.

[0064] The above-described one or more technical solutions in the embodiments of the present invention have at least one of the following technical effects:

[0065] This invention provides an image restoration method based on hierarchical multi-head attention. It proposes a hierarchical multi-head attention module, which introduces a ranking mechanism based on channel similarity and combines it with hierarchical subspace partitioning, so that each subspace contains independent information and has different sizes.

[0066] This invention designs a query-key cache update mechanism, which proposes a mechanism to enhance inter-head interaction, including intra-layer and inter-layer schemes: the intra-layer cache acts as a gating module to enhance useful information in the aggregated features of the head; the inter-layer cache modulates the attention weight of each head through historical attention scores. The advantages of this invention are that it can handle various image restoration tasks, including low-light enhancement, desnowing, and dehazing, and compared to other similar inventions, it has higher accuracy and better image restoration results.

[0067] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0068] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0069] Figure 1 This is a flowchart of an image restoration method based on hierarchical multi-head attention driven proposed in this invention.

[0070] Figure 2 This is a diagram illustrating the overall structure of an image restoration method based on hierarchical multi-head attention driven proposed in this invention.

[0071] Figure 3 This invention presents the specific structure and implementation method of a hierarchical multi-head attention mechanism equipped with query-key cache updates.

[0072] Figure 4 This is a before-and-after image comparison of the snow removal task in this invention.

[0073] Figure 5 This is a structural block diagram of an image restoration system based on hierarchical multi-head attention-driven architecture provided by the present invention.

[0074] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention.

[0075] Figure label:

[0076] 101. Shallow feature extraction module; 102. Deep feature extraction module; 103. Image restoration module; 810. Processor; 820. Communication interface; 830. Memory; 840. Communication bus. Detailed Implementation

[0077] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention. The following embodiments are used to illustrate this invention but cannot be used to limit the scope of this invention.

[0078] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0079] The following is combined with Figures 1 to 6 Description of the present invention

[0080] Implementation, for example Figure 1 and Figure 2 As shown in (a), Figure 1 Here is a flowchart of an image restoration method based on hierarchical multi-head attention, including the following steps:

[0081] S1: Input the degraded image into the convolutional layer to extract shallow features;

[0082] S2: Input the shallow features into the image restoration module to obtain deep features;

[0083] S3: Generate a residual image from the deep features using a convolutional layer;

[0084] S4: Add the residual image and the shallow features element by element to obtain the restored image.

[0085] Specifically, such as Figure 2 As shown in (a), the image restoration module includes the following steps:

[0086] S10: Input the shallow features into the encoder to obtain the encoder output;

[0087] S20: Input the encoder output into the decoder to obtain the decoder output;

[0088] S30: Input the decoder output into the refinement layer to obtain deep features.

[0089] The encoder includes a feedforward network module and a downsampled convolutional layer.

[0090] Among them, such as Figure 2 As shown in (a), the decoder includes the following steps:

[0091] S100: Input the encoder output into the hierarchical multi-head attention layer and update it using a query-key caching mechanism to obtain the first decoder result;

[0092] S200: Input the first decoder result into the feedforward network module to obtain the second decoder result;

[0093] S300: Upsample and convolve the result of the second decoder to obtain the result of the third decoder;

[0094] S400: The third decoder result and the encoder output are connected in a skip connection, and then convolution is performed to obtain the decoder output.

[0095] Specifically, such as Figure 2 (b) and Figure 3 As shown, the hierarchical multi-head attention layer includes the following steps:

[0096] S101: Divide the encoder output The channels are reordered in ascending order of Pearson similarity to obtain the channel space. , ,in, For the first The channel subspace of the layer, The channel ordinal number. , Total number of channels;

[0097] S102: Calculated based on the scaling point attention mechanism, the first decoder result is obtained:

[0098]

[0099]

[0100]

[0101]

[0102] in, Let be the attention function. For querying the matrix, The key matrix, For value matrices, As the first activation function, Let be the dimension of the key matrix. For matrix transpose, For the input tensor, To query the weight matrix of the matrix, The weight matrix is ​​the key matrix. The weight matrix is ​​the value matrix. For the first The weight matrix of the query matrix for the channel. For the first The weight matrix of the channel's key matrix. For the first The weight matrix of the channel's value matrix. For the first Attention value of the channel, To output the projection matrix, For HMHA output, This is a concatenation function.

[0103] This invention proposes a hierarchical multi-head attention module, which introduces a ranking mechanism based on channel similarity and combines it with hierarchical subspace partitioning so that each subspace contains independent information and has different sizes.

[0104] Specifically, the query-key caching mechanism updates include the following methods: intra-layer caching and inter-layer caching.

[0105] The in-layer cache includes the following steps:

[0106] S111: Calculate the modulation components within the layer :

[0107]

[0108] in, This is an in-layer cache value;

[0109] S112: Calculate key information for gating mechanism selection :

[0110]

[0111] in, For the second activation function, It is a convolution function. For element-wise multiplication;

[0112] S113: Reconstruct the key information selected by the gating mechanism to obtain... :

[0113]

[0114] in, Forward convolution, This is a backward convolution;

[0115] S114: Update in-layer cache:

[0116]

[0117] in, For the first Intra-layer matrix of the channel, For the first The query matrix of the channel. No. Channel key matrix;

[0118] The inter-layer caching includes the following steps:

[0119] S121: Calculate modulation components :

[0120]

[0121]

[0122]

[0123]

[0124] in, As a modulating component, To get attention, To adjust the inter-layer cache value, For offset components, The offset component weight matrix, For scale components, This is the scale component weight matrix;

[0125] S122: Calculation :

[0126]

[0127] S123: Update inter-layer cache values:

[0128]

[0129]

[0130] in, To adjust the inter-layer cache value, To adjust the size function, For hyperparameters, To update the inter-layer cache value, To update the inter-layer cache value.

[0131] The query-key caching mechanism first performs query-key cache modulation and updates within each layer. It filters important information through feature summation and gating mechanisms, and uses convolutional operations for feature compression and reconstruction. The result is then updated to the query-key cache to improve the utilization of information within each layer. In inter-layer information interaction, inter-layer query-key cache modulation and updates are calculated. First, the attention result of the current layer is summed with the historical cache, and the scaling and offset components of the features are calculated. Then, a gating mechanism is used to filter and optimize the features. Next, the cache contribution value of the current layer is calculated, and a weighted update mechanism combined with the historical cache is used to achieve dynamic fusion of inter-layer information, thereby improving the overall recovery effect.

[0132] This invention designs a query-key cache update mechanism, which proposes a mechanism to enhance inter-head interaction, including intra-layer and inter-layer schemes: the intra-layer cache acts as a gating module to enhance useful information in the aggregated features of the head; the inter-layer cache modulates the attention weight of each head through historical attention scores. The advantages of this invention are that it can handle various image restoration tasks, including low-light enhancement, desnowing, and dehazing, and compared to other similar inventions, it has higher accuracy and better image restoration results.

[0133] Specifically, such as Figure 2 As shown in (a), the refining layer includes the following steps:

[0134] S1000: Input the decoder output into the convolutional layer and update it using a query-key cache mechanism to obtain the first refined layer result;

[0135] S2000: Input the results of the first refined layer into the feedforward network module to obtain deep features.

[0136] This invention validates an image restoration method based on hierarchical multi-head attention:

[0137] The method described in this invention is compared with other methods that simultaneously process images from the LOL-v2-real and LOL-v2-syn datasets. The restoration performance is evaluated using two metrics: PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity Index). Higher PSNR and SSIM indicate better image restoration results.

[0138] Table 1 Comparison of Image Restoration Methods

[0139]

[0140] Among them, KinD (Kindling in Darkness: A Practical Low-Light Image Enhancement Method), EnGAN1 (Enlightening Generative Adversarial Network for Deep Illumination Enhancement without Paired Supervision), EnGAN2 (Expanded and Collaboratively Guided Architecture Search via Neural Generative Adversarial Network Inspired by Retina Mechanisms), Uformer (Unified U-shaped Transformer for Image Restoration), Restormer (Resolution-Efficient State-space Transformer for High-resolution Image Restoration), MIRNet (Multi-scale Interactive Refinement Network Learning Rich Features for Real Image Restoration and Enhancement), and Sparse (Sparsity-Aware Progressive) are also mentioned. Adaptive Refinement and Enhancement (learning rich features for real-world image restoration and enhancement), QuadPrior (Quadruple Physical Prior-driven Zero-reference Low-Light Enhancement via Physics-Informed Constraints), ManbaIR (Mamba-based Simple Baseline for Image Restoration in State-Space Paradigm),The methods used for image restoration include: a simple baseline in state-space modeling; SNR-Net (Signal-to-Noise Ratio-Aware Network for Low-Light Image Enhancement); Reuinexformer (Retinex-based One-stage Transformer for Low-Light Image Enhancement); MambaLLIE (Mamba-based Low-Light Image Enhancement with Global and Local State-Space Modeling for Implicit Retina Perception); and HINT, the method provided in this invention.

[0141] As shown in Table 1, this table compares the performance of this invention on the desnow-restored image task with 12 other proposed image restoration methods, including deep learning-based methods. The results demonstrate that the image restoration performance of this invention surpasses all previously published results across multiple tasks.

[0142] like Figure 4 As shown, Figure 4 This is a before-and-after image comparison of the present invention during a snow removal mission. Figure 4 (a) is a picture before the snow was removed. Figure 4 Image (b) is a picture after snow removal using the present invention. Figure 4 The advantages of this invention in snow removal tasks are clearly demonstrated.

[0143] like Figure 5 As shown, Figure 5 Here is a block diagram of an image restoration system based on hierarchical multi-head attention, including:

[0144] Shallow feature extraction module 101: Inputs the degraded image into the convolutional layer to extract shallow features;

[0145] Deep feature extraction module 102: Inputs the shallow features into the image restoration module to obtain deep features;

[0146] Image restoration module 103: Generates a residual image from the deep features using a convolutional layer; adds the residual image and the shallow features element by element to obtain the restored image.

[0147] Figure 6An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 6 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other via the communication bus 840. The processor 810 can call logical instructions in the memory 830 to execute an image restoration method based on hierarchical multi-head attention, the method including:

[0148] S1: Input the degraded image into the convolutional layer to extract shallow features;

[0149] S2: Input the shallow features into the image restoration module to obtain deep features;

[0150] S3: Generate a residual image from the deep features using a convolutional layer;

[0151] S4: Add the residual image and the shallow features element by element to obtain the restored image.

[0152] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0153] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, wherein when the program instructions are executed by a computer, the computer is able to execute an image restoration method based on hierarchical multi-head attention driven by the methods described above, the method comprising:

[0154] S1: Input the degraded image into the convolutional layer to extract shallow features;

[0155] S2: Input the shallow features into the image restoration module to obtain deep features;

[0156] S3: Generate a residual image from the deep features using a convolutional layer;

[0157] S4: Add the residual image and the shallow features element by element to obtain the restored image.

[0158] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the aforementioned image restoration method based on hierarchical multi-head attention driving, the method comprising:

[0159] S1: Input the degraded image into the convolutional layer to extract shallow features;

[0160] S2: Input the shallow features into the image restoration module to obtain deep features;

[0161] S3: Generate a residual image from the deep features using a convolutional layer;

[0162] S4: Add the residual image and the shallow features element by element to obtain the restored image.

[0163] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0164] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0165] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

[0166] It should be noted that the embodiments of this disclosure can be implemented using hardware, software, or a combination of both. The hardware portion can be implemented using dedicated logic; the software portion can be stored in memory and executed by a suitable instruction execution system, such as a microprocessor or dedicated-design hardware. Those skilled in the art will understand that the above-described devices and methods can be implemented using computer-executable instructions and / or included in processor control code, for example, such code provided on a programmable memory or a data carrier such as an optical or electronic signal carrier.

[0167] Furthermore, although the operation of the methods of this disclosure is described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Rather, the steps depicted in the flowcharts may be performed in a different order. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps. It should also be noted that the features and functions of two or more devices according to this disclosure may be embodied in one device. Conversely, the features and functions of one device described above may be further divided and embodied by multiple devices.

[0168] While this disclosure has been described with reference to several specific embodiments, it should be understood that this disclosure is not limited to the specific embodiments disclosed. This disclosure is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A layered multi-head attention driven image restoration method, characterized in that, Includes the following steps: S1: Input the degraded image into the convolutional layer to extract shallow features; S2: Input the shallow features into the image restoration module to obtain deep features, including the following steps: S10: Input the shallow features into the encoder to obtain the encoder output; S20: Input the encoder output into the decoder to obtain the decoder output, including the following steps: S100: The encoder output is input to the hierarchical multi-head attention layer, and a query-key caching mechanism is used to update the output, resulting in the first decoder. The query-key caching mechanism includes the following methods: intra-layer caching and inter-layer caching. The in-layer cache includes the following steps: S111: Calculate intra-layer modulation component : in, This is an in-layer cache value. For HMHA output; S112: Calculate key information for gating mechanism selection : in, For the second activation function, It is a convolution function. For element-wise multiplication; S113: Reconstruct the key information selected by the gating mechanism to obtain... : in, Forward convolution, This is a backward convolution; S114: Update in-layer cache: in, For the first In-layer matrix of the channel, For the first The query matrix of the channel. No. The channel's key matrix; The inter-layer caching includes the following steps: S121: Calculate modulation components : in, As a modulating component, To get attention, To adjust the inter-layer cache value, For offset components, The offset component weight matrix, For scale components, This is the scale component weight matrix; S122: Calculation : S123: Update inter-layer cache values: in, To adjust the inter-layer cache value, To adjust the size function, For hyperparameters, To update the inter-layer cache value, To update the inter-layer cache value; S200: Input the first decoder result into the feedforward network module to obtain the second decoder result; S300: Upsample and convolve the result of the second decoder to obtain the result of the third decoder; S400: The third decoder result and the encoder output are connected in a skip connection, and then convolution is performed to obtain the decoder output; S30: Input the decoder output into the refinement layer to obtain deep features; S3: Generate a residual image from the deep features using a convolutional layer; S4: Add the residual image and the shallow features element by element to obtain the restored image.

2. The image restoration method based on hierarchical multi-head attention-driven approach according to claim 1, characterized in that, The encoder includes a feedforward network module and a downsampled convolutional layer.

3. The image restoration method based on hierarchical multi-head attention-driven approach according to claim 2, characterized in that, The hierarchical multi-head attention layer includes the following steps: S101: Divide the encoder output The channels are reordered in ascending order of Pearson similarity to obtain the channel space. , ,in, For the first The channel subspace of the layer, The channel ordinal number. , Total number of channels; S102: Calculated based on the scaling point attention mechanism, the first decoder result is obtained: in, Let be the attention function. For querying the matrix, The key matrix, For value matrices, As the first activation function, Let be the dimension of the key matrix. For matrix transpose, For the input tensor, To query the weight matrix of the matrix, The weight matrix is ​​the key matrix. The weight matrix is ​​the value matrix. For the first The weight matrix of the query matrix for the channel. For the first The weight matrix of the channel's key matrix. For the first The weight matrix of the channel's value matrix. For the first Channel attention value, To output the projection matrix, This is a concatenation function.

4. The image restoration method based on hierarchical multi-head attention-driven approach according to claim 1, characterized in that, Step S30 includes the following steps: S1000: Input the decoder output into the convolutional layer and update it using a query-key cache mechanism to obtain the first refined layer result; S2000: Input the results of the first refined layer into the feedforward network module to obtain deep features.

5. An image restoration system based on hierarchical multi-head attention driving, for performing the image restoration method based on hierarchical multi-head attention driving as described in any one of claims 1 to 4, characterized in that, include: Shallow feature extraction module: Inputs the degraded image into the convolutional layer to extract shallow features; Deep feature extraction module: Inputs the shallow features into the image restoration module to obtain deep features; Image restoration module: Generates a residual image from the deep features using a convolutional layer; adds the residual image and the shallow features element by element to obtain the restored image.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the image restoration method based on hierarchical multi-head attention as described in any one of claims 1 to 4.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the image restoration method based on hierarchical multi-head attention as described in any one of claims 1 to 4.

Images