Image restoration method based on layered multi-head attention driving

Abstract

The invention relates to the technical field of image restoration, and provides an image restoration method based on layered multi-head attention driving. The method comprises the following steps: inputting a degraded image into a convolutional layer for extraction to obtain shallow layer features; inputting the shallow-layer features into an image recovery module to obtain deep-layer features; generating a residual image for the deep features by using a convolutional layer; and adding the residual image and the shallow feature element by element to obtain a restored image. According to the invention, by using a layered multi-head attention mechanism equipped with query-key cache updating, the image recovery and reply precision is improved, the image recovery effect is improved, and complex rain-containing, snow-containing and fog-containing image recovery can be executed.

Description

Technical Field

[0001] The present invention relates to the technical field of image restoration, and provides an image restoration method based on hierarchical multi-head attention driving. Background Art

[0002] Image restoration is a challenging task that aims to restore a clean image from a degraded input image. In recent years, convolutional neural network-based methods have become mainstream solutions due to their superior performance. However, the convolution operation, the fundamental building block of convolutional neural networks, has a limited receptive field and is independent of the input content, limiting the model's ability to estimate global dependencies. To overcome these limitations, Transformer-based models have been introduced to image restoration tasks and have achieved encouraging results. The Transformer's success is primarily due to its self-attention mechanism (SA), which models non-local relationships between pixels, which is crucial for restoring the global structure of an image. Numerous studies have focused on developing efficient variants of the self-attention mechanism to achieve high-quality output. Multi-head attention (MHA), which operates on evenly divided subspaces using multiple heads in parallel, has become a key component for improving computational efficiency and enhancing feature diversity.

[0003] While Transformer-based methods have achieved significant progress in image restoration tasks, their core mechanism, multi-head attention (MHA), still suffers from the following issues: In standard MHA, all heads are assigned a subspace of the same dimension for independent computation. Experiments show that different heads tend to focus on the same regions while neglecting certain degraded regions. This redundancy not only wastes computational resources but can also lead to suboptimal restoration results. The lack of an effective coordination mechanism between heads further exacerbates the redundancy issue. The feature information captured by different heads fails to fully interact and fuse, limiting the model's ability to learn diverse contextual relationships. Summary of the Invention

[0004] The present invention aims to solve at least one of the technical problems existing in the related art. To this end, the present invention provides an image restoration method based on hierarchical multi-head attention, which can handle various image restoration tasks, including low-light enhancement, snow removal, and defogging.

[0005] The present invention provides an image restoration method based on hierarchical multi-head attention driving, comprising the following steps: S1: Input the degraded image into the convolution layer to extract shallow features; S2: Input the shallow features into the image recovery module to obtain deep features; S3: Generate a residual image using a convolutional layer for the deep features; S4: Add the residual image and the shallow features element by element to obtain a restored image.

[0006] According to the present invention, a method for image restoration based on hierarchical multi-head attention driving is provided, wherein the image restoration module comprises the following steps: S10: Input the shallow features into the encoder to obtain the encoder output; S20: Inputting the encoder output into a decoder to obtain a decoder output; S30: Input the decoder output into the refinement layer to obtain deep features.

[0007] According to an image restoration method based on hierarchical multi-head attention driving provided by the present invention, the encoder includes a feedforward network module and a downsampling convolution layer.

[0008] According to an image restoration method based on hierarchical multi-head attention driving provided by the present invention, the decoder comprises the following steps: S100: Input the encoder output into the hierarchical multi-head attention layer and perform a query-key cache (QK-Cache) mechanism update to obtain a first decoder result; S200: Inputting the first decoder result into a feed-forward network (FFN) module to obtain a second decoder result; S300: Perform upsampling convolution on the second decoder result to obtain a third decoder result; S400: Perform a jump connection on the third decoder result and the encoder output, and then perform convolution to obtain a decoder output.

[0009] According to the present invention, a method for image restoration based on hierarchical multi-head attention driving is provided, wherein the hierarchical multi-head attention layer (HMHA) comprises the following steps: S101: Divide the encoder output into channels, and reorder them from small to large according to the Pearson similarity to obtain the channel space , ,in, For the The channel subspace of the layer, is the channel number, , is the total number of channels; S102: Calculate the first decoder result based on the zoom point attention mechanism: in, is the attention function, is the query matrix, is the bond matrix, is the value matrix, is the first activation function, is the dimension of the bond matrix, is the matrix transpose, is the input tensor, is the weight matrix of the query matrix, is the weight matrix of the bond matrix, is the weight matrix of the value matrix, For the The weight matrix of the query matrix of the channel, For the The weight matrix of the channel's key matrix, For the The weight matrix of the channel value matrix, For the The attention value of the channel, is the output projection matrix, For HMHA output, is the splicing function.

[0010] According to the image restoration method based on layered multi-head attention drive provided by the present invention, the query-key cache mechanism update includes the following methods: intra-layer cache and inter-layer cache, The intra-layer caching includes the following steps: S111: Calculate the modulation component within the layer : in, Cache value within the layer; S112: Calculate key information for gating mechanism selection : in, is the second activation function, is the convolution function, is element-wise multiplication; S113: Reconstruct key information selected from the gating mechanism to obtain : in, is the forward convolution, is backward convolution; S114: Update the cache in the layer: in, For the The intra-layer matrix of the channel, For the The query matrix of the channel, No. The channel's bond matrix; The inter-layer caching includes the following steps: S121: Calculate modulation components : in, For the modulation ingredients, To click on attention, To adjust the inter-layer cache value, is the offset component, is the bias component weight matrix, is the scale component, is the scale component weight matrix; S122: Calculation : S123: Update inter-layer cache value: in, To adjust the front-layer cache value, is the resizing function, is a hyperparameter, For the updated inter-layer cache value, This is the inter-layer cache value before updating.

[0011] According to the present invention, a method for image restoration based on hierarchical multi-head attention driving is provided, wherein the refinement layer comprises the following steps: S1000: Input the decoder output into the convolutional layer and perform a query-key cache mechanism update to obtain the first refinement layer result; S2000: Input the first refining layer result into the feedforward network module to obtain deep features.

[0012] The present invention also provides an image restoration system based on hierarchical multi-head attention drive, comprising: Shallow feature extraction module: inputs the degraded image into the convolution layer to extract shallow features; Deep feature extraction module: inputs the shallow features into the image restoration module to obtain deep features; Image restoration module: generates a residual image using a convolutional layer for the deep features; and adds the residual image and the shallow features element by element to obtain a restored image.

[0013] The present invention also provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the program, the steps of an image restoration system based on hierarchical multi-head attention drive as described above are implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of an image restoration system based on hierarchical multi-head attention drive as described above.

[0015] The above one or more technical solutions in the embodiments of the present invention have at least one of the following technical effects: The present invention provides an image restoration method based on hierarchical multi-head attention driving, which proposes a hierarchical multi-head attention module. The module introduces a ranking mechanism based on channel similarity and combines it with hierarchical subspace partitioning, so that each subspace contains independent information and has different sizes.

[0016] This invention designs a query-key cache update mechanism that enhances inter-head interaction, including both intra-layer and inter-layer approaches. The intra-layer cache acts as a gating module to enhance the useful information in the aggregated features of the heads; the inter-layer cache modulates the attention weight of each head using historical attention scores. This invention is advantageous in that it can handle a variety of image restoration tasks, including low-light enhancement, snow removal, and defogging. Compared to other similar inventions, this invention achieves higher accuracy and better image restoration results.

[0017] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned by practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 This is a flowchart of an image restoration method based on hierarchical multi-head attention drive proposed by the present invention.

[0020] Figure 2 This is the overall structural diagram of the image restoration method based on hierarchical multi-head attention drive proposed in this invention.

[0021] Figure 3 This is the specific structure and implementation method of the hierarchical multi-head attention mechanism equipped with query-key cache update proposed in the present invention.

[0022] Figure 4 This is a before-and-after comparison of images of the present invention on a snow removal task.

[0023] Figure 5 This is a structural block diagram of an image restoration system based on hierarchical multi-head attention drive provided by the present invention.

[0024] Figure 6 It is a structural schematic diagram of the electronic device provided by the present invention.

[0025] Reference numerals: 101. Shallow feature extraction module; 102. Deep feature extraction module; 103. Image restoration module; 810. Processor; 820. Communication interface; 830. Memory; 840. Communication bus. DETAILED DESCRIPTION

[0026] To make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below. Obviously, the embodiments described are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. The following embodiments are used to illustrate the present invention, but are not used to limit the scope of the present invention.

[0027] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiment of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0028] The following combination Figures 1 to 6 Describing the Invention Implementation example Figure 1 and Figure 2 As shown in (a), Figure 1 This is a flowchart of an image restoration method based on hierarchical multi-head attention drive, which includes the following steps: S1: Input the degraded image into the convolution layer to extract shallow features; S2: Input the shallow features into the image recovery module to obtain deep features; S3: Generate a residual image using a convolutional layer for the deep features; S4: Add the residual image and the shallow features element by element to obtain a restored image.

[0029] Specifically, such as Figure 2 As shown in (a), the image restoration module includes the following steps: S10: Input the shallow features into the encoder to obtain the encoder output; S20: Inputting the encoder output into a decoder to obtain a decoder output; S30: Input the decoder output into the refinement layer to obtain deep features.

[0030] The encoder includes a feedforward network module and a downsampling convolutional layer.

[0031] Among them, such as Figure 2 As shown in (a), the decoder includes the following steps: S100: Input the encoder output into the hierarchical multi-head attention layer and perform a query-key cache mechanism update to obtain a first decoder result; S200: Inputting the first decoder result into a feedforward network module to obtain a second decoder result; S300: Perform upsampling convolution on the second decoder result to obtain a third decoder result; S400: Perform a jump connection on the third decoder result and the encoder output, and then perform convolution to obtain a decoder output.

[0032] Specifically, such as Figure 2 (b) and Figure 3 As shown in Figure 2, the hierarchical multi-head attention layer consists of the following steps: S101: Divide the encoder output into channels, and reorder them from small to large according to the Pearson similarity to obtain the channel space , ,in, For the The channel subspace of the layer, is the channel number, , is the total number of channels; S102: Calculate the first decoder result based on the zoom point attention mechanism: in, is the attention function, is the query matrix, is the bond matrix, is the value matrix, is the first activation function, is the dimension of the bond matrix, is the matrix transpose, is the input tensor, is the weight matrix of the query matrix, is the weight matrix of the bond matrix, is the weight matrix of the value matrix, For the The weight matrix of the query matrix of the channel, For the The weight matrix of the channel's key matrix, For the The weight matrix of the channel value matrix, For the The attention value of the channel, is the output projection matrix, For HMHA output, is the splicing function.

[0033] This paper proposes a hierarchical multi-head attention module, which introduces a ranking mechanism based on channel similarity and combines it with hierarchical subspace partitioning, so that each subspace contains independent information and has different sizes.

[0034] Specifically, the query-key cache mechanism updates include the following methods: intra-layer cache and inter-layer cache, The intra-layer caching includes the following steps: S111: Calculate the modulation component within the layer : in, Cache value within the layer; S112: Calculate key information for gating mechanism selection : in, is the second activation function, is the convolution function, is element-wise multiplication; S113: Reconstruct key information selected from the gating mechanism to obtain : in, is the forward convolution, is backward convolution; S114: Update the cache in the layer: in, For the The intra-layer matrix of the channel, For the The query matrix of the channel, No. The channel's bond matrix; The inter-layer caching includes the following steps: S121: Calculate modulation components : in, For the modulation ingredients, To click on attention, To adjust the inter-layer cache value, is the offset component, is the bias component weight matrix, is the scale component, is the scale component weight matrix; S122: Calculation : S123: Update inter-layer cache value: in, To adjust the front-layer cache value, is the resizing function, is a hyperparameter, For the updated inter-layer cache value, This is the inter-layer cache value before updating.

[0035] The query-key cache mechanism first performs query-key cache modulation and updates within each layer, filtering important information through feature summation and gating mechanisms, and using convolution operations to compress and reconstruct features. Subsequently, the result is updated to the query-key cache to improve the utilization of information within the layer. During the inter-layer information exchange, inter-layer query-key cache modulation and updates are calculated. First, the current layer attention result is summed with the historical cache, and the scale and offset components of the features are calculated. Then, the gating mechanism is used to filter and optimize the features. Next, the cache contribution value of the current layer is calculated, and a weighted update mechanism is used in combination with the historical cache to achieve dynamic fusion of inter-layer information, thereby improving the overall recovery effect.

[0036] This invention designs a query-key cache update mechanism that enhances inter-head interaction, including both intra-layer and inter-layer approaches. The intra-layer cache acts as a gating module to enhance the useful information in the aggregated features of the heads; the inter-layer cache modulates the attention weight of each head using historical attention scores. This invention is advantageous in that it can handle a variety of image restoration tasks, including low-light enhancement, snow removal, and defogging. Compared to other similar inventions, this invention achieves higher accuracy and better image restoration results.

[0037] Specifically, such as Figure 2 As shown in (a), the refining layer includes the following steps: S1000: Input the decoder output into the convolutional layer and perform a query-key cache mechanism update to obtain the first refinement layer result; S2000: Input the first refining layer result into the feedforward network module to obtain deep features.

[0038] This paper verifies the image restoration method based on hierarchical multi-head attention drive: The method described in this paper was compared with other methods for simultaneously processing images from the LOL-v2-real and LOL-v2-syn datasets. The restoration performance was evaluated using Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM). Higher PSNR and SSIM indicate better image restoration.

[0039] Table 1 Comparison of image restoration methods

[0040] Among them, KinD (Kindling in Darkness: A Practical Low-Light Image Enhancement Method), EnGAN1 (Enlightening Generative Adversarial Network for Deep Illumination Enhancement without Paired Supervision), EnGAN2 (Expanded and Collaboratively Guided Architecture Search via Neural Generative Adversarial Network Inspired by Retina Mechanisms), Uformer (Unified U-shaped Transformer for Image Restoration), Restormer (Resolution-Efficient State-space Transformer for High-resolution Image Restoration), MIRNet (Multi-scale Interactive Refinement Network Learning Rich Features for Real Image Restoration and Enhancement), Sparse (Sparsity-Aware Progressive Adaptive Refinement and Enhancement, learning rich features for real image restoration and enhancement), QuardPrior (Quadruple Physical Prior-driven Zero-reference Low-Light Enhancement via Physics-Informed Constraints, achieving zero-reference low-light enhancement through physical quadruple priors), ManbaIR (Mamba-based Simple Baseline for Image Restoration in State-Space Paradigm,Other image restoration methods include SNR-Net (Signal-to-Noise Ratio-Aware Network for Low-Light Image Enhancement), Reuinexformer (Retinex-based One-stage Transformer for Low-Light Image Enhancement), and MambaLLIE (Mamba-based Low-Light Image Enhancement with Global and Local State-Space Modeling for Implicit Retina Perception). HINT is the method provided by the present invention.

[0041] As shown in Table 1, the present invention is compared with 12 other proposed image restoration methods, including deep learning-based methods, on the snow removal image restoration task. The results show that the present invention's image restoration performance on multiple tasks surpasses all previously published results.

[0042] like Figure 4 As shown, Figure 4 This is the before and after comparison of images in the snow removal task of the present invention. Figure 4 The middle (a) is the picture before the snow. Figure 4 (b) is a picture after snow removal using the present invention. Figure 4 The advantages of the present invention in snow removal tasks can be clearly reflected.

[0043] like Figure 5 As shown, Figure 5 This is a block diagram of an image restoration system based on hierarchical multi-head attention drive, including: Shallow feature extraction module 101: inputs the degraded image into the convolution layer to extract shallow features; Deep feature extraction module 102: inputs the shallow features into the image restoration module to obtain deep features; The image restoration module 103 generates a residual image by using a convolution layer for the deep features; and adds the residual image and the shallow features element by element to obtain a restored image.

[0044] Figure 6 An example of a physical structure diagram of an electronic device is shown below. Figure 6As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other via the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute an image restoration method based on hierarchical multi-head attention drive, the method comprising: S1: Input the degraded image into the convolution layer to extract shallow features; S2: Input the shallow features into the image recovery module to obtain deep features; S3: Generate a residual image using a convolutional layer for the deep features; S4: Add the residual image and the shallow features element by element to obtain a restored image.

[0045] Furthermore, the logic instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, mobile hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0046] On the other hand, the present invention also provides a computer program product, comprising a computer program stored on a non-transitory computer-readable storage medium, wherein the computer program comprises program instructions. When the program instructions are executed by a computer, the computer is capable of performing an image restoration method based on hierarchical multi-head attention drive provided by the above methods, the method comprising: S1: Input the degraded image into the convolution layer to extract shallow features; S2: Input the shallow features into the image recovery module to obtain deep features; S3: Generate a residual image using a convolutional layer for the deep features; S4: Add the residual image and the shallow features element by element to obtain a restored image.

[0047] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the above-mentioned image restoration method based on hierarchical multi-head attention driving, the method comprising: S1: Input the degraded image into the convolution layer to extract shallow features; S2: Input the shallow features into the image recovery module to obtain deep features; S3: Generate a residual image using a convolutional layer for the deep features; S4: Add the residual image and the shallow features element by element to obtain a restored image.

[0048] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0049] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0050] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

[0051] It should be noted that the embodiments of the present disclosure can be implemented by hardware, software, or a combination of software and hardware. The hardware portion can be implemented using dedicated logic; the software portion can be stored in a memory and executed by an appropriate instruction execution system such as a microprocessor or dedicated hardware. Those skilled in the art will understand that the above-mentioned devices and methods can be implemented using computer-executable instructions and / or contained in processor control code, for example, such code is provided on a programmable memory or a data carrier such as an optical or electronic signal carrier.

[0052] In addition, although the operations of the method of the present disclosure are described in a particular order in the accompanying drawings, this does not require or imply that these operations must be performed in this particular order, or that all the operations shown must be performed to achieve the desired result. On the contrary, the steps depicted in the flow chart can change the order of execution. Additionally or alternatively, certain steps can be omitted, multiple steps can be combined into one step, and / or one step can be decomposed into multiple steps. It should also be noted that the features and functions of two or more devices according to the present disclosure can be embodied in one device. Conversely, the features and functions of a device described above can be further divided into being embodied by multiple devices.

[0053] Although the present disclosure has been described with reference to several specific embodiments, it should be understood that the present disclosure is not limited to the specific embodiments disclosed. The present disclosure is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A hierarchical multi-head attention-driven image restoration method, characterized in that: The following steps are involved: S1: Input the degraded image into the convolution layer to extract shallow features; S2: Input the shallow features into the image recovery module to obtain deep features; S3: Generate a residual image using a convolutional layer for the deep features; S4: Add the residual image and the shallow features element by element to obtain a restored image.

2. The image restoration method based on hierarchical multi-head attention drive according to claim 1 is characterized in that: The image restoration module comprises the following steps: S10: Input the shallow features into the encoder to obtain the encoder output; S20: Inputting the encoder output into a decoder to obtain a decoder output; S30: Input the decoder output into the refinement layer to obtain deep features.

3. The image restoration method based on hierarchical multi-head attention drive according to claim 2 is characterized in that: The encoder includes a feedforward network module and a downsampling convolutional layer.

4. The image restoration method based on hierarchical multi-head attention drive according to claim 2 is characterized in that: The decoder comprises the following steps: S100: Input the encoder output into the hierarchical multi-head attention layer and perform a query-key cache mechanism update to obtain a first decoder result; S200: Inputting the first decoder result into a feedforward network module to obtain a second decoder result; S300: Perform upsampling convolution on the second decoder result to obtain a third decoder result; S400: Perform a jump connection on the third decoder result and the encoder output, and then perform convolution to obtain a decoder output.

5. The image restoration method based on hierarchical multi-head attention drive according to claim 4 is characterized in that: The hierarchical multi-head attention layer includes the following steps: S101: Divide the encoder output into channels, and reorder them from small to large according to the Pearson similarity to obtain the channel space , ,in, For the The channel subspace of the layer, is the channel number, , is the total number of channels; S102: Calculate the first decoder result based on the zoom point attention mechanism: in, is the attention function, is the query matrix, is the bond matrix, is the value matrix, is the first activation function, is the dimension of the bond matrix, is the matrix transpose, is the input tensor, is the weight matrix of the query matrix, is the weight matrix of the bond matrix, is the weight matrix of the value matrix, For the The weight matrix of the query matrix of the channel, For the The weight matrix of the channel's key matrix, For the The weight matrix of the channel value matrix, For the The attention value of the channel, is the output projection matrix, For HMHA output, is the splicing function.

6. The image restoration method based on hierarchical multi-head attention drive according to claim 5 is characterized in that: The query-key cache mechanism update includes the following methods: intra-layer cache and inter-layer cache, The intra-layer caching includes the following steps: S111: Calculate the modulation component within the layer : in, Cache value within the layer; S112: Calculate key information for gating mechanism selection : in, is the second activation function, is the convolution function, is element-wise multiplication; S113: Reconstruct key information selected from the gating mechanism to obtain : in, is the forward convolution, is backward convolution; S114: Update the cache in the layer: in, For the The intra-layer matrix of the channel, For the The query matrix of the channel, No. The channel's bond matrix; The inter-layer caching includes the following steps: S121: Calculate modulation components : in, For the modulation ingredients, To click on attention, To adjust the inter-layer cache value, is the offset component, is the bias component weight matrix, is the scale component, is the scale component weight matrix; S122: Calculation : S123: Update inter-layer cache value: in, To adjust the front-layer cache value, is the resizing function, is a hyperparameter, For the updated inter-layer cache value, This is the inter-layer cache value before updating.

7. The image restoration method based on hierarchical multi-head attention drive according to claim 2 is characterized in that: The refining layer includes the following steps: S1000: Input the decoder output into the convolutional layer and perform a query-key cache mechanism update to obtain the first refinement layer result; S2000: Input the first refining layer result into the feedforward network module to obtain deep features.

8. An image restoration system based on hierarchical multi-head attention driving, used to execute the image restoration method based on hierarchical multi-head attention driving according to any one of claims 1 to 7, characterized in that: include: Shallow feature extraction module: inputs the degraded image into the convolution layer to extract shallow features; Deep feature extraction module: inputs the shallow features into the image restoration module to obtain deep features; Image restoration module: generates a residual image using a convolutional layer for the deep features; and adds the residual image and the shallow features element by element to obtain a restored image.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the image restoration method based on hierarchical multi-head attention driving are implemented as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of an image restoration method based on hierarchical multi-head attention driving are implemented as described in any one of claims 1 to 7.

Images