Lightweight single image super-resolution method based on attention sharing and information distillation

Abstract

This invention discloses a lightweight single-image super-resolution method based on attention sharing and information distillation. The method includes: acquiring a low-resolution image to be processed; inputting the low-resolution image into a pre-trained super-resolution model to obtain a high-resolution image output by the super-resolution model; wherein the super-resolution model is obtained by training a pre-constructed deep learning network using a sample dataset; the sample dataset includes clear and blurred image pairs obtained by preprocessing the acquired original image data; and the deep learning network includes a shallow feature extraction module, an information distillation block stacking module, and an upsampling reconstruction module. This achieves lightweight image reconstruction.

Description

Technical Field

[0001] This invention relates to the field of image data processing or enhancement technology, and specifically to a lightweight single-image super-resolution method based on attention sharing and information distillation. Background Technology

[0002] Single-image super-resolution technology aims to reconstruct high-resolution images from low-resolution images using algorithms, and it has significant application value in fields such as medical image analysis, satellite remote sensing, and surveillance video enhancement. In recent years, deep learning-based single-image super-resolution methods have made significant progress. Among them, Transformer-based methods outperform traditional convolutional neural network methods due to their ability to capture long-distance pixel dependencies, but they suffer from high computational complexity and large parameter scale, making it difficult to meet the lightweight requirements of practical scenarios.

[0003] Therefore, providing a lightweight single-image resolution method based on attention sharing and information distillation to achieve lightweight image reconstruction has become a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] Therefore, embodiments of the present invention provide a lightweight single-image super-resolution method based on attention sharing and information distillation to solve at least one technical problem existing in the prior art.

[0005] This invention provides a lightweight single-image super-resolution method based on attention sharing and information distillation, the method comprising:

[0006] Acquire low-resolution images to be processed; The low-resolution image is input into a pre-trained super-resolution model to obtain a high-resolution image output by the super-resolution model. The super-resolution model is obtained by training a pre-constructed deep learning network using a sample dataset. The sample dataset includes clear and blurred image pairs obtained by preprocessing the collected original image data. The deep learning network includes a shallow feature extraction module, an information distillation block stacking module, and an upsampling reconstruction module.

[0007] In some embodiments, the training process of the super-resolution model includes: Raw image data is acquired and preprocessed to obtain data pairs of clear and blurry images. A sample dataset is constructed based on a large number of data pairs. The sample dataset is divided into a training set and a validation set according to a certain ratio; Initialize the network parameters of the deep learning network and set the hyperparameters; The low-resolution images in the training set are input into the initialization model. The low-resolution images are subjected to feature extraction, information distillation and upsampling reconstruction in the initialization model to output a super-resolution image. Based on a pre-built loss function, the loss between the super-resolution image and the clear image is calculated, and the model is iteratively optimized until a preset number of iterations or a preset loss is reached, at which point the iteration stops and the super-resolution model is obtained.

[0008] In some embodiments, the original image data is preprocessed, specifically including: Data of different blur types and degrees of blur, along with clear image data, were captured using a microscope as raw data. The collected raw data is classified into clear and blurry categories. Clear images are selected as the labels for the model, and the clear images are processed to simulate the blurring effect under a microscope, forming pairs of blurry and clear images.

[0009] In some embodiments, the shallow feature extraction module is used to extract shallow features Fs of the input image ILR through a 3×3 convolutional layer Hs, which maps the input image from RGB channels to multi-channel latent feature dimensions.

[0010] In some embodiments, the information distillation module and the 3x3 convolutional layer Hd, which are cascaded in the information distillation block stacking module, extract deep features Fd from shallow features Fs.

[0011] In some embodiments, the information distillation module includes: The local module applies a convolution operation to the input features to extract local features for use in the subsequent self-attention module. A spatial attention module enhances features by capturing pixel correlations within a single window and between different windows; The channel attention module calculates the similarity matrix of pixels within and between windows along the channel dimension.

[0012] In some embodiments, the information distillation module includes: Input features Features are obtained through the first local module and the spatial attention module. With attention matrix ,right Channel splitting yields refined features coarse features ; coarse features Features are obtained through the second local module and the spatial attention module. ,right Channel splitting yields refined features coarse features ; coarse features Features are obtained through the third local module and the spatial attention module. ; splicing , , The aggregated and refined features are obtained through 1×1 convolution, and then processed by the channel attention module and enhanced spatial attention operations, and finally combined with the input features. Fusion output.

[0013] In some embodiments, both the spatial attention module and the channel attention module employ a window-based self-attention method. The self-attention method includes operation at the meso-level and global level. At the meso-level, the pixel similarity matrix within a single window is calculated, and at the global level, the pixel similarity matrix between different windows is calculated, fusing intra-block and inter-block feature information.

[0014] The present invention also provides a lightweight single-image super-resolution device based on attention sharing and information distillation, the device comprising: An image acquisition unit is used to acquire low-resolution images to be processed. The result generation unit is used to input the low-resolution image into a pre-trained super-resolution model to obtain a high-resolution image output by the super-resolution model. The super-resolution model is obtained by training a pre-constructed deep learning network using a sample dataset. The sample dataset includes clear and blurred image pairs obtained by preprocessing the collected original image data. The deep learning network includes a shallow feature extraction module, an information distillation block stacking module, and an upsampling reconstruction module.

[0015] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method described above.

[0016] The lightweight single-image super-resolution method based on attention sharing and information distillation provided by this invention acquires a low-resolution image to be processed and inputs the low-resolution image into a pre-trained super-resolution model to obtain a high-resolution image output by the super-resolution model. The super-resolution model is obtained by training a pre-constructed deep learning network using a sample dataset. The sample dataset includes clear images and blurred image pairs obtained by preprocessing the acquired original image data. The deep learning network includes a shallow feature extraction module, an information distillation block stacking module, and an upsampling reconstruction module.

[0017] Thus, by combining an information distillation mechanism with a cross-module attention sharing strategy, this invention effectively captures long-distance dependencies in images while significantly reducing model parameters and computational load, solving the problem of balancing performance and efficiency in traditional super-resolution methods and achieving lightweight reconstruction. Through hierarchical feature distillation and attention matrix reuse, this model can adaptively handle low-resolution image degradation problems in different scenarios, achieving efficient and high-quality single-image super-resolution reconstruction. Attached Figure Description

[0018] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely exemplary, and those skilled in the art can derive other embodiments based on the provided drawings without creative effort.

[0019] The structures, proportions, sizes, etc. illustrated in this specification are only for the purpose of assisting those skilled in the art in understanding and reading the content disclosed herein, and are not intended to limit the conditions under which the present invention can be implemented. Therefore, they have no substantial technical significance. Any modifications to the structure, changes in the proportions, or adjustments to the size, without affecting the effects and objectives that the present invention can produce, should still fall within the scope of the technical content disclosed in the present invention.

[0020] Figure 1 This is one of the flowcharts for the lightweight single-image super-resolution method based on attention sharing and information distillation provided by the present invention; Figure 2 The second flowchart is a lightweight single-image super-resolution method based on attention sharing and information distillation provided by the present invention. Figure 3 This refers to the input image in an embodiment of the present invention; Figure 4 This is the output image in an embodiment of the present invention; Figure 5 This is a structural block diagram of a computer device provided by the present invention. Detailed Implementation

[0021] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] To address the aforementioned technical problems, this invention provides a lightweight single-image super-resolution method based on attention sharing and information distillation, including data acquisition and preprocessing, dataset establishment, super-resolution model construction, loss function design, model training and optimization, model validation and testing, and using the trained model to reconstruct low-resolution images into high-resolution images, ultimately obtaining high-resolution images with rich details and improved quality.

[0023] In one specific implementation, such as Figure 1 As shown, the lightweight single-image super-resolution method based on attention sharing and information distillation provided by this invention includes the following steps: S110: Acquire low-resolution images to be processed; S120: Input the low-resolution image into a pre-trained super-resolution model to obtain a high-resolution image output by the super-resolution model; The super-resolution model is obtained by training a pre-constructed deep learning network using a sample dataset. The sample dataset includes clear and blurred image pairs obtained by preprocessing the collected original image data. The deep learning network includes a shallow feature extraction module, an information distillation block stacking module, and an upsampling reconstruction module.

[0024] In this way, the method can directly perform end-to-end super-resolution reconstruction on any low-resolution image, with a simple process and convenient deployment. It adopts a modular network structure design, which takes into account the integrity of feature extraction, lightweight model and inference efficiency, and can stably adapt to a variety of complex degradation scenarios such as microscope images, while reducing hardware resource dependence while maintaining high reconstruction quality.

[0025] It is understood that the method provided by this invention comprises two parts, such as... Figure 2 The overall workflow shown depicts the model building and training process on the left and the model retrieval and usage process on the right. During model building and training, after data acquisition and high-resolution data filtering, the acquired raw images are converted and stored as a sample dataset. The sample dataset includes the retained original images, generated blur type 1, and generated blur type 2. The sample dataset is divided into a training set, a validation set, and a test set. The model is trained using the training set and validated using the validation set to obtain the final super-resolution model. During usage, the trained super-resolution model is deployed in the system, and the image data to be processed is input into the super-resolution model to obtain the output super-resolution image.

[0026] Specifically, the training process of the super-resolution model includes: Raw image data is acquired and preprocessed to obtain data pairs of clear and blurry images. A sample dataset is constructed based on a large number of data pairs. The sample dataset is divided into a training set and a validation set according to a certain ratio; Initialize the network parameters of the deep learning network and set the hyperparameters; The low-resolution images in the training set are input into the initialization model. The low-resolution images are subjected to feature extraction, information distillation and upsampling reconstruction in the initialization model to output a super-resolution image. Based on a pre-built loss function, the loss between the super-resolution image and the clear image is calculated, and the model is iteratively optimized until a preset number of iterations or a preset loss is reached, at which point the iteration stops and the super-resolution model is obtained.

[0027] In this way, the standardized training process ensures the model's convergence stability and generalization ability, and the division of the training set and validation set can effectively monitor the risk of model overfitting. Iterative optimization guided by the loss function can continuously reduce the difference between the reconstructed image and the real high-definition image, enabling the model to gradually have accurate detail restoration and texture reconstruction capabilities.

[0028] In some embodiments, the original image data is preprocessed, specifically including: Data of different blur types and degrees of blur, along with clear image data, were captured using a microscope as raw data. The collected raw data is classified into clear and blurry categories. Clear images are selected as the labels for the model, and the clear images are processed to simulate the blurring effect under a microscope, forming pairs of blurry and clear images.

[0029] In the data acquisition and preprocessing process, this invention addresses the blurring problem in images captured under a machine scanning microscope. The data sources include images with different blur types and degrees of blur, as well as clear images. The acquired raw data is categorized into clear and blurry images. Clear images are selected as the model's labels, and the blurring process under a microscope is simulated for clear images, creating pairs of blurry and clear images. This ensures the dataset closely matches the real degradation characteristics of microscope images, avoiding the mismatch between general datasets and practical application scenarios. By simulating degradation to generate paired samples, the model ensures uniform sample distribution and accurate labeling, significantly improving its adaptability, specificity, and reconstruction reliability for blurry microscope images.

[0030] Specifically, in the construction process of the aforementioned super-resolution model, the super-resolution model includes a shallow feature extraction module, an information distillation block, a stacking module, and an upsampling reconstruction module. The process is as follows: For a given input image I_LR, shallow features Fs are extracted through a 3×3 convolutional layer Hs, as shown below: ; This convolutional layer maps the input image from RGB channels to a multi-channel latent feature dimension. Then, a cascaded information distillation module and a 3x3 convolutional layer Hd extract deep features Fd from the shallow features Fs, as shown below:

[0031]

[0032]

[0033] in, Let represent the i-th information distillation module, A represent the attention matrix used for attention sharing, and N represent the number of information distillation modules. Each information distillation module contains multiple self-attention layers that extract meaningful features by considering long-range dependencies. These stacked information distillation modules refine the input features and finally extract deep features through a convolutional layer Hd. Then, the upsampling module Hup reconstructs the high-resolution output image ISR from the deep features Fd, as shown below: ; In this process, the pixel rearrangement layer remaps the latent feature dimensions back to the RGB channels and high-resolution space, thereby restoring the ISR of the output image.

[0034] Thus, the method provided by this invention rapidly extracts low-level visual features such as image edges, contours, brightness, and basic textures using lightweight convolutions, completing the transformation from RGB images to a high-dimensional latent feature space. Without increasing computational burden, it provides stable, rich, and dimensionally adapted basic features for subsequent deep feature extraction, improving the overall network feature utilization efficiency. Through multi-level stacking, features are refined layer by layer from shallow to deep and from coarse to fine, fully mining deep semantic and fine structural information of the image. While retaining effective features, redundant noise is filtered, significantly improving feature expression strength and discriminative power, providing high-quality feature support for the final high-resolution image reconstruction.

[0035] In some embodiments, the information distillation module includes: The local module applies a convolution operation to the input features to extract local features for use in the subsequent self-attention module. A spatial attention module enhances features by capturing pixel correlations within a single window and between different windows; The channel attention module calculates the similarity matrix of pixels within and between windows along the channel dimension.

[0036] In this way, feature representation is enhanced synergistically from three dimensions: local structure, spatial location, and feature channels. Local modules focus on fine details, spatial attention models pixel dependencies, and channel attention strengthens the weights of key feature channels. The three complement each other, enabling the model to capture local details, global structure, and important feature dimensions simultaneously, thereby improving feature effectiveness and reconstruction accuracy.

[0037] The information distillation module includes: Input features Features are obtained through the first local module and the spatial attention module. With attention matrix ,right Channel splitting yields refined features coarse features ; coarse features Features are obtained through the second local module and the spatial attention module. ,right Channel splitting yields refined features coarse features ; coarse features Features are obtained through the third local module and the spatial attention module. ; splicing , , The aggregated and refined features are obtained through 1×1 convolution, and then processed by the channel attention module and enhanced spatial attention operations, and finally combined with the input features. Fusion output.

[0038] In this way, feature layering and progressive refinement are achieved through channel splitting, effectively distinguishing key features from secondary features and reducing redundant calculations; multi-level refined feature splicing can fully retain effective information at different levels, and 1×1 convolution achieves efficient feature fusion; combining channel attention and enhanced spatial attention further strengthens feature expression, and residual fusion ensures that the original information is not lost, thereby improving the ability to restore details while reducing the number of parameters and computation.

[0039] Furthermore, both the spatial attention module and the channel attention module employ a window-based self-attention method. This self-attention method operates at both the meso-level and global levels. At the meso-level, the pixel similarity matrix within a single window is calculated, while at the global level, the pixel similarity matrix between different windows is calculated, fusing intra-block and inter-block feature information. In this way, the meso-level focuses on fine local features within the window, ensuring the complete reconstruction of small targets and texture details; the global level establishes long-distance pixel dependencies across windows, improving the consistency of large-scale structures and overall layout. This dual-level approach avoids the problem of high computational cost in global self-attention while addressing the deficiency of insufficient local receptive fields, thereby improving self-attention efficiency and enhancing the overall expressive power of the network.

[0040] Specifically, Information Distillation Blocks (IDBs) process features by hierarchically splitting them during the distillation step. These methods preserve the basic representation in a portion while passing the remaining features to subsequent modules, recognizing that some feature channels carry more critical information than others. Our designed information distillation module refines features progressively through splitting and concatenation, primarily consisting of Local Modules (LMs), Spatial Attention Modules (SAMs), and Channel Attention Modules (CAMs). Given input features... The first information distillation module refines features in the following way:

[0041]

[0042]

[0043]

[0044]

[0045] in, This represents the i-th LM. This represents the i-th SAM. Let represent the i-th spatial self-attention matrix, and `Split` denote channel splitting. The information distillation module refines features progressively by splitting them through channels. The LM applies convolution operations to the input features to extract local features for subsequent self-attention modules. Then, the SAM enhances the features by capturing pixel correlations within a single window and between different windows. After progressive feature refinement, the refined hierarchical features are aggregated through concatenation and 1x1 convolutions, as shown below:

[0046]

[0047] Wherein, CAM stands for Channel Attention Module. `ESA` stands for Enhanced Spatial Attention. Channel attention calculates the similarity matrix of pixels within and between windows along the channel dimension. The ESA module uses strided convolution and max pooling to achieve a large receptive field and rescale features, thereby enhancing features in spatial context. Simultaneously, the subsequent information distillation module utilizes an attention-sharing mechanism, efficiently performing self-attention layer operations using the spatial self-attention matrix calculated by the initial information distillation module. This allows subsequent information distillation modules to skip the calculation of the self-attention matrix by sharing the attention matrix from the initial information distillation module, as shown below:

[0048]

[0049]

[0050]

[0051]

[0052] The local module consists of pixel-wise convolution, depthwise convolution, and an activation module. The local module is located before the spatial attention module so that the spatial attention can use local information to compute the spatial correlation matrix.

[0053] Spatial attention and channel attention employ a window-based self-attention method to process input features into non-overlapping blocks. These two modules operate in two stages, referred to as the meso-level and global level. The meso-level self-attention layer computes the similarity matrix of pixels within a single block, while the global level self-attention layer computes the similarity matrix of pixels across different blocks. By combining intra-block and inter-block information, this method enhances the network's representational power, effectively capturing local information and long-range dependencies.

[0054] First, the input feature f is partitioned into meso-level and global-level windows. For the meso-level, the input feature is partitioned into PxP non-overlapping blocks, where P represents the partition size of the meso-level self-attention. For the global level, the input feature is partitioned into GxG non-overlapping blocks, where G represents the partition size of the global-level self-attention. Then, a linear projection is applied to the input feature to compute the self-attention matrix. Given the input feature f, compute the query... , key Sum As shown below:

[0055] Where Q, K, and V represent linear projection operations. Next, the self-attention matrix A is calculated as follows:

[0056]

[0057] in, represents the output of the self-attention layer, and FFN represents the feedforward network.

[0058] The computational burden of self-attention is a key inefficiency factor in super-resolution Transformers; however, stacking spatial self-attention layers is necessary. To reduce computational complexity, attention sharing and channel splitting techniques are introduced. Attention sharing allows self-attention layers to share the attention matrix, thus eliminating the need for spatial self-attention operations. This allows layers sharing the attention matrix to skip the computation of the self-attention matrix, reducing the parameters required for affinity matrix computation. Channel splitting limits the number of channels participating in spatial attention operations, which helps reduce computational burden and the number of parameters. The combination of attention sharing and channel splitting reduces the complexity of self-attention, allowing more layers to be stacked with fewer parameters.

[0059] In the above specific embodiments, the lightweight single-image super-resolution method based on attention sharing and information distillation provided by the present invention acquires a low-resolution image to be processed and inputs the low-resolution image into a pre-trained super-resolution model to obtain a high-resolution image output by the super-resolution model; wherein, the super-resolution model is obtained by training a pre-constructed deep learning network using a sample dataset; the sample dataset includes clear images and blurred image pairs obtained by preprocessing the acquired original image data; the deep learning network includes a shallow feature extraction module, an information distillation block stacking module, and an upsampling reconstruction module.

[0060] Thus, by combining an information distillation mechanism with a cross-module attention sharing strategy, this invention effectively captures long-distance dependencies in images while significantly reducing model parameters and computational load, solving the problem of balancing performance and efficiency in traditional super-resolution methods. Through hierarchical feature distillation and attention matrix reuse, this model can adaptively handle low-resolution image degradation problems in different scenarios, achieving efficient and high-quality single-image super-resolution reconstruction.

[0061] For ease of understanding, the overall implementation process and technical effects of the method provided by the present invention will be introduced below through an embodiment.

[0062] The lightweight single-image super-resolution method based on attention sharing and information distillation provided by this invention, such as... Figure 3 As shown, please follow these steps: 1. Data Acquisition and Preprocessing: This invention addresses the problem of image blurring under a machine scanning microscope. The data sources are data on different types and degrees of blurring captured under the microscope, as well as clear image data. The acquired raw data is classified into clear and blurry categories. Clear images are selected as labels for the model. The blurry images are simulated to mimic the blurring process under the microscope, forming blurry image and clear image pairs.

[0063] 2. Super-resolution model construction; the model includes a shallow feature extraction module, an information distillation block (IDB) stacking module, and an upsampling reconstruction module. The process is as follows: For a given input image ILR, shallow features Fs are extracted through a 3×3 convolutional layer Hs, as shown below:

[0064] This convolutional layer maps the input image from RGB channels to a multi-channel latent feature dimension. Then, a cascaded information distillation module and a 3x3 convolutional layer Hd extract deep features Fd from the shallow features Fs, as shown below:

[0065]

[0066]

[0067] in, Let represent the i-th information distillation module, A represent the attention matrix used for attention sharing, and N represent the number of information distillation modules. Each information distillation module contains multiple self-attention layers that extract meaningful features by considering long-range dependencies. These stacked information distillation modules refine the input features and finally extract deep features through a convolutional layer Hd. Then, the upsampling module Hup reconstructs the high-resolution output image ISR from the deep features Fd, as shown below:

[0068] In this process, the pixel rearrangement layer remaps the latent feature dimensions back to the RGB channels and high-resolution space, thereby restoring the ISR of the output image.

[0069] 3. Information Distillation Block (IDB) Information distillation processes features by hierarchically splitting them in a distillation step. These methods preserve the basic representation in a portion while passing the remaining features to subsequent modules, recognizing that some feature channels carry more critical information than others. The information distillation module provided in this invention refines features progressively through splitting and concatenation, and mainly consists of a Local Module (LM), a Spatial Attention Module (SAM), and a Channel Attention Module (CAM). Given input features... The first information distillation module refines features in the following way:

[0070]

[0071]

[0072]

[0073]

[0074] in, This represents the i-th LM. This represents the i-th SAM. Let represent the i-th spatial self-attention matrix, and `Split` denote channel splitting. The information distillation module refines features progressively by splitting them through channels. The LM applies convolution operations to the input features to extract local features for subsequent self-attention modules. Then, the SAM enhances the features by capturing pixel correlations within a single window and between different windows. After progressive feature refinement, the refined hierarchical features are aggregated through concatenation and 1x1 convolutions, as shown below:

[0075]

[0076] Wherein, CAM stands for Channel Attention Module. `ESA` stands for Enhanced Spatial Attention. Channel attention calculates the similarity matrix of pixels within and between windows along the channel dimension. The ESA module uses strided convolution and max pooling to achieve a large receptive field and rescale features, thereby enhancing features in spatial context. Simultaneously, the subsequent information distillation module utilizes an attention-sharing mechanism, efficiently performing self-attention layer operations using the spatial self-attention matrix calculated by the initial information distillation module. This allows subsequent information distillation modules to skip the calculation of the self-attention matrix by sharing the attention matrix from the initial information distillation module, as shown below:

[0077]

[0078]

[0079]

[0080]

[0081] The local module consists of pixel-wise convolution, depthwise convolution, and an activation module. The local module precedes the spatial attention module so that the spatial attention module can utilize local information to compute the spatial correlation matrix.

[0082] Spatial attention and channel attention employ a window-based self-attention method to process input features into non-overlapping blocks. These two modules operate in two stages, referred to as the meso-level and global level. The meso-level self-attention layer computes the similarity matrix of pixels within a single block, while the global level self-attention layer computes the similarity matrix of pixels across different blocks. By combining intra-block and inter-block information, this method enhances the network's representational power, effectively capturing local information and long-range dependencies.

[0083] First, the input feature f is partitioned into meso-level and global-level windows. For the meso-level, the input feature is partitioned into PxP non-overlapping blocks, where P represents the partition size of the meso-level self-attention. For the global level, the input feature is partitioned into GxG non-overlapping blocks, where G represents the partition size of the global-level self-attention. Then, a linear projection is applied to the input feature to compute the self-attention matrix. Given the input feature f, the query is computed... , key Sum As shown below:

[0084] Where Q, K, and V represent linear projection operations. Next, the self-attention matrix A is calculated as follows:

[0085]

[0086] in, represents the output of the self-attention layer, and FFN represents the feedforward network.

[0087] The computational burden of self-attention is a key inefficiency factor in super-resolution Transformers; however, stacking spatial self-attention layers is necessary. To reduce computational complexity, attention sharing and channel splitting techniques are introduced. Attention sharing allows self-attention layers to share the attention matrix, thus eliminating the need for spatial self-attention operations. This allows layers sharing the attention matrix to skip the computation of the self-attention matrix, reducing the parameters required for affinity matrix computation. Channel splitting limits the number of channels participating in spatial attention operations, which helps reduce computational burden and the number of parameters. The combination of attention sharing and channel splitting reduces the complexity of self-attention, allowing more layers to be stacked with fewer parameters.

[0088] like Figure 3 and Figure 4 As can be seen from the illustrated diagrams, this invention has the following advantages: (1) In order to adapt to complex low-resolution image degradation scenarios, this method solves the problem of balancing performance and efficiency in single-image super-resolution and overcomes the limitations of traditional methods in terms of insufficient reconstruction quality under lightweight design. By combining the information distillation mechanism with the attention sharing strategy, it can effectively capture long-distance dependencies of images to ensure reconstruction accuracy, while significantly reducing the computational load. It is suitable for low-resolution image super-resolution tasks with various degradation types.

[0089] (2) A feature processing mechanism combining attention sharing and channel segmentation is introduced to achieve efficient optimization of self-attention computation. This method not only achieves hierarchical refinement of features through the information distillation module, thereby improving the feature representation capability, but also reduces redundant computation of subsequent modules by reusing the attention matrix calculated by the initial information distillation module. While reducing the parameter scale and computational load, it retains the ability to capture image details, achieving efficient feature fusion and enhancement under a lightweight model.

[0090] (3) This method enhances the practicality of the model in resource-constrained scenarios and provides a super-resolution solution that combines flexibility and efficiency. It can generate high-quality high-resolution results on low-resolution images without relying on complex hardware configurations or large amounts of pre-trained data.

[0091] In addition to the methods described above, this invention also provides a lightweight single-image super-resolution device based on attention sharing and information distillation, the device comprising: An image acquisition unit is used to acquire low-resolution images to be processed. The result generation unit is used to input the low-resolution image into a pre-trained super-resolution model to obtain a high-resolution image output by the super-resolution model. The super-resolution model is obtained by training a pre-constructed deep learning network using a sample dataset. The sample dataset includes clear and blurred image pairs obtained by preprocessing the collected original image data. The deep learning network includes a shallow feature extraction module, an information distillation block stacking module, and an upsampling reconstruction module.

[0092] In some embodiments, the training process of the super-resolution model includes: Raw image data is acquired and preprocessed to obtain data pairs of clear and blurry images. A sample dataset is constructed based on a large number of data pairs. The sample dataset is divided into a training set and a validation set according to a certain ratio; Initialize the network parameters of the deep learning network and set the hyperparameters; The low-resolution images in the training set are input into the initialization model. The low-resolution images are subjected to feature extraction, information distillation and upsampling reconstruction in the initialization model to output a super-resolution image. Based on a pre-built loss function, the loss between the super-resolution image and the clear image is calculated, and the model is iteratively optimized until a preset number of iterations or a preset loss is reached, at which point the iteration stops and the super-resolution model is obtained.

[0093] In some embodiments, the original image data is preprocessed, specifically including: Data of different blur types and degrees of blur, along with clear image data, were captured using a microscope as raw data. The collected raw data is classified into clear and blurry categories. Clear images are selected as the labels for the model, and the clear images are processed to simulate the blurring effect under a microscope, forming pairs of blurry and clear images.

[0094] In some embodiments, the shallow feature extraction module is used to extract shallow features Fs of the input image ILR through a 3×3 convolutional layer Hs, which maps the input image from RGB channels to multi-channel latent feature dimensions.

[0095] In some embodiments, the information distillation module and the 3x3 convolutional layer Hd, which are cascaded in the information distillation block stacking module, extract deep features Fd from shallow features Fs.

[0096] In some embodiments, the information distillation module includes: The local module applies a convolution operation to the input features to extract local features for use in the subsequent self-attention module. A spatial attention module enhances features by capturing pixel correlations within a single window and between different windows; The channel attention module calculates the similarity matrix of pixels within and between windows along the channel dimension.

[0097] In some embodiments, the information distillation module includes: Input features Features are obtained through the first local module and the spatial attention module. With attention matrix ,right Channel splitting yields refined features coarse features ; coarse features Features are obtained through the second local module and the spatial attention module. ,right Channel splitting yields refined features coarse features ; coarse features Features are obtained through the third local module and the spatial attention module. ; splicing , , The aggregated and refined features are obtained through 1×1 convolution, and then processed by the channel attention module and enhanced spatial attention operations, and finally combined with the input features. Fusion output.

[0098] In some embodiments, both the spatial attention module and the channel attention module employ a window-based self-attention method. The self-attention method includes operation at the meso-level and global level. At the meso-level, the pixel similarity matrix within a single window is calculated, and at the global level, the pixel similarity matrix between different windows is calculated, fusing intra-block and inter-block feature information.

[0099] In the above specific embodiments, the lightweight single-image super-resolution device based on attention sharing and information distillation provided by the present invention acquires a low-resolution image to be processed and inputs the low-resolution image into a pre-trained super-resolution model to obtain a high-resolution image output by the super-resolution model; wherein, the super-resolution model is obtained by training a pre-constructed deep learning network using a sample dataset; the sample dataset includes clear images and blurred image pairs obtained by preprocessing the acquired original image data, and the deep learning network includes a shallow feature extraction module, an information distillation block stacking module, and an upsampling reconstruction module.

[0100] Thus, by combining an information distillation mechanism with a cross-module attention sharing strategy, this invention effectively captures long-distance dependencies in images while significantly reducing model parameters and computational load, solving the problem of balancing performance and efficiency in traditional super-resolution methods. Through hierarchical feature distillation and attention matrix reuse, this model can adaptively handle low-resolution image degradation problems in different scenarios, achieving efficient and high-quality single-image super-resolution reconstruction.

[0101] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and model predictions. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The model predictions of the computer device store static and dynamic information data. The network interface of the computer device is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements the steps in the above method embodiments.

[0102] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present invention and does not constitute a limitation on the computer device to which the present invention is applied. A specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0103] Corresponding to the above embodiments, this invention also provides a computer storage medium containing one or more program instructions. These one or more program instructions are used to execute the method described above.

[0104] The present invention also provides a computer program product, the computer program product including a computer program, the computer program being stored on a non-transitory computer-readable storage medium, and the computer being able to perform the above-described method when the computer program is executed by a processor.

[0105] In this embodiment of the invention, the processor can be an integrated circuit chip with signal processing capabilities. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0106] The various methods, steps, and logic diagrams disclosed in the embodiments of this invention can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this invention can be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The processor reads information from the storage medium and, in conjunction with its hardware, completes the steps of the above methods.

[0107] The storage medium can be memory, such as volatile memory or non-volatile memory, or may include both volatile and non-volatile memory.

[0108] Among them, non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory.

[0109] Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (Synchlink DRAM, SLDRAM), and direct memory bus RAM (DRRAM).

[0110] The storage media described in the embodiments of the present invention are intended to include, but are not limited to, these and any other suitable types of memory.

[0111] Those skilled in the art will recognize that, in one or more of the examples above, the functions described in this invention can be implemented using a combination of hardware and software. When applied as software, the corresponding functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transmission of computer programs from one place to another. Storage media can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0112] The above specific embodiments further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solution of the present invention should be included within the scope of protection of the present invention.

Claims

1. A lightweight single-image super-resolution method based on attention sharing and information distillation, characterized in that, The method includes: Acquire low-resolution images to be processed; The low-resolution image is input into a pre-trained super-resolution model to obtain a high-resolution image output by the super-resolution model. The super-resolution model is obtained by training a pre-constructed deep learning network using a sample dataset. The sample dataset includes clear and blurred image pairs obtained by preprocessing the collected original image data. The deep learning network includes a shallow feature extraction module, an information distillation block stacking module, and an upsampling reconstruction module.

2. The lightweight single-image super-resolution method based on attention sharing and information distillation according to claim 1, characterized in that, The training process of the super-resolution model includes: Raw image data is acquired and preprocessed to obtain data pairs of clear and blurry images. A sample dataset is constructed based on a large number of data pairs. The sample dataset is divided into a training set and a validation set according to a certain ratio; Initialize the network parameters of the deep learning network and set the hyperparameters; The low-resolution images in the training set are input into the initialization model. The low-resolution images are subjected to feature extraction, information distillation and upsampling reconstruction in the initialization model to output a super-resolution image. Based on a pre-built loss function, the loss between the super-resolution image and the clear image is calculated, and the model is iteratively optimized until a preset number of iterations or a preset loss is reached, at which point the iteration stops and the super-resolution model is obtained.

3. The lightweight single-image super-resolution method based on attention sharing and information distillation according to claim 2, characterized in that, The preprocessing of the raw image data specifically includes: Data of different blur types and degrees of blur, along with clear image data, were captured using a microscope as raw data. The collected raw data is classified into clear and blurry categories. Clear images are selected as the labels for the model, and the clear images are processed to simulate the blurring effect under a microscope, forming pairs of blurry and clear images.

4. The lightweight single-image super-resolution method based on attention sharing and information distillation according to claim 3, characterized in that, The shallow feature extraction module is used to extract shallow features Fs of the input image ILR through a 3×3 convolutional layer Hs, which maps the input image from RGB channels to multi-channel latent feature dimensions.

5. The lightweight single-image super-resolution method based on attention sharing and information distillation according to claim 4, characterized in that, The information distillation module and the cascaded information distillation module and the 3x3 convolutional layer Hd in the information distillation block stack module extract the deep feature Fd from the shallow feature Fs.

6. The lightweight single-image super-resolution method based on attention sharing and information distillation according to claim 5, characterized in that, The information distillation module includes: The local module applies a convolution operation to the input features to extract local features for use in the subsequent self-attention module. A spatial attention module enhances features by capturing pixel correlations within a single window and between different windows; The channel attention module calculates the similarity matrix of pixels within and between windows along the channel dimension.

7. The lightweight single-image super-resolution method based on attention sharing and information distillation according to claim 6, characterized in that, The information distillation module includes: Input features Features are obtained through the first local module and the spatial attention module. With attention matrix ,right Channel splitting yields refined features and coarse features ; coarse features Features are obtained through the second local module and the spatial attention module. ,right Channel splitting yields refined features coarse features ; coarse features Features are obtained through the third local module and the spatial attention module. ; splicing , , The aggregated and refined features are obtained through 1×1 convolution, and then processed by the channel attention module and enhanced spatial attention operations, and finally combined with the input features. Fusion output.

8. The method for evaluating the mimicry of breast milk fat globule membrane proteins based on multidimensional similarity weighted fusion according to claim 7, characterized in that, Both the spatial attention module and the channel attention module adopt a window-based self-attention method. The self-attention method includes meso-level and global-level operation. At the meso-level, the pixel similarity matrix within a single window is calculated, and at the global level, the pixel similarity matrix between different windows is calculated, fusing intra-block and inter-block feature information.

9. A lightweight single-image super-resolution device based on attention sharing and information distillation, characterized in that, The device includes: An image acquisition unit is used to acquire low-resolution images to be processed. The result generation unit is used to input the low-resolution image into a pre-trained super-resolution model to obtain a high-resolution image output by the super-resolution model. The super-resolution model is obtained by training a pre-constructed deep learning network using a sample dataset. The sample dataset includes clear and blurred image pairs obtained by preprocessing the collected original image data. The deep learning network includes a shallow feature extraction module, an information distillation block stacking module, and an upsampling reconstruction module.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method as described in any one of claims 1-8.

Images