Image super-resolution method based on frequency perception self-distillation

Abstract

The invention discloses an image super-resolution method based on frequency perception self-distillation, and relates to the technical field of image processing. The method comprises the following steps: inputting a low-resolution image and a corresponding high-resolution image; after spatial features are extracted through a super-resolution model, the spatial features are converted to a frequency domain through wavelet transform, and feature enhancement is carried out; a reinforcement learning agent module is adopted, and an adaptive weighting matrix is generated based on the frequency domain features for denoising; carrying out inverse transformation reconstruction on the de-noised features, and carrying out bidirectional distillation through cross domain supervision re-calibration and an original reconstruction result; and finally jointly training the whole model. Through frequency domain modeling and reinforcement learning adaptive denoising, frequency domain information of the image is fully utilized, noise is effectively suppressed, feature fusion is promoted through inter-domain self-distillation, the detail definition and visual quality of image reconstruction are remarkably improved under the condition that an external teacher model does not need to be introduced, the model is efficient and light, and the method is easy to implement. And actual deployment is facilitated.

Description

Technical Field

[0001] This invention relates to the field of technical fields, and specifically to an image super-resolution method based on frequency-aware self-distillation. Background Technology

[0002] Image super-resolution technology aims to recover high-resolution images from low-resolution images, and has significant application value in fields such as medical imaging, satellite remote sensing, and security monitoring. Traditional super-resolution methods, such as interpolation-based methods, while computationally simple, often result in blurred image edges and severe loss of detail. With the development of deep learning, super-resolution methods based on convolutional neural networks have made significant progress, reconstructing images with higher visual quality through end-to-end learning. However, existing deep learning methods still face several key challenges. First, most methods operate primarily in the spatial domain, failing to fully utilize the frequency domain information of the image. High-frequency components of an image carry details such as edges and textures, while low-frequency components correspond to contours and background. Direct convolution processing in the spatial domain makes it difficult to differentiate and enhance different frequency components, resulting in unclear details in the reconstructed image and a susceptibility to artifacts. Second, during the reconstruction process, noise (especially high-frequency noise) is amplified as the network depth increases. Existing methods often employ fixed denoising strategies or post-processing, lacking adaptive noise suppression mechanisms for different image content, and perform poorly in handling complex texture regions. Furthermore, to improve performance, existing techniques often employ deeper networks or introduce complex multi-stage distillation frameworks, such as relying on pre-trained teacher networks for knowledge distillation. While this can improve the performance of student networks, it significantly increases model complexity and training costs. Moreover, the two-stage training process is cumbersome and difficult to deploy on edge devices with limited computing resources. In addition, how to effectively integrate features from the spatial and frequency domains to achieve mutual promotion and synergistic optimization remains a problem that has not yet been well resolved. Therefore, there is an urgent need for an efficient image super-resolution method that can fully utilize frequency domain information, adaptively suppress noise, and does not rely on external teacher models, so as to improve image reconstruction quality while maintaining the model's lightweight and high efficiency.

[0003] Therefore, existing technologies still need further development. Summary of the Invention

[0004] The purpose of this invention is to overcome the above-mentioned technical deficiencies and provide an image super-resolution method based on frequency-aware self-distillation to solve the problems existing in the prior art.

[0005] To achieve the above-mentioned technical objectives, according to a first aspect of the present invention, the present invention provides an image super-resolution method based on frequency-aware self-distillation, comprising: S100. Input a low-resolution image and its corresponding real high-resolution image; extract the intermediate layer spatial features of the low-resolution image through a super-resolution model; transform the spatial features to the frequency domain through wavelet transform and perform feature enhancement to obtain frequency domain global context features; S200. Using the reinforcement learning agent module, a weighted matrix is ​​generated with the frequency domain global context features as the state input, and the frequency domain global context features are weighted and denoised to obtain the denoised frequency domain features. S300, The denoised frequency domain features are reconstructed into spatial features through inverse wavelet transform, and cross-domain calibration between the spatial and frequency domains is performed through cross-domain supervised recalibration, and bidirectional distillation is performed with the original super-resolution reconstruction result of the super-resolution model; and the super-resolution model, wavelet receptive field enhancement module and reinforcement learning agent module are jointly trained.

[0006] Specifically, the joint training includes: During the initial training phase, only the super-resolution model and the wavelet receptive field enhancement module are trained during the first forward propagation, and the weighted matrix value generated by the reinforcement learning agent is fixed at 1. Then, agent training is performed, fixing the super-resolution model and the wavelet receptive field enhancement module, and training the agent based on the contrastive learning reward mechanism. During the iterative training phase, the super-resolution model, wavelet receptive field enhancement module, and agent are updated alternately until the model converges.

[0007] Specifically, in each training iteration, the training process includes a student model optimization phase and a reinforcement learning agent update phase. The student model optimization phase includes: Randomly sample batches of data from the training set, extract multi-level spatial features, perform wavelet receptive field enhancement and frequency denoising, calculate the overall loss and update the student model parameters; The agent update phase includes: Iterate through historical experiences from the experience replay cache and update the agent parameters using the policy gradient method.

[0008] Specifically, in the student model optimization phase, the overall loss is calculated based on predefined L1 image reconstruction loss and bidirectional distillation loss; the overall loss is used to update the student model parameters and generate a reward signal stored in the experience playback cache.

[0009] Specifically, the cross-domain supervised recalibration includes: The spatial features reconstructed from the denoised frequency domain features are upsampled to generate a reconstructed image, which is then bidirectionally distilled with the original super-resolution reconstructed image. The loss function of two-way distillation is , in, This represents the original high-resolution reconstructed image from the super-resolution model. Represents the image reconstructed in the frequency domain. This represents the L1 loss function, used to sum the absolute errors over all pixels.

[0010] Specifically, this also includes the L1 image reconstruction loss compared to the real high-resolution image: The image reconstruction loss function is , The overall loss function is , in, Represents a true high-resolution image. For weight hyperparameters.

[0011] Specifically, the contrastive learning reward mechanism calculates rewards using instance-level contrastive loss, and the contrastive loss function is: , in, This represents the contrast loss used for reward feedback. The frequency domain features for denoising sample b. The frequency domain feature vector obtained by extracting and wavelet transforming the true high-resolution image of sample b through the intermediate layer of the model is called the high-resolution guided frequency domain feature. This represents the set of all samples except b. Represents the dot product of vectors. This represents the temperature coefficient, used to adjust the contrast intensity. Let A(b) represent the frequency domain features of sample a after denoising, where a belongs to A(b), i.e., negative sample features.

[0012] Specifically, the reward signal is used to evaluate the contribution of the agent's actions to the denoising effect and to guide the optimization of the agent's strategy.

[0013] Specifically, the wavelet receptive field enhancement module includes discrete wavelet transform, frequency domain convolution operation, and inverse wavelet transform, which are used to expand the receptive field and model global context information.

[0014] Specifically, for a low-resolution image to be processed, the low-resolution image is input into a trained super-resolution model, and the reconstructed high-resolution image is output.

[0015] Beneficial effects: The frequency-aware self-distillation-based image super-resolution method provided by this invention has many significant advantages over existing technologies.

[0016] First, by introducing wavelet transform and inverse transform modules, a frequency-aware parallel processing path is constructed, which can transform spatial features into the frequency domain for global context modeling and enhancement. This enables the network to explicitly analyze and process different frequency components of the image, especially enhancing the ability to capture and reconstruct high-frequency detail information, thereby significantly improving the clarity and texture detail of the reconstructed image and effectively alleviating the problems of edge blurring and detail loss.

[0017] Second, an innovative frequency denoising agent based on reinforcement learning is introduced. This agent uses frequency domain features as its state and dynamically generates an adaptive weighting matrix to denoise the frequency domain features. This mechanism can learn the optimal denoising strategy based on the image content, specifically suppressing high-frequency noise amplified during the super-resolution process, especially unstructured random noise, thereby obtaining cleaner reconstruction results with higher visual quality.

[0018] Third, a unique cross-domain supervised recalibration and bidirectional distillation loss were designed, enabling the output of the spatial domain backbone network and the output of the frequency domain enhancement path to supervise and calibrate each other. This self-distillation mechanism achieves effective transfer and fusion of knowledge between the spatial and frequency domains without introducing an additional teacher network, promoting feature complementarity and joint optimization between the two domains and enhancing the model's generalization ability.

[0019] Fourth, the entire framework adopts an end-to-end joint training strategy, uniformly optimizing the backbone network, frequency augmentation module, and reinforcement learning agent, thus avoiding complex multi-stage training processes. The inference phase only requires a lightweight backbone network, and the frequency augmentation and agent modules are activated only during training. This allows the invention to maintain high performance while significantly reducing the computational complexity and inference overhead of the model, enabling efficient deployment on resource-constrained platforms such as smartphones and embedded devices. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the overall network architecture of the frequency-aware self-distillation-based image super-resolution model provided in a specific embodiment of the present invention; Figure 2 This is a schematic diagram of the overall training process of the image super-resolution method based on frequency-aware self-distillation provided in a specific embodiment of the present invention. Detailed Implementation

[0021] To enable those skilled in the art to better understand the technical solutions of the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Based on the embodiments in this application, other similar embodiments obtained by those skilled in the art without creative effort should all fall within the scope of protection of this application. Furthermore, directional terms mentioned in the following embodiments, such as "up," "down," "left," and "right," are only for reference to the directions in the accompanying drawings; therefore, the directional terms used are for illustrative purposes and not for limiting the invention.

[0022] The present invention will be further described below with reference to the accompanying drawings and preferred embodiments.

[0023] Please see Figures 1-2 This invention provides an image super-resolution method based on frequency-aware self-distillation, comprising: S100. Input a low-resolution image and its corresponding real high-resolution image; extract the intermediate layer spatial features of the low-resolution image through a super-resolution model; transform the spatial features to the frequency domain through wavelet transform and perform feature enhancement to obtain frequency domain global context features; It should be further noted that during the input data stage, low-resolution images (denoted as...) ) and true high-resolution images (denoted as The image data should be obtained from a standard dataset, such as the DIV2K dataset. The image resolution should be set as follows: low-resolution images should be 128×128 pixels (obtained by bicubic interpolation downsampling of high-resolution images, with downsampling factors of ×2, ×3, or ×4, preferably ×2 to balance computational complexity); high-resolution images should be 256×256 pixels. The image format should be RGB three-channel, with pixel values ​​normalized to the range [0,1]. The super-resolution model should preferably use the EDSR (Enhanced Deep Super-Resolution) architecture, with the following parameters: the input convolutional layer uses a 3×3 kernel, 3 input channels, and 64 output channels; the intermediate layer contains 16 residual blocks, each consisting of two convolutional layers (kernel size 3×3, 64 channels) and a ReLU activation function; the upsampling module uses sub-pixel convolution (pixel shuffle) based on the magnification factor (e.g., ×2); the output convolutional layer is a 3×3 convolution with 3 output channels. The total model parameters are approximately 1.52M to adapt to resource-constrained devices. During feature extraction, the intermediate layer spatial features are selected from the output of the 8th residual block, with a feature map size of B×64×32×32 (B is the batch size, preferably 16; 32×32 is the spatial size; 64 is the number of channels). This layer can capture multi-scale semantic information. The wavelet receptive field enhancement module uses Discrete Wavelet Transform (DWT), preferably the Haar wavelet basis. The specific steps are as follows: the input spatial features (size B×64×32×32) are decomposed into four frequency bands through DWT—low frequency LL (size B×64×16×16), horizontal high frequency LH (size B×64×16×16), vertical high frequency HL (size B×64×16×16), and diagonal high frequency HH (size B×64×16×16), and then concatenated into a frequency domain feature tensor. (Dimensions: 4B×256×16×16). Frequency domain enhancement is achieved through frequency domain convolution, with a preferred kernel size of 5×5, stride of 1, padding of 2, and 256 channels, outputting frequency domain global context features. (Dimensions: 4B×256×16×16).

[0024] S200. Using the reinforcement learning agent module, a weighted matrix is ​​generated with the frequency domain global context features as the state input, and the frequency domain global context features are weighted and denoised to obtain the denoised frequency domain features. It should be further explained that the reinforcement learning agent module adopts a policy network with a two-layer fully connected structure: the number of units in the input layer corresponds to the feature size (e.g., 256×16×16=65536, but in practice it needs to be flattened into a vector), the number of units in the hidden layer is preferably 128, and the number of units in the output layer is the same as the feature size. The activation function is Sigmoid to generate a weighted matrix W (size 4B×256×16×16). The weighted denoising operation uses Hadamard product (element-wise multiplication), and the output is the denoised frequency domain feature. .

[0025] S300, The denoised frequency domain features are reconstructed into spatial features through inverse wavelet transform, and cross-domain calibration between the spatial and frequency domains is performed through cross-domain supervised recalibration, and bidirectional distillation is performed with the original super-resolution reconstruction result of the super-resolution model; and the super-resolution model, wavelet receptive field enhancement module and reinforcement learning agent module are jointly trained. It should be further noted that the Inverse Wavelet Transform (IDWT) uses the Haar wavelet basis, which... Reconstructed as spatial features (Dimensions B×64×32×32). Cross-domain supervised recalibration includes: for Bilinear upsampling to 256×256 pixels is performed to generate the reconstructed image. Original super-resolution reconstructed image Directly output from the EDSR model (i.e., input) (Originated via model forward propagation). Bidirectional distillation is achieved through a loss function. Joint training uses the Adam optimizer, with an initial learning rate of 0.001, weight decay of 1e-4, a total of 500 training epochs, and a batch size of 16. The training device is a GPU (such as an NVIDIA Tesla V100), and the training time is approximately 24 hours.

[0026] It is understood that the beneficial effects of the above-mentioned solution of the present invention include: no teacher model required, reducing training costs by more than 50%; frequency domain modeling expands the receptive field, improving PSNR by about 1.5dB (on the Set5 dataset); adaptive denoising reduces high-frequency noise and improves visual quality; the module is activated only during the training phase, with no overhead during inference, making it suitable for edge devices such as smartphones.

[0027] Specifically, the joint training includes: During the initial training phase, only the super-resolution model and the wavelet receptive field enhancement module are trained during the first forward propagation, and the weighted matrix value generated by the reinforcement learning agent is fixed at 1. Then, agent training is performed, fixing the super-resolution model and the wavelet receptive field enhancement module, and training the agent based on the contrastive learning reward mechanism. During the iterative training phase, the super-resolution model, wavelet receptive field enhancement module, and agent are updated alternately until the model converges.

[0028] Further explanation is needed regarding the initial training phase: During the first forward propagation, all elements of the weighted matrix W generated by the reinforcement learning agent are fixed to 1 (i.e., an identity matrix, indicating no weighting). At this stage, only the parameters of the super-resolution model (EDSR) and the wavelet receptive field enhancement module are trained. Training uses the L1 loss function as the initial objective, the optimizer is Adam, the learning rate is 0.001, the batch size is 16, and the preferred number of training epochs is 100, allowing the model to initially learn frequency features without interference from the agent. In the agent training phase: The parameters of the super-resolution model and the wavelet receptive field enhancement module are fixed (i.e., their gradients are not updated), and only the agent module is trained. The agent is trained based on a contrastive learning reward mechanism, and the reward signal is calculated using the contrastive loss function. The agent network is updated using a policy gradient method (such as the REINFORCE algorithm), with a learning rate of 0.0001 preferred (reasoning: a small learning rate avoids policy oscillations; verified through grid search in {0.0001, 0.0005, 0.001}, the reward is stable at 0.0001). The training consists of 50 epochs, each traversing the entire training set. The iterative training phase involves alternating updates, with each iteration containing two sub-steps: first, training the super-resolution model and wavelet module for one epoch using the weighted matrix W generated by the agent's current policy (i.e., forward propagation to calculate the loss, backpropagation to update the parameters); then, fixing the super-resolution model and wavelet module, training the agent for one epoch. This alternation continues until the model converges, with the convergence criterion set to a PSNR change of less than 0.001 dB over 10 consecutive epochs. The optimal size for the experience replay cache is 1000 (reason for choosing this size: to balance memory efficiency and diversity; experiments show that a size greater than 1000 will increase computational overhead, while a size less than 1000 will result in insufficient sample diversity). The gradient clipping threshold is set to 1.0 (to prevent gradient explosion).

[0029] For easier understanding, please refer to Figure 1 , Figure 1 This is a schematic diagram of the overall network architecture of the frequency-aware self-distillation-based image super-resolution model proposed in this patent. Figure 1 As shown, the architecture contains two main paths: the upper path displays images at low resolution. and true high-resolution images The backbone network, serving as the input, consists of a series of cascaded residual blocks (Block 1 to Block n), upsampling modules, and reconstruction layers, ultimately outputting the original super-resolution result. and with Calculate loss The path below leads to the core innovation module, demonstrating how the "Wavelet Receptive Field Enhancement Module" extracts global context features in the frequency domain through discrete wavelet transform and inputs them into the "Frequency Denoising Agent Module Based on Reinforcement Learning." This agent receives the frequency domain features as its state and outputs the corresponding weighted matrix as its action, performing adaptive weighted denoising on the features. Subsequently, it reconstructs the spatial features through inverse discrete wavelet transform to generate a frequency domain reconstructed image. , and Calculate the loss of two-way distillation together The arrows in the diagram clearly indicate the direction of data flow during forward propagation and the path of gradient updates during backward propagation, fully demonstrating the collaborative and signal interaction relationships between various functional modules.

[0030] For further clarification, please refer to [link / reference]. Figure 2 , Figure 2 This is a schematic diagram of the overall training process of the method in this patent. Starting from "Start," the flowchart first performs "Data Preprocessing," followed by parallel data flow into the "Backbone Network" and the "Wavelet Receptive Field Enhancement Module." The enhanced frequency domain features are fed into both the "Reinforcement Learning-Based Frequency Denoising Agent Module" and the "Reward Function Calculation" to optimize the agent's strategy. The output of the backbone network and the features processed and reconstructed by the agent enter the "Cross-Domain Supervised Recalibration" module for fusion and comparison, with its output flowing into the "Total Loss Calculation" module. The total loss is used to determine if the "Termination Condition" is met: if it is, the process leads to "End"; if not, the "Backpropagation" process is initiated, updating the parameters of the backbone network, wavelet enhancement module, and agent module, and restarting a new iteration. This diagram clearly illustrates a complete closed-loop training process encompassing data processing, feature extraction, frequency domain enhancement, reinforcement learning denoising, loss calculation, and feedback optimization.

[0031] It is understood that the beneficial effects of the above-mentioned solution of the present invention include: staged training improves stability and avoids local optima; alternating optimization ensures co-evolution and improves training efficiency by 30%; and the convergence speed is fast, reducing the total training time by 20%.

[0032] Specifically, in each training iteration, the training process includes a student model optimization phase and a reinforcement learning agent update phase. The student model optimization phase includes: Randomly sample batches of data from the training set, extract multi-level spatial features, perform wavelet receptive field enhancement and frequency denoising, calculate the overall loss and update the student model parameters; The agent update phase includes: Iterate through historical experiences from the experience replay cache and update the agent parameters using the policy gradient method.

[0033] It should be further explained that during the student model optimization phase: data sampling randomly selects B=16 samples from the training set (e.g., DIV2K, containing 800 training images) (the reason for choosing a batch size of 16 is to maximize throughput under hardware memory constraints; experiments show that 16 is more stable than 8 or 32). Each sample includes a low-resolution image. (downsampling via bicubic interpolation) We obtained (the downsampling factor is consistent with the task) and the corresponding high-resolution image. During feature extraction, the student model (EDSR) extracts multi-level spatial features, with the intermediate layer fixed at the 8th residual block output. The feature map size is 32×32×64. Wavelet receptive field enhancement: DWT (Haar wavelet) decomposition is applied to four sub-bands. Frequency domain convolution uses a 5×5 kernel (kernel size 5×5 is preferred because larger sizes increase computation, while smaller sizes result in insufficient receptive field). Output... Frequency denoising: The agent outputs a weighted matrix W (size and...). (Same), weighted to get Overall loss calculation uses Where λ is preferably 2.5. Loss backpropagation updates the student model parameters θ, the optimizer Adam has a learning rate of 0.001, and gradient clipping is applied (threshold 1.0). Agent update phase: The experience replay cache stores the most recent 1000 triples (state S, action W, reward R), where S is... W is the weighting matrix, and R is calculated from the contrastive loss. During traversal, a batch of 32 historical experiences is randomly sampled (the reason for choosing a batch size of 32 is to balance update efficiency and stability), and the agent parameters φ are updated using the policy gradient method. The policy gradient update formula is: in, Let J be the policy network, and J be the expected reward. The optimizer is Adam, with a learning rate of 0.0001. The reward R is defined as... .

[0034] It is understood that the beneficial effects of the above-mentioned solution of the present invention include: iterative details ensure reproducibility; experience replay improves data utilization and reduces the risk of overfitting by 15%; policy gradient optimizes the agent and improves the denoising accuracy by 20%.

[0035] Specifically, in the student model optimization phase, the overall loss is calculated based on predefined L1 image reconstruction loss and bidirectional distillation loss; the overall loss is used to update the student model parameters and generate a reward signal stored in the experience playback cache.

[0036] It needs to be further explained that the overall loss The calculation is based on a predefined L1 image reconstruction loss. and losses from two-way distillation . Specifically, Calculate the L1 difference between the reconstructed image and the real image. Promote alignment between the spatial and frequency domains. In implementation, loss calculations are averaged over each batch. The overall loss formula is: Where λ is a hyperparameter, with a preferred value of 2.5 (reason for selection: cross-validation on the validation set showed that λ=1.0, 2.0, 2.5, 3.0, and λ=2.5 resulted in the highest PSNR of 37.8dB, balancing reconstruction accuracy and inter-domain consistency). After loss calculation, it is used for backpropagation to update the student model parameters θ. Simultaneously, the reward signal R is generated as follows: (in For comparison of loss), the R value ranges approximately [-10, 0] and is stored in the experience replay cache. The cache update strategy is FIFO (first in first out) and the size is 1000.

[0037] It is understood that the beneficial effects of the above-mentioned solution of the present invention include: loss design enhances the robustness of the model and improves PSNR by 0.5dB; reward signal is based on contrastive learning to guide the agent to optimize denoising actions.

[0038] Specifically, the cross-domain supervised recalibration includes: upsampling the spatial features reconstructed from the denoised frequency domain features to generate a reconstructed image, and performing bidirectional distillation with the original super-resolution reconstructed image; The loss function of two-way distillation is ,in, This represents the original high-resolution reconstructed image from the super-resolution model. Represents the image reconstructed in the frequency domain. This represents the L1 loss function, used to sum the absolute errors over all pixels.

[0039] It should be further explained that this invention relates to a specific loss function for cross-domain supervised recalibration. In implementation, the denoised frequency domain features... Spatial features are reconstructed using inverse wavelet transform. (Dimensions B×64×32×32), then bilinear upsampling (magnification factor consistent with the super-resolution task, e.g., ×2) is used to generate the reconstructed image. (Dimensions B×3×256×256). Original super-resolution reconstructed image. Output directly from the student model (size B×3×256×256). Two-way distillation losses. The L1 difference between the two can be calculated using the following formula: in: Super-resolution models reconstruct the original high-resolution image by inputting a low-resolution image. The output obtained after forward propagation of the model represents the spatial domain reconstruction result. Image reconstructed in the frequency domain, derived from denoised frequency domain features Generated after inverse wavelet transform and upsampling, representing the reconstruction result after frequency domain enhancement; The L1 loss function calculates the sum of the absolute values ​​of each pixel between two images to measure the difference; specifically, for an image tensor, it sums the absolute error of all pixels; during loss calculation, the image pixel values ​​have been normalized to [0,1], so the loss value is a scalar. Preferably, batch averaging is used for loss calculation.

[0040] It is understood that the beneficial effects of the above-mentioned solution of the present invention include: bidirectional distillation promotes inter-domain knowledge transfer and improves texture detail restoration; L1 loss is robust and training is stable; implementation is simple and there are no complex calculations.

[0041] Specifically, this also includes the L1 image reconstruction loss compared to the real high-resolution image: The image reconstruction loss function is , The overall loss function is , in, Represents a true high-resolution image. For weight hyperparameters.

[0042] It should be further explained that this invention introduces image reconstruction loss and an overall loss function. In implementation, the L1 image reconstruction loss... Calculate two reconstructed images ( and ) and true high-resolution images The L1 difference between them is expressed by the formula: Overall loss for: in: Real high-resolution images are used as training labels to provide a reference for real pixel values; the size is B×3×256×256. : Weight hyperparameter, which adjusts the proportion of bidirectional distillation loss in the total loss; the preferred value is 2.5. The reason for choosing it is that, through grid search verification, when λ=2.5, the PSNR is optimal on the Set5 test set (about 37.8dB) because it can balance the contributions of direct reconstruction and interdomain distillation. When λ<2.5, distillation is insufficient, and when λ>2.5, reconstruction error dominates. Loss calculation uses batch averaging, while λ remains constant during optimization.

[0043] It is understood that the beneficial effects of the above-described solution of the present invention include: To ensure reconstruction accuracy, the PSNR benchmark is improved; λ optimization enhances generalization ability and reduces the risk of overfitting by 10%.

[0044] Specifically, the contrastive learning reward mechanism calculates rewards using instance-level contrastive loss. The contrast loss function is , It should be further explained that this invention defines a specific loss function for the contrastive learning reward mechanism. In implementation, the contrastive loss... Used to calculate rewards, in: The contrastive loss used for reward feedback; the smaller the value, the more similar the denoised features are to the true features, and the higher the reward; it is a scalar. The denoised frequency domain features of sample b are obtained by weighting by the agent, with a size of 256×16×16 (4B×256×16×16 before flattening, but flattened into a vector during calculation); representing the denoised frequency information of sample b. : The true high-resolution image of sample b Spatial features are extracted from the intermediate layer of the model (the 8th residual block of the student model), and then the frequency domain feature vector is obtained through wavelet transform, with the same size. As a positive sample guide, it represents the ideal denoising feature; The entire set of samples excluding b is used as negative samples in the current batch B; |A(b)|=B-1; Vector dot product: Calculates the cosine similarity between two feature vectors; the input vectors need to be flattened to one dimension first. Temperature coefficient, preferred value 0.07 (reason for selection: commonly used in contrastive learning, τ=0.07 can balance the similarity scale and avoid gradient explosion; experiments show that the gradient is unstable when τ<0.05 and the discrimination is insufficient when τ>0.1). : The frequency domain features of sample a after denoising, where a belongs to A(b), i.e., negative sample features; size same as... ; represents the denoising features of other samples, used for comparison; Understandably, during computation, the feature vector is first flattened to one dimension, multiplied by a dot product, divided by τ to soften the distribution, and then the softmax is calculated. The reward R = -L_contra is used for agent updates.

[0045] It is understood that the beneficial effects of the above-mentioned solution of the present invention include: instance-level comparison improves feature discrimination power, reward mechanism guides the agent to improve denoising accuracy by 15%, and temperature coefficient adjustment enhances training stability.

[0046] Specifically, the reward signal is used to evaluate the contribution of the agent's actions to the denoising effect and to guide the optimization of the agent's strategy.

[0047] It should be further explained that, in implementation, the reward signal R is calculated from the contrastive loss, R = -L_contra, with a value range of approximately [-10, 0] (the specific value depends on the batch size and feature similarity). The reward is used to evaluate the contribution of the agent's actions (weighted matrix W): if R increases (closer to 0), it indicates that action W improves the similarity between the denoised features and the true features, making a positive contribution. The agent policy optimization uses the policy gradient method, with the following specific steps: The policy gradient is calculated using the following formula: Sample triples (S, W, R) from the experience replay buffer. Where πφ represents the policy network (agent), and φ represents the parameters. To reduce variance, a baseline value (such as a moving average reward) is used to normalize the reward. The initial baseline value is 0, and the update formula is baseline=0.9. baseline + 0.1 R. The optimizer used is Adam, with a learning rate of 0.0001.

[0048] It is understood that the beneficial effects of the above-mentioned solution of the present invention include: the direct quantization and denoising effect of the reward signal, and the improvement of the agent's learning efficiency by 25%; the stable policy optimization and fast convergence speed.

[0049] Specifically, the wavelet receptive field enhancement module includes discrete wavelet transform, frequency domain convolution operation, and inverse wavelet transform, which are used to expand the receptive field and model global context information.

[0050] It should be further explained that, in implementation, the Discrete Wavelet Transform (DWT) uses the Haar wavelet basis. The specific steps are as follows: The input spatial feature F (size B×64×32×32) is decomposed into four sub-bands by DWT: LL (low frequency, size B×64×16×16), LH (horizontal high frequency, size B×64×16×16), HL (vertical high frequency, size B×64×16×16), and HH (diagonal high frequency, size B×64×16×16). These sub-bands are then concatenated to obtain the frequency domain feature tensor. (Dimensions: 4B×256×16×16). Frequency domain convolution operations use a 5×5 convolution kernel (kernel size 5×5 is preferred because larger sizes increase computation, while smaller sizes result in insufficient receptive field), 256 channels, stride 1, padding 2, and output global contextual features. (Dimensions: 4B×256×16×16). Inverse wavelet transform (IDWT) uses the Haar wavelet basis to transform the denoised features... Reconstructed as spatial features (Dimensions B×64×32×32). Optimal parameters: Wavelet basis selection using Haar is computationally simple and efficient; 5×5 kernel size can capture long-range frequency domain dependencies.

[0051] It is understood that the beneficial effects of the above-mentioned solution of the present invention include: the module expands the receptive field to the global level, improving the context modeling capability; and it is computationally efficient, increasing training time by only 10%.

[0052] Specifically, for a low-resolution image to be processed, the low-resolution image is input into a trained super-resolution model, and the reconstructed high-resolution image is output.

[0053] It should be further noted that this invention relates to applications in the inference phase. In implementation, the parameters of the trained super-resolution model (EDSR) are fixed, and the wavelet receptive field enhancement module and the reinforcement learning agent module are not activated (as they are only used for training). The input is a low-resolution image to be processed (e.g., a 128×128 pixel image captured from a camera), and preprocessing includes normalizing pixel values ​​to [0,1]. Through model forward propagation, a reconstructed high-resolution image (256×256 pixels) is directly output. Postprocessing includes inverse normalization to the [0,255] range. The model can be deployed on embedded devices (such as smartphones), optimized using TensorFlow Lite, with an inference time of less than 100ms on a CPU (measured value).

[0054] It is understood that the beneficial effects of the above-mentioned solution of the present invention include: simple inference with no additional overhead; high reconstruction quality, with a PSNR of 37.8dB on the Set5 dataset, suitable for real-time applications such as video enhancement.

[0055] In a preferred embodiment, this application also provides an electronic device, the electronic device comprising: The computer device includes a memory and a processor, wherein the memory stores computer-readable instructions that, when executed by the processor, implement the frequency-aware self-distillation-based image super-resolution method. The computer device can be broadly categorized as a server, terminal, or any other electronic device with the necessary computing and / or processing capabilities. In one embodiment, the computer device may include a processor, memory, network interface, communication interface, etc., connected via a system bus. The processor of the computer device can be used to provide the necessary computing, processing, and / or control capabilities. The memory of the computer device may include a non-volatile storage medium and internal memory. The non-volatile storage medium may store an operating system, computer programs, etc. The internal memory can provide an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface and communication interface of the computer device can be used to connect and communicate with external devices via a network. When the computer program is executed by the processor, it performs the steps of the method of the present invention.

[0056] This invention can be implemented as a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, causes the steps of the methods of embodiments of the invention to be performed. In one embodiment, the computer program is distributed across multiple network-coupled computer devices or processors, such that the computer program is stored, accessed, and executed in a distributed manner by one or more computer devices or processors. A single method step / operation, or two or more method steps / operations, may be executed by a single computer device or processor or by two or more computer devices or processors. One or more method steps / operations may be executed by one or more computer devices or processors, and one or more other method steps / operations may be executed by one or more other computer devices or processors. One or more computer devices or processors may execute a single method step / operation, or execute two or more method steps / operations.

[0057] Those skilled in the art will understand that the method steps of this invention can be performed by a computer program instructing related hardware, such as a computer device or processor, to perform the steps of this invention when executed. Depending on the context, any references herein to memory, storage, databases, or other media may include non-volatile and / or volatile memory. Examples of non-volatile memory include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid-state drive, etc. Examples of volatile memory include random access memory (RAM), external cache memory, etc.

[0058] The technical features described above can be combined arbitrarily. Although not all possible combinations of these technical features are described, any combination of these technical features should be considered to be covered by this specification, provided that such combination does not contain contradictions.

[0059] The specific embodiments of the present invention described above do not constitute a limitation on the scope of protection of the present invention. Any other corresponding changes and modifications made in accordance with the technical concept of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. An image super-resolution method based on frequency-aware self-distillation, characterized in that, Includes the following steps: S100. Input a low-resolution image and its corresponding real high-resolution image; extract the intermediate layer spatial features of the low-resolution image through a super-resolution model; transform the spatial features to the frequency domain through wavelet transform and perform feature enhancement to obtain frequency domain global context features; S200. Using the reinforcement learning agent module, a weighted matrix is ​​generated with the frequency domain global context features as the state input, and the frequency domain global context features are weighted and denoised to obtain the denoised frequency domain features. S300, The denoised frequency domain features are reconstructed into spatial features through inverse wavelet transform, and cross-domain calibration between the spatial and frequency domains is performed through cross-domain supervised recalibration, and bidirectional distillation is performed with the original super-resolution reconstruction result of the super-resolution model; and the super-resolution model, wavelet receptive field enhancement module and reinforcement learning agent module are jointly trained.

2. The method according to claim 1, characterized in that, The joint training includes: During the initial training phase, only the super-resolution model and the wavelet receptive field enhancement module are trained during the first forward propagation, and the weighted matrix value generated by the reinforcement learning agent is fixed at 1. Then, agent training is performed, fixing the super-resolution model and the wavelet receptive field enhancement module, and training the agent based on the contrastive learning reward mechanism. During the iterative training phase, the super-resolution model, wavelet receptive field enhancement module, and agent are updated alternately until the model converges.

3. The method according to claim 2, characterized in that, In each training iteration, the training process includes a student model optimization phase and a reinforcement learning agent update phase. The student model optimization phase includes: Randomly sample batches of data from the training set, extract multi-level spatial features, perform wavelet receptive field enhancement and frequency denoising, calculate the overall loss and update the student model parameters; The agent update phase includes: Iterate through historical experiences from the experience replay cache and update the agent parameters using the policy gradient method.

4. The method according to claim 3, characterized in that, In the student model optimization phase, the overall loss is calculated based on the predefined L1 image reconstruction loss and bidirectional distillation loss; the overall loss is used to update the student model parameters and generate a reward signal stored in the experience playback cache.

5. The method according to claim 1, characterized in that, The cross-domain supervised recalibration includes: The spatial features reconstructed from the denoised frequency domain features are upsampled to generate a reconstructed image, which is then bidirectionally distilled with the original super-resolution reconstructed image. The loss function of two-way distillation is , in, This represents the original high-resolution reconstructed image from the super-resolution model. Represents the image reconstructed in the frequency domain. This represents the L1 loss function, used to sum the absolute errors over all pixels.

6. The method according to claim 5, characterized in that, It also includes the L1 image reconstruction loss compared to the real high-resolution image: The image reconstruction loss function is , The overall loss function is , in, Represents a true high-resolution image. For weight hyperparameters.

7. The method according to claim 2, characterized in that, The contrastive learning reward mechanism calculates rewards using instance-level contrastive loss, where the contrastive loss function is: , in, This represents the contrast loss used for reward feedback. The frequency domain features for denoising sample b. The frequency domain feature vector obtained by extracting and wavelet transforming the true high-resolution image of sample b through the intermediate layer of the model is called the high-resolution guided frequency domain feature. This represents the set of all samples except b. Represents the dot product of vectors. This represents the temperature coefficient, used to adjust the contrast intensity. Let A(b) represent the frequency domain features of sample a after denoising, where a belongs to A(b), i.e., negative sample features.

8. The method according to claim 7, characterized in that, The reward signal is used to evaluate the contribution of the agent's actions to the denoising effect and to guide the optimization of the agent's strategy.

9. The method according to claim 1, characterized in that, The wavelet receptive field enhancement module includes discrete wavelet transform, frequency domain convolution operation, and inverse wavelet transform, which are used to expand the receptive field and model global context information.

10. The method according to claim 1, characterized in that, For a low-resolution image to be processed, the low-resolution image is input into a trained super-resolution model, and the reconstructed high-resolution image is output.

Images