Image restoration model compression method based on heterogeneous knowledge distillation

Abstract

The invention relates to the technical field of artificial intelligence, in particular to an image restoration model compression method based on heterogeneous knowledge distillation, which comprises the following steps: firstly, constructing a complex image restoration teacher network and a lightweight image restoration student network by respectively using an advanced visual conversion module and an efficient visual Mangbar module; and then designing Mangban-oriented heterogeneous knowledge distillation to migrate heterogeneous knowledge in the teacher network to the student network. The Mangban-oriented knowledge distillation mainly comprises a feature filtering module and a feature interface module. The two-way migration loss function can filter redundant feature information in the teacher network, and the two-way migration loss function is designed to convert heterogeneous features of the teacher network. And finally, performing an online image restoration test on the student network after distillation. According to the method, a lightweight and rapid general image restoration model can be realized, and the efficiency of the image restoration model under the condition that resources are limited is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an image restoration model compression method based on heterogeneous knowledge distillation. Background Art

[0002] Image restoration aims to recover a clean image from a polluted version caused by degradation factors (e.g., noise, blur, low illumination). As a fundamental computer vision task, IR is crucial for a wide range of real-world applications such as video surveillance and autonomous driving. IR is a severely ill-posed problem due to the existence of infinite feasible solutions. Therefore, traditional methods attempt to constrain the solution space by exploring various hand-crafted priors based on statistical observations, which lack robustness to complex scenes. In contrast, deep learning-based methods implicitly obtain general priors from abundant data, and these methods have achieved promising results.

[0003] Currently, developing universal models with advanced neural architectures is a hot research topic in order to unify various restoration tasks. In addition, the dominant neural architecture in the image restoration research field has shifted from convolutional neural networks to visual transformers. The basic component of the visual transformer is the multi-head self-attention mechanism, which wins the advantages of long-range dependency modeling and efficient parallel computing, but incurs quadratic complexity relative to the input size, which makes the visual transformer computationally expensive and memory intensive when applied to high-resolution images and limited-resource applications. Inspired by these problems, a group of compelling neural architectures have sprung up to challenge transformers in natural language processing and subsequently extended to computer vision. Notably, Mamba performs very competitively with transformers, but has linear time and constant memory complexity relative to the input size.

[0004] As a low-level computer vision task, developing a lightweight but fast image restoration model is very demanding. Driven by this, Li et al. (Huixia Li, Chenqian Yan, Shaohui Lin, Xiawu Zheng. Baochang Zhang, Fan Yang, and Rongrong Ji. PAMS: Quantized super-resolution via parameterized max scale. In ECCV, pages 564–580. Springer, 2020.) took the lead in using knowledge distillation to achieve model compression. However, their work only focuses on a subtask of image super-resolution and lacks a formal formula to promote research in this field. In addition, considering the complementary advantages of transformers and Mamba, the present application discloses an image restoration model compression method based on heterogeneous knowledge distillation, which is committed to developing a lightweight but fast general image restoration model to improve the efficiency of image restoration models under resource-constrained conditions. Summary of the invention

[0005] The purpose of the present invention is to solve the shortcomings existing in the prior art and propose an image restoration model compression method based on heterogeneous knowledge distillation. This method can realize a lightweight and fast general image restoration model and improve the efficiency of the image restoration model under resource-constrained conditions.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] A method for compressing an image restoration model based on heterogeneous knowledge distillation comprises the following steps:

[0008] Step 1: Build and pre-train a complex image restoration teacher network: Use the advanced visual transformation module to build an image restoration teacher network, and use the reconstruction loss function to complete the training of the teacher network;

[0009] Step 2: Construct an image restoration student network: Use the enhanced visual Mamba module to construct a lightweight image restoration student network;

[0010] Step 3: Design a feature filtering module: Use the visual conversion layer to build an autoencoder and train it using the self-reconstruction loss function;

[0011] Step 4: Design feature interface module: Use the visual Mamba module to transform the features of the teacher network, construct a one-way loss function from the teacher network to the student network, and use the visual conversion module to transform the features of the student network, and construct a one-way loss function from the student network to the teacher network;

[0012] Step 5. Construct a Mamba-guided knowledge distillation loss function: First, calculate the restored image output distillation loss function between the student network output and the teacher network output, then calculate the heterogeneous feature loss function and get the total training loss function.

[0013] Preferably, the specific process of step 1 is as follows:

[0014] S11: First, we build a high-level visual conversion module, which is composed of multi-head channel attention and multi-head spatial attention. Assume that the input feature of this module is represented as This module performs the following operations:

[0015]

[0016] in, represents the output features of the high-level visual transformation module, LN(·) and FFN(·) represent the regularization layer and multi-layer perceptron, respectively, MCA(·) and WSA(·) represent multi-head channel attention and multi-head spatial multi-head attention, respectively. The number of multi-head attention is assumed to be H. In the multi-head channel attention,

[0017] is divided into G parts, each of which is represented by Perform self-attention calculation on it, namely:

[0018]

[0019] in, represents the depthwise separable convolutional mapping function of the query channel, represents the depth-separable convolutional mapping function of the key-value channel, Represents the depth-separable convolutional mapping function of the value channel. d1 represents The number of channels in multi-head window attention, is divided into multiple small spatial windows, each part is represented as Perform self-attention calculation on it, namely:

[0020]

[0021] in, represents the linear mapping function of the query channel, represents the linear mapping function of the key-value channel, Represents the linear mapping function of the value channel. d2 represents The number of channels;

[0022] S12: The teacher network is represented by T, which consists of three modules: the bottom feature extraction module, the deep feature extraction module and the image reconstruction module. Given any input degraded image I i ∈R H×W×3 , the underlying feature extraction uses a 3×3 convolutional layer to convert it into features Among them, H, W, C represent the height, width and number of channels of the feature; the deep feature extraction module consists of an encoder and a decoder, each of which is divided into four stages. The encoder is composed of a high-level visual conversion module in step S11 and a downsampling layer stacked together, while the decoder is composed of a high-level visual conversion module in step S11 and an upsampling layer stacked together; assuming that the output of the last stage of the decoder is represented as The image reconstruction module uses a 3×3 convolutional layer to

[0023] Convert to residual image Will and I i Superimpose to obtain the output restoration image of the teacher network

[0024] S13: Restore the output image of the teacher network obtained in step S12 Assume that the corresponding clean image is represented as y i , construct the following loss function to complete the pre-training of the teacher network:

[0025]

[0026] Where N is the total number of training images and |·|1 represents the neighborhood distance function.

[0027] Preferably, the specific process of step 2 is as follows:

[0028] S21: First, we construct an enhanced visual Mamba module, which consists of an upper visual branch, a middle visual branch, and a lower visual branch. Assume that the input feature of this module is represented as The upper vision branch performs the following operations:

[0029]

[0030] Among them, Linear(·) represents the linear neural network layer, S wish (·) represents the activation function; the intermediate visual branch performs the following operations:

[0031]

[0032] Where LN(·) represents the regularization layer, C_SSM(·) represents the cross scan selection module, SiLU(·) represents the activation function, and Conv(·) represents the convolution function; the lower visual branch performs the following operations:

[0033]

[0034] Then the feature output value of the visual Mamba module is is represented as:

[0035]

[0036] Where Linear(·) represents a linear neural network layer, and ⊙ represents a Hadamard product operation;

[0037] S22: The student network is represented by S, which is also composed of three modules: the bottom feature extraction module, the deep feature extraction module and the image reconstruction module. Given any input degraded image I i ∈R H×W×3 , the underlying feature extraction module uses a 3×3 convolutional layer to convert it into features Where H, W, and C represent the height, width, and number of channels of the feature; the deep feature extraction module consists of an encoder and a decoder, each of which is divided into four stages. The encoder and the decoder are stacked by the enhanced visual Mamba module in step S21; assuming that the output of the last stage of the decoder is represented as The image reconstruction module uses a 3×3 convolutional layer to Convert to residual image Will and I i Superimpose to obtain the output restoration image of the student network

[0038] Preferably, the specific process of step 3 is as follows:

[0039] S31: The feature filtering module consists of an encoder and a decoder, which is composed of a visual conversion layer; the encoder and decoder play the role of feature mapping, represented by f(·) and g(·), respectively; assuming that the features of any stage in the teacher network are represented as The encoder maps this to The decoder then Mapping The mathematical description of the above process is:

[0040]

[0041] S32: The feature filtering module constructs the following self-reconstruction loss function:

[0042]

[0043] Where N is the total number of training images and |·|1 represents the neighborhood distance function.

[0044] Preferably, the specific process of step 4 is:

[0045] S41: The feature interface module uses the enhanced visual Mamba module in step S21 to convert the features of the teacher network into Mamba-type features, which are represented as And establish a one-way loss function from the teacher network to the student network. The mathematical description of the above process is:

[0046]

[0047] Where N is the total number of training images, EM(·) represents the enhanced visual Mamba module, and |·|1 represents the neighborhood distance function;

[0048] S42: The feature interface module uses the high-level visual transformation module in step S11 to transform the features of the teacher network into transformer-type features, which are represented as And establish a one-way loss function from the student network to the teacher network. The mathematical description of the above process is:

[0049]

[0050] Where N is the total number of training images, T(·) represents the high-level visual transformation module, and |·|1 represents the neighborhood distance function.

[0051] Preferably, the specific process of step 5 is as follows:

[0052] S51: Calculate the restored image output distillation loss function between the student output and the teacher network output, that is:

[0053]

[0054] Where N is the total number of training images, denote the output restored images of the teacher network and the student network, respectively, and |·|1 denotes the neighborhood distance function;

[0055] S52: Calculate the heterogeneous feature loss function, that is:

[0056]

[0057] Among them, α and β are adjustable parameters;

[0058] S52: Calculate the total loss function of student training, that is:

[0059] L O =L O+γL F

[0060] Among them, γ is an adjustable parameter based on the total loss function L O After completing the training of the student network, the trained student network performs online image restoration task testing.

[0061] Compared with the prior art, the present invention has the following beneficial effects:

[0062] The present invention can realize a lightweight and fast universal image restoration model, and can improve the efficiency of the image restoration model under resource-constrained conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0064] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings, so that those skilled in the art can better understand the advantages and features of the present invention, thereby making a clearer definition of the protection scope of the present invention. The embodiments described in the present invention are only a part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present invention.

[0065] A method for compressing an image restoration model based on heterogeneous knowledge distillation comprises the following steps:

[0066] Step 1: Build and pre-train a complex image restoration teacher network: Use the advanced visual transformation module to build an image restoration teacher network, and use the reconstruction loss function to complete the training of the teacher network;

[0067] Step 2: Construct an image restoration student network: Use the enhanced visual Mamba module to construct a lightweight image restoration student network;

[0068] Step 3: Design a feature filtering module: Use the visual conversion layer to build an autoencoder and train it using the self-reconstruction loss function;

[0069] Step 4: Design feature interface module: Use the visual Mamba module to transform the features of the teacher network, construct a one-way loss function from the teacher network to the student network, and use the visual conversion module to transform the features of the student network, and construct a one-way loss function from the student network to the teacher network;

[0070] Step 5. Construct a Mamba-guided knowledge distillation loss function: First, calculate the restored image output distillation loss function between the student network output and the teacher network output, then calculate the heterogeneous feature loss function and get the total training loss function.

[0071] Specifically, the specific process of step 1 is as follows:

[0072] S11: First, we build a high-level visual conversion module, which is composed of multi-head channel attention and multi-head spatial attention. Assume that the input feature of this module is represented as This module performs the following operations:

[0073]

[0074] in, represents the output features of the high-level visual transformation module, LN(·) and FFN(·) represent the regularization layer and multi-layer perceptron, respectively, MCA(·) and WSA(·) represent multi-head channel attention and multi-head spatial multi-head attention, respectively. The number of multi-head attention is assumed to be H. In the multi-head channel attention,

[0075] is divided into G parts, each of which is represented by Perform self-attention calculation on it, namely:

[0076]

[0077] in, represents the depthwise separable convolutional mapping function of the query channel, represents the depth-separable convolutional mapping function of the key-value channel, Represents the depth-separable convolutional mapping function of the value channel. d1 represents The number of channels in multi-head window attention, is divided into multiple small spatial windows, each part is represented as Perform self-attention calculation on it, namely:

[0078]

[0079] in, represents the linear mapping function of the query channel, represents the linear mapping function of the key-value channel, Represents the linear mapping function of the value channel. d2 represents The number of channels;

[0080] S12: The teacher network is represented by T, which consists of three modules: the bottom feature extraction module, the deep feature extraction module and the image reconstruction module. Given any input degraded image I i ∈R H×W×3 , the underlying feature extraction uses a 3×3 convolutional layer to convert it into features Among them, H, W, C represent the height, width and number of channels of the feature; the deep feature extraction module consists of an encoder and a decoder, each of which is divided into four stages. The encoder is composed of a high-level visual conversion module in step S11 and a downsampling layer stacked together, while the decoder is composed of a high-level visual conversion module in step S11 and an upsampling layer stacked together; assuming that the output of the last stage of the decoder is represented as The image reconstruction module uses a 3×3 convolutional layer to

[0081] Convert to residual image Will and I i Superimpose to obtain the output restoration image of the teacher network

[0082] S13: Restore the output image of the teacher network obtained in step S12 Assume that the corresponding clean image is represented as y i , construct the following loss function to complete the pre-training of the teacher network:

[0083]

[0084] Where N is the total number of training images and |·|1 represents the neighborhood distance function.

[0085] Specifically, the specific process of step 2 is as follows:

[0086] S21: First, we construct an enhanced visual Mamba module, which consists of an upper visual branch, a middle visual branch, and a lower visual branch. Assume that the input feature of this module is represented as The upper vision branch performs the following operations:

[0087]

[0088] Where Linear(·) represents a linear neural network layer, Swish(·) represents an activation function, and the intermediate visual branch performs the following operations:

[0089]

[0090] Where LN(·) represents the regularization layer, C_SSM(·) represents the cross scan selection module, SiLU(·) represents the activation function, and Conv(·) represents the convolution function; the lower visual branch performs the following operations:

[0091]

[0092] Then the feature output value of the visual Mamba module is is represented as:

[0093]

[0094] Where Linear(·) represents a linear neural network layer, and ⊙ represents a Hadamard product operation;

[0095] S22: The student network is represented by S, which is also composed of three modules: the bottom feature extraction module, the deep feature extraction module and the image reconstruction module. Given any input degraded image I i ∈R H×W×3 , the underlying feature extraction module uses a 3×3 convolutional layer to convert it into features Where H, W, and C represent the height, width, and number of channels of the feature; the deep feature extraction module consists of an encoder and a decoder, each of which is divided into four stages. The encoder and the decoder are stacked by the enhanced visual Mamba module in step S21; assuming that the output of the last stage of the decoder is represented as The image reconstruction module uses a 3×3 convolutional layer to Convert to residual image Will and I i Superimpose to obtain the output restoration image of the student network

[0096] Specifically, the specific process of step 3 is as follows:

[0097] S31: The feature filtering module consists of an encoder and a decoder, which is composed of a visual conversion layer; the encoder and decoder play the role of feature mapping, represented by f(·) and g(·), respectively; assuming that the features of any stage in the teacher network are represented as The encoder maps this to The decoder then Mapping The mathematical description of the above process is:

[0098]

[0099] S32: The feature filtering module constructs the following self-reconstruction loss function:

[0100]

[0101] Where N is the total number of training images and |·|1 represents the neighborhood distance function.

[0102] Specifically, the specific process of step 4 is:

[0103] S41: The feature interface module uses the enhanced visual Mamba module in step S21 to convert the features of the teacher network into Mamba-type features, which are represented as And establish a one-way loss function from the teacher network to the student network. The mathematical description of the above process is:

[0104]

[0105] Where N is the total number of training images, EM(·) represents the enhanced visual Mamba module, and |·|1 represents the neighborhood distance function;

[0106] S42: The feature interface module uses the high-level visual transformation module in step S11 to transform the features of the teacher network into transformer-type features, which are represented as And establish a one-way loss function from the student network to the teacher network. The mathematical description of the above process is:

[0107]

[0108] Where N is the total number of training images, T(·) represents the high-level visual transformation module, and |·|1 represents the neighborhood distance function.

[0109] Specifically, the specific process of step 5 is as follows:

[0110] S51: Calculate the restored image output distillation loss function between the student output and the teacher network output, that is:

[0111]

[0112] Where N is the total number of training images, denote the output restored images of the teacher network and the student network, respectively, and |·|1 denotes the neighborhood distance function;

[0113] S52: Calculate the heterogeneous feature loss function, that is:

[0114]

[0115] Among them, α and β are adjustable parameters;

[0116] S52: Calculate the total loss function of student training, that is:

[0117] L O =L O+γL F

[0118] Among them, γ is an adjustable parameter based on the total loss function L O After completing the training of the student network, the trained student network performs online image restoration task testing.

[0119] Example

[0120] Reference Figure 1 , the present invention takes the image classification task as an example, and its specific steps are as follows:

[0121] Step 1: First, build a high-level visual transformation module, which consists of multi-head channel attention and multi-head spatial attention. In the multi-head channel attention, the input feature is divided into G parts, each of which is represented by Perform self-attention calculation on it. In multi-head window attention, input feature is divided into multiple small spatial windows, each part is represented as Perform self-attention calculation on it. Then use 3×3 convolutional layers to establish the underlying feature extraction module. Stack the constructed high-level visual conversion module and the downsampling layer to obtain the encoder, and stack the constructed high-level visual conversion module and the upsampling layer to obtain the decoder. The encoder and decoder constitute the deep feature extraction module. Use 3×3 convolutional layers to establish the image reconstruction module. Finally, the coupling of the underlying feature extraction module, the deep feature extraction module and the image reconstruction module constitutes the image restoration teacher network T. For the output of the teacher network, the restored image

[0122] Construct a reconstruction loss function to complete the pre-training of the teacher network.

[0123] Step 2: First, build an enhanced visual Mamba module, which consists of an upper visual branch, a middle visual branch, and a lower visual branch. Use a 3×3 convolutional layer to build a bottom-level feature extraction module, and then stack the enhanced visual Mamba modules to obtain an encoder and a decoder. The encoder and decoder constitute a deep feature extraction module. Use a 3×3 convolutional layer to build an image reconstruction module. Finally, the coupling of the bottom-level feature extraction module, the deep feature extraction module, and the image reconstruction module constitutes the image restoration teacher network S. The output restoration image of the student network is

[0124] Step 3: Design a feature filtering module, which is mainly composed of an encoder and a decoder, and its basic component is the visual conversion layer. The encoder and decoder play the role of feature mapping, represented by f(·) and g(·) respectively. The encoder converts the teacher network features Mapping The decoder then Mapping At the same time, a self-reconstruction loss function is established.

[0125] Step 4: The feature interface module uses the enhanced visual Mamba module in step 2 to convert the features of the teacher network into Mamba-type features and establish a one-way loss function from the teacher network to the student network. Then, the high-level visual conversion module in step 1 is used to convert the features of the teacher network into transformer-type features, and a one-way loss function is established from the student network to the teacher network.

[0126] Step 5: First calculate the restored image output distillation loss function L between the student output and the teacher network output O , and then calculate the heterogeneous feature loss function L F , and finally calculate the total loss function of student training. Based on the total loss function L O After completing the training of the student network, the trained student network performs online image restoration task testing.

[0127] In summary, the present invention can realize a lightweight and fast universal image restoration model, and can improve the efficiency of the image restoration model under resource-constrained conditions.

[0128] The description and practice disclosed in the present invention are easy to think and understand for ordinary technicians in the technical field, and several improvements and modifications can be made without departing from the principles of the present invention. Therefore, modifications or improvements made without departing from the spirit of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. An image restoration model compression method based on heterogeneous knowledge distillation, characterized in that: The steps include: Step 1: Build and pre-train a complex image restoration teacher network: Use the advanced visual transformation module to build an image restoration teacher network, and use the reconstruction loss function to complete the training of the teacher network; Step 2: Construct an image restoration student network: Use the enhanced visual Mamba module to construct a lightweight image restoration student network; Step 3: Design a feature filtering module: Use the visual conversion layer to build an autoencoder and train it using the self-reconstruction loss function; Step 4: Design feature interface module: Use the visual Mamba module to transform the features of the teacher network, construct a one-way loss function from the teacher network to the student network, and use the visual conversion module to transform the features of the student network, and construct a one-way loss function from the student network to the teacher network; Step 5. Construct a Mamba-guided knowledge distillation loss function: First, calculate the restored image output distillation loss function between the student network output and the teacher network output, then calculate the heterogeneous feature loss function and get the total training loss function.

2. The image restoration model compression method based on heterogeneous knowledge distillation according to claim 1 is characterized in that: The specific process of step 1 is as follows: S11: First, we build a high-level visual conversion module, which is composed of multi-head channel attention and multi-head spatial attention. Assume that the input feature of this module is represented as This module performs the following operations: in, represents the output features of the high-level visual transformation module, LN(·) and FFN(·) represent the regularization layer and multi-layer perceptron, respectively, MCA(·) and WSA(·) represent multi-head channel attention and multi-head spatial multi-head attention, respectively. The number of multi-head attention is assumed to be H. In the multi-head channel attention, is divided into G parts, each of which is represented by Perform self-attention calculation on it, namely: in, represents the depthwise separable convolutional mapping function of the query channel, represents the depth-separable convolutional mapping function of the key-value channel, Represents the depth-separable convolutional mapping function of the value channel; d1 represents The number of channels in multi-head window attention, is divided into multiple small spatial windows, each part is represented as Perform self-attention calculation on it, namely: in, represents the linear mapping function of the query channel, represents the linear mapping function of the key-value channel, Represents the linear mapping function of the value channel, d2 represents The number of channels; S12: The teacher network is represented by T, which consists of three modules: the bottom feature extraction module, the deep feature extraction module and the image reconstruction module. Given any input degraded image I i ∈R H×W×3 , the underlying feature extraction uses a 3×3 convolutional layer to convert it into features Among them, H, W, C represent the height, width and number of channels of the feature; the deep feature extraction module consists of an encoder and a decoder, each of which is divided into four stages. The encoder is composed of a high-level visual conversion module in step S11 and a downsampling layer stacked together, while the decoder is composed of a high-level visual conversion module in step S11 and an upsampling layer stacked together; assuming that the output of the last stage of the decoder is represented as The image reconstruction module uses a 3×3 convolutional layer to Convert to residual image Will and I i Superimpose to obtain the output restoration image of the teacher network S13: Restore the output image of the teacher network obtained in step S12 Assume that the corresponding clean image is represented as y i , construct the following loss function to complete the pre-training of the teacher network: Where N is the total number of training images and |·|1 represents the neighborhood distance function.

3. The image restoration model compression method based on heterogeneous knowledge distillation according to claim 1 is characterized in that: The specific process of step 2 is as follows: S21: First, we construct an enhanced visual Mamba module, which consists of an upper visual branch, a middle visual branch, and a lower visual branch. Assume that the input feature of this module is represented as The upper vision branch performs the following operations: Where Linear(·) represents a linear neural network layer, swish(·) represents an activation function, and the intermediate visual branch performs the following operations: Where LN(·) represents the regularization layer, C_SSM(·) represents the cross scan selection module, SiLU(·) represents the activation function, and Conv(·) represents the convolution function; the lower visual branch performs the following operations: Then the feature output value of the visual Mamba module is is represented as: Where Linear(·) represents a linear neural network layer, and ⊙ represents a Hadamard product operation; S22: The student network is represented by S, which is also composed of three modules: the bottom feature extraction module, the deep feature extraction module and the image reconstruction module. Given any input degraded image I i ∈R H×W×3 , the underlying feature extraction module uses a 3×3 convolutional layer to convert it into features Where H, W, and C represent the height, width, and number of channels of the feature; the deep feature extraction module consists of an encoder and a decoder, each of which is divided into four stages. The encoder and the decoder are stacked by the enhanced visual Mamba module in step S21; assuming that the output of the last stage of the decoder is represented as The image reconstruction module uses a 3×3 convolutional layer to Convert to residual image Will and I i Superimpose to obtain the output restoration image of the student network 4. The image restoration model compression method based on heterogeneous knowledge distillation according to claim 1 is characterized in that: The specific process of step 3 is as follows: S31: The feature filtering module consists of an encoder and a decoder, which is composed of a visual conversion layer; the encoder and decoder play the role of feature mapping, represented by f(·) and g(·), respectively; assuming that the features of any stage in the teacher network are represented as The encoder maps this to The decoder then Mapping The mathematical description of the above process is: S32: The feature filtering module constructs the following self-reconstruction loss function: Where N is the total number of training images and |·|1 represents the neighborhood distance function.

5. The image restoration model compression method based on heterogeneous knowledge distillation according to claim 2 or 3, characterized in that: The specific process of step 4 is: S41: The feature interface module uses the enhanced visual Mamba module in step S21 to convert the features of the teacher network into Mamba-type features, which are represented as And establish a one-way loss function from the teacher network to the student network. The mathematical description of the above process is: Where N is the total number of training images, EM(·) represents the enhanced visual Mamba module, and |·|1 represents the neighborhood distance function; S42: The feature interface module uses the high-level visual transformation module in step S11 to transform the features of the teacher network into transformer-type features, which are represented as And establish a one-way loss function from the student network to the teacher network. The mathematical description of the above process is: Where N is the total number of training images, T(·) represents the high-level visual transformation module, and |·|1 represents the neighborhood distance function.

6. The image restoration model compression method based on heterogeneous knowledge distillation according to claim 1 is characterized in that: The specific process of step 5 is as follows: S51: Calculate the restored image output distillation loss function between the student output and the teacher network output, that is: Where N is the total number of training images, denote the output restored images of the teacher network and the student network, respectively, and |·|1 denotes the neighborhood distance function; S52: Calculate the heterogeneous feature loss function, that is: Among them, α and β are adjustable parameters; S52: Calculate the total loss function of student training, that is: THE O =L O +γL F Among them, γ is an adjustable parameter based on the total loss function L O After completing the training of the student network, the trained student network performs online image restoration task testing.

Images