Image enhancement method, device and storage medium based on dual-teacher knowledge distillation

Abstract

The present invention discloses an image enhancement method, device and storage medium based on dual-teacher knowledge distillation, and belongs to the field of digital image processing technology. The method proposes an underwater image enhancement network with dual-teacher knowledge distillation, and the framework divides the image processing problem into multiple small problems; a Freq‑UIE network is designed and a visual-based Fourier processing module is designed to add frequency domain information to different layers of the network to achieve the effect of edge and detail enhancement; in order to better match and distill the features of multiple teachers and students, a knowledge collection and distillation module is proposed to process the features of students and multiple teachers at each scale. Experimental results show that compared with the prior art, the present invention can well enhance the details and texture information of the image, and the color restoration is more accurate and the edges are clearer.

Description

Technical Field

[0001] The present invention relates to an image enhancement method, a device and a storage medium based on dual-teacher knowledge distillation, and belongs to the technical field of digital image processing. Background Art

[0002] Image enhancement technology is one of the contents of image processing. It is a processing technology that improves the readability of photographic images. It is determined based on the characteristics of human eyes' observation of light brightness and aims to improve the readability of images.

[0003] Underwater image enhancement is crucial for underwater environmental monitoring and detection. However, due to the scattering of suspended particles such as plankton and tiny particles, the original underwater images have blurred details, reduced contrast, and a hazy appearance. In addition, light of different wavelengths encounters different degrees of absorption when propagating in water. In particular, red light has a longer wavelength and is more easily absorbed. In contrast, blue and green light have shorter wavelengths and can penetrate deep underwater. This phenomenon leads to increased color distortion and limited visual range in deeper waters, which poses a major challenge to the implementation of vision-based technologies. Therefore, underwater image enhancement and restoration techniques have received widespread attention, and many researchers have proposed various methods to improve the quality of underwater images.

[0004] At the same time, as deep learning models become increasingly complex, the computational requirements and resource consumption of the models have also increased significantly, which limits their deployment in many practical application scenarios. To solve this problem, knowledge distillation technology came into being and has demonstrated its unique advantages in the field of image enhancement. Traditional knowledge distillation usually relies on a single teacher model to guide the learning of the student model. Its knowledge coverage and diversity are relatively limited, resulting in the student model being unable to learn certain key information, which in turn affects the model's extraction of key features, further leading to unsatisfactory image enhancement results. Summary of the invention

[0005] In order to optimize the image enhancement effect, the present invention provides an image enhancement method, comprising:

[0006] Step 1: When the number of training times of the current image enhancement model is less than or equal to half of the preset number of training times, the first image to be processed is passed into the first teacher model, and the second image to be processed is passed into the second teacher model, and they are processed respectively. At the same time, the first image to be processed and the second image to be processed are passed into the student model for processing; the first teacher model and the second teacher model are used to perform different image enhancement tasks respectively;

[0007] Step 2: Input the feature maps of each layer obtained by the first teacher model, the second teacher model and the student model into the knowledge collection and distillation module CDK for processing, project the features extracted by the teacher model and the features of the same scale of the student model into the common feature space using progressive features, and process the features of different scales separately;

[0008] Step 3: Based on the features processed in step 2, images output by the teacher model and the student model are obtained, and the difference between the enhanced results of the teacher model and the enhanced results of the teacher model is calculated using L1 loss. At the same time, soft contrast regularization is used, and the results predicted by the teacher model are used as positive samples, the degraded images are used as negative samples, and the images generated by the student network are used as samples. The student model learns features under the guidance of the teacher model;

[0009] Step 4: When the current training number is greater than half of the preset training number, the first image to be processed and the second image to be processed are input into the student model, and the student model is trained using the loss function;

[0010] Step 5: Use the trained student model to perform image enhancement tasks on the image to be processed;

[0011] The first teacher model, the second teacher model and the student model all adopt the Freq-UIE network, and the Freq-UIE network includes an encoder and a decoder. The frequency domain low-frequency feature map of the input image is added to the shallow layer of the encoder, and the frequency domain high-frequency feature map of the input image is added to the deep layer of the encoder.

[0012] Optionally, the frequency domain low-frequency feature map and the frequency domain high-frequency feature map are respectively obtained through a vision-based Fourier processing module, and the feature extraction process includes:

[0013] (1) Convert the input image into Fourier coefficients F through Fourier transform, and divide the coefficients into real and imaginary parts;

[0014] (2) Use MLP Mixer to divide the imaginary and real images into patches with a side length of P and flatten them into one-dimensional sequences respectively;

[0015] (3) These sequences are processed separately by two channel mixers to obtain real feature and imaginary feature sequences;

[0016] (4) After the image is processed using the linear layer, the obtained feature map is used to extract frequency domain information through CMB;

[0017] (5) Use layer normalization, 1D convolution, and activation function PRelu to refine high-frequency features;

[0018] (6) Connect the real features and the imaginary features along the feature channel dimension so that the model can process the real and imaginary features in the frequency domain at the same time;

[0019] (7) Use 3D convolution kernel filtering to reduce the number of channels of the feature map while capturing the temporal and spatial correlation of the feature map, compressing the frequency domain features after 3D convolution into a bilateral grid containing affine numbers;

[0020] (8) Use a convolutional network to process the original image to obtain a guided image with a guiding function;

[0021] (9) using a slicing operation to query the affine transformation matrix information in the bilateral grid through the guide image, and dynamically adjusting according to the guide image to obtain an output image;

[0022] (10) The low-frequency information and the high-frequency information are filtered out using a low-frequency filter and a high-frequency filter respectively, to obtain the frequency-domain low-frequency feature map and the frequency-domain high-frequency feature map.

[0023] Optionally, the processing process of the knowledge collection and distillation module CDK includes:

[0024] Step 21: Project the features extracted by the teacher module and the features of the same scale of the student model into a common feature space using progressive feature projection PFP, and process the features of different scales separately; use L1 loss to calculate the distance between the two projections, and the formula is as follows:

[0025]

[0026] in, N is the total number of encoders, and They are the encoders of the corresponding teacher network and student network. n Feature map of layer PFP projection, Indicates i A teacher model;

[0027] Step 22: Use bidirectional feature matching to constrain the learned features, project the features of the teacher network back to the original space through inverse progressive projection, and use L1 loss to calculate the difference between the inverse projected features and the original features. The formula is as follows:

[0028]

[0029] in, is the back-projection feature operation, Represents the original feature;

[0030] Step 23: Calculate the total loss of the CDK module, defined as:

[0031]

[0032] in, Represents the total loss of the CDK module.

[0033] Optionally, the step 3 calculates the difference between the enhanced result of the teacher model and the enhanced result of the student model, and uses soft contrast regularization at the same time, takes the result predicted by the teacher model as a positive sample, takes the degraded image as a negative sample, and takes the image generated by the student network as a sample, and the formula is as follows:

[0034]

[0035] in, and Student Model S and teacher model T i The prediction results, express The degraded image of the class, The formula definition is as follows:

[0036]

[0037] in Represent the prediction results, positive samples and negative samples respectively, · represents the dot product operation, represents VGG-19 feature extraction, represents the scale temperature, n Represents the total number of negative samples;

[0038] The total loss at this stage The formula is as follows:

[0039]

[0040] in, Represents the difference between the augmented results of the teacher model and the augmented results of the student model.

[0041] Optionally, the step 4 uses hard contrast regularization, taking the reference image of the image as a positive sample, all degraded images as negative samples, and the image of the student model as a sample, and the calculation formula is as follows:

[0042]

[0043] in, and are the prediction results and reference images of the student model, respectively. represents all categories of degraded images;

[0044] Total loss The formula is as follows:

[0045]

[0046] in is the L1 loss between the reference image and the prediction result of the student model.

[0047] Optionally, the output of the second layer of the Freq-UIE network encoder is added to the frequency domain low-frequency feature map.

[0048] Optionally, the output of the fourth layer of the Freq-UIE network encoder is added to the frequency domain high-frequency feature map.

[0049] Optionally, the The value of is 0.07.

[0050] The present invention provides an image enhancement device, characterized in that it comprises a memory and a processor;

[0051] The memory is used to store computer programs;

[0052] The processor is used to implement the image enhancement method as described in any one of the above items when executing the computer program.

[0053] The present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the image enhancement method as described in any one of the above items is implemented.

[0054] The beneficial effects of the present invention are:

[0055] The present invention proposes an underwater image enhancement model with dual-teacher knowledge distillation, which divides the problem of underwater image processing into multiple small problems and can simultaneously handle different problems of underwater image enhancement. Through the knowledge distillation model, the image feature map information can be better guided by the mature teacher model to guide the student model to learn, so that the correct information can be learned more accurately.

[0056] The present invention designs a Freq-UIE network and a visual Fourier processing module. The visual Fourier processing module is based on the human visual mechanism, and uses low-frequency filters and high-frequency filters respectively to extract different information in the frequency domain. According to the human visual system mechanism, low-frequency information is added to the second layer of the network, and high-frequency information is added to the fourth layer of the network. Frequency domain information is added to different layers of the network to achieve the effect of edge and detail enhancement.

[0057] In order to better match and distill the features of multiple teachers and students, a knowledge collection and distillation (CDK) module is proposed to process the features of students and multiple teachers at each scale, calculate the differences between the student network features and different teacher networks, and use bidirectional feature matching to constrain the learned features, thereby improving the stability of the teacher network.

[0058] Experimental results show that the present invention uses a knowledge collection and distillation module, which can simultaneously handle image enhancement tasks such as image dehazing and color correction; using a vision-based Fourier processing module, the present invention can better enhance the details and texture information of the image compared to the prior art, and the color restoration is more accurate and the edges are clearer. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0060] Figure 1 This is a network architecture diagram of DTKD-UIE provided in Example 1 of the present invention.

[0061] Figure 2 It is a schematic diagram of the CDK module processing process provided in the first embodiment of the present invention.

[0062] Figure 3 This is an architecture diagram of the Freq-UIE network provided in Example 1 of the present invention.

[0063] Figure 4 It is a structural diagram of a vision-based Fourier processing module provided in Example 1 of the present invention.

[0064] Figure 5 This is a flow chart of the image enhancement method based on dual-teacher knowledge distillation provided in Example 1 of the present invention.

[0065] Figure 6 This is a comparison chart of the enhancement effects of the present invention and other image enhancement methods. DETAILED DESCRIPTION

[0066] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0067] Embodiment 1:

[0068] This embodiment provides an image enhancement method, including:

[0069] Step 1: When the number of training times of the current image enhancement model is less than or equal to half of the preset number of training times, the image to be dehazed is passed to the dehazed teacher model, and the image to be color processed is passed to the color correction teacher model for processing respectively. At the same time, the image to be dehazed and the image to be color processed are passed to the student model for processing;

[0070] Step 2: Input the feature maps of each layer obtained by the dehazing teacher model, color correction teacher model and student model into the knowledge collection and distillation module CDK for processing. The features extracted by the teacher model and the features of the same scale of the student model are projected into the common feature space using progressive features, and the features of different scales are processed separately.

[0071] Step 3: Based on the features processed in step 2, the images output by the teacher model and the student model are obtained. The L1 loss is used to calculate the difference between the enhanced results of the teacher model and the enhanced results of the teacher model. At the same time, soft contrast regularization is used. The results predicted by the teacher model are used as positive samples, the degraded images are used as negative samples, and the images generated by the student network are used as samples. The student model learns features under the guidance of the teacher model.

[0072] Step 4: When the current training times are greater than half of the preset training times, the image to be dehazed and the image to be color processed are input into the student model, and the student model is trained using the loss function;

[0073] Step 5: Use the trained student model to perform dehazing and color correction on the image to be processed;

[0074] The dehazing teacher model, color correction teacher model and student model all use the Freq-UIE network. The Freq-UIE network includes an encoder and a decoder. The frequency domain low-frequency feature map of the input image is added to the shallow layer of the encoder, and the frequency domain high-frequency feature map of the input image is added to the deep layer of the encoder.

[0075] Embodiment 2:

[0076] This embodiment provides an image enhancement method based on dual-teacher knowledge distillation, and proposes a dual-teacher knowledge distillation image enhancement network DTKD-UIE. The framework divides the image processing problem into multiple small problems, which are mainly divided into two sub-tasks: underwater image dehazing and underwater image color correction in this embodiment; a Freq-UIE network is designed and a vision-based Fourier processing module is designed. The vision-based Fourier processing module adds frequency domain information to different layers of the network based on the human visual mechanism to achieve the effect of edge and detail enhancement; in order to better match and distill the features of multiple teachers and students, a knowledge collection and distillation (CDK) module is proposed to process the features of students and multiple teachers at each scale.

[0077] The structure of the image enhancement network DTKD-UIE in this embodiment is as follows Figure 1 As shown, the training of the network is divided into two stages. The training process is as follows Figure 5 shown.

[0078] During the training process, when the current training is less than half of the training times, it is the first stage of training. In this stage, the student model learns features under the guidance of the teacher model, including the following steps:

[0079] Step 1: The foggy image is passed into the defogging teacher model, and the color processed image is passed into the color correction teacher model for processing respectively. At the same time, the two categories of images are passed into the student model for processing. Both the student model and the teacher model use the Freq-UIE network. The structure of the Freq-UIE network is as follows: Figure 3 As shown, the purpose of this step is to obtain feature maps of different scales of the network.

[0080] Step 11: Pass the input to the processing block of the first layer of the network. Each block consists of a LayerNorm layer, then processed by 1*1 convolution and 3*3 deconvolution, and then used SimpleGate. Then it is processed by a simple channel attention module and then used by 1*1 convolution. The result is added to the input as a feature. Figure 1 , further through the LayerNorm layer, a 1*1 convolution layer, a SimpleGate layer, and finally the above results are processed by 1*1 convolution and combined with the feature Figure 1 Add them together to get a block processing result, and process them according to the number of blocks in the figure to get the first layer result.

[0081] Step 12: After the second layer is processed by the block, it is added to the frequency domain low-frequency feature map.

[0082] Step 13: The processing flow of the third layer is the same as that of the first layer;

[0083] Step 14: After the fourth layer is processed by the block, it is added to the frequency domain high-frequency feature map.

[0084] In the shallow layer of the encoder (the second layer), the image still maintains a high resolution. At this time, adding low-frequency information helps the network quickly capture the overall structure and main information of the image. In the deep layer of the encoder (the fourth layer), the image has been downsampled many times, the resolution is reduced, but the semantic information is richer. At this time, adding high-frequency information can make up for the image details lost in the downsampling process. Adding low-frequency information in the first layer and high-frequency information in the third or fourth layer may cause the network to rely too much on low-frequency information in the shallow layer and ignore the importance of high-frequency information. If low-frequency information is added in the second layer and high-frequency information is added in the third layer, it may destroy the effective fusion of low-frequency and high-frequency information in the decoder. If low-frequency information is added in the first and second layers, and high-frequency information is added in the third and fourth layers, it may cause information redundancy and waste of computing resources.

[0085] Among them, the frequency domain low-frequency feature map and the frequency domain high-frequency feature map are obtained through the visual Fourier processing module (VFP), and the network structures of VFP (L) and VFP (H) are as follows: Figure 4 As shown, the feature extraction process is as follows:

[0086] (1) Convert the input image into Fourier coefficients F through Fourier transform, and divide the coefficients into real and imaginary parts;

[0087] (2) Use MLP Mixer to divide the imaginary and real images into patches with a side length of P and flatten them into one-dimensional sequences respectively.

[0088] (3) These sequences are processed separately through two channel mixers to obtain real feature and imaginary feature sequences.

[0089] (4) After processing the image using the linear layer, the obtained feature map is used to extract frequency domain information through CMB.

[0090] (5) Use a Layer Normalization and two 1D Conv and PRelu to refine the high-frequency features to obtain the final output.

[0091] (6) Connect the real features and the imaginary features along the feature channel dimension so that the model can process the real and imaginary features in the frequency domain at the same time.

[0092] (7) Use 3D convolution kernel filtering to reduce the number of channels of the feature map while capturing the temporal and spatial correlation of the feature map. The frequency domain features after 3D convolution are compressed into a bilateral grid containing affine numbers. The reshaped frequency domain features can be regarded as a 16x16x16 bilateral grid, where each grid cell contains 3 digits.

[0093] (8) Use a convolutional network to process the original image and obtain a guided image with a guiding function.

[0094] (9) Use the slicing operation to query the affine transformation matrix information in the bilateral grid through the guidance map, and dynamically adjust the output image according to the guidance map. This can generate a high-quality feature map as the output of the frequency domain branch at a very low computational cost.

[0095] (10) Use low-frequency filters and high-frequency filters to filter out low-frequency information and high-frequency information respectively, and obtain the final frequency domain low-frequency feature map and frequency domain high-frequency feature map.

[0096] Step 2: Transfer the feature maps of each layer obtained in step 1 to the knowledge collection and distillation module (CDK) for processing. The specific steps are as follows:

[0097] Step 21: The features of multiple teachers and the features of the same scale of the student network are projected into the common feature space using progressive feature projection (PFP), and the features of different scales are processed separately; in order to make the projected features in the common feature space closer, the L1 loss is used to calculate the distance between the two projections, and the formula is as follows:

[0098]

[0099] in, N is the total number of encoders, and They are the encoders of the corresponding teacher network and student network. n Feature map of layer PFP projection.

[0100] Step 22: Use bidirectional feature matching to constrain the learned features, thereby enhancing the robustness of learning from the teacher network. Project the features of the teacher network back to the original space through inverse progressive projection, and use L1 loss to calculate the difference between the inverse projected features and the original features, as follows:

[0101]

[0102] in, is the back-projection feature operation, Represents the original feature.

[0103] Step 23: Calculate the total loss of the CDK module ( L CDK ), defined as:

[0104]

[0105] Step 3: Based on the images output by the teacher model and the student model after step 2, a simple L1 loss is used to calculate the difference between the results enhanced by the teacher network and the results enhanced by the student network. At the same time, soft contrast regularization is used, and the results predicted by the teacher network are used as positive samples, degraded images are used as negative samples, and images generated by the student network are used as samples. The formula is as follows:

[0106]

[0107] in, and Student Network S and teacher network T i The prediction results, express The degraded image of the class, The formula definition is as follows:

[0108]

[0109] in Represent the prediction results, positive samples and negative samples respectively, · represents the dot product operation, represents VGG-19 feature extraction, represents the scale temperature, and its value in this embodiment is 0.07, n Represents the total number of negative samples.

[0110] The total loss at this stage The formula is as follows:

[0111]

[0112] Represents the difference between the enhanced results of the teacher model and the enhanced results of the student model (i.e., the L1 loss calculation result).

[0113] When the current training is greater than half of the training times, the student model does not learn features under the guidance of the teacher model:

[0114] Step 4: Pass the input to the student model and get the final image after Freq-UIE processing. Calculate the L1 loss between the image and the reference image (ground truth). Use hard contrast regularization at the same time. Take the reference image of the image as the positive sample, all degraded images as negative samples, and the image of the student model as the sample. The calculation formula is as follows:

[0115]

[0116] in and are the prediction results and reference images of the student network respectively. Represents the degraded images of all categories.

[0117] Total loss The formula is as follows:

[0118]

[0119] in That is, the L1 loss between the reference image (ground truth) and the student network prediction result.

[0120] In this embodiment, the training of the DTKD-UIE network mainly includes the early student network training under the guidance of the teacher, and the later independent training of the student network after it is relatively mature. Since the students are not mature in the early stage, they are trained under the guidance of the teacher model. The teacher model is a set of training weights. The student network is trained by using different teacher networks at the same time in each epoch. The dehazed image and the color corrected image are enhanced by different teachers respectively. The student uses the same network to enhance the image and processes the features through the collection and distillation modules. During this period, we regard the teacher's prediction results as reliable results and use regularization loss for loss calculation. After the end of this period, the student network is considered to be mature and can process and obtain ideal results by itself. Therefore, it no longer relies on the teacher network, regards the ground truth as a reliable result, and uses regularization loss for calculation.

[0121] In order to demonstrate the technical effect of the image enhancement method proposed in this invention, experiments were conducted on the ImageNet and UIEB datasets and compared with the methods in the prior art. The comparison results are shown in the figure. Figure 6 As shown, (a) is the input image Input; (b) UDCP; (c) FUnIE-GAN; (d) Water-Net; (e) Ucolor; (f) MLFcGAN; (g) PUIE-Net; (h) U-shape; (i) the present invention; (j) reference image.

[0122] It can be seen from the first two images that the method of the present invention has the best overall performance on the ImageNet dataset. In the image above, although the color correction effect of FUnIE-net is better, there will be artifacts at the edge, the color of MLFCGAN is better, but the edge is fuzzy, and the edge blurring of waternet and PUIENet is better, but the color is biased. In the image below, although the overall color of FUnIE-net and MLFCGAN is also close to the gt image, in the enlarged detail image, the method of the present invention is more accurate in detail color enhancement.

[0123] As can be seen from the last two images, among these methods, the proposed method has the best overall performance on the UIEB dataset. In the above image, although the visual effect of Ucolor looks the best, when the details are enlarged, the method is not accurate enough in enhancing the color edge details, and the blue edge of the gt image is displayed green, while the proposed method is more accurate in edge color correction and clearer. In the image below, our method keeps the color closest to the gt image while ensuring the clarity of the scale.

[0124] In addition, the effectiveness of image enhancement is verified by calculating the PSNR and SSIM of the test set. The effect comparison between the method of the present invention and the existing method is shown in Table 1. In the table, testA and test B are randomly selected from 128 pairs of images in ImageNet (Underwater_imagenet) and 90 pairs of images in "UIEB" as test sets, denoted as testA and testB.

[0125] Table 1: Comparison results between the method of the present invention and the existing method

[0126]

[0127] It can be seen from Table 1 that the present invention has the best effect compared with the existing methods. In testA, the effect of FUnIE-net is second only to the present invention, and in testB, the effect of PUIENet is second only to the present invention. PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity) are two commonly used image quality evaluation indicators, and the higher the better.

[0128] Some steps in the embodiments of the present invention may be implemented using software, and the corresponding software program may be stored in a readable storage medium, such as a CD or a hard disk.

[0129] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An image enhancement method, characterized in that: The method comprises: Step 1: When the number of training times of the current image enhancement model is less than or equal to half of the preset number of training times, the first image to be processed is passed into the first teacher model, and the second image to be processed is passed into the second teacher model, and they are processed respectively, and at the same time, the first image to be processed and the second image to be processed are passed into the student model for processing; the first teacher model and the second teacher model are used to perform different image enhancement tasks; Step 2: Input the feature maps of each layer obtained by the first teacher model, the second teacher model and the student model into the knowledge collection and distillation module CDK for processing, project the features of the same scale extracted by the teacher model and the student model into the common feature space using progressive features, and process the features of different scales separately; Step 3: After step 2, the images output by the teacher model and the student model are obtained, and the difference between the enhanced results of the teacher model and the student model is calculated using the L1 loss. At the same time, soft contrast regularization is used, and the results predicted by the teacher model are used as positive samples, the degraded pictures are used as negative samples, and the images generated by the student model are used as samples. The student model learns features under the guidance of the teacher model; Step 4: When the current training number is greater than half of the preset training number, the first image to be processed and the second image to be processed are input into the student model, and the student model is trained using the loss function; Step 5: Use the trained student model to perform image enhancement tasks on the image to be processed; The first teacher model, the second teacher model and the student model all adopt the Freq-UIE network, and the Freq-UIE network includes an encoder and a decoder. The frequency domain low-frequency feature map of the input image is added to the shallow layer of the encoder, and the frequency domain high-frequency feature map of the input image is added to the deep layer of the encoder.

2. The image enhancement method according to claim 1, characterized in that: The frequency domain low-frequency feature map and the frequency domain high-frequency feature map are respectively obtained through a vision-based Fourier processing module, and the feature extraction process includes: (1) Convert the input image into Fourier coefficients F through Fourier transform, and divide the coefficients into real and imaginary parts; (2) Use MLP Mixer to divide the imaginary and real images into patches with side length P and flatten them into one-dimensional sequences respectively; (3) These sequences are processed separately by two channel mixers to obtain real feature and imaginary feature sequences; (4) After processing the image using the linear layer, extract the frequency domain information from the obtained feature map; (5) Use layer normalization, 1D convolution and activation function PRelu to refine high-frequency features; (6) Connect the real features and the imaginary features along the feature channel dimension so that the model can simultaneously process the real and imaginary features in the frequency domain; (7) Use 3D convolution kernel filtering to reduce the number of channels of the feature map while capturing the temporal and spatial correlation of the feature map, compressing the frequency domain features after 3D convolution into a bilateral grid containing affine numbers; (8) Use a convolutional network to process the original image to obtain a guided image with a guiding function; (9) using a slicing operation to query the affine transformation matrix information in the bilateral grid through the guide image, and dynamically adjusting according to the guide image to obtain an output image; (10) The low-frequency information and the high-frequency information are filtered out using a low-frequency filter and a high-frequency filter respectively, to obtain the frequency-domain low-frequency feature map and the frequency-domain high-frequency feature map.

3. The image enhancement method according to claim 1, characterized in that: The processing process of the knowledge collection and distillation module CDK includes: Step 21: Project the features of the same scale extracted by the teacher model and the student model into the common feature space using the progressive feature PFP, and process the features of different scales separately; use the L1 loss to calculate the distance between the two projections, and the formula is as follows: in, N is the total number of encoders, and They are the encoders of the corresponding teacher network and student network. n Feature map of layer PFP projection, Indicates i A teacher model; Step 22: Use bidirectional feature matching to constrain the learned features, project the features of the teacher network back to the original space through inverse progressive projection, and use L1 loss to calculate the difference between the inverse projected features and the original features. The formula is as follows: in, is the back-projection feature operation, Represents the original feature; Step 23: Calculate the total loss of the CDK module, defined as: in, Represents the total loss of the CDK module.

4. The image enhancement method according to claim 3, characterized in that: The step 3 calculates the difference between the enhancement results of the teacher model and the student model, and uses soft contrast regularization at the same time. The result predicted by the teacher model is used as a positive sample, the degraded image is used as a negative sample, and the image generated by the student model is used as a sample. The formula is as follows: in, and Student Model S and teacher model T i The prediction results, express The degraded image of the class, The formula definition is as follows: in Represent the prediction results, positive samples and negative samples respectively, · represents the dot product operation, represents VGG-19 feature extraction, represents the scale temperature, n Represents the total number of negative samples; The total loss at this stage The formula is as follows: in, Represents the difference between the augmented results of the teacher model and the augmented results of the student model.

5. The image enhancement method according to claim 1, characterized in that: The step 4 uses hard contrast regularization, takes the reference image of the image as the positive sample, all degraded images as negative samples, and the image of the student model as the sample, and the calculation formula is as follows: in, and are the prediction results and reference images of the student model, respectively. represents all categories of degraded images; Total loss The formula is as follows: in is the L1 loss between the reference image and the prediction result of the student model.

6. The image enhancement method according to claim 1, characterized in that: The output of the second layer of the Freq-UIE network encoder is added to the frequency domain low-frequency feature map.

7. The image enhancement method according to claim 1, characterized in that: The output of the fourth layer of the Freq-UIE network encoder is added to the frequency domain high-frequency feature map.

8. The image enhancement method according to claim 4, characterized in that: Said The value of is 0.

07.

9. An image enhancement device, characterized in that: including memory and processor; The memory is used to store computer programs; The processor is used to implement the image enhancement method according to any one of claims 1 to 8 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the image enhancement method according to any one of claims 1 to 8 is implemented.

Images