Lightweight social platform image restoration method and system based on dual-branch frequency and space fusion, and readable storage medium

Abstract

The invention discloses a lightweight social platform image restoration method and system based on dual-branch frequency and space fusion, and a readable storage medium. Comprising the following steps: constructing and training a lightweight social platform image restoration network model based on dual-branch frequency and space fusion; and inputting the low-quality image into a lightweight social platform image restoration network model based on double-branch frequency and space fusion for image restoration to obtain a high-quality image. According to the method, the image definition is remarkably improved, the training process is stabilized, and the robustness and the high efficiency of the model are enhanced.

Description

Technical Field

[0001] The present invention belongs to the technical field of image restoration, and more specifically, relates to a lightweight social platform image restoration method, system and readable storage medium based on dual-branch frequency and space fusion. Background Art

[0002] With the widespread popularity of social media platforms, people are increasingly using handheld devices to communicate with friends, share dynamics and participate in interactions. However, when these images are uploaded to social media platforms, they are often affected by various compression artifacts and unpredictable noise degradation, resulting in a significant decrease in image quality. Therefore, how to effectively reduce computational complexity and speed up inference time while achieving excellent visual effects, and provide users with a smoother and more efficient experience on mobile devices has become an important and meaningful task. In recent years, many super-resolution (SR) models based on convolutional neural networks (CNNs) have explored various methods to reduce model complexity. For example, FSRCNN and ESPCN use a post-upsampling strategy to predefine inputs to reduce computational burden. ShuffleMixer reduces model complexity and latency by utilizing large kernel convolution and channel shuffling techniques. BSRN replaces standard convolutions with carefully designed depthwise separable convolutions and enhances model capabilities by integrating two effective attention mechanisms. Spatially adaptive feature mixture networks develop an effective SR model by integrating the principles of convolution and self-attention. Although these methods have made substantial progress in model efficiency, there is still potential for further improvement in reconstruction performance. Dealing with the trade-off between computational efficiency and performance improvement is an important and promising area of ​​research.

[0003] Blind super-resolution technology is also a key method to solve this type of problem. Blind super-resolution technology (Blind SR) generally aims to recover high-resolution images from low-resolution images of unknown degradation. It does not need to know the specific type or degree of image degradation in advance, and can cope with complex and diverse degradation situations. It plays an important role in image restoration and improving visual quality, and is particularly suitable for scenarios where the degradation process cannot be accurately modeled. Most methods solve this problem in an explicit or implicit way: implicit methods aim to learn degradation networks directly from low-resolution images in the real world, while explicit methods try to approximate real low-resolution images by designing synthetic degradation processes. BSRGAN and Real-ESRGAN designed complex degradation pipelines to synthesize degraded images, thereby enhancing the generalization ability of existing SR models, which demonstrated significant progress in the field of blind super-resolution.

[0004] The invention patent with the prior art publication number CN118822846A proposes a method for improving lightweight social platform image restoration based on dual-branch frequency and space fusion, which belongs to the field of lightweight social platform image restoration based on dual-branch frequency and space fusion, including: selecting a suitable image data set; dividing the data set according to a suitable ratio; improving the shallow feature extraction module; making corresponding improvements to the deep feature extraction module, using a multi-scale feature extraction module to extract features from the feature map, and enhancing the feature extraction ability of the network; using an improved efficient channel attention network to calculate the importance of each channel of the feature map, and improving the feature expression ability of the algorithm; using a multi-level feature fusion mechanism to strengthen the network's use of intermediate layer feature information and enhance the reconstruction ability of the algorithm; setting training-related parameters, training the model, and saving weights; loading the training weights with the smallest loss function of the validation set, and testing the model reconstruction performance. This scheme can improve the quality of the reconstructed image to a certain extent and obtain a more realistic super-resolution image. However, its accuracy and the efficiency of model training still need to be improved. Summary of the invention

[0005] In order to overcome the problem of how to improve the accuracy and efficiency of the image restoration network in the prior art, the present invention provides a lightweight social platform image restoration method, system and readable storage medium based on dual-branch frequency and space fusion.

[0006] The first aspect of the present invention provides a lightweight social platform image restoration method based on dual-branch frequency and space fusion, comprising the following steps:

[0007] The training data is cropped into small image blocks, and the small image blocks are processed using a preset second-order degradation model to obtain low-resolution image blocks as training data sets;

[0008] Using the preset lightweight restoration network as the generator and the frequency-domain-based discriminative gating network as the discriminator, a lightweight social platform image restoration network model based on dual-branch frequency and space fusion is constructed;

[0009] The training data set is used to train a lightweight social platform image restoration network model based on dual-branch frequency and space fusion, wherein the generator is used to convert the image to be processed into a high-quality image, and the discriminator is used to discriminate the high-quality image generated by the generator during model training, and the discrimination result is fed back and used to adjust the generator;

[0010] The low-quality image is input into the trained lightweight social platform image restoration network model based on dual-branch frequency and space fusion to restore the image and obtain a high-quality image.

[0011] Furthermore, the preset second-order degradation model includes: two blur modules, two size adjustment modules, two noise adding modules, and two hybrid compression modules, wherein the hybrid compression module is constructed by: on the basis of the Real-ESRGAN synthetic degradation algorithm, combining the JPEG compression method and the prediction-based algorithm to construct the hybrid compression module; reducing the strength of the default parameters of the Real-ESRGAN algorithm in the hybrid compression module to obtain the adjusted hybrid compression module;

[0012] Using the preset second-order degradation model to process small image blocks includes the following steps:

[0013] Input the small image block into the first blur module, the first size adjustment module, and the first noise adding module in sequence, output the image block after noise addition, input the image block after noise addition into the first hybrid compression module, and output the processed small image block;

[0014] The processed small image blocks are sequentially input into the second blur module, the second size adjustment module and the second noise adding module to obtain the noisy image blocks, and the noisy image blocks are input into the second hybrid compression module to output the low-resolution image blocks.

[0015] Furthermore, the preset lightweight restoration network includes: a convolution kernel, several parallel dual-branch frequency and spatial domain basis feature fusion modules, and an upsampling module. The image to be processed is sequentially passed through the convolution kernel, several parallel dual-branch frequency and spatial domain basis feature fusion modules, and an upsampling module to output a high-quality image.

[0016] Furthermore, the dual-branch frequency and space domain basis feature fusion module includes a spatial adaptive feature mixing network and a frequency domain basis discriminant gating network running in parallel, the spatial adaptive feature mixing network is used to extract the spatial features of the image features, and the frequency domain basis discriminant gating network is used to extract the frequency features of the image features.

[0017] Furthermore, the method for converting the image to be processed into a high-quality image comprises the following steps:

[0018] Input the image to be processed into the convolution kernel for convolution, and output the primary features of the image;

[0019] The primary features of the image are input into the spatial adaptive feature hybrid network, and the spatial features in the spatial domain are output; the primary features of the image are input into the frequency domain basis discriminant gating network, and the frequency features in the frequency domain are output;

[0020] The spatial features in the spatial domain and the frequency features in the frequency domain are fused to obtain fused features;

[0021] The fused features are input into the upsampling module, and the residual connection is used to capture the high-frequency details of the image to obtain a high-quality image.

[0022] Furthermore, the method of extracting frequency features in the frequency domain from primary features of the image using a frequency domain basis discriminant gating network comprises the following steps:

[0023] The input image primary feature X is convolved point by point by 1×1 to double the number of channels, and the spatial local context is encoded using 3×3 deep convolution, expressed as:

[0024] X1=DW_Conv 3x3 (Conv 1x1 (X)

[0025] Among them, X1 represents the features after 1x1 convolution and 3x3 depth convolution, Conv 1x1 (·) represents 1×1 point-wise convolution, DW_Conv 3x3 (·) represents 3×3 depthwise convolution;

[0026] The features are converted to the frequency domain and the fast Fourier transform is used to split the features into two independent components along the channel dimension. The expression is:

[0027] X F =F(X1)

[0028]

[0029] Among them, X F Represents the features converted to the frequency domain by fast Fourier transform, represents the features after segmentation along the channel dimension, F(·) represents fast Fourier transform, and Split(·) represents the channel segmentation operation;

[0030] For a component after segmentation Apply a 1×1 point-wise convolution and use the convolution result as the gate of the second component to selectively retain the relevant frequency information, expressed as:

[0031]

[0032] Among them, ⊙ represents pixel-by-pixel multiplication, Represents the features output after the above steps;

[0033] Apply the GELU activation function for nonlinear mapping, the expression is:

[0034]

[0035] Among them, X out Represents the frequency characteristics of the final output, F -1 (·) denotes inverse fast Fourier transform, and G(·) denotes GELU function.

[0036] Furthermore, the method for constructing the frequency domain-based discriminant gating network is as follows: adding a frequency domain-based discriminant gating module between the convolution module and the layer normalization module in the discriminator based on the U-net structure, and replacing the layer normalization module with a spectral normalization module.

[0037] Furthermore, the training strategy of the lightweight social platform image restoration network model based on dual-branch frequency and space fusion is a training strategy based on a generative adversarial network.

[0038] A second aspect of the present invention provides a lightweight social platform image restoration system based on dual-branch frequency and space fusion, including a memory and a processor, wherein the memory includes a lightweight social platform image restoration method program based on dual-branch frequency and space fusion, and when the lightweight social platform image restoration method program based on dual-branch frequency and space fusion is executed by the processor, the steps of a lightweight social platform image restoration method based on dual-branch frequency and space fusion are implemented.

[0039] The third aspect of the present invention provides a computer-readable storage medium, which includes a lightweight social platform image restoration method program based on dual-branch frequency and space fusion. When the lightweight social platform image restoration method program based on dual-branch frequency and space fusion is executed by a processor, it implements the steps of a lightweight social platform image restoration method based on dual-branch frequency and space fusion.

[0040] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0041] The present invention designs a dual-branch frequency and space domain-based feature fusion network as a generator of a lightweight social platform image restoration network model based on dual-branch frequency and space fusion, integrates the gated dynamic network in the frequency domain and the feature selection technology in the space domain, and can effectively explore the frequency domain and space domain features at the same time, achieve better image restoration effects, and effectively restore the clarity of the image. In addition, the present invention also adds a discriminator to the lightweight social platform image restoration network model based on dual-branch frequency and space fusion, and uses the discriminator to judge the accuracy and efficiency of the image restoration of the generator, and then feeds back the judgment results. Based on the feedback information, the generator and the discriminator are adjusted, which effectively stabilizes the training dynamics and improves the robustness of the model and the ability to restore detailed textures. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to make the purpose and technical solution of the present invention clearer, the present invention provides the following drawings and descriptions:

[0043] Figure 1A flow chart of a lightweight social platform image restoration method based on dual-branch frequency and space fusion provided by an embodiment of the present invention;

[0044] Figure 2 A flow chart of synthetic degradation data based on a second-order degradation model provided by an embodiment of the present invention;

[0045] Figure 3 A schematic diagram of a feature fusion module of dual-branch frequency and space domain basis provided in an embodiment of the present invention;

[0046] Figure 4 A schematic diagram of a frequency domain discriminant gating network provided by an embodiment of the present invention;

[0047] Figure 5 A schematic diagram of a frequency domain based discriminator provided by an embodiment of the present invention;

[0048] Figure 6 This is a diagram showing a qualitative comparison between a dual-branch frequency and space fusion network and a blind super-resolution model provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0049] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0050] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0051] Embodiment 1:

[0052] The present invention provides a lightweight social platform image restoration method based on dual-branch frequency and space fusion, such as Figure 1 The figure shows a flow chart of a lightweight social platform image restoration method based on dual-branch frequency and space fusion. The specific steps are as follows:

[0053] S1: The training data is cropped into small image patches, and then the second-order degradation model is used to generate the corresponding low-resolution (LR) image patches as the training dataset.

[0054] In a specific embodiment, the training set of DIV2K, Flickr2K, and the 600 collected data sets are used as training data, the HR images are cropped into image blocks of 256×256 pixels, and the corresponding low-resolution image blocks are generated using the second-order degradation model proposed in the present invention. Through this step, a synthetic low-quality image with a sense of reality is generated for training and optimizing the image restoration model.

[0055] More specifically, the preset second-order degradation model includes: two blur modules, two size adjustment modules, two noise adding modules, and two hybrid compression modules, wherein the hybrid compression module is constructed by: on the basis of the Real-ESRGAN synthetic degradation algorithm, combining the JPEG compression method and the prediction-based algorithm to construct the hybrid compression module; reducing the strength of the default parameters of the Real-ESRGAN algorithm in the hybrid compression module to avoid excessive blur or noise in the synthetic degraded image, and obtaining the adjusted hybrid compression module;

[0056] Using the preset second-order degradation model to process small image blocks includes the following steps:

[0057] Input the small image block into the first blur module, the first size adjustment module, and the first noise adding module in sequence, output the image block after noise addition, input the image block after noise addition into the first hybrid compression module, and output the processed small image block;

[0058] The processed small image blocks are sequentially input into the second blur module, the second size adjustment module and the second noise adding module to obtain the noisy image blocks, and the noisy image blocks are input into the second hybrid compression module to output the low-resolution image blocks.

[0059] It should be noted that the DIV2K dataset is a high-quality dataset designed for image super-resolution tasks and is widely used in research and development in the field of computer vision. The dataset contains 800 high-resolution (HR) training images as a training set and 100 high-resolution validation images as a validation set. Each image has extremely high clarity and is very suitable for training and evaluating super-resolution algorithms.

[0060] Flickr2K is a high-quality dataset for image restoration tasks, containing 2,650 2K resolution images. These images are mainly used to train and evaluate super-resolution algorithms, especially for research on converting low-resolution images to high-resolution images2. The Flickr2K dataset is provided by Flickr and is a large extended dataset containing 800 training images, 100 validation images, and 100 test images.

[0061] Real-ESRGAN stands for Enhanced Super-Resolution Generative Adversarial Network, which is an enhanced super-resolution generative adversarial network.

[0062] It should be noted that the present invention provides a second-order degradation model. The process of synthesizing degradation data based on the second-order degradation model is as follows: Figure 2 As shown. The present invention first combines the JPEG compression method and the prediction-based algorithm to construct a wide range of compression degradation space, called "hybrid compression". In addition, the strength of the default parameters in Real-ESRGAN is reduced to avoid excessive blur or noise in the synthetic degraded images. In addition, 650 high-quality pictures are collected. This high-quality selfie picture dataset enhances the robustness of the network when processing selfie pictures on social media platforms.

[0063] S2: Using the preset lightweight restoration network as the generator and the frequency domain-based discriminative gating network as the discriminator, a lightweight social platform image restoration network model based on dual-branch frequency and space fusion is constructed.

[0064] More specifically, since not all frequency-domain-based features contribute to image restoration, exploring effective frequency information is crucial to improving image restoration quality. In order to effectively distinguish and utilize these key frequency features, the present invention provides a frequency-domain-based discriminant gating network, i.e., a frequency-domain-based discriminant, which can effectively determine the low-frequency and high-frequency information that should be retained when restoring a potential clear image. The frequency-domain-based discriminator is specifically: a frequency-domain-based discriminant gating module is added between the convolution module and the layer normalization module in the discriminator based on the U-net structure, and the layer normalization module is replaced by a spectral normalization module to form a frequency-domain-based discriminator, such as Figure 5 shown.

[0065] It should be noted that in the existing field of generative adversarial network super-resolution technology, although most methods can generate realistic images that conform to human perception, to generate high-quality images, they need to rely on a discriminator that can provide accurate gradient feedback to ensure the restoration of details of local textures in the image. Traditional methods usually use a U-Net-based discriminator to evaluate the authenticity at the pixel level and provide detailed pixel-level feedback to the generator. However, the direct application of such discriminators may lead to instability in the training process. The main reason is the difference in model capacity between the generator and the discriminator, which makes the generator unable to effectively keep up with the discriminator's ability to distinguish authenticity. In order to alleviate training instability, the present invention attempts to reduce the number of channels in each convolutional layer of the discriminator, but this will weaken the discriminator's ability to accurately identify real details. To this end, the present invention proposes a discriminator network based on the frequency domain, such as Figure 4 As shown in Figure 1, the network is designed to capture the detail differences in the frequency domain during image restoration, thereby more accurately evaluating the quality of image detail restoration. By fine-tuning the frequency features, the frequency domain-based discriminator significantly enhances the model's performance in image detail discrimination, thereby improving the stability and effectiveness of the entire GAN system in generating high-quality super-resolution images.

[0066] The preset lightweight restoration network includes: a convolution kernel, a plurality of parallel dual-branch frequency and space domain feature fusion modules, and an upsampling module. The dual-branch frequency and space feature fusion module includes a spatial adaptive feature hybrid network and a frequency domain basis discriminant gating network running in parallel.

[0067] More specifically, the method for generating a high-quality image from an image to be processed comprises the following steps:

[0068] Input the image to be processed into the convolution kernel for convolution, and output the primary features of the image;

[0069] The primary features of the image are input into the spatial adaptive feature hybrid network, and the spatial features in the spatial domain are output; the primary features of the image are input into the frequency domain basis discriminant gating network, and the frequency features in the frequency domain are output;

[0070] The spatial features in the spatial domain and the frequency features in the frequency domain are fused to obtain fused features;

[0071] The fused features are input into the upsampling module, and the residual connection is used to capture the high-frequency details of the image to obtain a high-quality image.

[0072] It should be noted that in order to fully utilize the advantages of spatial feature interaction, the present invention proposes a dual-branch frequency and spatial domain feature fusion module, such as Figure 3As shown. This module integrates the spatial adaptive feature hybrid network with the frequency domain basis discriminant gating network. The spatial adaptive feature hybrid network is a lightweight network module that learns long-range dependencies from multi-scale feature representations to better explore features that are more useful for high-quality image reconstruction. The dual-branch frequency and spatial domain basis feature fusion module of the present invention enables spatial and frequency domain features to be extracted simultaneously to achieve a more comprehensive image restoration effect.

[0073] More specifically, the frequency domain features are extracted from the primary features of the image using the frequency domain basis discriminant gating network, such as Figure 4 As shown, the following steps are included:

[0074] The input image primary feature X is convolved point by point by 1×1 to double the number of channels, and the spatial local context is encoded using 3×3 deep convolution, expressed as:

[0075] X1=DW_Conv 3x3 (Conv 1x1 (X)

[0076] Among them, X1 represents the features after 1x1 convolution and 3x3 depth convolution, Conv 1x1 (·) represents 1×1 point-wise convolution, DW_Conv 3x3 (·) represents 3×3 depthwise convolution;

[0077] The features are converted to the frequency domain and the fast Fourier transform is used to split the features into two independent components along the channel dimension. The expression is:

[0078] X F =F(X1)

[0079]

[0080] Among them, X F Represents the features converted to the frequency domain by fast Fourier transform, represents the features after segmentation along the channel dimension, F(·) represents fast Fourier transform, and Split(·) represents the channel segmentation operation;

[0081] A 1×1 point-wise convolution is applied to one of the split components, and the convolution result is used as the gate of the second component to selectively retain the relevant frequency information. The expression is:

[0082]

[0083] Among them, ⊙ represents pixel-by-pixel multiplication, Represents the features output after the above steps;

[0084] Apply the GELU activation function for nonlinear mapping, the expression is:

[0085]

[0086] Among them, X out Represents the frequency characteristics of the final output, F -1 (·) denotes inverse fast Fourier transform, and G(·) denotes GELU function.

[0087] S3: A lightweight social platform image restoration network model based on dual-branch frequency and space fusion is trained using a training data set. The generator is used to convert the image to be processed into a high-quality image. The discriminator is used to discriminate the high-quality images generated by the generator during model training, and the discrimination results are fed back and used to adjust the generator.

[0088] More specifically, the training strategy of the lightweight social platform image restoration network model based on dual-branch frequency and space fusion is a training strategy based on a generative adversarial network.

[0089] The training is based on a lightweight social platform image restoration network model based on dual-branch frequency and space fusion, including the following steps:

[0090] S3.1: Train the generator and discriminator using the training dataset.

[0091] The training dataset is used to train the generator of the feature fusion module with a dual-branch frequency and spatial domain basis. First, features are extracted through 3x3 convolution, and the spatial adaptive feature mixing network is used to dynamically select representative image features in the spatial domain. The frequency domain basis discriminant gating network is used to select frequency information that is helpful for image restoration. Finally, the two types of information are fused to better assist subsequent image restoration work.

[0092] Then, in the second stage, the generator is trained using a frequency-domain-based discriminative gating network to improve the quality and detail restoration capabilities of the images generated by the generator and stabilize the training process.

[0093] S3.2: Collect data for model testing and build a test set.

[0094] S3.3: Input the test set into the lightweight social platform image restoration network model based on dual-branch frequency and space fusion to test the restoration effect of the generator.

[0095] In a specific embodiment, 100 DIV2K validation sets and 60 high-quality selfie datasets we collected were used as test sets, which were transmitted through WeChat to obtain real LR images. This was used to test and evaluate the effects of our method and other methods for qualitative and quantitative comparison.

[0096] In a specific embodiment, PSNR (Peak signal-to-noise ratio) and SSIM (structural similarity) are used to measure the fidelity of image reconstruction, while LPIPS (Learned Perceptual Image Patch Similarity), NIQE (Natural Image Quality Evaluator) and MUSIQ (Multi-scale Image Quality Transformer) are used to evaluate the perceptual quality.

[0097] In addition, this embodiment uses the number of model parameters (#Params), the number of floating-point operations (#Flops), and the average running time (#Avg.Time) to verify the model efficiency.

[0098] S4: Input the low-quality image into the trained lightweight social platform image restoration network model based on dual-branch frequency and space fusion to restore the image and obtain a high-quality image.

[0099] In a specific embodiment, Figure 6 As shown in the figure, on the DIV2K dataset, after the synthetic degradation process of the present invention, the dual-branch frequency and space fusion network (DBFSNet) is qualitatively compared with the blind super-resolution model of the prior art. From the image point of view, the image restoration effect of the method of the present invention is more accurate.

[0100] The present invention first develops a dual-branch frequency and space fusion network (DBFSNet). The network integrates the gated dynamic network in the frequency domain and the feature selection technology in the spatial domain, which can effectively explore the frequency domain and spatial domain features at the same time, achieve better image restoration effects, and effectively restore the clarity of the image. Secondly, in response to the training instability problem caused by the traditional GAN ​​discriminator in a lightweight network, the present invention provides a lightweight frequency domain discriminator to improve the training stability and detail recovery ability of the model. In addition, the present invention also adjusts the second-order degradation model of synthetic degraded data to better simulate the real-life scenario of social media image degradation, and collects a small data set suitable for the present invention, and verifies the efficiency and accuracy of the method through experiments.

[0101] The restoration results of the lightweight social platform image restoration model based on dual-branch frequency and space fusion provided by the present invention have achieved results that are competitive with advanced large model solutions in terms of quality and fidelity. The present invention achieves the best effect at the lowest cost and achieves a good balance between model efficiency and restoration effect.

[0102] Embodiment 2:

[0103] The present embodiment provides a lightweight social platform image restoration system based on dual-branch frequency and space fusion, including a memory and a processor, wherein the memory includes a lightweight social platform image restoration method program based on dual-branch frequency and space fusion, and when the lightweight social platform image restoration method program based on dual-branch frequency and space fusion is executed by the processor, the steps of a lightweight social platform image restoration method based on dual-branch frequency and space fusion as described in Example 1 are implemented.

[0104] Embodiment 3:

[0105] The present embodiment provides a computer-readable storage medium, which includes a lightweight social platform image restoration method program based on dual-branch frequency and space fusion. When the lightweight social platform image restoration method program based on dual-branch frequency and space fusion is executed by a processor, the steps of a lightweight social platform image restoration method based on dual-branch frequency and space fusion as described in Example 1 are implemented.

[0106] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A lightweight social platform image restoration method based on dual-branch frequency and space fusion, characterized in that: The following steps are involved: The training data is cropped into small image blocks, and the small image blocks are processed using a preset second-order degradation model to obtain low-resolution image blocks as training data sets; Using the preset lightweight restoration network as the generator and the frequency-domain-based discriminative gating network as the discriminator, a lightweight social platform image restoration network model based on dual-branch frequency and space fusion is constructed; The training data set is used to train a lightweight social platform image restoration network model based on dual-branch frequency and space fusion, wherein the generator is used to convert the image to be processed into a high-quality image, and the discriminator is used to discriminate the high-quality image generated by the generator during model training, and the discrimination result is fed back and used to adjust the generator; The low-quality image is input into the trained lightweight social platform image restoration network model based on dual-branch frequency and space fusion to restore the image and obtain a high-quality image.

2. According to claim 1, a lightweight social platform image restoration method based on dual-branch frequency and space fusion is characterized in that: The preset second-order degradation model includes: two blur modules, two size adjustment modules, two noise adding modules, and two hybrid compression modules, wherein the hybrid compression module is constructed by: on the basis of the Real-ESRGAN synthetic degradation algorithm, combining the JPEG compression method and the prediction-based algorithm to construct the hybrid compression module; reducing the strength of the default parameters of the Real-ESRGAN algorithm in the hybrid compression module to obtain the adjusted hybrid compression module; Using the preset second-order degradation model to process small image blocks includes the following steps: Input the small image block into the first blur module, the first size adjustment module, and the first noise adding module in sequence, output the image block after noise addition, input the image block after noise addition into the first hybrid compression module, and output the processed small image block; The processed small image blocks are sequentially input into the second blur module, the second size adjustment module and the second noise adding module to obtain the noisy image blocks, and the noisy image blocks are input into the second hybrid compression module to output the low-resolution image blocks.

3. According to claim 1, a lightweight social platform image restoration method based on dual-branch frequency and space fusion is characterized in that: The preset lightweight restoration network includes: a convolution kernel, a plurality of parallel dual-branch frequency and space domain basis feature fusion modules, and an upsampling module; the method for processing an image using the preset lightweight restoration network includes the following steps: Input the image to be processed into the convolution kernel and output the primary features of the image; Input the primary features of the image into several parallel dual-branch frequency and space domain feature fusion modules, and output the fusion features; The fused features are input into the upsampling module to output high-quality images.

4. According to claim 3, a lightweight social platform image restoration method based on dual-branch frequency and space fusion is characterized in that: The dual-branch frequency and space domain feature fusion module includes a spatial adaptive feature mixing network and a frequency domain basis discriminant gating network running in parallel. The spatial adaptive feature mixing network is used to extract the spatial features of the primary features of the image, and the frequency domain basis discriminant gating network is used to extract the frequency features of the primary features of the image.

5. According to claim 4, a lightweight social platform image restoration method based on dual-branch frequency and space fusion is characterized in that: The method for converting an image to be processed into a high-quality image comprises the following steps: Input the image to be processed into the convolution kernel for convolution, and output the primary features of the image; The primary features of the image are input into the spatial adaptive feature hybrid network, and the spatial features in the spatial domain are output; the primary features of the image are input into the frequency domain basis discriminant gating network, and the frequency features in the frequency domain are output; The spatial features in the spatial domain and the frequency features in the frequency domain are fused to obtain fused features; The fused features are input into the upsampling module, and the residual connection is used to capture the high-frequency details of the image to obtain a high-quality image.

6. According to claim 5, a lightweight social platform image restoration method based on dual-branch frequency and space fusion is characterized in that: The frequency domain features are extracted from the primary features of the image using a frequency domain basis discriminant gating network, including the following steps: The input image primary feature X is convolved point by point by 1×1 to double the number of channels, and the spatial local context is encoded using 3×3 deep convolution, expressed as: X1=DW_Conv 3x3 (Conv. 1x1 (X)) Among them, X1 represents the features after 1x1 convolution and 3x3 depth convolution, Conv 1x1 (·) represents 1×1 point-wise convolution, DW_Conv 3x3 (·) represents 3×3 depthwise convolution; The features are converted to the frequency domain and the fast Fourier transform is used to split the features into two independent components along the channel dimension. The expression is: X F =F(X1) Among them, X F Represents the features converted to the frequency domain by fast Fourier transform, represents the features after segmentation along the channel dimension, F(·) represents fast Fourier transform, and Split(·) represents the channel segmentation operation; For a component after segmentation Apply 1×1 point-by-point convolution to obtain the convolution result, and use the convolution result as the gate of the second component to selectively retain the relevant frequency information, expressed as: Among them, ⊙ represents pixel-by-pixel multiplication, Represents the features output after the above steps; Apply the GELU activation function for nonlinear mapping, the expression is: Among them, X out Represents the frequency characteristics of the final output, F -1 (·) denotes inverse fast Fourier transform, and G(·) denotes GELU function.

7. According to claim 1, a lightweight social platform image restoration method based on dual-branch frequency and space fusion is characterized in that: The method for constructing the frequency domain-based discriminant gating network is as follows: adding a frequency domain-based discriminant gating module between a convolution module and a layer normalization module in a discriminator based on a U-net structure, and replacing the layer normalization module with a spectral normalization module.

8. According to claim 1, a lightweight social platform image restoration method based on dual-branch frequency and space fusion is characterized in that: The training strategy of the lightweight social platform image restoration network model based on dual-branch frequency and space fusion is a training strategy based on generative adversarial networks.

9. A lightweight social platform image restoration system based on dual-branch frequency and space fusion, characterized in that: It includes a memory and a processor, wherein the memory includes a lightweight social platform image restoration method program based on dual-branch frequency and space fusion, and when the lightweight social platform image restoration method program based on dual-branch frequency and space fusion is executed by the processor, the steps of a lightweight social platform image restoration method based on dual-branch frequency and space fusion as described in any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a lightweight social platform image restoration method program based on dual-branch frequency and space fusion. When the lightweight social platform image restoration method program based on dual-branch frequency and space fusion is executed by a processor, the steps of a lightweight social platform image restoration method based on dual-branch frequency and space fusion as described in any one of claims 1 to 8 are implemented.

Images