Image super-resolution reconstruction method and system
By adopting the residual dense block and channel attention mechanism of jump connection technology in the super-resolution image reconstruction algorithm, combined with the back projection loss function, the problem of insufficient texture detail recovery in the existing technology is solved, and a more refined high-frequency texture detail recovery effect is achieved.
Patent Information
- Application Number
- CN202210407697.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-04-19
- Publication Date
- 2025-03-25
- Estimated Expiration
- 2042-04-19
AI Technical Summary
The existing super-resolution image reconstruction algorithm has shortcomings in texture detail recovery, resulting in smooth texture details of the high-frequency reconstructed images.
The residual dense block of jump connection technology is used to build a basic network model, and the weight parameters of the input features are calculated in combination with the channel attention mechanism, and a back projection loss function is constructed to enhance the perceived quality of the image and texture detail recovery.
Through the above method, the reconstructed super-resolution image is closer to the original high-resolution image, with more finer high-frequency texture details.
Smart Images

Figure CN114677282B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image reconstruction, and in particular to an image super-resolution reconstruction method and system. Background Art
[0002] With the development of deep learning, more and more scholars have begun to pay attention to using deep neural networks to solve the SR (superresolution) problem. Compared with traditional methods, the performance of SR algorithms based on deep learning has been significantly improved. Among them, algorithms such as SRGAN (Super Resolution Generative Adversarial Network) and ESRGAN (Enhanced SRGAN) take perceptual loss as the optimization target, and improve the visual effect of the reconstructed image by minimizing the error in feature space rather than pixel space. However, the problem of loss of high-frequency texture detail information has not been optimally solved. Existing work has achieved outstanding results in the field of SR, but there are still some shortcomings in texture details. The reconstructed image has the problem of smooth high-frequency texture details. Summary of the invention
[0003] The object of the present invention is to provide an image super-resolution reconstruction method and system to reconstruct an image with finer high-frequency texture details.
[0004] To achieve the above object, the present invention provides the following solutions:
[0005] An image super-resolution reconstruction method, comprising:
[0006] Building a basic network model for image super-resolution reconstruction; the basic network model includes a feature extraction layer, a plurality of residual dense blocks, a channel attention module and an upsampling layer connected in sequence;
[0007] Considering the perceptual difference between the low-resolution image projected by the super-resolution image and the input low-resolution image, a back-projection loss function is constructed, and the total loss function for basic network model training is established based on the back-projection loss function;
[0008] Based on the total loss function, the basic network model is trained to obtain an image super-resolution reconstruction network model;
[0009] The low-resolution image to be reconstructed is input into the image super-resolution reconstruction network model, and a super-resolution image is output.
[0010] Optionally, the perceptual difference between the low-resolution image projected by the super-resolution image and the input low-resolution image is considered, a back-projection loss function is constructed, and a total loss function for training the basic network model is established based on the back-projection loss function, specifically including:
[0011] Based on the mean absolute error distance, a pixel-level content loss function is defined as l1=E||I SR -I HR ||1; where l1 represents content loss, E represents expectation, and I HR represents the real high-resolution image, I SR represents a super-resolution image;
[0012] Considering the perceptual difference between the low-resolution image projected by the super-resolution image and the input low-resolution image, the back-projection loss function is defined as l BP =E||σ(I SR )-σ(I HR )||1; where l BP represents the back-projection loss, σ(I SR ) represents the projection function from the super-resolution image to the low-resolution image, σ(I HR ) represents the projection function from the real high-resolution image to the low-resolution image;
[0013] According to the content loss function and the back-projection loss function, a total loss function is constructed as l total = l BP +l per +λl1+ηl GAN ; Among them, l total represents the total loss, l per represents the perceptual loss defined based on the VGG network, l GAN represents the adversarial loss between the generator and the discriminator, and λ and η represent the first and second balance coefficients, respectively.
[0014] Optionally, the residual dense block adopts a residual scaling technique, and the residual scaling coefficient β=0.2;
[0015] The number of residual dense blocks is 23.
[0016] Optionally, the step of inputting the low-resolution image to be reconstructed into the image super-resolution reconstruction network model and outputting the super-resolution image specifically includes:
[0017] Using the formula Feature extraction is performed on the low-resolution image to be reconstructed; wherein, represents the extracted features, f 3×3 represents the convolution operation with a convolution kernel size of 3×3, I LRrepresents the low-resolution image to be reconstructed;
[0018] The extracted features are mapped through multiple residual dense blocks, and the formula is used Get high-level semantic features; among them, represents the high-level semantic features after the i+1th residual dense block, represents the high-level semantic features after the i-th residual dense block, g(·) represents the internal operation function of the Dense Block, β represents the residual scaling factor, and λ represents the features obtained after the second Dense Block inside the i+1-th residual dense block. μ represents the feature obtained after passing through the first Dense Block inside the i+1th residual dense block.
[0019] Based on high-level semantic features, channel attention is used to calculate the function Get the features after the channel attention module; among them, represents the features after the channel attention module, CA() represents the channel attention calculation function, Represents the high-level semantic features after 23 residual dense blocks;
[0020] According to the features after the channel attention module, upsampling operation is used Output super-resolution image; where I SR represents a super-resolution image, and Sam represents an upsampling operation.
[0021] An image super-resolution reconstruction system, comprising:
[0022] A basic network model building module, used to build a basic network model for image super-resolution reconstruction; the basic network model includes a feature extraction layer, a plurality of residual dense blocks, a channel attention module and an upsampling layer connected in sequence;
[0023] A back-projection loss function construction module is used to consider the perceptual difference between the low-resolution image projected by the super-resolution image and the input low-resolution image, construct a back-projection loss function, and establish a total loss function for basic network model training based on the back-projection loss function;
[0024] A training module, used for training the basic network model based on the total loss function to obtain an image super-resolution reconstruction network model;
[0025] The super-resolution image output module is used to input the low-resolution image to be reconstructed into the image super-resolution reconstruction network model and output a super-resolution image.
[0026] Optionally, the back-projection loss function construction module specifically includes:
[0027] Content loss function definition submodule, which is used to define a pixel-level content loss function based on the mean absolute error distance as l1=E||I SR -I HR ||1; where l1 represents content loss, E represents expectation, and I HR represents the real high-resolution image, I SR represents a super-resolution image;
[0028] The back-projection loss function is used to establish a submodule to consider the perceptual difference between the low-resolution image projected by the super-resolution image and the input low-resolution image. The back-projection loss function is defined as l BP =E||σ(I SR )-σ(I HR )||1; where l BP represents the back-projection loss, σ(I SR ) represents the projection function from the super-resolution image to the low-resolution image, σ(I HR ) represents the projection function from the real high-resolution image to the low-resolution image;
[0029] The total loss function construction submodule is used to construct a total loss function of l according to the content loss function and the back projection loss function. total = l BP +l per +λl1+ηl GAN ; Among them, l total represents the total loss, l per represents the perceptual loss defined based on the VGG network, l GAN represents the adversarial loss between the generator and the discriminator, and λ and η represent the first and second balance coefficients, respectively.
[0030] Optionally, the residual dense block adopts a residual scaling technique, and the residual scaling coefficient β=0.2;
[0031] The number of residual dense blocks is 23.
[0032] Optionally, the super-resolution image output module specifically includes:
[0033] The feature extraction submodule is used to use the formula Feature extraction is performed on the low-resolution image to be reconstructed; wherein, represents the extracted features, f 3×3 represents the convolution operation with a convolution kernel size of 3×3, I LR represents the low-resolution image to be reconstructed;
[0034] The high-level semantic feature acquisition submodule is used to map the extracted features through multiple residual dense blocks and use the formula Get high-level semantic features; among them, represents the high-level semantic features after the i+1th residual dense block, represents the high-level semantic features after the i-th residual dense block, g(·) represents the internal operation function of the Dense Block, β represents the residual scaling factor, and λ represents the features obtained after the second Dense Block inside the i+1-th residual dense block. μ represents the feature obtained after passing through the first Dense Block inside the i+1th residual dense block.
[0035] The channel attention calculation submodule is used to calculate the channel attention function based on high-level semantic features. Get the features after the channel attention module; among them, represents the features after the channel attention module, CA() represents the channel attention calculation function, Represents the high-level semantic features after 23 residual dense blocks;
[0036] The upsampling operation submodule is used to use the upsampling operation based on the features after passing through the channel attention module. Output super-resolution image; where I SR represents a super-resolution image, and Sam represents an upsampling operation.
[0037] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:
[0038] The present invention discloses a method and system for image super-resolution reconstruction, which adopts a residual dense block of skip connection technology to construct a basic network model for image super-resolution reconstruction, calculates the weight parameters of input features with the help of a channel attention mechanism, and constructs a back-projection loss function that considers the perceptual difference between a low-resolution image projected by a super-resolution image and an input low-resolution image, so that the super-resolution image reconstructed by the present invention is closer to the original high-resolution image and has finer high-frequency texture details. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0040] Figure 1 A flowchart of an image super-resolution reconstruction method provided by the present invention;
[0041] Figure 2 This is a schematic diagram of the structure of the image super-resolution reconstruction network model provided by the present invention. DETAILED DESCRIPTION
[0042] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0043] The object of the present invention is to provide an image super-resolution reconstruction method and system to reconstruct an image with finer high-frequency texture details.
[0044] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.
[0045] To solve the problem of high-frequency texture detail information loss in super-resolution reconstructed images, the present invention provides an image super-resolution reconstruction method, such as Figure 1 As shown, including:
[0046] Step S1, building a basic network model for image super-resolution reconstruction; the basic network model includes a feature extraction layer, multiple residual dense blocks, a channel attention module and an upsampling layer connected in sequence.
[0047] In order to better restore the perceived quality of the image, the core structure of the basic network model of the present invention adopts the RRDB (Residual Dense Block) in ESRGAN, combining a multi-level residual network and dense connections, such as Figure 2 In addition, RRDB adopts residual scaling technology, which promotes network training and prevents instability caused by excessive network depth by multiplying the residual by a constant between 0 and 1. The residual scaling coefficient β of the present invention is 0.2.
[0048] Figure 2 In the figure, LR means low resolution, Conv means convolution, RRDB means Residual-in-Residual Dense Block, Dense Block means dense block, LReLU means non-linear activation layer, Upsampling means upsampling, and SR means super resolution.
[0049] Step S2, considering the perceptual difference between the low-resolution image projected by the super-resolution image and the input low-resolution image, constructing a back-projection loss function, and establishing a total loss function during basic network model training based on the back-projection loss function.
[0050] 1) In order to ensure that the SR image is as close as possible to the HR (high resolution) image, a pixel-level content loss function is defined based on the mean absolute error distance as l1 = E||I SR -I HR ||1; where l1 represents content loss, E represents expectation, and I HR represents the real high-resolution image, I SR represents a super-resolution image;
[0051] However, this only constrains from one direction, so the present invention considers constraining from two directions.
[0052] 2) The ideal result of super-resolution is that the SR image is exactly the same as the HR image. In this case, the LR image projected by the SR image should also be exactly the same as the LR image projected by the HR image. In other words, if you want to ensure that the SR image is as close to the HR image as possible, then the LR image projected by the SR should also be as close to the LR image projected by the HR. Based on this, considering the perceptual difference between the low-resolution image projected by the super-resolution image and the input low-resolution image, the back-projection loss function is defined as l BP =E||σ(I SR )-σ(I HR )||1;
[0053] Among them, l BP represents the back-projection loss, σ(I SR ) represents the projection function from the super-resolution image to the low-resolution image, σ(I HR ) represents the projection function from the real high-resolution image to the low-resolution image;
[0054] 3) Based on the content loss function and the back-projection loss function, the total loss function is constructed as l total = l BP +l per +λl1+ηl GAN ;
[0055] Among them, l total represents the total loss, l per represents the perceptual loss defined based on the VGG network, l GAN represents the adversarial loss between the generator and the discriminator, λ and η represent the first and second balance coefficients, respectively. Preferably, λ and η are 0.02 and 0.005, respectively.
[0056] Step S3: Based on the total loss function, the basic network model is trained to obtain an image super-resolution reconstruction network model.
[0057] Step S4: input the low-resolution image to be reconstructed into the image super-resolution reconstruction network model, and output a super-resolution image.
[0058] Given a low-resolution image The network is expected to output a high-resolution image with a corresponding magnification factor of s
[0059] The general process of the network is as follows:
[0060] 1) Using the formula Perform feature extraction on the low-resolution image to be reconstructed;
[0061] in, represents the extracted features, f 3×3 represents the convolution operation with a convolution kernel size of 3×3, I LR represents the low-resolution image to be reconstructed;
[0062] 2) The extracted features are mapped through multiple residual dense blocks, and the formula is used Get high-level semantic features;
[0063] in, represents the high-level semantic features after the i+1th residual dense block, represents the high-level semantic features after the i-th residual dense block, g(·) represents the internal operation function of the Dense Block, β represents the residual scaling factor, and λ represents the features obtained after the second Dense Block inside the i+1-th residual dense block. μ represents the feature obtained after passing through the first Dense Block inside the i+1th residual dense block.
[0064] 3) Based on high-level semantic features, use channel attention to calculate the function Obtain the features after passing through the channel attention module; through the channel attention mechanism, enhance the importance of the effective information of the input features and suppress unimportant information features.
[0065] in, represents the features after the channel attention module, CA() represents the channel attention calculation function, Represents the high-level semantic features after 23 residual dense blocks;
[0066] 4) Based on the features after the channel attention module, use upsampling operation Output super-resolution image;
[0067] Among them, I SR represents a super-resolution image, and Sam represents an upsampling operation.
[0068] Image super-resolution (SR) reconstruction is a fundamental problem in the field of computer vision, and has wide applications in pedestrian re-identification, security monitoring, satellite and medical imaging. SR refers to the generation of a corresponding high-resolution (HR) image from one or a group of low-resolution (LR) images. In recent years, SR algorithms based on deep learning have been widely proposed, greatly improving the performance of SR. However, most of these algorithms use mean absolute error (MAE) or mean squared error (MSE) as the loss function for optimization, and only rely on the pixel distance between the SR image and the HR image for constraints, but have limited ability to capture perceptually relevant differences. Therefore, blurred images with high peak signal-to-noise ratio (PSNR) are usually generated, and the recovery of texture detail information cannot be guaranteed.
[0069] Since the game theory of Generative Adversarial Network (GAN) has achieved great success in the field of image generation, SR algorithms based on GAN have also been proposed to restore more realistic SR images. For example, in order to improve the perceptual quality of the restored image, SRGAN (Super Resolution Generative Adversarial Network, SRGAN) and ESRGAN (Enhanced SRGAN, ESRGAN) proposed to use the perceptual loss function as the optimization target, which consists of the adversarial loss between the generator and the discriminator and the content loss driven by perceptual similarity, which improves the overall visual quality of the reconstructed image to a certain extent. However, the problem of loss of texture detail information has not been optimally solved. In addition, the training of GAN may be unstable, resulting in deformed reconstructed images, such as distorted geometric structures. Therefore, the present invention aims to solve these problems from two aspects.
[0070] The algorithm of the present invention builds a basic model with residual dense blocks using skip connection technology, calculates the weight parameters of input features with the help of channel attention mechanism, and takes the perceptual loss of fused back-projection loss function as the optimization target. In addition, in order to stabilize GAN network training, gradient normalization (GN) is introduced into the SR model framework.
[0071] Compared with the previous SR algorithms driven by perceptual loss, the algorithm of the present invention is not limited to the perceptually relevant differences between the SR image and the HR image, but also considers the perceptually relevant differences between the low-resolution image mapped by the SR image and the input LR image, which undoubtedly adds a constraint condition on the basis of the original constraint condition. Therefore, the SR image restored by the algorithm of the present invention is closer to the original HR image and has finer texture information.
[0072] The present invention also provides an image super-resolution reconstruction system, comprising:
[0073] A basic network model building module, used to build a basic network model for image super-resolution reconstruction; the basic network model includes a feature extraction layer, a plurality of residual dense blocks, a channel attention module and an upsampling layer connected in sequence;
[0074] A back-projection loss function construction module is used to consider the perceptual difference between the low-resolution image projected by the super-resolution image and the input low-resolution image, construct a back-projection loss function, and establish a total loss function for basic network model training based on the back-projection loss function;
[0075] A training module, used for training the basic network model based on the total loss function to obtain an image super-resolution reconstruction network model;
[0076] The super-resolution image output module is used to input the low-resolution image to be reconstructed into the image super-resolution reconstruction network model and output a super-resolution image.
[0077] The back-projection loss function building block includes:
[0078] Content loss function definition submodule, which is used to define a pixel-level content loss function based on the mean absolute error distance as l1=E||I SR -I HR ||1; where l1 represents content loss, E represents expectation, and I HR represents the real high-resolution image, I SR represents a super-resolution image;
[0079] The back-projection loss function is used to establish a submodule to consider the perceptual difference between the low-resolution image projected by the super-resolution image and the input low-resolution image. The back-projection loss function is defined as l BP =E||σ(I SR )-σ(I HR )||1; where l BP represents the back-projection loss, σ(I SR ) represents the projection function from the super-resolution image to the low-resolution image, σ(I HR ) represents the projection function from the real high-resolution image to the low-resolution image;
[0080] The total loss function construction submodule is used to construct a total loss function of l according to the content loss function and the back projection loss function. total = l BP +l per +λl1+ηl GAN ; Among them, l total represents the total loss, l per represents the perceptual loss defined based on the VGG network, l GAN represents the adversarial loss between the generator and the discriminator, and λ and η represent the first and second balance coefficients, respectively.
[0081] The residual dense block adopts residual scaling technology, and the residual scaling factor β = 0.2;
[0082] The number of residual dense blocks is 23.
[0083] The super-resolution image output module specifically includes:
[0084] The feature extraction submodule is used to use the formula Feature extraction is performed on the low-resolution image to be reconstructed; wherein, represents the extracted features, f 3×3 represents the convolution operation with a convolution kernel size of 3×3, I LR represents the low-resolution image to be reconstructed;
[0085] The high-level semantic feature acquisition submodule is used to map the extracted features through multiple residual dense blocks and use the formula Get high-level semantic features; among them, represents the high-level semantic features after the i+1th residual dense block, represents the high-level semantic features after the i-th residual dense block, g(·) represents the internal operation function of the Dense Block, β represents the residual scaling factor, and λ represents the features obtained after the second Dense Block inside the i+1-th residual dense block. μ represents the feature obtained after passing through the first Dense Block inside the i+1th residual dense block.
[0086] The channel attention calculation submodule is used to calculate the channel attention function based on high-level semantic features. Get the features after the channel attention module; among them, represents the features after the channel attention module, CA() represents the channel attention calculation function, Represents the high-level semantic features after 23 residual dense blocks;
[0087] The upsampling operation submodule is used to use the upsampling operation based on the features after passing through the channel attention module. Output super-resolution image; where I SR represents a super-resolution image, and Sam represents an upsampling operation.
[0088] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.
[0089] The present invention uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only used to help understand the method and core ideas of the present invention. At the same time, for those skilled in the art, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.
Claims
1. A method for super-resolution image reconstruction, characterized in that: include: Building a basic network model for image super-resolution reconstruction; the basic network model includes a feature extraction layer, a plurality of residual dense blocks, a channel attention module and an upsampling layer connected in sequence; Considering the perceptual difference between the low-resolution image projected by the super-resolution image and the input low-resolution image, a back-projection loss function is constructed, and the total loss function for basic network model training is established based on the back-projection loss function; Based on the total loss function, the basic network model is trained to obtain an image super-resolution reconstruction network model; Inputting the low-resolution image to be reconstructed into the image super-resolution reconstruction network model, and outputting a super-resolution image; The method considers the perceptual difference between the low-resolution image projected by the super-resolution image and the input low-resolution image, constructs a back-projection loss function, and establishes a total loss function for basic network model training based on the back-projection loss function, specifically including: Based on the mean absolute error distance, a pixel-level content loss function is defined as e1=E‖I SR -I HR ||1; where l1 represents content loss, E represents expectation, and I HR represents the real high-resolution image, I SR represents a super-resolution image; Considering the perceptual difference between the low-resolution image projected by the super-resolution image and the input low-resolution image, the back-projection loss function is defined as l BP =E||σ(I SR )-σ(I HR )||1; where l BP represents the back-projection loss, σ(I SR ) represents the projection function from the super-resolution image to the low-resolution image, σ(I HR ) represents the projection function from the real high-resolution image to the low-resolution image; According to the content loss function and the back-projection loss function, a total loss function is constructed as l total = l BP +l per +αl1+ηl GAN ; Among them, l total represents the total loss, l per represents the perceptual loss defined based on the VGG network, l GAN represents the adversarial loss between the generator and the discriminator, α and η represent the first and second balance coefficients, respectively.
2. The image super-resolution reconstruction method according to claim 1, characterized in that: The residual dense block adopts residual scaling technology, and the residual scaling coefficient β=0.2; The number of residual dense blocks is 23.
3. The image super-resolution reconstruction method according to claim 2, characterized in that: The step of inputting the low-resolution image to be reconstructed into the image super-resolution reconstruction network model and outputting the super-resolution image specifically includes: Using the formula Feature extraction is performed on the low-resolution image to be reconstructed; wherein, represents the extracted features, f 3×3 represents the convolution operation with a convolution kernel size of 3×3, I LR represents the low-resolution image to be reconstructed; The extracted features are mapped through multiple residual dense blocks, and the formula is used Get high-level semantic features; among them, represents the high-level semantic features after the i+1th residual dense block, represents the high-level semantic features after the i-th residual dense block, g(·) represents the internal operation function of the Dense Block, β represents the residual scaling factor, and λ represents the features obtained after the second Dense Block inside the i+1-th residual dense block. μ represents the feature obtained after passing through the first Dense Block inside the i+1th residual dense block. Based on high-level semantic features, channel attention is used to calculate the function Get the features after the channel attention module; among them, represents the features after the channel attention module, CA() represents the channel attention calculation function, Represents the high-level semantic features after 23 residual dense blocks; According to the features after the channel attention module, upsampling operation is used Output super-resolution image; where I SR represents a super-resolution image, and Sam represents an upsampling operation.
4. An image super-resolution reconstruction system, characterized in that: include: A basic network model building module, used to build a basic network model for image super-resolution reconstruction; the basic network model includes a feature extraction layer, a plurality of residual dense blocks, a channel attention module and an upsampling layer connected in sequence; A back-projection loss function construction module is used to consider the perceptual difference between the low-resolution image projected by the super-resolution image and the input low-resolution image, construct a back-projection loss function, and establish a total loss function for basic network model training based on the back-projection loss function; A training module, used for training the basic network model based on the total loss function to obtain an image super-resolution reconstruction network model; A super-resolution image output module, used for inputting the low-resolution image to be reconstructed into the image super-resolution reconstruction network model, and outputting a super-resolution image; The back-projection loss function construction module specifically includes: Content loss function definition submodule, which is used to define a pixel-level content loss function based on the mean absolute error distance as l1=E||I SR -I HR ||1; where l1 represents content loss, E represents expectation, and I HR represents the real high-resolution image, I SR represents a super-resolution image; The back-projection loss function is used to establish a submodule to consider the perceptual difference between the low-resolution image projected by the super-resolution image and the input low-resolution image. The back-projection loss function is defined as l BP =E||σ(I SR )-σ(I HR )||1; where l BP represents the back-projection loss, σ(I SR ) represents the projection function from the super-resolution image to the low-resolution image, σ(I HR ) represents the projection function from the real high-resolution image to the low-resolution image; The total loss function construction submodule is used to construct a total loss function of l according to the content loss function and the back projection loss function. total = l BP +l per +αl1+ηl GAN ; Among them, l total represents the total loss, l per represents the perceptual loss defined based on the VGG network, l GAN represents the adversarial loss between the generator and the discriminator, α and η represent the first and second balance coefficients, respectively.
5. The image super-resolution reconstruction system according to claim 4, characterized in that: The residual dense block adopts residual scaling technology, and the residual scaling coefficient β=0.2; The number of residual dense blocks is 23.
6. The image super-resolution reconstruction system according to claim 5, characterized in that: The super-resolution image output module specifically includes: The feature extraction submodule is used to use the formula Feature extraction is performed on the low-resolution image to be reconstructed; wherein, represents the extracted features, f 3×3 represents the convolution operation with a convolution kernel size of 3×3, I LR represents the low-resolution image to be reconstructed; The high-level semantic feature acquisition submodule is used to map the extracted features through multiple residual dense blocks and use the formula Get high-level semantic features; among them, represents the high-level semantic features after the i+1th residual dense block, represents the high-level semantic features after the i-th residual dense block, g(·) represents the internal operation function of DenseBlock, β represents the residual scaling factor, and λ represents the features obtained after the second Dense Block inside the i+1-th residual dense block. μ represents the feature obtained after passing through the first Dense Block inside the i+1th residual dense block. The channel attention calculation submodule is used to calculate the channel attention function based on high-level semantic features. Get the features after the channel attention module; among them, represents the features after the channel attention module, CA() represents the channel attention calculation function, Represents the high-level semantic features after 23 residual dense blocks; The upsampling operation submodule is used to use the upsampling operation based on the features after passing through the channel attention module. Output super-resolution image; where I SR represents a super-resolution image, and Sam represents an upsampling operation.
Citation Information
Patent Citations
Super-resolution face image reconstruction method based on SRGAN network
CN110211045A
Image super-resolution reconstruction method and system based on edge detection
CN111062872A