Image super-resolution reconstruction method based on semi-coupled deep convolutional dictionary learning

Abstract

This invention discloses a method for image super-resolution reconstruction based on semi-coupled deep convolutional dictionary learning, comprising: preparing input training set data; constructing a semi-coupled deep convolutional dictionary learning model; constructing and training a deep neural network to solve the model, obtaining a trained coefficient mapping network, dictionary, and coefficients; and reconstructing a low-resolution image to obtain a high-resolution image using the trained coefficient mapping network, dictionary, and coefficients. This invention fully leverages the learning capabilities of deep learning and the interpretability of dictionary learning to perform super-resolution reconstruction on low-resolution images, improving the image resolution and thereby enhancing the image display effect.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to an image super-resolution reconstruction method based on semi-coupled deep convolutional dictionary learning. Background Art

[0002] Research shows that the vast majority of information in human activities is perceived through the visual system. As a medium for recording and transmitting visual information, images offer advantages such as intuitive imagery, rich information content, and ease of understanding. They have crucial applications in numerous fields, including astronomical observation, location tracking, medical imaging, and traffic monitoring. With the advent of the mobile internet era and the continuous advancement of information technology, the demand for image quality and resolution is growing, leading to an exponential increase in image data volume. On the one hand, the increasing resolution of captured images significantly increases storage space and bandwidth, but without the adoption of efficient image compression techniques, this will significantly increase the pressure on storage space and bandwidth, leading to a sharp decline in the number of images that can be stored. On the other hand, during image acquisition and transmission, compressed images may suffer from compression distortion and low resolution, hindering subsequent image processing and failing to meet the requirements of practical applications. This poses significant challenges to existing technologies and equipment.

[0003] Super-resolution (SR) reconstruction uses software technology to reconstruct existing low-resolution images, improving the image resolution while restoring the lost high-frequency details of the image. It has the characteristics of low cost and strong practicality, and has become a research hotspot in the field of image processing.

[0004] Current deep neural networks have achieved promising results in image super-resolution. Researchers have proposed the ESRGAN method (Enhanced super-resolution generative adversarial networks). However, due to the inherent black-box nature of deep neural networks, many deep learning models lack reliable theoretical guarantees. Traditional model-based methods, such as image super-resolution via sparse representation, offer excellent flexibility and interpretability. Deep unfolding models combine the advantages of both model-based and deep learning-based approaches. Existing deep unfolding networks have demonstrated the feasibility of this approach, with some even surpassing the performance of deep neural networks on image processing tasks. Deep convolutional dictionary learning (DCDicL) has also been proposed for image denoising. However, existing convolutional dictionary learning-based super-resolution methods rely on artificially set priors. However, utilizing this intuitive prior information can overlook potential information that characterizes image structure. Summary of the Invention

[0005] Technical problem solved: The present invention proposes an image super-resolution reconstruction method based on semi-coupled deep convolutional dictionary learning, which fully utilizes the learning ability of deep learning and the interpretability of dictionary learning to perform super-resolution reconstruction on low-resolution images, improve the resolution of the image, and thus enhance the image display effect.

[0006] Technical solution:

[0007] An image super-resolution reconstruction method based on semi-coupled deep convolutional dictionary learning, the image super-resolution reconstruction method comprising the following steps:

[0008] S1, prepare the input training set data: degrade the HR image to obtain the corresponding LR image, and construct the training set {X i , Y i}, where X i is the LR image, Y i is a HR image, the subscript i represents the i-th LR image or HR image; LR represents low resolution, and HR represents high resolution;

[0009] S2, build a semi-coupled deep convolutional dictionary learning model:

[0010]

[0011] In formula (1), X and Y represent the LR image and HR image, D x and D y are dictionaries of LR images and HR images respectively, A x 、A y is the representation coefficient corresponding to the LR image and the HR image, is the prior information of the coefficient, φ(·) is the prior information of the dictionary, * represents convolution, λ represents the regularization parameter, σ x , σ y 、 are the coefficients of each item, f y (·) is the coefficient mapping function between the LR image and the HR image representation coefficients;

[0012] S3, build and train a deep neural network to solve the semi-coupled deep convolutional dictionary learning model. The deep neural network includes four modules: Net-X, Net-Y, Net-F, and Hyper-Net. Net-X is used to learn the representation coefficient A of the LR image. x Dictionary D corresponding to the LR image x , Net-Y is used to learn the representation coefficient A of HR images y Dictionary D corresponding to the HR image y , Net-F is used to train and learn A x to A y The coefficient mapping function f y (·) Hyper-Net is used to learn the hyperparameters in the model at each iteration stage. The coefficient mapping loss function is used to constrain the difference between the mapped representation coefficients and the representation coefficients of the HR image. The reconstruction loss function is used to constrain the difference between the predicted reconstructed image and the original HR image. The network model parameters are continuously adjusted until the model converges after T iterations. The model training is completed and the trained coefficient mapping network, dictionary and coefficients are obtained.

[0013] S4: Input a LR image to be super-resolved into the trained model to obtain the representation coefficient of the LR image in the Tth iteration stage Then the trained coefficient mapping function After mapping and training Perform convolution to obtain the final reconstructed high-resolution image:

[0014] Furthermore, in step S1, the process of preparing input data includes the following sub-steps:

[0015] S11, degrade the HR image using the following formula to obtain the corresponding LR image:

[0016] X=(Y*k)↓s +n

[0017] Where X is the LR image, Y is the HR image, * is the convolution operation, k is the blur kernel, ↓ s represents s-fold downsampling, and n is additive white Gaussian noise;

[0018] S12, performing null value upsampling on the LR image obtained by the degradation process so that the size thereof is consistent with the size of the corresponding HR image;

[0019] S13, segmenting the HR image and the LR image into image blocks of a preset size, collecting the segmented images to obtain a training set, and using the training set to train the network. The image block size and the number of batch samples are related to the device performance of the network carrier.

[0020] Furthermore, we introduce auxiliary variables The semi-quadratic splitting algorithm is used to iteratively solve equation (1). In the tth iteration stage, the solution process of equation (1) is to calculate equations (2) to (10) in sequence:

[0021]

[0022] in

[0023]

[0024]

[0025] in

[0026]

[0027]

[0028]

[0029]

[0030]

[0031] Construct a deep neural network Net-F and calculate A according to formula (3) and formula (5) x and A y Training learning mapping function Make

[0032]

[0033] Among them, α x , α y , βx , β y , is the hyperparameter learned by the Hyper-Net network, vec represents the vectorization operation, that is, converting the matrix into a column vector, vec -1 represents the inverse quantization operation, unf represents the expansion of the matrix in the convolution operation into the matrix form of the product operation, F(·) and F -1 (·) denotes 2D Fourier transform and inverse transform, respectively, represents the complex conjugate of the 2D Fourier transform, Indicates element-by-element multiplication, and the division involved is also element-by-element division. In formula (3) and formula (5), Indicates that the solution is obtained by constructing a deep neural network Net-A In formula (7) and formula (9), Indicates that the solution is obtained by constructing a deep neural network Net-D The superscript (t) indicates the t-th iteration stage, and the superscript T is the matrix transpose.

[0034] Furthermore, the Net-X includes Head-Net-X, Three modules;

[0035] Head-Net-X adopts the Head-Net network structure. The Head-Net network structure is: including at least 2 convolutional layers, with 1 ReLu activation function between each 2 convolutional layers, and each convolutional layer contains at least 64 channels; in the 0th iteration stage, the LR image, the blur kernel k of the degradation model, and the noise standard deviation σ are input to Head-Net-X to obtain the initialized representation coefficients and a dictionary initialized with 0

[0036] The Net-A network structure is adopted. The Net-A network structure is: including 7 modules, each module contains n r Residual units, each residual unit contains at least two convolutional layers and jump connections, with a ReLu activation function between every two convolutional layers. Stride convolution is used to downsample feature maps in the convolutional layers of the first three modules, and transposed convolution is used to upsample feature maps in the convolutional layers of the last three modules. The convolutional layers of the first four modules have 64, 128, 256, and 512 channels respectively. The process of processing data in the tth iteration stage is as follows: input the auxiliary variable obtained by formula (2) Hyperparameters, get the representation coefficients

[0037] The Net-D network structure is adopted. The Net-D network structure is as follows: it includes at least 6 convolutional layers, with a ReLu activation function between every two convolutional layers, each convolutional layer contains at least 16 channels, and a skip connection is set between the input of the first convolutional layer and the output of the last convolutional layer; The process of processing data in the tth iteration stage is as follows: input the auxiliary variable obtained by formula (6) Hyperparameters, get dictionary

[0038] Furthermore, Net-Y includes Head-Net-Y, In the third part, Head-Net-Y is used to obtain the initial representation coefficients in the 0th iteration stage. and dictionary In the tth iteration stage, by get pass get

[0039] Head-Net-Y uses the Head-Net network structure; the process of Head-Net-Y processing data is as follows: input HR image, get the initialized representation coefficient and a dictionary initialized with 0

[0040] Adopt Net-A network structure; The process of processing data in the tth iteration stage is as follows: input the auxiliary variable obtained by formula (4) Hyperparameters, get the representation coefficients

[0041] Adopt Net-D network structure; The process of processing data in the tth iteration stage is as follows: input the auxiliary variables solved by formula (8) Hyperparameters, get

[0042] Furthermore, Net-F adopts the Net-A network structure; the process of Net-F processing data in the tth iteration stage is: input get And constrain the loss function by mapping the coefficients and The difference between Updated to

[0043] Furthermore, the coefficient mapping loss function is:

[0044]

[0045] where θ F Indicates the parameters of the Net-F network, is the representation coefficient of the LR image in the t-th iteration stage, is the representation coefficient of the HR image at the t-th iteration stage, is the coefficient mapping function of the t-1th iteration stage, N is the number of batch samples, and the subscript i represents the i-th LR image or HR image.

[0046] Furthermore, the Hyper-Net network structure is as follows: it includes at least 4 convolutional layers, contains a ReLu activation function between every 2 convolutional layers, and has a SoftPlus layer after the last convolutional layer; the process of Hyper-Net processing data in the tth iteration stage is: input noise standard deviation σ to obtain hyperparameters.

[0047] Furthermore, the reconstruction loss function is:

[0048]

[0049] Where θ represents the parameters of the network, is the dictionary of HR images in the t-th iteration stage, and is the representation coefficient of the t-th stage LR image and HR image, * represents convolution, is the coefficient mapping function, Y i is the i-th HR image, and N is the number of batch samples.

[0050] Beneficial effects:

[0051] First, the image super-resolution reconstruction method based on semi-coupled deep convolutional dictionary learning of the present invention fully utilizes the learning ability of deep neural networks to learn the prior information of representation coefficients and dictionaries. At the same time, it also extracts representation coefficient features through the network and learns coefficient mapping functions, which can better maintain the structure and texture elements in the image and effectively improve the image reconstruction quality.

[0052] Second, the image super-resolution reconstruction method based on semi-coupled deep convolutional dictionary learning of the present invention has a black box characteristic of deep neural networks. Many deep learning models do not have reliable theoretical guarantees. In addition to utilizing the deep learning ability, the present invention also makes full use of the interpretability of traditional dictionary learning methods, so that the network structure designed in the present invention is also theoretically interpretable.

[0053] Third, the image super-resolution reconstruction method based on semi-coupled deep convolutional dictionary learning of the present invention integrates the U-Net network and the Residual network in the core module of the main network, so that the model can learn richer global features and has stronger learning ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 This is a diagram of the overall structure of the image super-resolution reconstruction model according to an embodiment of the present invention.

[0055] Figure 2 This is a diagram of the Head-Net network structure of an embodiment of the present invention.

[0056] Figure 3 This is a diagram of the Net-A network structure of an embodiment of the present invention.

[0057] Figure 4 This is a diagram of the Net-D network structure of an embodiment of the present invention.

[0058] Figure 5 This is a diagram of the Hyper-Net network structure of an embodiment of the present invention.

[0059] Figure 6 Schematic diagram of the original high-resolution image.

[0060] Figure 7 Schematic diagram of the image obtained after processing by the bicubic method.

[0061] Figure 8 Schematic diagram of the image obtained after processing by the SRMD method.

[0062] Figure 9 Schematic diagram of the image obtained after processing by the USRNet method.

[0063] Figure 10 Schematic diagram of the image obtained after processing by the SwinIR method.

[0064] Figure 11 Schematic diagram of an image obtained after processing according to an embodiment of the present invention. DETAILED DESCRIPTION

[0065] The following examples may enable those skilled in the art to more fully understand the present invention, but are not intended to limit the present invention in any way.

[0066] This embodiment discloses an image super-resolution reconstruction method based on semi-coupled deep convolutional dictionary learning, which specifically includes the following steps:

[0067] Step 1: Prepare input training set data.

[0068] Obtain public datasets for training and testing; in this embodiment, three public datasets, DIV2K, Flickr2K, and CBSD68, are used, where DIV2K and Flickr2K contain 800 and 2650 images respectively for training, and CBSD68 contains 68 images for testing; degrade the HR images to obtain the corresponding LR images, and use them to construct the training set {X i , Y i}, where X i is the LR image, Y i is a HR image, and the subscript i represents the i-th LR or HR image.

[0069] The degradation model formula for degradation treatment is: X = (Y*k)↓ s +n; where X is the LR image, Y is the HR image, * is the convolution operation, k is the blur kernel, ↓ s represents downsampling by a factor of s, and n is additive white Gaussian noise.

[0070] In this embodiment, the blur kernel k is set to an isotropic Gaussian blur kernel, the downsampling factor s is set to 2, and the noise standard deviation is set to a random number between 0 and 5.

[0071] In this embodiment, the LR image obtained by the degradation processing is null-upsampled (i.e., the LR image is padded with zero values) to make it consistent with the size of the corresponding HR image; the HR image and the LR image are segmented into image blocks of size 64×64, and the segmented images are aggregated to obtain a training set; when using the training set to train the network, the number of batch samples N is set to 32.

[0072] Step 2: Build and solve the semi-coupled deep convolutional dictionary learning model.

[0073] The constructed semi-coupled deep convolutional dictionary learning model is formula (1), by introducing auxiliary variables The semi-quadratic splitting algorithm is used to iteratively solve equation (1). In the t-th iteration stage (Stage-t), the solution process of equation (1) is to calculate equations (2) to (10) in sequence.

[0074] Step 3: Build and train a deep neural network to solve the semi-coupled deep convolutional dictionary learning model to obtain the trained coefficient mapping function, dictionary, and representation coefficients.

[0075] like Figure 1 As shown in Figure 2, the deep neural network for solving the semi-coupled deep convolutional dictionary learning model includes four modules: Net-X, Net-Y, Net-F, and Hyper-Net, where Net-X is used to learn the representation coefficient A of the LR image. x Dictionary D corresponding to the LR imagex , Net-Y is used to learn the representation coefficient A of HR images y Dictionary D corresponding to the HR image y , Net-F is used to train and learn A x to A y The coefficient mapping function f y (·), Hyper-Net is used to learn the hyperparameters in the model at each iteration stage. The coefficient mapping loss function is used to constrain the difference between the mapped representation coefficients and the representation coefficients of the HR image, and the reconstruction loss function is used to constrain the difference between the reconstructed image and the original HR image. The network model parameters are continuously adjusted until the model converges after T iterations, completing the model training.

[0076] Net-X includes Head-Net-X, Three modules, using Head-Net-X in the 0th iteration to get the initial representation coefficients and dictionary In the tth iteration stage, by get pass get

[0077] Head-Net-X uses the Head-Net network structure; the module processes data as follows: input LR image, blur kernel k of degradation model, noise standard deviation σ, and obtain the initialized representation coefficient and a dictionary initialized with 0

[0078] like Figure 2 As shown in the figure, the Head-Net network structure is as follows: it includes 2 convolutional layers, with 1 ReLu activation function between the two convolutional layers, and each convolutional layer contains 64 channels.

[0079] The Net-A network structure is used; the process of processing data in the tth iteration stage of this module is as follows: input the auxiliary variable obtained by formula (2) Hyperparameters, get the representation coefficients That is the solution of formula (3).

[0080] like Figure 3As shown in the figure, the Net-A network structure is as follows: it includes 7 modules, each module contains 4 residual units, each residual unit contains 2 convolutional layers and a skip connection, and there is a Relu activation function between the two convolutional layers. The convolutional layers of the first 3 modules use strided convolution to downsample the feature maps, and the convolutional layers of the last 3 modules use transposed convolution to upsample the feature maps. The convolutional layers of the first 4 modules have 64, 128, 256, and 512 channels respectively.

[0081] The Net-D network structure is used; the process of processing data in the tth iteration stage of this module is as follows: input the auxiliary variable obtained by formula (6) Hyperparameters, get dictionary That is the solution of formula (7).

[0082] like Figure 4 As shown in the figure, the Net-D network structure is as follows: it includes 6 convolutional layers, with a ReLu activation function between every 2 convolutional layers, each convolutional layer contains 16 channels, and a jump connection is set between the input of the first convolutional layer and the output of the last convolutional layer.

[0083] Net-Y includes Head-Net-Y, In the third part, Head-Net-Y is used to obtain the initial representation coefficients in the 0th iteration stage. and dictionary In the tth iteration stage, by get pass get

[0084] Head-Net-Y uses the Head-Net network structure; the process of data processing in this module is as follows: input HR image, obtain the initialized representation coefficient and a dictionary initialized with 0

[0085] The Net-A network structure is used; the process of processing data in the tth iteration stage of this module is as follows: input the auxiliary variable obtained by formula (4) Hyperparameters, get the representation coefficients That is the solution of formula (5).

[0086] The Net-D network structure is used; the process of processing data in the tth iteration stage of this module is as follows: input the auxiliary variables solved by formula (8) Hyperparameters, get That is the solution of formula (9).

[0087] Net-F uses the Net-A network structure; the module processes data in the tth iteration stage as follows: input get And constrain the loss function by mapping the coefficients and The difference between Updated to The coefficient mapping loss function is: where θ F Indicates the parameters of the Net-F network, is the representation coefficient of the LR image in the t-th iteration stage, is the representation coefficient of the HR image at the t-th iteration stage, is the coefficient mapping function of the t-1th iteration stage, N is the number of batch samples, and the subscript i represents the i-th LR or HR image.

[0088] like Figure 5 As shown in the figure, the Hyper-Net network structure is as follows: it includes 4 convolutional layers, with a ReLu activation function between every two convolutional layers, a SoftPlus layer after the last convolutional layer, and each convolutional layer contains 256 channels; the process of processing data in the tth iteration stage of this module is: input noise standard deviation σ, and obtain hyperparameters.

[0089] Using the reconstruction loss function The difference between the constrained reconstructed image and the original HR image; where θ represents the parameters of the network, is the dictionary of HR images in the t-th iteration stage, and is the representation coefficient of the t-th stage LR image and HR image, * represents convolution, is the coefficient mapping function, Y i is the HR image, N is the number of batch samples, and the subscript i represents the i-th in the LR or HR image;

[0090] In this embodiment, when training a deep neural network for solving a semi-coupled deep convolutional dictionary learning model, the ADAM optimization method and back propagation are used to optimize and adjust the parameters in the network, and the initial learning rate is 10 -4 , the learning rate decay cycle is 20,000 times, and the number of training iterations is 40,000 times.

[0091] Step 4: Use the trained coefficient mapping function, dictionary and representation coefficients to reconstruct the low-resolution image to obtain a high-resolution image.

[0092] Input a LR image to be super-resolved into the trained model to obtain the representation coefficient of the LR image in the Tth iteration stage Then the trained coefficient mapping function After mapping and training Perform convolution to obtain the reconstructed high-resolution image:

[0093] The following table 1 and Figures 6 to 11 The present invention is further illustrated by the effect evaluation of the embodiments.

[0094] In this embodiment, Peak Signal-to-Noise Ratio (PSNR) and Structure Similarity (SSIM) are used as objective evaluation indicators of the results;

[0095] In this example, images from the CBSD68 dataset were used as test images. Table 1 lists the average PSNR and SSIM of the images obtained by this example and other advanced image super-resolution methods. As can be seen from Table 1, the results obtained by this example method are significantly improved compared to those of the other methods.

[0096] Table 1

[0097] Objective evaluation indicators Bicubic SRMD USRNet SwinIR The method of this embodiment PSNR 23.53 26.21 28.47 23.69 29.41 SSIM 0.6145 0.7693 0.8096 0.6084 0.8553

[0098] Figures 6 to 11 The following are the original HR image and the super-resolution images obtained using Bicubic, SRMD, USRNet, SwinIR, and the method of this embodiment. As can be seen from the figure, the resulting images of the Bicubic and SwinIR methods clearly have blurred details. The resulting images of the SRMD and USRnet methods have clearer details, but some local areas have blurred details. The resulting image of the method of this embodiment has richer high-frequency details. The visual effects and objective evaluation indicators of the results of the method of this embodiment show that the method of the present invention can achieve better image super-resolution reconstruction quality while taking into account the interpretability of the network.

[0099] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Any simple modification or equivalent change made to the above embodiment based on the technical essence of the present invention shall fall within the scope of protection of the present invention.

Claims

1. An image super-resolution reconstruction method based on semi-coupled deep convolutional dictionary learning, characterized in that: The image super-resolution reconstruction method comprises the following steps: S1, prepare the input training set data: degrade the HR image to obtain the corresponding LR image, and construct the training set {X i , Y i }, where X i is the LR image, Y i is a HR image, the subscript i represents the i-th LR image or HR image; LR represents low resolution, and HR represents high resolution; S2, build a semi-coupled deep convolutional dictionary learning model: In formula (1), X and Y represent the LR image and HR image, D x and D y are dictionaries of LR images and HR images respectively, A x 、A y is the representation coefficient corresponding to the LR image and the HR image, is the prior information of the coefficient, φ(·) is the prior information of the dictionary, * represents convolution, λ represents the regularization parameter, σ x , σ y 、 are the coefficients of each item, f y (·) is the coefficient mapping function between the LR image and the HR image representation coefficients; S3, build and train a deep neural network to solve the semi-coupled deep convolutional dictionary learning model. The deep neural network includes four modules: Net-X, Net-Y, Net-F, and Hyper-Net. Net-X is used to learn the representation coefficient A of the LR image. x Dictionary D corresponding to the LR image x , Net-Y is used to learn the representation coefficient A of HR images y Dictionary D corresponding to the HR image y , Net-F is used to train and learn A x to A y The coefficient mapping function f y (·) Hyper-Net is used to learn the hyperparameters in the model at each iteration stage. The coefficient mapping loss function is used to constrain the difference between the mapped representation coefficients and the representation coefficients of the HR image. The reconstruction loss function is used to constrain the difference between the predicted reconstructed image and the original HR image. The network model parameters are continuously adjusted until the model converges after T iterations. The model training is completed and the trained coefficient mapping network, dictionary and coefficients are obtained. S4: Input a LR image to be super-resolved into the trained model to obtain the representation coefficient of the LR image in the Tth iteration stage Then the trained coefficient mapping function After mapping and training Perform convolution to obtain the final reconstructed high-resolution image:

2. The image super-resolution reconstruction method based on semi-coupled deep convolutional dictionary learning according to claim 1 is characterized in that In step S1, the process of preparing input data includes the following sub-steps: S11, degrade the HR image using the following formula to obtain the corresponding LR image: X=(Y*k)↓ s +n Where X is the LR image, Y is the HR image, * is the convolution operation, k is the blur kernel, ↓ s represents s-fold downsampling, and n is additive white Gaussian noise; S12, performing null value upsampling on the LR image obtained by the degradation process so that the size thereof is consistent with the size of the corresponding HR image; S13, dividing the HR image and the LR image into image blocks of a preset size, collecting the divided images to obtain a training set, and using the training set to train the network in batches. The image block size and the number of batch samples are related to the device performance of the network carrier.

3. The image super-resolution reconstruction method based on semi-coupled deep convolutional dictionary learning according to claim 1 is characterized in that Introducing auxiliary variables The semi-quadratic splitting algorithm is used to iteratively solve equation (1). In the tth iteration stage, the solution process of equation (1) is to calculate equations (2) to (10) in sequence: in in Construct a deep neural network Net-F and calculate A according to formula (3) and formula (5) x and A y Training learning mapping function Make Among them, α x , α y , β x , β y , is the hyperparameter learned by the network Hyper-Net, vec represents the vectorization operation, that is, converting the matrix into a column vector, vec -1 represents the inverse quantization operation, unf represents the expansion of the matrix in the convolution operation into the matrix form of the product operation, F(·) and F -1 (·) denotes 2D Fourier transform and inverse transform, respectively, represents the complex conjugate of the 2D Fourier transform, Indicates element-by-element multiplication, and the division involved is also element-by-element division. In formula (3) and formula (5), Indicates that the solution is obtained by constructing a deep neural network Net-A In formula (7) and formula (9), Indicates that the solution is obtained by constructing a deep neural network Net-D The superscript (t) indicates the t-th iteration stage, and the superscript T is the matrix transpose.

4. The image super-resolution reconstruction method based on semi-coupled deep convolutional dictionary learning according to claim 3 is characterized in that The Net-X includes Head-Net-X, Three modules; Head-Net-X adopts the Head-Net network structure. The Head-Net network structure is: including at least 2 convolutional layers, with 1 ReLu activation function between each 2 convolutional layers, and each convolutional layer contains at least 64 channels; in the 0th iteration stage, the LR image, the blur kernel k of the degradation model, and the noise standard deviation σ are input to Head-Net-X to obtain the initialized representation coefficients and a dictionary initialized with 0 The Net-A network structure is adopted. The Net-A network structure is: including 7 modules, each module contains n r Residual units, each residual unit contains at least two convolutional layers and jump connections, with a ReLu activation function between every two convolutional layers. Stride convolution is used to downsample feature maps in the convolutional layers of the first three modules, and transposed convolution is used to upsample feature maps in the convolutional layers of the last three modules. The convolutional layers of the first four modules have 64, 128, 256, and 512 channels respectively. The process of processing data in the tth iteration stage is as follows: input the auxiliary variable obtained by formula (2) Hyperparameters, get the representation coefficients The Net-D network structure is adopted. The Net-D network structure is as follows: it includes at least 6 convolutional layers, with a ReLu activation function between every two convolutional layers, each convolutional layer contains at least 16 channels, and a skip connection is set between the input of the first convolutional layer and the output of the last convolutional layer; The process of processing data in the tth iteration stage is as follows: input the auxiliary variable obtained by formula (6) Hyperparameters, get dictionary 5. The image super-resolution reconstruction method based on semi-coupled deep convolutional dictionary learning according to claim 3 is characterized in that Net-Y includes Head-Net-Y, In the third part, Head-Net-Y is used to obtain the initial representation coefficients in the 0th iteration stage. and dictionary In the tth iteration stage, by get pass get Head-Net-Y uses the Head-Net network structure; the process of Head-Net-Y processing data is as follows: input HR image, get the initialized representation coefficient and a dictionary initialized with 0 Adopt Net-A network structure; The process of processing data in the tth iteration stage is as follows: input the auxiliary variable obtained by formula (4) Hyperparameters, get the representation coefficients Adopt Net-D network structure; The process of processing data in the tth iteration stage is as follows: input the auxiliary variables solved by formula (8) Hyperparameters, get 6. The image super-resolution reconstruction method based on semi-coupled deep convolutional dictionary learning according to claim 3 is characterized in that Net-F uses the Net-A network structure; the process of Net-F processing data in the tth iteration stage is as follows: input get And constrain the loss function by mapping the coefficients and The difference between Updated to 7. The image super-resolution reconstruction method based on semi-coupled deep convolutional dictionary learning according to claim 6 is characterized in that The coefficient mapping loss function is: where θ F Indicates the parameters of the Net-F network, is the representation coefficient of the LR image in the t-th iteration stage, is the representation coefficient of the HR image at the t-th iteration stage, is the coefficient mapping function of the t-1th iteration stage, N is the number of batch samples, and the subscript i represents the i-th LR image or HR image.

8. The image super-resolution reconstruction method based on semi-coupled deep convolutional dictionary learning according to claim 3 is characterized in that The Hyper-Net network structure is as follows: it includes at least 4 convolutional layers, contains a ReLu activation function between every two convolutional layers, and has a SoftPlus layer after the last convolutional layer; the process of Hyper-Net processing data in the tth iteration stage is: input the noise standard deviation σ to obtain the hyperparameter.

9. The image super-resolution reconstruction method based on semi-coupled deep convolutional dictionary learning according to claim 1, characterized in that The reconstruction loss function is: Where θ represents the parameters of the network, is the dictionary of HR images in the t-th iteration stage, and is the representation coefficient of the t-th stage LR image and HR image, * represents convolution, is the coefficient mapping function, Y i is the i-th HR image, and N is the number of batch samples.

Images