Image super-resolution reconstruction method of generative adversarial network based on LSTM dense connection

Abstract

The invention discloses an image super-resolution reconstruction method of a generative adversarial network based on LSTM dense connection, and the method comprises the steps: carrying out the residual connection of a multi-scale feature extraction module and an attention mechanism which are in dense connection, and constructing a generator of the attention mechanism; the method comprises the following steps: densely connecting a plurality of LSTM networks to construct a discriminator of a long short-term memory network; combining the generator of the attention mechanism and the discriminator of the long and short-term memory network to construct a generative adversarial network model; performing fine-tuning training on a generator of an attention mechanism and a discriminator of a long and short-term memory network in the generative adversarial network model by adopting a data set of a low-resolution image and a real high-resolution image; performing super-resolution reconstruction on the target image by adopting the trained generative adversarial network model to obtain a super-resolution image; the problem that the feature gradient disappears in the transmission process is avoided, and a generator is guided to generate a super-resolution image with richer details.

Description

Technical Field

[0001] The present invention relates to the technical field of image super-resolution reconstruction, and in particular to an image super-resolution reconstruction method based on a LSTM densely connected generative adversarial network. Background Art

[0002] In response to the problem of image analysis, researchers proposed to solve the problem through deep learning methods, which are mainly divided into three directions: target detection method based on lesions; image segmentation method based on lesions and image super-resolution method based on lesion features; target detection method based on lesions attempts to detect lesions through the Yolo series algorithm, but the target detection method implemented by Yolo is not accurate; image segmentation method based on lesions is mainly based on U-Net and improved algorithms, but this algorithm belongs to supervised learning and is heavily dependent on data labels; image super-resolution method based on lesion features mainly uses super-resolution technology, which increases the resolution of low-resolution lesion digital image space to obtain high-resolution methods. At present, it is mainly divided into methods based on deep convolution, methods based on wavelet transform and methods based on generative adversarial networks. Super-resolution methods based on deep convolutional neural networks mainly complete feature extraction of low-resolution images through deep convolutional networks, and then map them to the size of super-resolution images through upsampling to generate super-resolution images, but this method is prone to chessboard effect problems in two-dimensional images. Super-resolution methods based on wavelet transforms decompose low-resolution images by wavelet decomposition, extract high-frequency and low-frequency information, and then generate high-resolution images through interpolation and reconstruction processes. However, this method requires the selection of a wavelet basis suitable for a specific application, and different wavelet bases have a significant impact on the reconstruction effect. The super-resolution method based on the generative adversarial network uses a generative deep learning method, but it is less effective for sequence image data. Summary of the invention

[0003] Purpose of the invention: In order to overcome the shortcomings of the prior art, the present invention provides an image super-resolution reconstruction method based on an LSTM densely connected generative adversarial network, which integrates a densely connected multi-scale feature extraction module and a generator with an attention mechanism, and integrates a long short-term memory network LSTM and a densely connected discriminator to obtain a super-resolution image with richer details.

[0004] Technical solution: To achieve the above purpose, the image super-resolution reconstruction method based on LSTM densely connected generative adversarial network of the present invention comprises the following steps:

[0005] Step 1: Perform residual connection between the densely connected multi-scale feature extraction module and the attention mechanism to construct a generator of the attention mechanism;

[0006] Step 2: Construct a discriminator of the long short-term memory network by densely connecting several LSTM networks;

[0007] Step 3: Combine the generator of the attention mechanism and the discriminator of the long short-term memory network to construct a generative adversarial network model;

[0008] Step 4: Use low-resolution images and real high-resolution image datasets to fine-tune the generator of the attention mechanism and the discriminator of the long short-term memory network in the generative adversarial network model;

[0009] Step 5: Use the trained generative adversarial network model to perform super-resolution reconstruction on the target image to obtain a super-resolution image.

[0010] Furthermore, in the step one, the generator of the attention mechanism includes a first residual connection module, a second residual connection module, a third residual connection module and an attention mechanism AG module; the input end of the first residual connection module inputs the initial image, the output end of the first residual connection module is transmitted and connected to the input end of the second residual connection module, and the output end of the second residual connection module is transmitted and connected to the input end of the third residual connection module; the image data outputted from the output end of the third residual connection module is transmitted to one input end of the attention mechanism AG module after multiplying by the coefficient β, and the initial image is input to the other input end of the attention mechanism AG module; the output end of the attention mechanism AG module serves as the output end of the generator of the attention mechanism.

[0011] Furthermore, the first residual connection module, the second residual connection module and the third residual connection module have the same structure, and all include a densely connected multi-scale feature extraction module and an attention mechanism AG module; the input end of the multi-scale feature extraction module and one input end of the attention mechanism AG module are both used as input ends of the residual connection module, and the image data output from the output end of the multi-scale feature extraction module is transmitted to the other input end of the attention mechanism AG module after multiplying by a coefficient β, and the output end of the attention mechanism AG module is used as the output end of the residual connection module.

[0012] Furthermore, the multi-scale feature extraction module connects the low-resolution image to features of different scales through dense connections, and fuses features of different levels. The calculation process is as follows:

[0013] x l =F D (x l-1 )+x0+x1+…+x l-1

[0014] In the formula, x l is the output feature, x0 is the initial feature; F DRepresents the densely connected nonlinear structure of the current layer, including the Conv layer and the ReLU function.

[0015] Furthermore, the output feature x of the multi-scale feature extraction module is l The result of multiplying by the coefficient β and the initial feature x0 are used as the input feature G of the attention mechanism respectively. i and P i ; Input feature G i and P i After 1×1×1 convolution, the input feature G i and P i The corresponding position features are added, the negative features are suppressed by the ReLU function, and then the features are normalized by the Sigmoid function. i l With the initial feature P i Multiply each channel to get the new feature P after assigning weights l ; The calculation process is as follows:

[0016] P i l =Sigmoid(ReLu(W g T ×G i +W g T ×P i ))

[0017] P l =P i l *P i

[0018] Where W g and W g is the identity matrix, G i and P i Represents the low-order and high-order features of the i-layer input, P i is the initial feature, P l It is the feature output by the attention mechanism AG module.

[0019] Furthermore, the multi-scale feature extraction module includes a feature extraction module combining several Conv layers and ReLU functions; the several feature extraction modules are connected in sequence, and the input end of each feature extraction module in the several feature extraction modules is densely connected to the output end of its feature extraction module and the output ends of all the feature extraction modules connected thereto.

[0020] Furthermore, in the step 2, the discriminator of the long short-term memory network includes several LSTM network modules and a fully connected layer; the several LSTM network modules are connected in sequence, and the output end of each LSTM network module in the several LSTM network modules is densely connected with the output ends of all the LSTM network modules connected thereto; the input end of the first LSTM network in the several LSTM network modules is used as the input end of the discriminator of the long short-term memory network, and the output end of the last LSTM network module in the several LSTM network modules is transmitted to the input layer of the fully connected layer; the output layer of the fully connected layer normalizes the features through the Sigmoid function and outputs the output of the discriminator of the long short-term memory network.

[0021] Furthermore, the output results of several densely connected LSTM network modules are fused at different times. The calculation process is as follows:

[0022] O t =h t +h t-1 +h t-2 +h t-3 +…+h1

[0023] In the formula, O t Represents the output after dense connection, and all output results before and after time t are densely connected.

[0024] Beneficial effects: In the image super-resolution reconstruction method based on the LSTM densely connected generative adversarial network of the present invention, the generator of the attention mechanism merges low-order features with high-order features and increases the weights of important features; the discriminator of the long short-term memory network replaces the deep convolutional network with the long short-term memory densely connected network to improve the global feature extraction capability; at the same time, through the densely connected multi-scale feature extraction module and the discriminator of the long short-term memory network, the problem of feature gradient vanishing and gradient explosion that occurs during long-term transmission is avoided. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a flowchart of the image super-resolution reconstruction method based on the LSTM densely connected generative adversarial network;

[0026] Figure 2 To generate a structural diagram of the adversarial network model;

[0027] Figure 3 This is the structural diagram of the generator of the attention mechanism;

[0028] Figure 4 This is the structure diagram of the attention mechanism gate;

[0029] Figure 5 This is the structural diagram of the discriminator of the long short-term memory network;

[0030] Figure 6 This is a comparison image of the brain super-resolution reconstruction effect;

[0031] Figure 7 This is a comparison chart of the classification accuracy of the discriminator;

[0032] Figure 8 This is a comparison chart of the ablation experiment results based on the LSTM densely connected generative adversarial network. DETAILED DESCRIPTION

[0033] The present invention will be further described below in conjunction with the accompanying drawings.

[0034] like Figure 1 As shown in FIG. 1 , the image super-resolution reconstruction method based on the LSTM densely connected generative adversarial network includes the following steps:

[0035] Step 1: Perform residual connection between the densely connected multi-scale feature extraction module and the attention mechanism to construct a generator of the attention mechanism;

[0036] Step 2: Construct a discriminator of the long short-term memory network by densely connecting several LSTM networks;

[0037] Step 3: Combine the generator of the attention mechanism and the discriminator of the long short-term memory network to construct a generative adversarial network model;

[0038] Step 4: Preprocess the MRI brain data set to generate training set, test set and validation set respectively; use the data set of low-resolution images and real high-resolution images, that is, the training set, to fine-tune the generator of the attention mechanism and the discriminator of the long short-term memory network in the generative adversarial network model;

[0039] Step 5: Use the trained generative adversarial network model to super-reconstruct the target image to obtain a super-resolution image with richer details.

[0040] like Figure 2 As shown, in the generative adversarial network model, the low-resolution image is converted into a super-resolution image through the generator of the attention mechanism, and the super-resolution image generated by the generator of the attention mechanism is compared with the real high-resolution image in the judgement of the long short-term memory network to judge the super-resolution image that is closer to the reality; this method can generate more realistic detail features and can better process discrete image data; the judgement of the long short-term memory network guides the generator to generate ultra-high-resolution images with richer picture details.

[0041] like Figure 3As shown, in the step one, the generator of the attention mechanism includes a first residual connection module, a second residual connection module, a third residual connection module and an attention mechanism AG module; the input end of the first residual connection module inputs the initial image, the output end of the first residual connection module is transmitted and connected to the input end of the second residual connection module, and the output end of the second residual connection module is transmitted and connected to the input end of the third residual connection module; the image data outputted from the output end of the third residual connection module is transmitted to one input end of the attention mechanism AG module after multiplying by the coefficient β, and the initial image is input to the other input end of the attention mechanism AG module; the output end of the attention mechanism AG module serves as the output end of the generator of the attention mechanism, and the coefficient β is set according to demand.

[0042] like Figure 3 As shown, the first residual connection module, the second residual connection module and the third residual connection module have the same structure, and all include a multi-scale feature extraction module and an attention mechanism AG module for dense connection; the input end of the multi-scale feature extraction module and one input end of the attention mechanism AG module are both used as the input end of the residual connection module, and the image data outputted from the output end of the multi-scale feature extraction module is transmitted to the other input end of the attention mechanism AG module after being multiplied by the coefficient β, and the output end of the attention mechanism AG module is used as the output end of the residual connection module.

[0043] like Figure 3 As shown, the multi-scale feature extraction module connects the low-resolution image to different scale features through dense connection, and fuses the features of different levels. The calculation process is as follows:

[0044] x l =F D (x l-1 )+x0+x1+…+x l-1

[0045] In the formula, x l is the output feature, x0 is the initial feature; F D Represents the densely connected nonlinear structure of the current layer, including the Conv layer and the ReLU function.

[0046] like Figure 4 As shown, the output feature x of the multi-scale feature extraction module is l The result of multiplying by the coefficient β and the initial feature x0 are used as the input feature G of the attention mechanism respectively. i and P i ; Input feature G i and P i After 1×1×1 convolution, the input feature G i and P iThe corresponding position features are added, the negative features are suppressed by the ReLU function, a 1×1×1 convolution layer is passed, and then the features are normalized by the Sigmoid function. i l With the initial feature P i Multiply each channel to get the new feature P after assigning weights l ; The calculation process is as follows:

[0047] P i l =Sigmoid(ReLu(W g T ×G i +W g T ×P i ))

[0048] P l =P i l *P i

[0049] Where W g and W g is the unit matrix, used to adjust the feature size; G i and P i Represents the low-order and high-order features of the i-layer input, P i is the initial feature, P l It is the feature output by the attention mechanism AG module.

[0050] The multi-scale feature extraction module includes a feature extraction module combining several Conv layers and ReLU functions; the several feature extraction modules are connected in sequence, and the input end of each feature extraction module in the several feature extraction modules is densely connected to the output end of the feature extraction module and the output ends of the feature extraction modules used after it.

[0051] The generator of the attention mechanism not only includes three residual connection modules, but more residual connection modules can be added according to the needs to extract features from the image data; in the multi-scale feature extraction module Dense Block, several feature extraction modules obtained by combining the Conv layer and the ReLU function are connected in sequence, the input end of the first feature extraction module is used as the input end of the multi-scale feature extraction module, and the output end of the first feature extraction module is transmitted and connected to the input end of the second feature extraction module until the input end of the last feature extraction module is connected; the output end of the last feature extraction module is processed by a Conv layer and output as the output of the multi-scale feature extraction module. At the same time, the input end of the first feature extraction module is densely connected to the output end of the first feature extraction module, the output end of the second feature extraction module, and the output end of the last feature extraction module; the input end of the second feature extraction module is densely connected to the output end of the second feature extraction module, the output end of the third feature extraction module, and the output end of the last feature extraction module; and so on, the feature extraction modules connected in sequence are densely connected. The generator of the attention mechanism combines low-order features with high-order features and increases the weight of important features. At the same time, dense connections are used to ensure that features are continuously extracted in the multi-scale feature extraction module to prevent feature gradients from disappearing.

[0052] like Figure 5 As shown, in the step 2, the discriminator of the long short-term memory network includes several LSTM network modules and a fully connected layer; the several LSTM network modules are connected in sequence, and the output end of each LSTM network module in the several LSTM network modules is densely connected with the output ends of all the LSTM network modules connected thereto; the input end of the first LSTM network in the several LSTM network modules is used as the input end of the discriminator of the long short-term memory network, and the output end of the last LSTM network module in the several LSTM network modules is transmitted to the input layer of the fully connected layer; the output layer of the fully connected layer uses the Sigmoid function to normalize the features and outputs them as the output of the discriminator of the long short-term memory network. The LSTM is the long short-term memory network.

[0053] like Figure 5As shown, the input end of the first LSTM network module among several LSTM network modules inputs real high-resolution images and low-resolution images respectively, and generates super-resolution images through the generator of the attention mechanism; the output end of the first LSTM network module is connected to the input end of the second LSTM network module, and the output end of the second LSTM network module is connected to the third LSTM network module, and then connected to the last LSTM network module in sequence. The output end of the first LSTM network module is densely connected to the output end of the second LSTM network module, the output end of the third LSTM network module, and the output end of the last LSTM network module; similarly, the output end of the second LSTM network module is densely connected to the output end of the third LSTM network module, the output end of the fourth LSTM network module, and the output end of the last LSTM network module; and so on, the LSTM network modules connected in sequence are densely connected. The long short-term memory dense connection network replaces the deep convolutional network as the discriminator of the long short-term memory network to improve the global feature extraction capability.

[0054] like Figure 5 As shown in the figure, the long short-term memory dense connection network structure is mainly divided into forget gate, input gate, update gate, output gate and dense connection. According to the input sequence image features, its working principle is to input the image data into the long short-term memory model according to the time series, the super-resolution image sequence and the real high-resolution image sequence, and the current input data x t and the output value h at the previous moment t-1 Input into the forget gate, the calculation process is as follows:

[0055] f t =σ(W f ·[h t-1 ,x t ]+b f )

[0056] Where W f is the weight matrix of the forget gate, [h t-1 ,x t ] means connecting two vectors into a longer vector, b f is the bias term of the forget gate, and σ is the Sigmoid function.

[0057] The forget gate determines the cell state h at the previous moment t-1 The current moment is partially retained, so the forget gate can partially retain the state of the previous moment. The input gate calculates the unit state of the current input based on the previous output and this input. The calculation process is as follows:

[0058] i t =σ(W i ·[h t-1 ,xt ]+b i )

[0059] Where W i is the weight matrix of the input gate, [h t-1 ,x t ] means connecting two vectors into a longer vector, b i is the bias term of the input gate, and σ is the Sigmoid function.

[0060] The update gate determines the input of the cell at the current moment, and the cell state C at the current moment t The calculation process is as follows:

[0061]

[0062] In the formula, is the candidate memory cell state, W C is the weight matrix of the update gate, b C is the bias term of the update gate.

[0063] The current cell state C t The last cell state C t-1 Multiply the forget gate f element-wise t , and then use the current input candidate memory cell state Element-wise multiplication of the input gate i t , and finally add the two products. In this way, the candidate memory cell state of the current input can be and long-term memory C t-1 Combined together to form a new unit state C t Due to the control of the forget gate, the global feature information can be saved. At the same time, due to the control of the input gate, it can prevent the current irrelevant content from entering the memory. Finally, the output gate controls the current state C t Output state h to LSTM t , the calculation process is as follows:

[0064] o t =σ(W o ·[h t-1 ,x t ]+b o )

[0065] h t =o t *tanh(C t )

[0066] Where W o is the weight matrix of the update gate, b o is the bias term of the update gate.

[0067] The output results of several densely connected LSTM network modules are fused at different times. The calculation process is as follows:

[0068] O t =h t +h t-1 +h t-2 +h t-3 +…+h1

[0069] In the formula, O t Represents the output after dense connection, and all output results before and after time t are densely connected.

[0070] Example

[0071] The BraTs2021 dataset is a large-scale brain multimodal MRI glioma segmentation dataset, including 8160 MRI scans of 2040 patients; each patient contains MR images of four modalities: T1, T1Gd, T2, and T2-FLAIR. Since only the super-resolution reconstruction of medical imaging images is completed, T2-FLAIR modality data is selected, and the initial data type of the dataset is nii.gz. And it needs to be converted into PNG format. A nii.gz data sample can be divided into 155 PNG format images of size 240x240. After data processing and cleaning, 54,528 images are retained and divided into training set, test set, and validation set according to the ratio of 70%, 20%, and 10%.

[0072] like Figure 6 As shown in the figure, the comparison effect diagrams of different methods and the method of the present invention on the same data set are shown. Through comparative analysis, the LSTMAGAN of the present invention, the image super-resolution reconstruction method based on the generative adversarial network of LSTM dense connection and attention mechanism, is compared with other methods such as Bicubic, SRGAN, MedGAN, ESRGAN, TGAN, DISGAN and other methods. The LSTMAGAN method of the present invention has better effect than other methods in evaluation indicators such as PNSR peak signal-to-noise ratio and SSIM structural similarity index. Figure 6 It can be seen that in the super-high resolution images obtained by super-resolution reconstruction of the three images, both the PNSR and SSIM values ​​of the method of this scheme are higher than those of other methods. The quality of the super-resolution image obtained by the LSTMAGAN method of the present invention is better, which can prove the superiority of the present invention.

[0073] like Figure 7As shown in the figure, in order to verify the classification effect of the long short-term memory dense connection structure in the discriminator, the present invention applies different comparison methods to the cranial brain data set; the classification effect of the discriminator is measured by the discrimination accuracy. Figure 7 It is shown in the figure that the classification accuracy of the discriminator of the method proposed in the present invention is gradually improved during the iteration process during the training process, and the highest accuracy value is achieved after 100 rounds of training; Figure 7 It is shown in the figure that when the number of training times is greater, the accuracy of the discriminator of the long short-term memory network of the present invention is higher than that of the discriminator of other methods. It can be clearly shown that after reaching the required number of training times, the discriminator accuracy of the long short-term memory network of this scheme is higher.

[0074] like Figure 8 As shown in the figure, the LSTMAGAN in this scheme is compared with the image super-resolution reconstruction method based on the generative adversarial network of LSTM dense connection and attention mechanism by ablation experiment; Figure 8 From left to right in the figure, four experiments are performed in order: ablation of LSTM long short-term memory network and Attention mechanism in the generative adversarial network; retention of LSTM long short-term memory network and ablation of Attention mechanism; ablation of LSTM long short-term memory network and retention of Attention mechanism; retention of LSTM long short-term memory network and Attention mechanism. The results are evaluated by PNSR and SSIM evaluation indicators. Figure 8 It can be seen that whether adding and retaining the LSTM long short-term memory network or the Attention mechanism will improve the PNSR and SSIM evaluation indicators of the images generated by the generative adversarial network that does not have either of them. The combination of the LSTM long short-term memory network and the Attention mechanism performs the generation of the generator and the discrimination of the discriminator, guiding the generative adversarial network to generate better images, and the PNSR and SSIM evaluation indicators are significantly improved compared with the generative adversarial network that does not have either of them. Therefore, it can be obtained that the super-resolution image quality obtained by the combination of the LSTM long short-term memory network and the Attention mechanism is better; the method of the present invention is more effective.

[0075] The above is only a description of the preferred embodiments of the present invention. Ordinary technicians in this technical field can make several modifications and optimizations based on the above disclosure without departing from the above basic principles. These improvements and optimizations should be regarded as the understood protection scope of the present invention.

Claims

1. An image super-resolution reconstruction method based on a LSTM densely connected generative adversarial network, characterized by: The following steps are involved: Step 1: Perform residual connection between the densely connected multi-scale feature extraction module and the attention mechanism to construct a generator of the attention mechanism; Step 2: Construct a discriminator of the long short-term memory network by densely connecting several LSTM networks; Step 3: Combine the generator of the attention mechanism and the discriminator of the long short-term memory network to construct a generative adversarial network model; Step 4: Use low-resolution images and real high-resolution image datasets to fine-tune the generator of the attention mechanism and the discriminator of the long short-term memory network in the generative adversarial network model; Step 5: Use the trained generative adversarial network model to perform super-resolution reconstruction on the target image to obtain a super-resolution image.

2. The image super-resolution reconstruction method based on the LSTM densely connected generative adversarial network according to claim 1 is characterized in that: In the step one, the generator of the attention mechanism includes a first residual connection module, a second residual connection module, a third residual connection module and an attention mechanism AG module; the input end of the first residual connection module inputs the initial image, the output end of the first residual connection module is transmitted and connected to the input end of the second residual connection module, and the output end of the second residual connection module is transmitted and connected to the input end of the third residual connection module; the image data outputted from the output end of the third residual connection module is transmitted to one input end of the attention mechanism AG module after multiplying by the coefficient β, and the initial image is input to the other input end of the attention mechanism AG module; the output end of the attention mechanism AG module serves as the output end of the generator of the attention mechanism.

3. The image super-resolution reconstruction method based on the LSTM densely connected generative adversarial network according to claim 2 is characterized in that: The first residual connection module, the second residual connection module and the third residual connection module have the same structure, and all include a densely connected multi-scale feature extraction module and an attention mechanism AG module; the input end of the multi-scale feature extraction module and one input end of the attention mechanism AG module are both used as the input end of the residual connection module, and the image data outputted from the output end of the multi-scale feature extraction module is transmitted to the other input end of the attention mechanism AG module after being multiplied by the coefficient β, and the output end of the attention mechanism AG module is used as the output end of the residual connection module.

4. The image super-resolution reconstruction method based on the LSTM densely connected generative adversarial network according to claim 3 is characterized in that: The multi-scale feature extraction module connects the low-resolution image to different scale features through dense connection, and fuses the features of different levels. The calculation process is as follows: x l =F D (x l-1 )+x0+x1+…+x l-1 In the formula, x l is the output feature, x0 is the initial feature; F D Represents the densely connected nonlinear structure of the current layer, including the Conv layer and the ReLU function.

5. The image super-resolution reconstruction method based on the LSTM densely connected generative adversarial network according to claim 3 is characterized in that: The output feature x of the multi-scale feature extraction module l The result of multiplying by the coefficient β and the initial feature x0 are used as the input feature G of the attention mechanism respectively. i and P i ; Input feature G i and P i After 1×1×1 convolution, the input feature G i and P i The corresponding position features are added, the negative features are suppressed by the ReLU function, and then the features are normalized by the Sigmoid function. i l With the initial feature P i Multiply each channel to get the new feature P after assigning weights l ; The calculation process is as follows: P i l =Sigmoid(ReLu(W g T ×G i +W g T ×P i )) P l =P i l *P i Where W g and W g is the identity matrix, G i and P i Represents the low-order and high-order features of the i-layer input, P i is the initial feature, P l It is the feature output by the attention mechanism AG module.

6. The image super-resolution reconstruction method based on the LSTM densely connected generative adversarial network according to claim 3 is characterized in that: The multi-scale feature extraction module includes a feature extraction module combining several Conv layers and ReLU functions; Several feature extraction modules are connected in sequence, and the input end of each feature extraction module in the several feature extraction modules is densely connected with the output end of the feature extraction module and the output ends of the feature extraction modules connected thereto.

7. The image super-resolution reconstruction method based on the LSTM densely connected generative adversarial network according to claim 1 is characterized in that: In the step 2, the discriminator of the long short-term memory network includes a plurality of LSTM network modules and a fully connected layer; the plurality of LSTM network modules are connected in sequence, and the output end of each LSTM network module in the plurality of LSTM network modules is densely connected to the output ends of all LSTM network modules connected thereto; the input end of the first LSTM network in the plurality of LSTM network modules is used as the input end of the discriminator of the long short-term memory network, and the output end of the last LSTM network module in the plurality of LSTM network modules is transmitted to the input layer of the fully connected layer; the output layer of the fully connected layer normalizes the features through a Sigmoid function and outputs the output of the discriminator of the long short-term memory network.

8. The image super-resolution reconstruction method based on the LSTM densely connected generative adversarial network according to claim 7 is characterized in that: The output results of several densely connected LSTM network modules are fused at different times. The calculation process is as follows: O t =h t +h t-1 +h t-2 +h t-3 +…+h1 In the formula, O t Represents the output after dense connection, and all output results before and after time t are densely connected.

Images