A super-resolution reconstruction method, apparatus, and storage medium
By performing feature extraction and model training on the training set images, and combining depthwise separable convolution and attention modules, the problem of large network depth and number of parameters in image super-resolution reconstruction in existing technologies is solved, achieving efficient image feature information acquisition and resolution enhancement.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-07-21
- Publication Date
- 2026-03-13
AI Technical Summary
Existing image super-resolution technologies suffer from problems such as excessively deep network structures, large number of parameters, inability to identify key information in the spatial domain, and neglect of local information in the channel domain, resulting in poor image feature extraction and classification performance.
Feature extraction and analysis are performed using multiple training images to build a training model. Image reconstruction is performed through an encoder and decoder. Feature information is enhanced by using depthwise separable convolutional layers, channel attention modules, and spatial attention modules. The model is optimized by combining a loss function to achieve super-resolution reconstruction.
It improves the range of image feature information acquisition and feature information enhancement, reduces the number of model parameters, and improves the image resolution reconstruction effect.
Smart Images
Figure CN115409700B_ABST
Abstract
Description
Technical Field
[0001] This invention relates primarily to the field of image processing technology, and more specifically to a super-resolution reconstruction method, apparatus, and storage medium. Background Technology
[0002] Image super-resolution (SR) technology aims to obtain a clear high-resolution (HR) image from a single low-resolution (LR) image input. As a classic image restoration problem, SR remains a challenging research area in machine vision. Currently, image super-resolution (SR) techniques mainly rely on interpolation, reconstruction, and learning methods to improve image resolution. However, interpolation methods do not consider image degradation models, which significantly reduces the degree of image restoration; while reconstruction methods consider image degradation models, they have certain uncertainties. Learning-based methods utilize a large amount of training data to obtain the correspondence between low-resolution and high-resolution images, and by learning the mapping relationship, the SR rate of images is greatly improved.
[0003] Current image super-resolution techniques based on learning methods generally suffer from problems such as excessively deep network structures and a large number of parameters. Furthermore, the spatial domain in attention mechanisms cannot identify key information (edges), and the channel domain ignores the local information of each channel. Therefore, there is still room for improvement in areas such as feature classification and parameter complexity reduction, from low-level feature extraction to high-level feature extraction. Summary of the Invention
[0004] The technical problem to be solved by the present invention is to provide a super-resolution reconstruction method, apparatus and storage medium to address the shortcomings of the prior art.
[0005] The technical solution of the present invention to solve the above-mentioned technical problems is as follows: A super-resolution reconstruction method, comprising the following steps:
[0006] S1: Import multiple training set images and multiple test set images, and perform feature extraction and analysis on each of the training set images to obtain the features of the image to be trained corresponding to each of the training set images;
[0007] S2: Construct a training model by training the training model using multiple features of the images to be trained, and obtain the trained model;
[0008] S3: Test the trained model based on all the test set images to obtain the super-resolution reconstruction model;
[0009] S4: Import the image to be reconstructed, reconstruct the image using the super-resolution reconstruction model to obtain a super-resolution reconstructed image, and use the super-resolution reconstructed image as the super-resolution reconstruction result.
[0010] Another technical solution of the present invention to solve the above-mentioned technical problems is as follows: A super-resolution reconstruction device, comprising:
[0011] The feature extraction and analysis module is used to import multiple training set images and multiple test set images, and perform feature extraction and analysis on each of the training set images to obtain the features of the image to be trained corresponding to each of the training set images.
[0012] The model training module is used to construct a training model and train the training model using multiple features of the images to be trained to obtain the trained model.
[0013] The model testing module is used to test the trained model based on all test set images to obtain the super-resolution reconstruction model.
[0014] The super-resolution reconstruction result acquisition module is used to import the image to be reconstructed, reconstruct the image to be reconstructed using the super-resolution reconstruction model, obtain the super-resolution reconstructed image, and use the super-resolution reconstructed image as the super-resolution reconstruction result.
[0015] Another technical solution of the present invention to solve the above-mentioned technical problems is as follows: a super-resolution reconstruction device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the super-resolution reconstruction method described above is implemented.
[0016] Another technical solution of the present invention to solve the above-mentioned technical problems is as follows: a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the super-resolution reconstruction method as described above.
[0017] The beneficial effects of this invention are as follows: by extracting and analyzing the features of the training set images, the features of the image to be trained are obtained; by training the training model with the features of the image to be trained, the trained model is obtained; by testing the trained model with the test set images, the super-resolution reconstruction model is obtained; and by reconstructing the image of the image to be reconstructed using the super-resolution reconstruction model, the super-resolution reconstruction result is obtained. This invention has the advantages of wide image feature information acquisition range, enhanced feature information, low parameter quantity, and good image resolution improvement effect. While improving the image feature information acquisition range, it also enhances the feature attributes of the information. Attached Figure Description
[0018] Figure 1 A flowchart illustrating a super-resolution reconstruction method provided in an embodiment of the present invention;
[0019] Figure 2 A schematic diagram illustrating the process of constructing and training a model according to an embodiment of the present invention;
[0020] Figure 3 This is a schematic diagram illustrating the steps of a super-resolution reconstruction method provided in an embodiment of the present invention;
[0021] Figure 4 This is a block diagram of a super-resolution reconstruction device provided in an embodiment of the present invention. Detailed Implementation
[0022] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.
[0023] Figure 1 This is a flowchart illustrating a super-resolution reconstruction method provided in an embodiment of the present invention.
[0024] like Figure 1 As shown, a super-resolution reconstruction method includes the following steps:
[0025] S1: Import multiple training set images and multiple test set images, and perform feature extraction and analysis on each of the training set images to obtain the features of the image to be trained corresponding to each of the training set images;
[0026] S2: Construct a training model by training the training model using multiple features of the images to be trained, and obtain the trained model;
[0027] S3: Test the trained model based on all the test set images to obtain the super-resolution reconstruction model;
[0028] S4: Import the image to be reconstructed, reconstruct the image using the super-resolution reconstruction model to obtain a super-resolution reconstructed image, and use the super-resolution reconstructed image as the super-resolution reconstruction result.
[0029] It should be understood that a common dataset for training (i.e., multiple training set images and multiple test set images) is determined, and shallow feature information of low-resolution images is obtained.
[0030] It should be understood that the common datasets used (i.e., the plurality of training set images and the plurality of test set images) include DIV2K, Set5, Set14, B100, Urban100 and Manga109, wherein DIV2K is used as the training set (i.e., the plurality of training set images) and Set5, Set14, B100, Urban100 and Manga109 are used as the test set (i.e., the plurality of test set images).
[0031] Specifically, a common dataset for training (i.e., multiple training set images and multiple test set images) is determined. The datasets (i.e., multiple training set images and multiple test set images) include DIV2K, Set5, Set14, B100, Urban100, and Manga109, where DIV2K contains 800 training images (i.e., multiple training set images), and Set5, Set14, B100, Urban100, and Manga109 contain 328 test images (i.e., multiple test set images). The 800 images in DIV2K are processed by flipping, cropping, etc., to obtain 3000 training images for model training. Set5, Set14, B100, Urban100, and Manga109 are used as test datasets, each containing one high-resolution image and one low-resolution image downsampled by 2x, 3x, or 4x.
[0032] In the above embodiments, the features of the image to be trained are obtained by feature extraction and analysis of the training set images. The trained model is obtained by training the training model using the features of the image to be trained. The super-resolution reconstruction model is obtained by testing the trained model using the test set images. The super-resolution reconstruction result is obtained by reconstructing the image to be reconstructed using the super-resolution reconstruction model. This method has the advantages of wide image feature information acquisition range, enhanced feature information, low parameter quantity, and good image resolution improvement effect. While improving the image feature information acquisition range, it also enhances the feature attributes of the information.
[0033] Optionally, as an embodiment of the present invention, step S1, which involves performing feature extraction and analysis on each of the training set images to obtain the features of the image to be trained corresponding to each of the training set images, includes:
[0034] Low-resolution features are extracted from each of the training set images using a convolutional neural network to obtain low-resolution features corresponding to each of the training set images.
[0035] The first formula is used to extract the training image features from each of the low-resolution features, thereby obtaining the training image features corresponding to each of the training set images.
[0036] H0 = f dsc (I LR ),
[0037] Where H0 represents the features of the image to be trained, f dsc () represents a depthwise convolutional layer, I LR This is a low-resolution feature.
[0038] It should be understood that depthwise separable convolutional layers are used to extract shallow feature information. A depthwise separable convolution is a (3×3) kernel-sized convolutional layer, and its expression is:
[0039] H0 = f dsc (I LR )
[0040] Among them, f dsc () represents a depthwise convolutional layer, I LR H0 is the low-resolution feature (i.e., the low-resolution feature), and H0 is the extracted feature information (i.e., the image feature to be trained).
[0041] In the above embodiments, low-resolution features are obtained by extracting low-resolution features from training set images through a convolutional neural network, and the features of the training image are obtained by extracting the low-resolution features from the training image features through the first formula. These features are used to obtain shallow feature information of low-resolution images, thereby improving the range of image feature information acquisition and enhancing the feature attributes of the information.
[0042] Optionally, as an embodiment of the present invention, the training model includes an encoder and a decoder, and the process of step S2 includes:
[0043] The encoder encodes the features of each of the images to be trained, thereby obtaining the encoded features corresponding to each of the training set images, specifically as follows:
[0044] The second formula is used to encode each of the features of the image to be trained, thereby obtaining the encoded image features corresponding to each of the training set images. The second formula is:
[0045] H(H0)=F(H0)+H0,
[0046] Where H(H0) represents the encoded image features, H0 represents the image features to be trained, and F(H0) represents the expectation function;
[0047] The decoder decodes each of the encoded image features to obtain the decoded image features corresponding to each of the training set images.
[0048] The training model is subjected to loss analysis based on all decoded image features and all training image features to obtain the trained model.
[0049] It should be understood that the encoder is built based on residual blocks.
[0050] Specifically, the residual blocks are standard residual blocks used on ResNets. By introducing richer target information through the fitting function, the network learns more comprehensive content. The expression for the fitting function is:
[0051] H(x)=F(x)+x
[0052] Where F(x) is the expectation function, which is a mapping relationship composed of two convolutional layers; x is the input feature information (i.e., the features of the image to be trained).
[0053] In the above embodiments, encoded image features are obtained by encoding the features of the image to be trained using the second method, and decoded image features are obtained by decoding the encoded image features using the decoder. The trained model is obtained by analyzing the loss value of the training model based on all the decoded image features and all the features of the image to be trained. This introduces richer target information, enabling the network to learn more content. It has advantages such as a wide range of image feature information acquisition, enhanced feature information, low parameter quantity, and good image resolution improvement effect. While improving the range of image feature information acquisition, it also enhances the feature attributes of the information.
[0054] Optionally, as an embodiment of the present invention, such as Figures 1 to 2 As shown, the decoder includes a first 3×3 convolutional layer, a channel attention module, a spatial attention module, and a first 1×1 convolutional layer. The process of decoding each of the encoded image features to obtain the decoded image features corresponding to each of the training set images includes:
[0055] The first 3×3 convolutional layer is used to extract features from each of the encoded image features to obtain the extracted image features corresponding to each of the training set images.
[0056] The channel attention module performs feature fusion analysis on each of the extracted features to obtain the first fusion feature corresponding to each of the training set images.
[0057] The spatial attention module performs spatial attention feature fusion analysis on each of the extracted image features to obtain a second fusion feature corresponding to each of the training set images.
[0058] The first 1×1 convolutional layer merges each of the first fusion features and the second fusion features corresponding to each of the training set images to obtain the merged features corresponding to each of the training set images.
[0059] Each of the merged features is upsampled to obtain the decoded image features corresponding to each of the training set images.
[0060] It should be understood that the decoder is constructed by fusing spatial domain attention (i.e., the spatial attention module) and channel domain attention (i.e., the channel attention module).
[0061] It should be understood that the channel attention module and the spatial attention module are designed using the interdependence between channels and the spatial location of their features.
[0062] Specifically, such as Figure 2 As shown, during the model's decoding process, a (3×3) convolutional layer (i.e., the first 3×3 convolutional layer) is first used to refine the image features (i.e., the encoded image features) to further enhance them. Then, the refined feature information (i.e., the encoded image features) is divided into two parts and input into the spatial attention module and the channel attention module, respectively. After the feature information (i.e., the encoded image features) is enhanced by the spatial attention module and the channel attention module, the feature descriptors output by the two attention mechanisms (i.e., the first fused feature and / or the second fused feature) are fused together to adaptively adjust the feature representation. Finally, the features (i.e., the merged features) are upsampled based on PixelShuffle in the sub-pixel convolutional module.
[0063] Specifically, to recover high-level details of the SR, the network is more discriminative of different local regions, including those that are more important and harder to reconstruct. The model designs channel attention modules and spatial attention modules by leveraging the interdependencies between channels and the spatial location of features. The spatial attention module and channel attention module use two different pools. For a W×H×C feature map (i.e., the image features after feature extraction), the channel module receives 1×1×C attention weights, and the spatial module receives W×H×1 attention weights.
[0064] In the above embodiments, the decoded image features are obtained by decoding the encoded image features through the decoder. The image features are further enhanced by utilizing the interdependence between channels and the spatial location of the features, and the features can be adaptively adjusted.
[0065] Optionally, as an embodiment of the present invention, such as Figures 1 to 2 As shown, the channel attention module includes a second 1×1 convolutional layer, a first ReLU activation function layer, and a second 3×3 convolutional layer. The process of performing channel attention feature fusion analysis on each of the extracted image features through the channel attention module to obtain the first fused features corresponding to each of the training set images includes:
[0066] The number of image channels is adjusted for each of the extracted features according to the preset number of channels, so as to obtain the image channel-adjusted features corresponding to each of the training set images;
[0067] The second 1×1 convolutional layer extracts features from the adjusted features of each of the image channels to obtain the first feature-extracted features corresponding to each of the training set images.
[0068] The first ReLU activation function layer is used to normalize each of the first features extracted from the first feature, so as to obtain the first normalized features corresponding to each of the training set images.
[0069] The second 3×3 convolutional layer is used to extract features from each of the feature-extracted image features to obtain the second feature-extracted features corresponding to each of the training set images.
[0070] The first normalized features and the second features extracted from each of the training set images are fused to obtain the first fused features corresponding to each of the training set images.
[0071] It should be understood that adjusting the number of image channels refers to adjusting the number of channels in an image. This can be done by reducing the number of channels in the image. For example, a grayscale image has 1 channel and a color image has 3 channels. Reducing the number of channels in the color image to 1 would be considered adjusting the number of image channels.
[0072] In the above embodiments, the first fused feature is obtained by fusing and analyzing the channel attention features of the image features after each feature is extracted by the channel attention module. The image features are further enhanced by utilizing the interdependence between channels and the spatial position of features, and the features can be adaptively adjusted.
[0073] Optionally, as an embodiment of the present invention, such as Figures 1 to 2 As shown, the spatial attention module includes a third 1×1 convolutional layer, a second ReLU activation function layer, and a third 3×3 convolutional layer. The process of performing spatial attention feature fusion analysis on each of the extracted image features through the spatial attention module to obtain the second fused features corresponding to each of the training set images includes:
[0074] The image size of each extracted feature is adjusted according to the preset width and preset height to obtain the image size adjusted feature corresponding to each training set image;
[0075] The third 1×1 convolutional layer is used to extract features from each of the image size-adjusted features to obtain the third feature-extracted features corresponding to each of the training set images.
[0076] The second ReLU activation function layer is used to normalize each of the extracted third features to obtain the second normalized features corresponding to each of the training set images.
[0077] The third 3×3 convolutional layer extracts features from each of the feature-extracted image features to obtain the fourth feature-extracted features corresponding to each of the training set images.
[0078] The second normalized features and the fourth features extracted from each of the training set images are fused to obtain the second fused features corresponding to each of the training set images.
[0079] It should be understood that adjusting the image size can be understood as follows: For example, if an image is 1600*1200, with a width of 1600 and a height of 1200, and the preset width is 1280 and the preset height is 768, then adjusting the 1600*1200 image to a 1280*768 image is an adjustment of the image size.
[0080] In the above embodiments, the spatial attention feature fusion analysis of the image features after feature extraction is performed by the spatial attention module to obtain the second fused feature. The image features are further enhanced by utilizing the interdependence between channels and the spatial position of the features, and the features can be adaptively adjusted.
[0081] Optionally, as an embodiment of the present invention, the process of performing loss value analysis on the training model based on all decoded image features and all training image features to obtain the trained model includes:
[0082] The loss function corresponding to each training set image is obtained by calculating the loss function of each decoded image feature and the corresponding training set image feature using the third equation. The third equation is as follows:
[0083]
[0084] in,
[0085] Where, diff = ||output - GT||,
[0086] Where loss is the loss function, diff is the error value between the decoded image features and the training image features, thresh is the threshold set by the alpha function, output is the decoded image features, and GT is the training image features;
[0087] The parameters of the training model are updated according to all the loss functions to obtain the updated training model. The updated training model is then used as the training model, and the process returns to step S1 until the preset number of iterations is reached. The updated training model is then used as the post-trained model.
[0088] It should be understood that the loss function outputs a reconstructed high-resolution image.
[0089] It should be understood that the loss function of the present invention can increase the penalty for incorrectly predicted gray pixels and decrease the penalty for correct predictions; that is, the greater the deviation from the expected value, the greater the penalty should be; the closer to the expected value, the smaller the penalty should be.
[0090] Specifically, the expression for the loss function is:
[0091] diff = ||output-GT||
[0092]
[0093]
[0094] Where, diff is the error value between the output image features (i.e., the decoded image features) and the original image features GT (i.e., the training image features), and thresh is the threshold set by the alpha function.
[0095] In the above embodiments, the trained model is obtained by analyzing the loss value of the training model using all decoded image features and all training image features. This can increase the penalty for incorrectly predicted gray pixels and reduce the penalty for correct predictions.
[0096] Optionally, as another embodiment of the present invention, ablation experiments are conducted on the fusion attention and loss function, and the reconstruction method is evaluated. For simplicity, the present invention only trains on 800 training sets in the DIV2K database to verify that the channel fusion method and the constructed loss function are useful for improving image super-resolution reconstruction. Finally, the method of the present invention is compared with state-of-the-art super-resolution reconstruction networks on five benchmark datasets (Set5, Set14, B100, Urban100, and Manga109) to verify that the method of the present invention has good image SR reconstruction performance while maintaining a small number of model parameters.
[0097] Specifically, ablation experiments were conducted on the fusion attention and loss function to evaluate the reconstruction method. Only 800 training sets from the DIV2K database were used to verify the usefulness of the channel fusion method and the constructed loss function in improving image super-resolution reconstruction. Finally, the method of this invention was compared with state-of-the-art super-resolution reconstruction networks on five benchmark datasets (Set5, Set14, B100, Urban100, and Manga109) to verify that the method of this invention achieves good image SR reconstruction results while maintaining a small number of model parameters.
[0098] Alternatively, as another embodiment of the present invention, the present invention uses the shallow features of low-resolution images as input to the model, and enhances the image features by fusing spatial attention modules and channel attention modules based on the residual module. The number of model parameters is reduced based on the residual structure, and the model is adaptively adjusted by using the constructed loss function, thereby achieving high-quality reconstruction of image resolution.
[0099] Alternatively, as another embodiment of the present invention, such as Figure 3 As shown, this invention determines a common dataset for training; acquires shallow feature information from low-resolution images; constructs an encoder based on residual blocks; fuses spatial domain attention and channel domain attention; constructs a loss function to output the reconstructed high-resolution image; conducts ablation experiments on the fused attention and loss function; and evaluates the reconstruction method. This invention has advantages such as a wide range of image feature information acquisition, enhanced feature information, low parameter count, and good image resolution improvement. While increasing the range of image feature information acquisition, it also enhances the feature attributes of the information.
[0100] Figure 4 This is a block diagram of a super-resolution reconstruction device provided in an embodiment of the present invention.
[0101] Alternatively, as another embodiment of the present invention, such as Figure 4 As shown, a super-resolution reconstruction device includes:
[0102] The feature extraction and analysis module is used to import multiple training set images and multiple test set images, and perform feature extraction and analysis on each of the training set images to obtain the features of the image to be trained corresponding to each of the training set images.
[0103] The model training module is used to construct a training model and train the training model using multiple features of the images to be trained to obtain the trained model.
[0104] The model testing module is used to test the trained model based on all test set images to obtain the super-resolution reconstruction model.
[0105] The super-resolution reconstruction result acquisition module is used to import the image to be reconstructed, reconstruct the image to be reconstructed using the super-resolution reconstruction model, obtain the super-resolution reconstructed image, and use the super-resolution reconstructed image as the super-resolution reconstruction result.
[0106] Optionally, another embodiment of the present invention provides a super-resolution reconstruction apparatus, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the super-resolution reconstruction method as described above. This apparatus may be a computer or similar device.
[0107] Optionally, another embodiment of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the super-resolution reconstruction method as described above.
[0108] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.
[0109] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described apparatus and unit can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.
[0110] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.
[0111] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of the present invention, depending on actual needs.
[0112] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0113] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0114] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A super-resolution reconstruction method, characterized in that, Includes the following steps: S1: Import multiple training set images and multiple test set images, and perform feature extraction and analysis on each of the training set images to obtain the features of the image to be trained corresponding to each of the training set images; S2: Construct a training model by training the training model using multiple features of the images to be trained, and obtain the trained model; S3: Test the trained model based on all the test set images to obtain the super-resolution reconstruction model; S4: Import the image to be reconstructed, reconstruct the image using the super-resolution reconstruction model to obtain a super-resolution reconstructed image, and use the super-resolution reconstructed image as the super-resolution reconstruction result; The training model includes an encoder and a decoder, and step S2 includes the following: The encoder encodes the features of each of the images to be trained, thereby obtaining the encoded features corresponding to each of the training set images, specifically as follows: The second formula is used to encode each of the features of the image to be trained, thereby obtaining the encoded image features corresponding to each of the training set images. The second formula is: H ( )= F ( )+ ， in, H ( ) represents the encoded image features. For the features of the image to be trained, F ( Let ) be the expectation function. F ( The mapping relationship consists of two convolutional layers. The decoder decodes each of the encoded image features to obtain the decoded image features corresponding to each of the training set images. Based on all the features of the decoded images and all the features of the images to be trained, the training model is analyzed for loss value to obtain the trained model. The process of performing loss value analysis on the training model based on all decoded image features and all training image features to obtain the trained model includes: The loss function corresponding to each training set image is obtained by calculating the loss function of each decoded image feature and the corresponding training set image feature using the third equation. The third equation is as follows: ， in, , in, , in, For loss function, This represents the error value between the decoded image features and the features of the image to be trained. The threshold set for the alpha function, For the features of the decoded image, Features of the image to be trained; The parameters of the training model are updated according to all the loss functions to obtain the updated training model. The updated training model is then used as the training model, and the process returns to step S1 until the preset number of iterations is reached. The updated training model is then used as the post-trained model.
2. The super-resolution reconstruction method according to claim 1, characterized in that, In step S1, the process of performing feature extraction and analysis on each of the training set images to obtain the features of the training image corresponding to each of the training set images includes: Low-resolution features are extracted from each of the training set images using a convolutional neural network to obtain low-resolution features corresponding to each of the training set images. The first formula is used to extract the training image features from each of the low-resolution features, thereby obtaining the training image features corresponding to each of the training set images. ， in, For the features of the image to be trained, It is a deep convolutional layer. This is a low-resolution feature.
3. The super-resolution reconstruction method according to claim 1, characterized in that, The decoder includes a first 3×3 convolutional layer, a channel attention module, a spatial attention module, and a first 1×1 convolutional layer. The process of decoding each of the encoded image features using the decoder to obtain the decoded image features corresponding to each of the training set images includes: The first 3×3 convolutional layer is used to extract features from each of the encoded image features to obtain the extracted image features corresponding to each of the training set images. The channel attention module performs feature fusion analysis on each of the extracted features to obtain the first fusion feature corresponding to each of the training set images. The spatial attention module performs spatial attention feature fusion analysis on each of the extracted image features to obtain a second fusion feature corresponding to each of the training set images. The first 1×1 convolutional layer merges each of the first fusion features and the second fusion features corresponding to each of the training set images to obtain the merged features corresponding to each of the training set images. Each of the merged features is upsampled to obtain the decoded image features corresponding to each of the training set images.
4. The super-resolution reconstruction method according to claim 3, characterized in that, The channel attention module includes a second 1×1 convolutional layer, a first ReLU activation function layer, and a second 3×3 convolutional layer. The process of performing channel attention feature fusion analysis on each of the extracted image features through the channel attention module to obtain the first fused feature corresponding to each of the training set images includes: The number of image channels is adjusted for each of the extracted features according to the preset number of channels, so as to obtain the image channel-adjusted features corresponding to each of the training set images; The second 1×1 convolutional layer extracts features from the adjusted features of each of the image channels to obtain the first feature-extracted features corresponding to each of the training set images. The first ReLU activation function layer is used to normalize each of the first features extracted from the first feature, so as to obtain the first normalized features corresponding to each of the training set images. The second 3×3 convolutional layer is used to extract features from each of the feature-extracted image features to obtain the second feature-extracted features corresponding to each of the training set images. The first normalized features and the second features extracted from each of the training set images are fused to obtain the first fused features corresponding to each of the training set images.
5. The super-resolution reconstruction method according to claim 3, characterized in that, The spatial attention module includes a third 1×1 convolutional layer, a second ReLU activation function layer, and a third 3×3 convolutional layer. The process of performing spatial attention feature fusion analysis on each of the extracted image features through the spatial attention module to obtain the second fused features corresponding to each of the training set images includes: The image size of each extracted feature is adjusted according to the preset width and preset height to obtain the image size adjusted feature corresponding to each training set image; The third 1×1 convolutional layer is used to extract features from each of the image size-adjusted features to obtain the third feature-extracted features corresponding to each of the training set images. The second ReLU activation function layer is used to normalize each of the extracted third features to obtain the second normalized features corresponding to each of the training set images. The third 3×3 convolutional layer extracts features from each of the feature-extracted image features to obtain the fourth feature-extracted features corresponding to each of the training set images. The second normalized features and the fourth features extracted from each of the training set images are fused to obtain the second fused features corresponding to each of the training set images.
6. A super-resolution reconstruction device, characterized in that, include: The feature extraction and analysis module is used to import multiple training set images and multiple test set images, and perform feature extraction and analysis on each of the training set images to obtain the features of the image to be trained corresponding to each of the training set images; The model training module is used to construct a training model and train the training model using multiple features of the images to be trained to obtain the trained model. The model testing module is used to test the trained model based on all test set images to obtain the super-resolution reconstruction model. The super-resolution reconstruction result acquisition module is used to import the image to be reconstructed, reconstruct the image to be reconstructed using the super-resolution reconstruction model, obtain the super-resolution reconstructed image, and use the super-resolution reconstructed image as the super-resolution reconstruction result. The training model includes an encoder and a decoder, and the model training module is specifically used for: The encoder encodes the features of each of the images to be trained, thereby obtaining the encoded features corresponding to each of the training set images, specifically as follows: The second formula is used to encode each of the features of the image to be trained, thereby obtaining the encoded image features corresponding to each of the training set images. The second formula is: H ( )= F ( )+ ， in, H ( ) represents the encoded image features. For the features of the image to be trained, F ( Let ) be the expectation function. F ( The mapping relationship consists of two convolutional layers. The decoder decodes each of the encoded image features to obtain the decoded image features corresponding to each of the training set images. Based on all the features of the decoded images and all the features of the images to be trained, the training model is analyzed for loss value to obtain the trained model. In the model training module, the process of performing loss value analysis on the training model based on all decoded image features and all training image features to obtain the trained model includes: The loss function corresponding to each training set image is obtained by calculating the loss function of each decoded image feature and the corresponding training set image feature using the third equation. The third equation is as follows: ， in, , in, , in, For loss function, This represents the error value between the decoded image features and the features of the image to be trained. The threshold set for the alpha function, For the features of the decoded image, Features of the image to be trained; The parameters of the training model are updated according to all the loss functions to obtain the updated training model. The updated training model is then used as the training model and returned to the feature extraction and analysis module until a preset number of iterations is reached. The updated training model is then used as the post-trained model.
7. A super-resolution reconstruction system, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the super-resolution reconstruction method as described in any one of claims 1 to 5.
8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the super-resolution reconstruction method as described in any one of claims 1 to 5.