An image super-resolution reconstruction method, a terminal and a storage medium

Abstract

The application discloses an image super-resolution reconstruction method, a terminal and a storage medium. The method comprises the following steps: performing feature extraction on a to-be-reconstructed image to obtain spatial domain features and frequency domain features of the to-be-reconstructed image; generating frequency domain attention features of the to-be-reconstructed image according to the frequency domain features of the to-be-reconstructed image; splicing the spatial domain features, the frequency domain features and the frequency domain attention features of the to-be-reconstructed image to obtain fusion features; inputting the fusion features into a lightweight image super-resolution network to obtain super-resolution features and reconstruct an image based on the super-resolution features to obtain a super-resolution image; and the super-resolution features comprise low-frequency super-resolution features, horizontal high-frequency super-resolution features, vertical high-frequency super-resolution features and diagonal high-frequency super-resolution features. Through the above scheme, the efficiency of image super-resolution reconstruction is improved, the calculation resources are saved, the effect of image super-resolution reconstruction is improved, the resolution of the reconstructed super-resolution image is higher, and the application is suitable for increasing user demand.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image super-resolution reconstruction method, terminal, and storage medium. Background Technology

[0002] As people's living standards improve, there is a strong demand for high-definition images and videos, especially in the experience of smart devices, where high-resolution image and video displays enhance visual perception. In the field of computer vision, high-resolution images and videos are beneficial for downstream tasks to acquire more detailed information from images and videos, raising the upper limit of downstream tasks. However, during image acquisition, some outdated equipment or harsh environmental factors can lead to low-resolution images and loss of image details; during network transmission, bandwidth limitations can also cause high-resolution images to be compressed into low-resolution images for transmission. Image super-resolution reconstruction technology reconstructs low-resolution images into high-resolution images, thereby meeting people's demand for high-resolution images.

[0003] Image super-resolution technology has great application value in mobile devices, medical imaging, and network image transmission, such as 2K display, medical diagnosis, and video conferencing.

[0004] To improve performance, image super-resolution reconstruction techniques often suffer from high computational costs and long processing times. Therefore, lightweight image super-resolution reconstruction techniques have emerged. Current lightweight image super-resolution reconstruction techniques can reduce computational costs and save computation time to some extent. However, while these techniques improve image reconstruction efficiency and save computational resources, their reconstruction results are unsatisfactory, producing low-quality super-resolution images that fail to meet the increasingly demanding requirements of users.

[0005] Therefore, how to improve the effect of image super-resolution reconstruction while ensuring efficiency and solving computational resource issues has become an urgent technical problem to be solved. Summary of the Invention

[0006] The main objective of this invention is to provide an image super-resolution reconstruction method, terminal, and computer-readable storage medium, aiming to solve the technical problem in the prior art that, while improving image reconstruction efficiency and saving computing resources, the image super-resolution reconstruction effect is poor and the reconstructed super-resolution quality is low.

[0007] To achieve the above objectives, embodiments of the present invention provide an image super-resolution reconstruction method, the method comprising:

[0008] The image to be reconstructed is acquired, and features are extracted from the image to be reconstructed to obtain the spatial domain features and frequency domain features of the image to be reconstructed.

[0009] The frequency domain features include: low-frequency frequency domain features, horizontal high-frequency frequency domain features, vertical high-frequency frequency domain features, and diagonal high-frequency frequency domain features;

[0010] Based on the frequency domain features of the image to be reconstructed, generate frequency domain attention features of the image to be reconstructed;

[0011] The spatial domain features, frequency domain features, and frequency domain attention features of the image to be reconstructed are concatenated to obtain the fused features of the image to be reconstructed.

[0012] The fused features are input into a lightweight image super-resolution network to obtain the super-resolution features of the image to be reconstructed.

[0013] The super-resolution features include: low-frequency super-resolution features, horizontal high-frequency super-resolution features, vertical high-frequency super-resolution features, and diagonal high-frequency super-resolution features;

[0014] Image reconstruction is performed based on the super-resolution features to obtain a super-resolution image of the image to be reconstructed.

[0015] Optionally, the step of extracting features from the image to be reconstructed to obtain the spatial and frequency domain features of the image to be reconstructed specifically includes:

[0016] The image to be reconstructed is input into a preset spatial feature extraction network, and convolution calculations are performed through the 1*1 convolution kernel operator, the 3*3 convolution kernel operator and the 5*5 convolution kernel operator of the preset spatial feature extraction network to obtain the first spatial feature, the second spatial feature and the third spatial feature of the image to be reconstructed.

[0017] The first spatial feature, the second spatial feature, and the third spatial feature are added together and downsampled to obtain the spatial features of the image to be reconstructed.

[0018] Optionally, the step of extracting features from the image to be reconstructed to obtain the spatial and frequency domain features of the image to be reconstructed specifically includes:

[0019] The image to be reconstructed is subjected to wavelet transform to obtain the frequency domain features of the image to be reconstructed.

[0020] Optionally, frequency domain attention features of the image to be reconstructed are generated based on the frequency domain features of the image to be reconstructed, specifically including:

[0021] The high-frequency domain features are obtained by splicing the horizontal high-frequency domain features, the vertical high-frequency domain features, and the diagonal high-frequency domain features.

[0022] The high-frequency domain features and the low-frequency domain features are respectively mapped to obtain the corresponding high-frequency mapped features and low-frequency mapped features.

[0023] Affine calculation is performed based on the high-frequency mapping features and the low-frequency mapping features to obtain the similarity weight matrix;

[0024] The high-frequency mapping replication features are weighted according to the similarity weight matrix to obtain the weighted high-frequency mapping replication features;

[0025] The high-frequency mapping replication feature is obtained by copying the high-frequency mapping replication feature;

[0026] The frequency domain attention features of the image to be reconstructed are generated based on the low-frequency domain features and the weighted high-frequency mapping copy features.

[0027] Optionally, the high-frequency mapping replication features are weighted according to the similarity weight matrix to obtain weighted high-frequency mapping replication features, specifically including:

[0028] The similarity weight matrix is ​​normalized to obtain a normalized similarity weight matrix;

[0029] The high-frequency mapping replication features are weighted according to the normalized similarity weight matrix to obtain the weighted high-frequency mapping replication features.

[0030] Optionally, an affine calculation is performed based on the high-frequency mapping features and the low-frequency mapping features to obtain a similarity weight matrix, specifically including:

[0031] Select any one element from the low-frequency mapping features as the first low-frequency element to be calculated;

[0032] Based on the position of the first low-frequency element to be calculated, determine the element corresponding to the cross-shaped part in the high-frequency mapping feature, and use it as the first high-frequency element to be calculated.

[0033] Based on the first low-frequency element to be calculated and the first high-frequency element to be calculated, determine the similarity weight corresponding to the first low-frequency element to be calculated.

[0034] The similarity weight matrix is ​​formed based on the similarity weight corresponding to each element in the low-frequency mapping feature.

[0035] Optionally, the step of weighting the high-frequency mapping replication features according to the similarity weight matrix to obtain weighted high-frequency mapping replication features specifically includes:

[0036] Select an element from the cross-shaped portion corresponding to any position in the high-frequency mapping replication feature as the second high-frequency element to be calculated;

[0037] Based on the selected high-frequency mapping replication features, determine the element corresponding to the cross-shaped part of the corresponding position in the similarity weight matrix, and use it as the second low-frequency element to be calculated.

[0038] The second high-frequency element to be calculated and the second low-frequency element to be calculated are multiplied by a dot product to obtain the weighted feature points.

[0039] The weighted high-frequency mapping replication feature is formed by weighting the feature points of each element in the high-frequency mapping replication feature.

[0040] Optionally, the step of inputting the fused features into a lightweight image super-resolution network to obtain the super-resolution features of the image to be reconstructed specifically includes:

[0041] The fused features are input into a branch of the first image super-resolution network to obtain the low-frequency super-resolution features of the image to be reconstructed; and

[0042] The fused features are input into the second image super-resolution network branch to obtain the horizontal high-frequency super-resolution features of the image to be reconstructed;

[0043] The fused features are input into the third image super-resolution network branch to obtain the vertical high-frequency super-resolution features of the image to be reconstructed;

[0044] The fused features are input into the fourth image super-resolution network branch to obtain the diagonal high-frequency super-resolution features of the image to be reconstructed;

[0045] The lightweight image super-resolution network includes: a first image super-resolution network branch, a second image super-resolution network branch, a third image super-resolution network branch, and a fourth image super-resolution network branch.

[0046] Optionally, the step of reconstructing the image based on the super-resolution features to obtain a super-resolution image of the image to be reconstructed specifically includes:

[0047] According to the preset feature channels, the low-frequency super-resolution features, horizontal high-frequency super-resolution features, vertical high-frequency super-resolution features and diagonal high-frequency super-resolution features are connected to obtain spliced ​​features;

[0048] The stitched features are mapped to a spatial super-resolution image through inverse wavelet transform, which serves as the super-resolution image corresponding to the image to be reconstructed.

[0049] Optionally, the lightweight image super-resolution network is generated by the following method:

[0050] Obtain training samples, which include: sample images, true super-resolution images of the sample images, and fusion features of the sample images;

[0051] The fusion features of the sample images are obtained by splicing the frequency domain features, spatial domain features, and frequency domain attention features of the sample images; the frequency domain features of the sample images include: low frequency domain features, horizontal high frequency domain features, vertical high frequency domain features, and diagonal high frequency domain features.

[0052] The fusion features of the sample images are input into a preset lightweight neural network to obtain the super-resolution features of the sample images;

[0053] The super-resolution features of the sample images include: low-frequency super-resolution features, horizontal high-frequency super-resolution features, vertical high-frequency super-resolution features, and diagonal high-frequency super-resolution features.

[0054] Image reconstruction is performed based on the super-resolution features to obtain a generated super-resolution image of the sample image;

[0055] Based on the real super-resolution image and the generated super-resolution image of the sample image, the network parameters of the preset lightweight neural network are corrected, and the preset super-resolution network corresponding to the fusion feature input of the next sample image in the next training sample is executed to obtain the super-resolution features of the sample image. Based on the super-resolution features, image reconstruction is performed to obtain the generated super-resolution image of the next sample image. This process continues until the training status of the preset lightweight neural network meets the preset conditions to obtain the trained lightweight image super-resolution network.

[0056] Optionally, the step of correcting the network parameters of the preset super-resolution network based on the real super-resolution image and the generated super-resolution image of the sample image specifically includes:

[0057] Feature extraction is performed on the real super-resolution image of the sample image to obtain the frequency domain features of the real super-resolution image;

[0058] The frequency domain features of the real super-resolution image and the super-resolution features of the generated super-resolution image are respectively subjected to loss calculation to obtain the corresponding loss values;

[0059] Based on each of the loss values ​​of the sample images, determine the total loss value of the sample images;

[0060] The network parameters of the preset lightweight neural network are corrected based on the total loss value.

[0061] To achieve the above objectives, embodiments of the present invention also provide a computer-readable storage medium, characterized in that the computer-readable storage medium stores one or more programs, which can be executed by one or more processors to implement the steps in the image super-resolution reconstruction method as described in any of the above claims.

[0062] To achieve the above objectives, embodiments of the present invention also provide a terminal, the terminal comprising: a memory and a processor; the memory storing a computer-readable program executable by the processor; the processor executing the computer-readable program implementing the steps of the image super-resolution reconstruction method as described above.

[0063] This invention acquires the spatial, frequency, and frequency-domain attention features of the image to be reconstructed, and then concatenates these features to obtain fused features. These fused features are then input into a lightweight image super-resolution network to obtain super-resolution features, which include low-frequency, horizontal, vertical, and diagonal high-frequency features. Finally, image reconstruction is performed based on these super-resolution features to obtain a super-resolution image. As can be seen, the lightweight image super-resolution network improves the efficiency of image super-resolution reconstruction and saves computational resources. Furthermore, concatenating the spatial, frequency, and frequency-domain attention features of the image to be reconstructed to obtain fused features, and then outputting these fused features through the lightweight image super-resolution network to generate a super-resolution image, improves the effect of image super-resolution reconstruction. The quality of the reconstructed super-resolution image is improved, making it more suitable for the growing needs of users. In other words, it improves the effect of image super-resolution reconstruction while saving computational resources. Attached Figure Description

[0064] Figure 1 A flowchart of the image super-resolution reconstruction method provided in this embodiment of the invention;

[0065] Figure 2 A flowchart of step S101 provided in an embodiment of the present invention;

[0066] Figure 3 This is a schematic diagram of the structure of the preset spatial feature extraction network provided in an embodiment of the present invention;

[0067] Figure 4 A schematic diagram of frequency domain features provided in an embodiment of the present invention;

[0068] Figure 5 Another schematic diagram of the frequency domain features provided in an embodiment of the present invention;

[0069] Figure 6 This is another flowchart illustrating the acquisition of frequency domain features provided in an embodiment of the present invention;

[0070] Figure 7 A flowchart of step S102 provided in an embodiment of the present invention;

[0071] Figure 8 A flowchart of step S704 provided in an embodiment of the present invention;

[0072] Figure 9 This is a schematic diagram of the process for obtaining frequency domain attention features according to an embodiment of the present invention;

[0073] Figure 10 A flowchart of step S705 provided in an embodiment of the present invention;

[0074] Figure 11 A flowchart of step S104 provided in an embodiment of the present invention;

[0075] Figure 12 A flowchart of step S105 provided in an embodiment of the present invention.

[0076] Figure 13 A schematic diagram of the splicing features provided in an embodiment of the present invention;

[0077] Figure 14 The image super-resolution reconstruction method provided in this embodiment of the invention is illustrated in the diagram.

[0078] Figure 15 A flowchart for constructing a lightweight image super-resolution network provided in an embodiment of the present invention;

[0079] Figure 16 A flowchart of step S1504 provided in an embodiment of the present invention;

[0080] Figure 17 A framework diagram of a lightweight image super-resolution network provided for embodiments of the invention;

[0081] Figure 18 This is a schematic diagram of the terminal structure provided in an embodiment of the present invention. Detailed Implementation

[0082] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0083] This invention provides an image super-resolution reconstruction method. Figure 1 A flowchart of the image super-resolution reconstruction method provided in the embodiments of the present invention is shown below. Figure 1 As shown, this image super-resolution reconstruction method includes at least the following steps:

[0084] S101, acquire the image to be reconstructed, and extract features from the image to obtain the spatial and frequency domain features of the image to be reconstructed.

[0085] The frequency domain features consist of one low-frequency feature and three high-frequency features, namely: low-frequency frequency domain feature, horizontal high-frequency frequency domain feature, vertical high-frequency frequency domain feature, and diagonal high-frequency frequency domain feature.

[0086] S102, Generate frequency domain attention features of the image to be reconstructed based on the frequency domain features of the image to be reconstructed.

[0087] In this embodiment of the invention, the frequency domain features of the image to be reconstructed can be input into a preset frequency domain attention network to obtain frequency domain attention features. The preset frequency domain attention network weights the channels of the frequency domain features. Simply put, frequency domain attention features are features obtained by reweighting the frequency domain features; more important regions have higher weights, thus allowing the preset frequency domain attention network to focus more on important regions.

[0088] S103, the spatial domain features, frequency domain features and frequency domain attention features of the image to be reconstructed are spliced ​​together to obtain the fused features of the image to be reconstructed.

[0089] In this embodiment of the invention, the spatial domain features, frequency domain features, and frequency domain attention features of the image to be reconstructed all have the same feature dimension, which is H×W. Here, H represents the image height, and W represents the image width.

[0090] Specifically, the spatial domain features, frequency domain features, and frequency domain attention features of the image to be reconstructed can be concatenated at the feature channel level to obtain the fused features of the image to be reconstructed. It should be noted that there are no specific requirements for the concatenation order of the spatial domain features, frequency domain features, and frequency domain attention features of the image to be reconstructed. For example, low-frequency domain features + horizontal high-frequency domain features + vertical high-frequency domain features + diagonal high-frequency domain features + spatial features + frequency domain attention features; spatial features + frequency domain attention features + low-frequency domain features + horizontal high-frequency domain features + vertical high-frequency domain features + diagonal high-frequency domain features, etc.

[0091] Simply put, it involves concatenating the above features together. For example, two vectors a = [1,2] and b = [3,4] can be concatenated to obtain c = [a,b] = [1,2,3,4], thus obtaining the fused feature c.

[0092] For example:

[0093] Spatial features S∈[S1,H,W], where S1 represents the number of channels, H represents the image height, and W represents the image width;

[0094] The frequency domain feature F∈[S2,H,W], where S2 represents the number of channels, H represents the image height, and W represents the image width;

[0095] Frequency domain attention features A∈[S3,H,W], where S3 represents the number of channels, H represents the image height, and W represents the image width;

[0096] Then, the fusion feature P∈[S4,H,W], where S4 represents the number of channels, S4=S1+S2+S3, H represents the image height, and W represents the image width.

[0097] The number of channels is determined by the number of convolution kernels. An image passing through one convolution kernel will output a feature map of size H*W; an image passing through N convolution kernels will output N feature maps of size H*W. Each feature map stacked together is represented as [N, H, W].

[0098] S104. Input the fused features into the lightweight image super-resolution network to obtain the super-resolution features of the image to be reconstructed.

[0099] The lightweight image super-resolution networks are all pre-trained lightweight image super-resolution networks. The super-resolution features include: low-frequency super-resolution features, horizontal high-frequency super-resolution features, vertical high-frequency super-resolution features, and diagonal high-frequency super-resolution features. The lightweight image super-resolution networks can be lightweight EDSR, RSAN, etc.

[0100] S105, image reconstruction is performed based on super-resolution features to obtain a super-resolution image of the image to be reconstructed.

[0101] The image super-resolution reconstruction method provided in this invention fuses the spatial domain features, frequency domain features, and frequency domain attention features of the image to be reconstructed to obtain fused features. These fused features are then input into a preset lightweight image super-resolution network to obtain one low-frequency super-resolution feature and three high-frequency super-resolution features. Image reconstruction is then performed based on these low-frequency and high-frequency features to obtain a super-resolution image of the image to be reconstructed. This method improves the image super-resolution reconstruction effect while maintaining lightweight image super-resolution reconstruction, ensuring the quality of the reconstructed image and enhancing the user experience.

[0102] In some embodiments of the present invention, such as Figure 2 As shown, step S101 above can be achieved through at least the following steps:

[0103] S201, the image to be reconstructed is input into a preset spatial feature extraction network, and convolution calculations are performed by the 1*1 convolution kernel operator, the 3*3 convolution kernel operator and the 5*5 convolution kernel operator of the preset spatial feature extraction network to obtain the first spatial feature, the second spatial feature and the third spatial feature of the image to be reconstructed.

[0104] In this embodiment of the invention, the aforementioned preset spatial feature extraction network can be a convolutional neural network. It is understood that the aforementioned preset spatial feature extraction network can also be other neural networks, as long as they can automatically extract the spatial features of the image.

[0105] S202, the first spatial feature, the second spatial feature and the third spatial feature are added together and downsampled to obtain the spatial features of the image to be reconstructed.

[0106] Specifically, such as Figure 3 As shown, the image to be reconstructed (i.e. Figure 3 The low-resolution image in the image is input into the 1*1 convolution kernel operator, 3*3 convolution kernel operator and 5*5 convolution kernel operator of the preset spatial feature extraction network for convolution calculation to obtain spatial feature 1 (first spatial feature), spatial feature 2 (second spatial feature) and spatial feature 3 (third spatial feature). Spatial feature 1, spatial feature 2 and spatial feature 3 are added together to obtain the added spatial feature. The added spatial feature is then downsampled by a preset interpolation method to obtain the spatial features of the image to be reconstructed.

[0107] In this embodiment of the invention, features are extracted using convolution kernels of different sizes, which means extracting features at different scales. Therefore, features extracted using multiple convolution kernels of different sizes are richer and more semantically informative than features extracted using a single convolution kernel. It is understood that other sizes of convolution kernels can also be used.

[0108] The aforementioned preset interpolation method can be a bicubic interpolation method. It is understood that, in addition to bicubic interpolation, other existing interpolation methods can also be used for downsampling in this embodiment of the invention, such as bilinear interpolation. Compared to other bilinear interpolation methods, bicubic interpolation can create smoother image edges, which is beneficial for improving the effect of image super-resolution reconstruction, and it is simple, efficient, and computationally less demanding.

[0109] S203, perform wavelet transform on the image to be reconstructed to obtain the frequency domain features of the image to be reconstructed.

[0110] In this embodiment of the invention, the image to be constructed can be transformed to the wavelet domain using wavelet transform to obtain frequency domain features. For example, the frequency domain features can be obtained by performing second-order wavelet decomposition on the image to be constructed using the Haar wavelet operator, such as... Figure 4 ,5 As shown, A represents low-frequency domain characteristics, H represents horizontal high-frequency domain characteristics, V represents vertical high-frequency domain characteristics, and D represents diagonal high-frequency domain characteristics.

[0111] Furthermore, the image to be constructed can be filtered and downsampled horizontally first, and then filtered and downsampled vertically to obtain four sub-images of the first-order wavelet transform: approximation sub-image, horizontal detail sub-image, vertical detail sub-image, and diagonal detail sub-image.

[0112] Repeat the above process for the approximation subgraph of the image to be constructed until the second-order wavelet decomposition is obtained, that is, the frequency domain features of the image to be constructed are obtained.

[0113] like Figure 6 As shown, Figure a is the image to be constructed, Figure b is the result after the first-level wavelet transform, where A represents the downsampled image, H represents the details in the horizontal direction, V represents the details in the vertical direction, and D represents the details in the diagonal direction. Figure c is the result after the second-level wavelet transform, where A1 represents the low-frequency domain features, H1 represents the horizontal high-frequency domain features, V1 represents the vertical high-frequency domain features, and D1 represents the diagonal high-frequency domain features.

[0114] It should be noted that the above filtering may include at least one of high-pass filtering and low-pass filtering. Furthermore, the above embodiment provides a second-order wavelet transform, but in the embodiments of this invention, it can be replaced with other wavelet transforms, without specific limitation. It is understood that in the embodiments of this invention, the spatial domain features of the image to be reconstructed can be obtained first through steps S201 and S202, and then the frequency domain features of the image to be reconstructed can be obtained through step S203; or, the frequency domain features of the image to be reconstructed can be obtained first through step S203, and then the spatial domain features of the image to be reconstructed can be obtained through steps S201 and S202; or, the spatial domain features of the image to be reconstructed obtained through steps S201 and S202, and the frequency domain features of the image to be reconstructed obtained through step S203, can be obtained simultaneously, without specific limitation in the embodiments of this invention.

[0115] In some embodiments of the present invention, such as Figure 7 As shown, step S102 above can be achieved through at least the following steps:

[0116] S701 splices together the horizontal high-frequency domain features, the vertical high-frequency domain features, and the diagonal high-frequency domain features to obtain the high-frequency domain features.

[0117] In this embodiment of the invention, the three high-frequency features in the frequency domain features of the image to be constructed are spliced ​​together, namely, the horizontal high-frequency domain feature H, the vertical high-frequency domain feature V, and the diagonal high-frequency domain feature D, to obtain the high-frequency domain feature M = [H, V, D].

[0118] S702 performs feature mapping on high-frequency domain features to obtain high-frequency mapped features.

[0119] The high-frequency feature M is mapped by a 1×1 convolution kernel operator to obtain the high-frequency mapped feature K.

[0120] S703 performs feature mapping on low-frequency domain features to obtain low-frequency mapped features.

[0121] Similarly, the low-frequency domain feature A is mapped using a 1×1 convolution kernel operator to obtain the low-frequency mapped feature Q.

[0122] In this embodiment of the invention, the high-frequency mapping feature K and the low-frequency mapping feature Q obtained by feature mapping have the same feature dimension, wherein the feature dimension size can be represented as H×M.

[0123] The aforementioned feature mapping of high-frequency and low-frequency features can map these features from high-dimensional to low-dimensional, which can further save time and computational resources when reconstructing images based on these features.

[0124] It is understood that step S702 can be restricted before step S703 is executed; or step S703 can be executed first, followed by step S702; or steps S702 and S703 can be executed simultaneously. No specific limitation is made in the embodiments of the present invention.

[0125] S704. Based on the above high-frequency mapping features and the above low-frequency mapping features, affine calculation is performed to obtain the similarity weight matrix.

[0126] Affine calculations are performed on the high-frequency mapping feature K and the low-frequency mapping feature Q to obtain the similarity weight matrix S. The size of the similarity weight matrix is ​​also H×M.

[0127] In this embodiment of the invention, affine calculations are performed on the high-frequency mapping features and the aforementioned low-frequency mapping features, which can simply and efficiently calculate the correlation between different feature points.

[0128] Furthermore, such as Figure 8 As shown, step S704 above can be achieved through at least the following steps:

[0129] S801, select any element from the low-frequency mapping features as the first low-frequency element to be calculated.

[0130] In this embodiment of the invention, the low-frequency mapping feature Q has multiple elements, each with its corresponding position information. The element at any position P in the low-frequency mapping feature Q is denoted as Q. p .

[0131] like Figure 9 As shown, select any element Q from the low-frequency mapping feature Q. p Q, as the first low-frequency element to be calculated p .

[0132] S802, based on the position of the first low-frequency element to be calculated, determine the element corresponding to the cross-shaped part in the high-frequency mapping feature, and use it as the first high-frequency element to be calculated.

[0133] In this embodiment of the invention, the high-frequency mapping feature K also has multiple elements, each with its corresponding position information. The element at any position P in the high-frequency mapping feature K is denoted as K. p .

[0134] like Figure 9 As shown, based on the first low-frequency element Q to be calculated p The position P is used to determine the element K at position P in the high-frequency mapping feature K. p Obtain the element K in the high-frequency mapping feature K. p The elements in the cross-shaped intersection are taken as the first high-frequency elements to be calculated, totaling (H+M-1) elements, all denoted as K. p .

[0135] S803, determine the similarity weight of the first low-frequency element to be calculated based on the first low-frequency element to be calculated and its corresponding first high-frequency element to be calculated.

[0136] The similarity weight S corresponding to the first low-frequency element to be calculated can be obtained by using the following formula. p :

[0137] S p =Q p ×K p .

[0138] S804. Based on the similarity weight of each element in the low-frequency mapping features, the above similarity weight matrix is ​​formed.

[0139] In this embodiment of the invention, each element Q in the low-frequency mapping feature K is calculated sequentially through the above steps S801-S803. p The similarity weights are used to obtain the Q of each element in the low-frequency mapping feature K. p The similarity weights are then determined. Then, based on each element Q in the low-frequency mapping feature K... pThe similarity weights and positions are used to form a similarity weight matrix S, such as... Figure 9 As shown.

[0140] S705, based on the similarity weight matrix, the high-frequency mapping replication features are weighted to obtain the weighted high-frequency mapping replication features.

[0141] In this embodiment of the invention, by weighting the high-frequency mapping replication features according to the similarity weight matrix, the network can adaptively focus on more important features.

[0142] Specifically, the similarity weight matrix can be normalized first, and then the high-frequency mapping replication features can be weighted according to the normalized similarity weight matrix to obtain the weighted high-frequency mapping replication features.

[0143] In this embodiment of the invention, the similarity weight matrix S can be normalized using the Softmax function to obtain the normalized similarity weight matrix S', where:

[0144] S' = Softmax(S).

[0145] Normalizing the similarity weight matrix S further improves the accuracy of image super-resolution reconstruction.

[0146] The aforementioned high-frequency mapping replication feature is obtained by copying the high-frequency mapping feature and is identical to the high-frequency mapping feature.

[0147] In some embodiments of the present invention, such as Figure 10 As shown, step S705 above can be achieved through at least the following steps:

[0148] S1001, select the element of the cross-shaped part corresponding to any position in the high-frequency mapping replication feature as the second high-frequency element to be calculated.

[0149] The high-frequency mapping replication feature Y of the image to be constructed has multiple elements, each with its corresponding position. The vertical high-frequency element corresponding to any position P in the high-frequency mapping replication feature Y is denoted as Y. p .

[0150] like Figure 9 As shown, select the element Y of the cross-shaped part corresponding to any position P in the high-frequency mapping replication feature Y. p , as the second high-frequency element to be calculated.

[0151] S1002, based on the above selection of any position in the high-frequency mapping replication features, determine the element corresponding to the cross-shaped part of the corresponding position in the similarity weight matrix, as the second low-frequency element to be calculated.

[0152] like Figure 9 As shown, based on any position P in the selected high-frequency mapping replication features, the matrix element corresponding to the cross-shaped part of position P in the normalized similarity weight matrix S' is determined as the second low-frequency element S' to be calculated. p ,like Figure 9 As shown.

[0153] S1003, perform dot product calculation on the second high-frequency element to be calculated and its corresponding second low-frequency element to be calculated to obtain the feature point corresponding to the second high-frequency element to be calculated.

[0154] like Figure 9 As shown, the second high-frequency element Y to be calculated is... p With the second low-frequency element S to be calculated p Perform dot product calculation to obtain the weighted feature point Y. p Among them, the weighted feature point Y' p That is, multiplying and summing the products of each element, we get:

[0155] Y' p =∑Y p (i)*S' p (i).

[0156] S1004, Based on the weighted feature points of each element in the high-frequency mapping replication feature, a weighted high-frequency mapping replication feature is formed.

[0157] In this embodiment of the application, through the above steps S1001-S1003, the element Y at each position in the high-frequency mapping replication feature Y is calculated sequentially. p Corresponding feature point Y' p This is how we obtain the element Y at each position in Y. p Feature point Y' p Then, take the element Y at each position. p Feature point Y' p The weighted high-frequency mapping replication feature Y' is composed of, for example Figure 9 As shown.

[0158] Since different high-frequency features have different amounts of information, weighting the high-frequency mapping replication features can make the network pay more attention to features with more semantic meaning, thereby enhancing the network's expressive power.

[0159] S706, based on the low-frequency mapping features and the weighted high-frequency mapping replication features, generates frequency domain attention features for the image to be reconstructed.

[0160] like Figure 9As shown, the low-frequency mapping feature Q is added to the weighted vertical high-frequency domain feature V' to obtain the frequency domain attention feature of the image to be reconstructed.

[0161] As described above, the frequency domain features include one low-frequency feature and three low-frequency features, namely: low-frequency frequency domain feature, horizontal high-frequency frequency domain feature, vertical high-frequency frequency domain feature, and diagonal high-frequency frequency domain feature. Based on this, the preset lightweight image super-resolution network provided in this embodiment of the invention includes four super-resolution network branches, namely: a first image super-resolution network branch, a second image super-resolution network branch, a third image super-resolution network branch, and a fourth image super-resolution network branch.

[0162] The first image super-resolution network branch, denoted as G1, has N0 channels and outputs Y1, with supervision defined as the low-frequency part of wavelet decomposition (i.e., low-frequency frequency domain features).

[0163] The second image super-resolution network branch, denoted as G2, has N0 / 2 channels and outputs Y2. The supervision is the horizontal high-frequency part of wavelet decomposition (i.e., horizontal high-frequency domain features).

[0164] The third image super-resolution network branch, denoted as G3, has N0 / 2 channels and outputs Y3. The supervision is the vertical high-frequency part of wavelet decomposition (i.e., vertical high-frequency domain features).

[0165] The fourth image super-resolution network branch, denoted as G3, has N0 / 2 channels and outputs Y4. The supervision is the high-frequency part of the diagonal wavelet decomposition (i.e., the high-frequency features of the diagonal).

[0166] By inputting the fusion features into the four image super-resolution network branches mentioned above, one low-frequency super-resolution feature and three high-frequency super-resolution features can be obtained.

[0167] Specifically, such as Figure 11 As shown, step S104 above can be achieved through at least the following steps:

[0168] S1101, input the fused features into the first image super-resolution network branch to obtain the low-frequency super-resolution features of the image to be constructed.

[0169] S1102, the fused features are input into the second image super-resolution network branch to obtain the horizontal high-frequency super-resolution features of the image to be constructed.

[0170] S1103, input the fused features into the third image super-resolution network branch to obtain the vertical high-frequency feature super-resolution of the image to be constructed.

[0171] S1104, input the fused features into the fourth image super-resolution network branch to obtain the diagonal high-frequency super-resolution features of the image to be constructed.

[0172] As can be seen from the above, the embodiment of the present invention uses a lightweight image super-resolution network, and the four image super-resolution network branches included are also lightweight, for example, 0.1 times that of the original image super-resolution network, thereby reducing computational overhead and inference time, so as to achieve image super-resolution reconstruction efficiently.

[0173] In some embodiments of the present invention, such as Figure 12 As shown, step S105 above can be achieved through at least the following steps:

[0174] S1201, according to the preset feature channels, the low-frequency super-resolution features, horizontal high-frequency super-resolution features, vertical high-frequency features and diagonal high-frequency super-resolution features are spliced ​​together to obtain spliced ​​features.

[0175] like Figure 13 As shown, the low-frequency super-resolution feature Y1, the horizontal high-frequency super-resolution feature Y2, the vertical high-frequency feature Y3, and the diagonal high-frequency super-resolution feature Y4 obtained above are spliced ​​together according to the preset feature channels to obtain the spliced ​​features (i.e., spliced ​​features).

[0176] S1202 uses inverse wavelet transform to map the stitched features into a spatial super-resolution image, which serves as the super-resolution image corresponding to the image to be constructed.

[0177] For example, such as Figure 14 As shown, the image to be constructed (i.e. Figure 14 The image super-resolution construction method provided in this embodiment of the invention obtains the corresponding super-resolution block from the small block to be constructed in the low-resolution image.

[0178] In some embodiments of the present invention, a method for constructing a lightweight image super-resolution network is also provided, such as... Figure 15 As shown, constructing a lightweight image super-resolution network may include at least the following steps:

[0179] S1501, Obtain training samples.

[0180] The training samples include: sample images, real super-resolution images of sample images, and fusion features of sample images.

[0181] The fusion features of the sample image are obtained by splicing the frequency domain features, spatial domain features, and frequency domain attention features of the sample image; the frequency domain features of the sample image include: low frequency domain features, horizontal high frequency domain features, vertical high frequency domain features, and diagonal high frequency domain features.

[0182] It should be noted that the technical solution for generating the fusion features of the image to be constructed has been described in the above embodiments. The fusion features of the sample image can be obtained by referring to the above embodiments, and will not be described in detail here.

[0183] S1502, input the fusion features of the sample image into a preset lightweight neural network to obtain the super-resolution features of the sample image.

[0184] The aforementioned preset lightweight neural network includes four lightweight neural network branches: the first lightweight neural network branch, the second lightweight neural network branch, the third lightweight neural network branch, and the fourth lightweight neural network branch.

[0185] Specifically, the first lightweight neural network branch is used to supervise the low-frequency domain features in the fusion features of the sample images; the second lightweight neural network branch is used to supervise the horizontal high-frequency domain features in the fusion features of the sample images; the third lightweight neural network branch is used to supervise the vertical high-frequency domain features in the fusion features of the sample images; and the fourth lightweight neural network branch is used to supervise the diagonal high-frequency domain features in the fusion features of the sample images.

[0186] The fusion features of the above sample images are input into four lightweight neural network branches respectively to obtain super-resolution features of the sample images, including one low-frequency super-resolution feature and three high-frequency super-resolution features, namely: low-frequency super-resolution feature, horizontal high-frequency super-resolution feature, vertical high-frequency super-resolution feature, and diagonal high-frequency super-resolution feature.

[0187] In this embodiment of the invention, a neural network of the prior art can be used, and the number of parameters and computational cost of the existing neural network can be reduced (for example, reduced to 0.1 times), and it can be used as a branch of a lightweight neural network to construct the above-mentioned preset lightweight neural network.

[0188] S1503, based on the super-resolution features of the sample image, performs image reconstruction to obtain the generated super-resolution image of the sample image.

[0189] The above embodiments have already described the scheme for image reconstruction based on super-resolution features, and will not be repeated here. Please refer to the above embodiments.

[0190] S1504, based on the real super-resolution image and the generated super-resolution image of the sample image, the network parameters of the preset lightweight neural network are corrected, and the preset lightweight neural network is executed by inputting the fusion features of the next sample image in the next training sample to obtain the super-resolution features of the next sample image, and image reconstruction is performed based on the super-resolution features to obtain the generated super-resolution image of the next sample image, until the training status of the preset lightweight neural network meets the preset conditions, and the trained lightweight image super-resolution network is obtained.

[0191] As can be seen from the above, after correcting the network parameters of the preset lightweight neural network based on the real super-resolution image and the generated super-resolution image of the sample image, the steps S1501-S1502 are continued, that is, the training samples are continued to be obtained to train the preset lightweight neural network.

[0192] The aforementioned preset conditions may include a loss function value meeting preset requirements or the number of training iterations reaching a preset number. Preset requirements can be determined based on the accuracy of the lightweight image super-resolution network, which will not be detailed here. The preset number of iterations can be the maximum number of training iterations for the lightweight neural network, for example, 5000 iterations.

[0193] In some embodiments of the present invention, such as Figure 16 As shown, the step S1504 above, which involves correcting the network parameters of the preset lightweight neural network based on the real super-resolution image and the generated super-resolution image of the sample image, can be achieved through at least the following steps:

[0194] S1601, extract features from the real super-resolution image of the sample image to obtain the frequency domain features of the real super-resolution image.

[0195] The frequency domain features of a true super-resolution image include one low-frequency feature and three high-frequency features, namely: low-frequency frequency domain feature, horizontal high-frequency frequency domain feature, vertical high-frequency frequency domain feature, and diagonal high-frequency frequency domain feature.

[0196] S1602, calculate the loss of the frequency domain features of the real super-resolution image and the super-resolution features of the generated super-resolution image respectively, and obtain the corresponding loss values.

[0197] Specifically, the low-frequency domain features of the real super-resolution image and the low-frequency super-resolution features of the generated super-resolution image are used to calculate the loss, and the first loss value is obtained.

[0198] The loss is calculated by comparing the horizontal high-frequency domain features of the real super-resolution image with the horizontal high-frequency super-resolution features of the generated super-resolution image to obtain the second loss value.

[0199] The loss is calculated by combining the vertical high-frequency domain features of the real super-resolution image with the vertical high-frequency super-resolution features of the generated super-resolution image to obtain the third loss value.

[0200] The fourth loss value is obtained by calculating the loss between the high-frequency diagonal features of the real super-resolution image and the high-frequency super-resolution features of the generated super-resolution image.

[0201] In other words, the above loss values ​​include: the first loss value, the second loss value, the third loss value, and the fourth loss value.

[0202] In this embodiment of the invention, the loss value can be calculated using a loss function (e.g., L1 loss function).

[0203] S1603, Calculate the total loss value of the sample image based on the loss values ​​of each sample image.

[0204] The total loss value of the sample images is the sum of the individual loss values ​​of the sample images, which is:

[0205] Total loss value = First loss value + Second loss value + Third loss value + Fourth loss value.

[0206] S1604X1303 corrects the network parameters of the preset lightweight neural network based on the total loss value.

[0207] In this embodiment of the invention, the network parameters of each lightweight neural network branch in the preset lightweight neural network can be corrected by using the total loss value.

[0208] Figure 17 This is a framework diagram of a lightweight image super-resolution network provided in an embodiment of the present invention, as shown below. Figure 17 As shown, the low-resolution image is processed through feature extraction, feature fusion, and feature super-resolution to obtain super-resolution features (1 low-frequency and 3 high-frequency features). Then, image reconstruction is performed using the super-resolution features, and the loss of the super-resolution features is calculated to adjust the lightweight image super-resolution network.

[0209] In this embodiment of the invention, a lightweight neural network is trained using the fusion features of sample images in the training samples to obtain a lightweight image super-resolution network. The fusion features of the sample images are composed of the frequency domain features and spatial domain features of the sample images. Therefore, by training the lightweight neural network with the fusion features of the sample images and the real super-resolution images, a lightweight image super-resolution network for image super-resolution reconstruction can be obtained. This allows for the rapid acquisition of high-quality super-resolution images through the lightweight image super-resolution network, thus improving the quality of image super-resolution reconstruction.

[0210] Based on the image super-resolution reconstruction method provided in the above embodiments, this invention also provides a computer-readable storage medium storing one or more programs, which can be executed by one or more processors to implement the steps in the image super-resolution reconstruction method described in the above embodiments.

[0211] Based on the image super-resolution reconstruction method provided in the above embodiments, this invention also provides a terminal, such as... Figure 18As shown, it includes at least one processor 30; a display screen 31; and a memory 32, and may also include a communication interface 33 and a bus 34. The processor 30, display screen 31, memory 32, and communication interface 33 can communicate with each other via the bus 34. The display screen 31 is configured to display a preset user guide interface in the initial setup mode. The communication interface 33 can transmit information. The processor 30 can call logical instructions in the memory 32 to execute the image super-resolution reconstruction method described in the above embodiment.

[0212] Furthermore, the logical instructions in the aforementioned memory 32 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium.

[0213] The memory 32, as a computer-readable storage medium, can be configured to store software programs, computer-executable programs, such as program instructions or modules corresponding to the methods in the embodiments of this disclosure. The processor 30 executes functional applications and data processing by running the software programs, instructions, or modules stored in the memory 32, thereby implementing the methods in the above embodiments.

[0214] The memory 32 may include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a given function; the data storage area may store data created based on the use of the terminal. Furthermore, the memory 32 may include high-speed random access memory (RAM) and non-volatile memory. Examples include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks; these can also be transient storage media.

[0215] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the terminal and medium embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0216] The terminals and media provided in this application are one-to-one with the methods. Therefore, the terminals and media also have similar beneficial technical effects as their corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the terminals and media will not be repeated here.

[0217] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0218] Of course, those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.). The program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The computer-readable storage medium can be a memory, magnetic disk, optical disk, etc.

[0219] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. An image super-resolution reconstruction method, characterized in that, The method includes: The image to be reconstructed is acquired, and features are extracted from the image to be reconstructed to obtain the spatial domain features and frequency domain features of the image to be reconstructed. The frequency domain features include: low-frequency frequency domain features, horizontal high-frequency frequency domain features, vertical high-frequency frequency domain features, and diagonal high-frequency frequency domain features; Based on the frequency domain features of the image to be reconstructed, generate frequency domain attention features of the image to be reconstructed; The spatial domain features, frequency domain features, and frequency domain attention features of the image to be reconstructed are concatenated to obtain the fused features of the image to be reconstructed. The fused features are input into a lightweight image super-resolution network to obtain the super-resolution features of the image to be reconstructed. The super-resolution features include: low-frequency super-resolution features, horizontal high-frequency super-resolution features, vertical high-frequency super-resolution features, and diagonal high-frequency super-resolution features; Image reconstruction is performed based on the super-resolution features to obtain a super-resolution image of the image to be reconstructed.

2. The image super-resolution reconstruction method according to claim 1, characterized in that, The step of extracting features from the image to be reconstructed to obtain its spatial and frequency domain features specifically includes: The image to be reconstructed is input into a preset spatial feature extraction network, and convolution calculations are performed through the 1*1 convolution kernel operator, the 3*3 convolution kernel operator and the 5*5 convolution kernel operator of the preset spatial feature extraction network to obtain the first spatial feature, the second spatial feature and the third spatial feature of the image to be reconstructed. The first spatial feature, the second spatial feature, and the third spatial feature are added together and downsampled to obtain the spatial features of the image to be reconstructed.

3. The image super-resolution reconstruction method according to claim 1, characterized in that, The step of extracting features from the image to be reconstructed to obtain its spatial and frequency domain features specifically includes: The image to be reconstructed is subjected to wavelet transform to obtain the frequency domain features of the image to be reconstructed.

4. The image super-resolution reconstruction method according to claim 1, characterized in that, Based on the frequency domain features of the image to be reconstructed, frequency domain attention features of the image to be reconstructed are generated, specifically including: The high-frequency domain features are obtained by splicing the horizontal high-frequency domain features, the vertical high-frequency domain features, and the diagonal high-frequency domain features. The high-frequency domain features and the low-frequency domain features are respectively mapped to obtain the corresponding high-frequency mapped features and low-frequency mapped features. Affine calculation is performed based on the high-frequency mapping features and the low-frequency mapping features to obtain the similarity weight matrix; The high-frequency mapping replication features are weighted according to the similarity weight matrix to obtain the weighted high-frequency mapping replication features; The high-frequency mapping replication feature is obtained by copying the high-frequency mapping replication feature; The frequency domain attention features of the image to be reconstructed are generated based on the low-frequency domain features and the weighted high-frequency mapping copy features.

5. The image super-resolution reconstruction method according to claim 4, characterized in that, The high-frequency mapping replication features are weighted according to the similarity weight matrix to obtain the weighted high-frequency mapping replication features, which specifically include: The similarity weight matrix is ​​normalized to obtain a normalized similarity weight matrix; The high-frequency mapping replication features are weighted according to the normalized similarity weight matrix to obtain the weighted high-frequency mapping replication features.

6. The image super-resolution reconstruction method according to claim 4, characterized in that, Affine calculation is performed based on the high-frequency mapping features and the low-frequency mapping features to obtain a similarity weight matrix, specifically including: Select any one element from the low-frequency mapping features as the first low-frequency element to be calculated; Based on the position of the first low-frequency element to be calculated, determine the element corresponding to the cross-shaped part in the high-frequency mapping feature, and use it as the first high-frequency element to be calculated. Based on the first low-frequency element to be calculated and the first high-frequency element to be calculated, determine the similarity weight corresponding to the first low-frequency element to be calculated. The similarity weight matrix is ​​formed based on the similarity weight corresponding to each element in the low-frequency mapping feature.

7. The image super-resolution reconstruction method according to claim 4, characterized in that, The step of weighting the high-frequency mapping replication features according to the similarity weight matrix to obtain the weighted high-frequency mapping replication features specifically includes: Select an element from the cross-shaped portion corresponding to any position in the high-frequency mapping replication feature as the second high-frequency element to be calculated; Based on the selected high-frequency mapping replication features, determine the element corresponding to the cross-shaped part of the corresponding position in the similarity weight matrix, and use it as the second low-frequency element to be calculated. The second high-frequency element to be calculated and the second low-frequency element to be calculated are multiplied by a dot product to obtain the weighted feature points. The weighted high-frequency mapping replication feature is formed by weighting the feature points of each element in the high-frequency mapping replication feature.

8. The image super-resolution reconstruction method according to claim 1, characterized in that, The step of inputting the fused features into a lightweight image super-resolution network to obtain the super-resolution features of the image to be reconstructed specifically includes: The fused features are input into a branch of the first image super-resolution network to obtain the low-frequency super-resolution features of the image to be reconstructed; and The fused features are input into the second image super-resolution network branch to obtain the horizontal high-frequency super-resolution features of the image to be reconstructed; The fused features are input into the third image super-resolution network branch to obtain the vertical high-frequency super-resolution features of the image to be reconstructed; The fused features are input into the fourth image super-resolution network branch to obtain the diagonal high-frequency super-resolution features of the image to be reconstructed; The lightweight image super-resolution network includes: a first image super-resolution network branch, a second image super-resolution network branch, a third image super-resolution network branch, and a fourth image super-resolution network branch.

9. The image super-resolution reconstruction method according to claim 1, characterized in that, The process of image reconstruction based on the super-resolution features to obtain a super-resolution image of the image to be reconstructed specifically includes: According to the preset feature channels, the low-frequency super-resolution features, horizontal high-frequency super-resolution features, vertical high-frequency super-resolution features and diagonal high-frequency super-resolution features are connected to obtain spliced ​​features; The stitched features are mapped to a spatial super-resolution image through inverse wavelet transform, which serves as the super-resolution image corresponding to the image to be reconstructed.

10. The image super-resolution reconstruction method according to claim 1, characterized in that, The lightweight image super-resolution network is generated using the following method: Obtain training samples, which include: sample images, true super-resolution images of the sample images, and fusion features of the sample images; The fusion features of the sample images are obtained by splicing the frequency domain features, spatial domain features, and frequency domain attention features of the sample images; the frequency domain features of the sample images include: low frequency domain features, horizontal high frequency domain features, vertical high frequency domain features, and diagonal high frequency domain features. The fusion features of the sample images are input into a preset lightweight neural network to obtain the super-resolution features of the sample images; The super-resolution features of the sample images include: low-frequency super-resolution features, horizontal high-frequency super-resolution features, vertical high-frequency super-resolution features, and diagonal high-frequency super-resolution features. Image reconstruction is performed based on the super-resolution features to obtain a generated super-resolution image of the sample image; Based on the real super-resolution image and the generated super-resolution image of the sample image, the network parameters of the preset lightweight neural network are corrected, and the preset lightweight neural network corresponding to the fusion feature input of the next sample image in the next training sample is executed to obtain the super-resolution features of the sample image. Based on the super-resolution features, image reconstruction is performed to obtain the generated super-resolution image of the next sample image. This process continues until the training status of the preset lightweight neural network meets the preset conditions to obtain the trained lightweight image super-resolution network.

11. The image super-resolution reconstruction method according to claim 10, characterized in that, The step of correcting the network parameters of the preset lightweight neural network based on the real super-resolution image and the generated super-resolution image of the sample image specifically includes: Feature extraction is performed on the real super-resolution image of the sample image to obtain the frequency domain features of the real super-resolution image; The frequency domain features of the real super-resolution image and the super-resolution features of the generated super-resolution image are respectively subjected to loss calculation to obtain the corresponding loss values; Based on each of the loss values ​​of the sample images, determine the total loss value of the sample images; The network parameters of the preset lightweight neural network are corrected based on the total loss value.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs, which can be executed by one or more processors to implement the steps in the image super-resolution reconstruction method as described in any one of claims 1-11.

13. A terminal, characterized in that, The terminal includes: a memory and a processor; the memory stores a computer-readable program that can be executed by the processor; when the processor executes the computer-readable program, it implements the steps of the image super-resolution reconstruction method as described in any one of claims 1-11.

Images