Image super-resolution reconstruction method, electronic device and storage medium based on FSRCNN and OPE

Abstract

A method, electronic device and storage medium for image super-resolution reconstruction based on FSRCNN and OPE, belonging to the technical field of image super-resolution reconstruction. In order to solve the problem of independent training models of image super-resolution algorithms at different upsampling rates, the present invention inputs low-resolution images in a data set into an FSRCNN network, extracts features of the image through a convolutional layer, outputs feature maps and inputs them into an OPE module, encodes the feature maps to obtain orthogonal position codes, and then uses the obtained orthogonal position codes to upsample the feature maps, calculates pixel values ​​of super-resolution images SR through linear combinations of OPE modules, generates preliminary SR images for post-processing, and obtains images after super-resolution processing; performs performance evaluation and calculates loss functions, and adjusts the FSRCNN network structure and parameters of the OPE module through back propagation according to the results of the loss function. The present invention saves training time.

Description

Technical Field

[0001] The present invention belongs to the technical field of image super-resolution reconstruction, and specifically relates to an image super-resolution reconstruction method based on FSRCNN and OPE, an electronic device and a storage medium. Background Art

[0002] Image resolution is an important indicator to measure the details and clarity of an image. High-resolution images can provide more information. In military, medical, industrial and daily life, the demand for high-resolution images is growing. However, due to technical and conditional limitations, the images actually acquired often have blur and noise problems, making it difficult to achieve the ideal resolution. Traditional methods of improving sensor technology are costly and inflexible, and are only applicable to the capture of new images, and cannot improve the resolution of existing low-resolution images. Therefore, it is particularly important to explore other methods to improve image resolution. Image super-resolution technology can restore high-resolution images from low-resolution images, improve image clarity, reduce blur and noise, and is widely used in high-definition television, medical imaging, satellite imaging, monitoring, and biometric information recognition.

[0003] This technology uses image processing technology to mine hidden information and generate high-resolution images. At present, this technology is mainly divided into three categories: interpolation-based methods, reconstruction-based methods, and deep learning-based methods. Interpolation-based methods infer unknown pixel values ​​through known pixel values, including nearest neighbor interpolation, bilinear interpolation, and bicubic interpolation. These methods are simple and fast, but have limitations in recovering high-frequency details. Reconstruction-based methods use image prior knowledge as constraints, and improve the performance of linear models and edge blur problems through iterative back projection, maximum a posteriori probability method, and convex set projection methods. Deep learning-based methods learn the mapping relationship between low-resolution and high-resolution images, and use neighborhood embedding, sparse representation, anchored neighbor regression, and deep learning methods to achieve image super-resolution reconstruction. Deep learning methods such as SRCNN learn image features through convolutional neural networks to achieve efficient super-resolution reconstruction, but face challenges such as model size, training time, and memory consumption. In addition, existing deep learning-based image super-resolution algorithms usually require independent models to be trained for specific upsampling rates, which limits the versatility of the algorithm in processing images of different scales. Summary of the invention

[0004] The problem to be solved by the present invention is that the existing image super-resolution algorithm requires independent training models at different upsampling rates. An image super-resolution reconstruction method, electronic device and storage medium based on FSRCNN and OPE are proposed.

[0005] To achieve the above object, the present invention is implemented through the following technical solutions:

[0006] An image super-resolution reconstruction method based on FSRCNN and OPE comprises the following steps:

[0007] S1. Collect high-resolution images or obtain high-resolution images based on the DIV2K Dataset open source dataset, obtain high-resolution images HR and corresponding low-resolution images LR after preprocessing, and construct a dataset for training image super-resolution reconstruction methods based on FSRCNN and OPE;

[0008] S2. Construct FSRCNN network structure and OPE module;

[0009] S3. Input the low-resolution image in the data set obtained in step S1 into the FSRCNN network, extract the features of the image through the convolution layer, and output the feature map F;

[0010] S4. Input the feature map obtained in step S3 into the OPE module, encode the feature map to obtain the orthogonal position code, and then use the obtained orthogonal position code to upsample the feature map, calculate the pixel value of the super-resolution image SR through the linear combination of the OPE module, and generate a preliminary SR image;

[0011] S5. Post-processing the preliminary SR image obtained in step S4 to obtain a super-resolution processed image;

[0012] S6. Perform performance evaluation on the super-resolution processed image obtained in step S5 and calculate the loss function. According to the result of the loss function, adjust the FSRCNN network structure and the parameters of the OPE module through back propagation, use the data set for iterative training, and output the reconstruction result from the LR image to the SR image.

[0013] Furthermore, the specific implementation method of step S1 includes the following steps:

[0014] S1.1. Downsample the high-resolution image to obtain a low-resolution image LR by using the bicubic interpolation method, and adjust the reduction ratio of the bicubic interpolation method to obtain low-resolution images LR with three ratios of 1 / 2, 1 / 4, and 1 / 8, and place HR and LR in different folders respectively;

[0015] S1.2. After normalizing the HR and LR obtained in step S1.1, the pixel value range is [0,1], and a data set for training the image super-resolution reconstruction method based on FSRCNN and OPE is obtained.

[0016] Furthermore, the specific implementation method of step S2 includes the following steps:

[0017] S2.1. Construct the FSRCNN network structure: the first layer is a 3x3 convolution layer, using 32 convolution kernels, followed by a ReLU activation function, the second layer is a 3x3 maximum pooling layer, the third layer is a 3x3 convolution layer, using 64 convolution kernels, followed by a ReLU activation function, and the fourth layer is a 1x1 convolution layer for compressing feature maps;

[0018] S2.2. Construct the OPE module: select sine and cosine functions as orthogonal bases, then set the maximum frequency and encoding length, select Adam as the optimizer for model training, and set the initial learning rate lr to 0.001.

[0019] Furthermore, the expression for extracting the features of the image in step S3 is:

[0020]

[0021] Among them, x m,n is the pixel value in the input image, is the weight of the convolution kernel, b i,j is the bias term, f is the ReLU activation function, y i,j is the pixel value of the output feature map, M×N is the size of the convolution kernel, where m and n represent the position coordinates of the pixel in the image, m is the horizontal coordinate, n is the vertical coordinate, M is the width of the image, and N is the height of the image;

[0022] The output of the FSRCNN network is a feature map F, and the dimension of F is H×W×C, where H is the height of the feature map, W is the width of the feature map, and C is the number of channels of the feature map.

[0023] Furthermore, the specific implementation method of step S4 includes the following steps:

[0024] S4.1. Define a set of orthogonal basis functions including orthogonal basis functions in the horizontal direction and orthogonal basis functions in the vertical direction, expressed as:

[0025]

[0026]

[0027] Among them, φ p (x) is the cosine basis function in the horizontal direction, φ q (y) is the cosine basis function in the vertical direction, ψ p (x) is the sine basis function in the horizontal direction, ψ q (y) is the sine basis function in the vertical direction, p and q are the frequency indexes of the orthogonal basis function, x and y are the position of the feature map in the horizontal direction and the vertical direction, respectively;

[0028] S4.2. For each pair of orthogonal basis functions, calculate the projection coefficient of the feature map on the orthogonal basis function, expressed as:

[0029]

[0030] Among them, F m,n,c is the pixel value of the feature map at position m, n and channel c, φ p (m) and φ q (n) are the values ​​of the sine basis function at positions m and n, respectively, ψ p (m) and ψ q (n) are the values ​​of the cosine basis function at positions m and n respectively;

[0031] S4.3. The feature map is upsampled using the projection coefficients of the feature map obtained on the orthogonal basis function. The pixel values ​​of the super-resolution image SR are calculated through the linear combination of the OPE modules to generate a preliminary SR image. The calculation expression is:

[0032]

[0033] Among them, I i,j is the pixel value of the high-resolution image at position i, j, K and L are the number of orthogonal basis functions in the horizontal and vertical directions, P p,q are the coefficients of the pth and qth basis functions in the encoded feature map.

[0034] Furthermore, step S5 performs post-processing on the preliminary SR image, uses the non-local mean technique to remove noise, then uses the Sobel operator to perform high-pass filtering, and finally obtains the super-resolution processed image through histogram equalization.

[0035] Furthermore, the specific implementation method of step S6 includes the following steps:

[0036] S6.1. Performance evaluation: The performance of the reconstructed super-resolution image is evaluated using the peak signal-to-noise ratio (PSNR) and the structural similarity (SSIM) indicators. The calculation expression of PSNR is:

[0037]

[0038] Among them, MAX I is the maximum pixel value of the image, MSE is the mean square error, and the calculation formula of MSE is:

[0039]

[0040] Among them, I HR is the original high-resolution image, I SRis the image after super-resolution processing, M and N are the number of rows and columns of the image respectively;

[0041] The calculation expression of SSIM is:

[0042]

[0043] in: and are the means of the original high-resolution image and the image after super-resolution processing, and are the variances of the original high-resolution image and the image after super-resolution processing, is the covariance of the original high-resolution image and the image after super-resolution processing, C1 and C2 are constants used to stabilize the denominator of SSIM;

[0044] S6.2. Set MSE and SSIM to jointly construct the loss function of model training, the expression is:

[0045] Loss=α·MSE(I,K)+(1-α)·(1-SSIM(I,K))

[0046] Among them, α is the weight parameter, ranging from [0,1]. The larger the Loss value, the worse the super-resolution image reconstructed by the model.

[0047] An electronic device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of an image super-resolution reconstruction method based on FSRCNN and OPE are implemented.

[0048] A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the image super-resolution reconstruction method based on FSRCNN and OPE is implemented.

[0049] Beneficial effects of the present invention:

[0050] The image super-resolution reconstruction method based on FSRCNN and OPE described in the present invention can handle the image super-resolution problem of any scale without the need to train an independent model for each scale, thus saving training time and computing resources.

[0051] The image super-resolution reconstruction method based on FSRCNN and OPE described in the present invention improves the accuracy of the upsampling process by utilizing the OPE module, reduces the blur and distortion introduced by the traditional interpolation method, and improves the clarity and visual quality of the image.

[0052] The image super-resolution reconstruction method based on FSRCNN and OPE described in the present invention can handle image super-resolution tasks of any scale by training the model only once.

[0053] The present invention discloses an image super-resolution reconstruction method based on FSRCNN and OPE, wherein the lightweight design of the FSRCNN network ensures the computational efficiency of the reconstruction process.

[0054] The effectiveness and potential of the image super-resolution reconstruction method based on FSRCNN and OPE in practical applications. With the continuous development of technology, the method of the present invention is expected to play an important role in more fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 This is a flow chart of an image super-resolution reconstruction method based on FSRCNN and OPE according to the present invention;

[0056] Figure 2 A flowchart of the training process of the method of the present invention;

[0057] Figure 3 Schematic diagram of the FSRCNN and OPE model structure described in the present invention. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the present invention more clear, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention, that is, the specific embodiments described are only part of the embodiments of the present invention, rather than all of the specific embodiments. The components of the specific embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations, and the present invention can also have other embodiments.

[0059] Therefore, the following detailed description of the specific embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents the selected specific embodiments of the present invention. Based on the specific embodiments of the present invention, all other specific embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present invention.

[0060] In order to further understand the content, features and effects of the present invention, the following specific implementation methods are given as examples, and the attached Figure 1 -Attached Figure 3 The detailed instructions are as follows: Specific implementation method one:

[0062] An image super-resolution reconstruction method based on FSRCNN and OPE comprises the following steps:

[0063] S1. Collect high-resolution images or obtain high-resolution images based on the DIV2K Dataset open source dataset, obtain high-resolution images HR and corresponding low-resolution images LR after preprocessing, and construct a dataset for training image super-resolution reconstruction methods based on FSRCNN and OPE;

[0064] Furthermore, the specific implementation method of step S1 includes the following steps:

[0065] S1.1. Downsample the high-resolution image to obtain a low-resolution image LR by using the bicubic interpolation method, and adjust the reduction ratio of the bicubic interpolation method to obtain low-resolution images LR with three ratios of 1 / 2, 1 / 4, and 1 / 8, and place HR and LR in different folders respectively;

[0066] S1.2. After normalizing the HR and LR obtained in step S1.1, the pixel value range is [0,1], and a data set for training the image super-resolution reconstruction method based on FSRCNN and OPE is obtained.

[0067] S2. Construct FSRCNN network structure and OPE module;

[0068] Furthermore, the specific implementation method of step S2 includes the following steps:

[0069] S2.1. Construct the FSRCNN network structure: the first layer is a 3x3 convolution layer, using 32 convolution kernels, followed by a ReLU activation function, the second layer is a 3x3 maximum pooling layer, the third layer is a 3x3 convolution layer, using 64 convolution kernels, followed by a ReLU activation function, and the fourth layer is a 1x1 convolution layer for compressing feature maps;

[0070] S2.2. Construct the OPE module: select sine and cosine functions as orthogonal bases, then set the maximum frequency and encoding length, select Adam as the optimizer for model training, and set the initial learning rate lr to 0.001.

[0071] S3. Input the low-resolution image in the data set obtained in step S1 into the FSRCNN network, extract the features of the image through the convolution layer, and output the feature map F;

[0072] Furthermore, the expression for extracting the features of the image in step S3 is:

[0073]

[0074] Among them, x m,nis the pixel value in the input image, is the weight of the convolution kernel, b i,j is the bias term, f is the ReLU activation function, y i,j is the pixel value of the output feature map, M×N is the size of the convolution kernel, where m and n represent the position coordinates of the pixel in the image, m is the horizontal coordinate, n is the vertical coordinate, M is the width of the image, and N is the height of the image;

[0075] The output of the FSRCNN network is a feature map F, and the dimension of F is H×W×C, where H is the height of the feature map, W is the width of the feature map, and C is the number of channels of the feature map.

[0076] S4. Input the feature map obtained in step S3 into the OPE module, encode the feature map to obtain the orthogonal position code, and then use the obtained orthogonal position code to upsample the feature map, calculate the pixel value of the super-resolution image SR through the linear combination of the OPE module, and generate a preliminary SR image;

[0077] Furthermore, the specific implementation method of step S4 includes the following steps:

[0078] S4.1. Define a set of orthogonal basis functions including orthogonal basis functions in the horizontal direction and orthogonal basis functions in the vertical direction, expressed as:

[0079]

[0080]

[0081] Among them, φ p (x) is the cosine basis function in the horizontal direction, φ q (y) is the cosine basis function in the vertical direction, ψ p (x) is the sine basis function in the horizontal direction, ψ q (y) is the sine basis function in the vertical direction, p and q are the frequency indexes of the orthogonal basis function, x and y are the position of the feature map in the horizontal direction and the vertical direction, respectively;

[0082] S4.2. For each pair of orthogonal basis functions, calculate the projection coefficient of the feature map on the orthogonal basis function, expressed as:

[0083]

[0084] Among them, F m,n,c is the pixel value of the feature map at position m, n and channel c, φ p (m) and φ q (n) are the values ​​of the sine basis function at positions m and n, respectively, ψp (m) and ψ q (n) are the values ​​of the cosine basis function at positions m and n respectively;

[0085] S4.3. The feature map is upsampled using the projection coefficients of the feature map obtained on the orthogonal basis function. The pixel values ​​of the super-resolution image SR are calculated through the linear combination of the OPE modules to generate a preliminary SR image. The calculation expression is:

[0086]

[0087] Among them, I i,j is the pixel value of the high-resolution image at position i, j, K and L are the number of orthogonal basis functions in the horizontal and vertical directions, P p,q are the coefficients of the pth and qth basis functions in the encoded feature map;

[0088] S5. Post-processing the preliminary SR image obtained in step S4 to obtain a super-resolution processed image;

[0089] Furthermore, step S5 performs post-processing on the preliminary SR image, uses the non-local mean technique to remove noise, then uses the Sobel operator to perform high-pass filtering, and finally obtains the image after super-resolution processing through histogram equalization;

[0090] S6. The super-resolution processed image obtained in step S5 is subjected to performance evaluation and a loss function is calculated. According to the result of the loss function, the FSRCNN network structure and the parameters of the OPE module are adjusted by back propagation, and iterative training is performed using the data set to output the reconstruction result from the LR image to the SR image.

[0091] Furthermore, the specific implementation method of step S6 includes the following steps:

[0092] S6.1. Performance evaluation: The performance of the reconstructed super-resolution image is evaluated using the peak signal-to-noise ratio (PSNR) and the structural similarity (SSIM) indicators. The calculation expression of PSNR is:

[0093]

[0094] Among them, MAX I is the maximum pixel value of the image, MSE is the mean square error, and the calculation formula of MSE is:

[0095]

[0096] Among them, I HR is the original high-resolution image, I SR is the image after super-resolution processing, M and N are the number of rows and columns of the image respectively;

[0097] The calculation expression of SSIM is:

[0098]

[0099] in: and are the means of the original high-resolution image and the image after super-resolution processing, and are the variances of the original high-resolution image and the image after super-resolution processing, is the covariance of the original high-resolution image and the image after super-resolution processing, C1 and C2 are constants used to stabilize the denominator of SSIM;

[0100] S6.2. Set MSE and SSIM to jointly construct the loss function of model training, the expression is:

[0101] Loss=α·MSE(I,K)+(1-α)·(1-SSIM(I,K))

[0102] Among them, α is the weight parameter, ranging from [0,1]. The larger the Loss value, the worse the super-resolution image reconstructed by the model.

[0103] Furthermore, the dataset is used for iterative training. Figure 2 As shown, the specific implementation method includes the following steps:

[0104] S6.2.1. Input the training set, validation set, upsampling rate, and number of iterations epoch into the constructed FSRCNN network structure and OPE module for training to obtain the training results;

[0105] S6.2.2. Use the training results to calculate the average peak signal-to-noise ratio (PSNR) of the validation set, and then determine whether the constructed FSRCNN network structure and OPE module are the best model. If yes, update the best model; otherwise, update the number of iterations for the next round of training until the best model is output.

[0106] This implementation is used for the following verification: a 160x160 pixel face image is input into the model as a test case, and the upsampling multiples of the model are set to 2, 4, and 6 times, respectively, to obtain HR images of three resolutions. Through the above implementation steps, we obtain a high-resolution face image, whose resolution is increased from 160x160 pixels to 320x320 and other pixels. Only one training is required to obtain images of various resolutions. The reconstructed HR image has significant improvements in details, textures, edges, etc., and there are no obvious artifacts or noise. The quantitative evaluation indicators PSNR and SSIM have reached a high level. Specific implementation method 2:

[0108] An electronic device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of an image super-resolution reconstruction method based on FSRCNN and OPE described in a specific implementation method 1 are implemented.

[0109] The computer device of the present invention may be a device including a processor and a memory, such as a single chip microcomputer including a central processing unit. Furthermore, the processor is used to implement the steps of the above-mentioned image super-resolution reconstruction method based on FSRCNN and OPE when executing the computer program stored in the memory.

[0110] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0111] The memory may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Specific implementation method three:

[0113] A computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, an image super-resolution reconstruction method based on FSRCNN and OPE described in a specific implementation method 1 is implemented.

[0114] The computer-readable storage medium of the present invention can be any form of storage medium that can be read by a processor of a computer device, including but not limited to non-volatile memory, volatile memory, ferroelectric memory, etc. The computer-readable storage medium stores a computer program. When the processor of the computer device reads and executes the computer program stored in the memory, the steps of the above-mentioned image super-resolution reconstruction method based on FSRCNN and OPE can be implemented.

[0115] The computer program includes computer program code, which may be in source code form, object code form, executable file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0116] It should be noted that relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0117] Although the present application has been described above with reference to specific embodiments, various modifications may be made thereto and parts thereof may be replaced with equivalents without departing from the scope of the present application. In particular, as long as there is no structural conflict, the various features in the specific embodiments disclosed in the present application may be used in combination with each other in any manner, and the fact that these combinations are not exhaustively described in this specification is only for the sake of omitting space and saving resources. Therefore, the present application is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. An image super-resolution reconstruction method based on FSRCNN and OPE, characterized in that: The steps include: S1. Collect high-resolution images or obtain high-resolution images based on the DIV2K Dataset open source dataset, obtain high-resolution images HR and corresponding low-resolution images LR after preprocessing, and construct a dataset for training image super-resolution reconstruction methods based on FSRCNN and OPE; S2. Construct FSRCNN network structure and OPE module; The specific implementation method of step S2 includes the following steps: S2.

1. Construct the FSRCNN network structure: the first layer is a 3x3 convolution layer, using 32 convolution kernels, followed by a ReLU activation function, the second layer is a 3x3 maximum pooling layer, the third layer is a 3x3 convolution layer, using 64 convolution kernels, followed by a ReLU activation function, and the fourth layer is a 1x1 convolution layer for compressing feature maps; S2.

2. Construct the OPE module: select sine and cosine functions as orthogonal bases, then set the maximum frequency and encoding length, select Adam as the optimizer for model training, and set the initial learning rate lr to 0.001; S3. Input the low-resolution image in the data set obtained in step S1 into the FSRCNN network, extract the features of the image through the convolution layer, and output the feature map F; S4. Input the feature map obtained in step S3 into the OPE module, encode the feature map to obtain the orthogonal position code, and then use the obtained orthogonal position code to upsample the feature map, calculate the pixel value of the super-resolution image SR through the linear combination of the OPE module, and generate a preliminary SR image; The specific implementation method of step S4 includes the following steps: S4.

1. Define a set of orthogonal basis functions including orthogonal basis functions in the horizontal direction and orthogonal basis functions in the vertical direction, expressed as: Among them, φ p (x) is the cosine basis function in the horizontal direction, φ q (y) is the cosine basis function in the vertical direction, ψ p (x) is the sine basis function in the horizontal direction, ψ q (y) is the sine basis function in the vertical direction, p and q are the frequency indexes of the orthogonal basis function, x and y are the position of the feature map in the horizontal direction and the vertical direction, respectively; S4.

2. For each pair of orthogonal basis functions, calculate the projection coefficient of the feature map on the orthogonal basis function, expressed as: Among them, F m,n,c is the pixel value of the feature map at position m, n and channel c, φ p (m) and φ q (n) are the values ​​of the sine basis function at positions m and n, respectively, ψ p (m) and ψ q (n) are the values ​​of the cosine basis function at positions m and n respectively; S4.

3. The feature map is upsampled using the projection coefficients of the feature map obtained on the orthogonal basis function. The pixel values ​​of the super-resolution image SR are calculated through the linear combination of the OPE modules to generate a preliminary SR image. The calculation expression is: Among them, I i,j is the pixel value of the high-resolution image at position i, j, K and L are the number of orthogonal basis functions in the horizontal and vertical directions, P p,q are the coefficients of the pth and qth basis functions in the encoded feature map; S5. Post-processing the preliminary SR image obtained in step S4 to obtain a super-resolution processed image; S6. Perform performance evaluation on the super-resolution processed image obtained in step S5 and calculate the loss function. According to the result of the loss function, adjust the FSRCNN network structure and the parameters of the OPE module through back propagation, use the data set for iterative training, and output the reconstruction result from the LR image to the SR image.

2. The image super-resolution reconstruction method based on FSRCNN and OPE according to claim 1, characterized in that: The specific implementation method of step S1 includes the following steps: S1.

1. Downsample the high-resolution image to obtain a low-resolution image LR by using the bicubic interpolation method, and adjust the reduction ratio of the bicubic interpolation method to obtain low-resolution images LR with three ratios of 1 / 2, 1 / 4, and 1 / 8, and place HR and LR in different folders respectively; S1.

2. After normalizing the HR and LR obtained in step S1.1, the pixel value range is [0,1], and a data set for training the image super-resolution reconstruction method based on FSRCNN and OPE is obtained.

3. The image super-resolution reconstruction method based on FSRCNN and OPE according to claim 2, characterized in that: The expression of the feature of the image extracted in step S3 is: Among them, x m,n is the pixel value in the input image, is the weight of the convolution kernel, b i,j is the bias term, f is the ReLU activation function, y i,j is the pixel value of the output feature map, M×N is the size of the convolution kernel, where m and n represent the position coordinates of the pixel in the image, m is the horizontal coordinate, n is the vertical coordinate, M is the width of the image, and N is the height of the image; The output of the FSRCNN network is a feature map F, and the dimension of F is H×W×C, where H is the height of the feature map, W is the width of the feature map, and C is the number of channels of the feature map.

4. The image super-resolution reconstruction method based on FSRCNN and OPE according to claim 3 is characterized in that: Step S5 performs post-processing on the preliminary SR image, uses the non-local mean technique to denoise, then uses the Sobel operator to perform high-pass filtering, and finally obtains the super-resolution processed image through histogram equalization.

5. The image super-resolution reconstruction method based on FSRCNN and OPE according to claim 4, characterized in that: The specific implementation method of step S6 includes the following steps: S6.

1. Performance evaluation: The performance of the reconstructed super-resolution image is evaluated using the peak signal-to-noise ratio (PSNR) and the structural similarity (SSIM) indicators. The calculation expression of PSNR is: Among them, MAX I is the maximum pixel value of the image, MSE is the mean square error, and the calculation formula of MSE is: Among them, I HR is the original high-resolution image, I SR is the image after super-resolution processing, M and N are the number of rows and columns of the image respectively; The calculation expression of SSIM is: in: and are the means of the original high-resolution image and the image after super-resolution processing, and are the variances of the original high-resolution image and the image after super-resolution processing, is the covariance of the original high-resolution image and the image after super-resolution processing, C1 and C2 are constants used to stabilize the denominator of SSIM; S6.

2. Set MSE and SSIM to jointly construct the loss function of model training, the expression is: Loss=α·MSE(I,K)+(1-α)·(1-SSIM(I,K)) Among them, α is the weight parameter, ranging from [0,1]. The larger the Loss value, the worse the super-resolution image reconstructed by the model.

6. An electronic device, characterized in that: It includes a memory and a processor, the memory stores a computer program, and the processor implements the steps of an image super-resolution reconstruction method based on FSRCNN and OPE as described in any one of claims 1 to 5 when executing the computer program.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the image super-resolution reconstruction method based on FSRCNN and OPE described in any one of claims 1 to 5 is implemented.

Images