Image super-resolution processing method and device, electronic equipment and storage medium

Abstract

The application provides an image super-resolution processing method and device, electronic equipment and storage medium. The method comprises the following steps: receiving a user input image to be processed; inputting the image to be processed into a lightweight neural network model, extracting detailed features of the image to be processed by the lightweight neural network, and fusing the extracted detailed feature map and the image to be processed to obtain a super-resolution image corresponding to the image to be processed. The user can directly process image data online on the terminal device, and the lightweight neural network composed of a convolution layer, a plurality of structure blocks and an up-sampling layer can greatly improve the image processing speed, avoid time-consuming network transmission and achieve real-time image processing.

Description

Image super-resolution processing methods, apparatus, electronic devices and storage media Technical Field

[0001] This application relates to the field of image processing, and more specifically, to an image super-resolution processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the rapid development of science and information technology, the application of images and videos has spread to all areas of human society. People's requirements for image clarity and quality are also increasing, and the demand for high-resolution images and videos is gradually growing. Image super-resolution technology therefore has important applications. Image super-resolution technology can recover high-resolution images from low-resolution images.

[0003] In existing technologies, most image super-resolution algorithms are based on deep learning and use relatively large convolutional neural networks, which can only be processed in the cloud. At the same time, the long processing time and network transmission time make it impossible to process video in real time. Summary of the Invention

[0004] The purpose of this application is to address the shortcomings of the prior art by providing an image super-resolution processing method, apparatus, electronic device, and storage medium to achieve online image processing and improve image processing efficiency.

[0005] To achieve the above objectives, the technical solutions adopted in the embodiments of this application are as follows:

[0006] In a first aspect, embodiments of this application provide an image super-resolution processing method, the method comprising:

[0007] Receive the image to be processed from the user input;

[0008] The image to be processed is input into a lightweight neural network model, which extracts detailed features from the image and fuses the extracted detailed feature maps with the image to be processed to obtain a super-resolution image corresponding to the image to be processed. The lightweight neural network model includes convolutional layers, multiple structural blocks, and upsampling layers connected in sequence. Each structural block includes multiple residual convolution processing modules and a total compressed convolution module connected in sequence. Each residual convolution processing module includes an attention residual module and a compressed convolution module. The output of the compressed convolution module of each residual convolution processing module is connected to the total compressed convolution module.

[0009] Optionally, the step of inputting the image to be processed into a lightweight neural network model, and having the lightweight neural network model extract detailed features from the image to be processed, includes:

[0010] The image to be processed is input into the convolutional layer for convolution operation to obtain the convolutional feature map;

[0011] The convolutional feature map is input into the first structural block, and is processed sequentially by each residual convolution processing module and the total compression convolution module in the first structural block to obtain the processed feature map.

[0012] The processed feature map is processed sequentially by each subsequent structure block to obtain the feature map to be sampled;

[0013] The feature map to be sampled is input into the upsampling layer for upsampling operation to obtain the detail feature map.

[0014] Optionally, the convolutional feature map is input into the first structural block, and then processed sequentially by the residual convolution processing modules and the overall compression convolution module in the first structural block to obtain the processed feature map, including:

[0015] The convolutional feature map is input into the attention residual module of the first residual convolution processing module in the first structural block to obtain the initial residual feature map.

[0016] The initial residual feature map is input into the attention residual module in the residual convolution processing module after the first residual convolution processing module, and the attention residual modules in each residual convolution processing module after the first residual convolution processing module perform residual processing in sequence to obtain the residual feature map output by each attention residual module.

[0017] The initial residual feature map is input into the compressed convolution module of the first residual convolution processing module to obtain the initial structure block convolution feature map. The compressed convolution modules in each residual convolution processing module after the first residual convolution processing module perform convolution processing on the residual feature maps output by each attention residual module to obtain the structure block convolution feature maps output by each compressed convolution module.

[0018] The initial convolutional feature map of the structural blocks and the convolutional feature maps of the structural blocks output by each compressed convolution module are input into the total compressed convolution module for convolution processing to obtain the processed feature map.

[0019] Optionally, the step of inputting the convolutional feature map into the attention residual module of the first residual convolution processing module in the first structural block to obtain the initial residual feature map includes:

[0020] The convolutional feature map is input into the attention residual module of the first residual convolution processing module in the first structural block. The attention residual module determines the channel variance and channel mean of the convolutional feature map. The channel mean and channel variance are summed. The sum is then subjected to erosion convolution, dilation convolution and normalization processing in sequence to obtain the normalized feature map.

[0021] The convolutional feature map is then subjected to convolution processing to obtain a new convolutional feature map;

[0022] The convolutional feature map, the normalized feature map, and the new convolutional feature map are fused to obtain the initial residual feature map.

[0023] Optionally, the lightweight neural network model further includes an upsampling layer and a downsampling layer, wherein the input of the downsampling layer is connected to the output of the upsampling layer, and the output of the downsampling layer is connected to the input of the convolutional layer.

[0024] The image to be processed is input into a lightweight neural network model, which extracts detailed features from the image, including:

[0025] The image to be processed is input into the up-resampled layer and the down-resampled layer, and the up-resampled layer and the down-resampled layer perform sampling processing in sequence to obtain the sampled feature map;

[0026] The sampled feature map is input into the convolutional layer, and the detailed feature map is obtained through processing by the convolutional layer, each of the structural blocks, and the upsampling layer.

[0027] Optionally, after inputting the image to be processed into a lightweight neural network model, and having the lightweight neural network model extract detailed features from the image to be processed, the process further includes:

[0028] If the image to be processed is a frame from a video, then the detail feature map is stored in the cache pool;

[0029] After receiving the next frame of the image to be processed, the next frame is input into the lightweight neural network model. The lightweight neural network model extracts detail features from the next frame based on the detail feature maps stored in the cache pool. Based on the extracted detail feature maps of the next frame and the next frame, the super-resolution image corresponding to the next frame is obtained, and the detail feature maps of the next frame are stored in the cache pool.

[0030] Optionally, the lightweight neural network model extracts detail features from the subsequent frame image based on the detail features of the image to be processed stored in the cache pool, including:

[0031] The next frame image is input to the up-resampled layer, and the convolutional feature map is obtained through the up-resampled layer, the down-resampled layer, and the convolutional layer.

[0032] The detailed feature map stored in the cache pool is input into the first video compression convolutional layer of the lightweight neural network model to obtain the first video compression convolutional feature map;

[0033] The convolutional feature map and the first video convolutional feature map are input into the second video compression convolutional layer of the lightweight neural network model for convolution processing to obtain the second video convolutional feature map.

[0034] The second video convolutional feature map is input into the first structural block of the lightweight neural network model, and is processed sequentially by each residual convolution processing module and the total compression convolution module in the first structural block to obtain the processed feature map.

[0035] The processed feature map is processed sequentially by each subsequent structure block to obtain the feature map to be sampled;

[0036] The feature map to be sampled is input into the upsampling layer for upsampling processing to obtain the detail feature map of the next frame image.

[0037] Secondly, embodiments of this application also provide an image super-resolution processing apparatus, the apparatus comprising:

[0038] The receiving module is used to receive the image to be processed input by the user;

[0039] The processing module is used to input the image to be processed into a lightweight neural network model, whereby the lightweight neural network extracts detail features from the image to be processed and fuses the extracted detail feature maps with the image to be processed to obtain a super-resolution image corresponding to the image to be processed. The lightweight neural network model includes convolutional layers, multiple structural blocks, and upsampling layers connected in sequence. Each structural block includes multiple residual convolution processing modules and a total compression convolution module connected in sequence. Each residual convolution processing module includes an attention residual module and a compression convolution module. The output of the compression convolution module of each residual convolution processing module is connected to the total compression convolution module.

[0040] Optionally, the processing module is specifically used for:

[0041] The image to be processed is input into the convolutional layer for convolution operation to obtain the convolutional feature map;

[0042] The convolutional feature map is input into the first structural block, and is processed sequentially by each residual convolution processing module and the total compression convolution module in the first structural block to obtain the processed feature map.

[0043] The processed feature map is processed sequentially by each subsequent structure block to obtain the feature map to be sampled;

[0044] The feature map to be sampled is input into the upsampling layer for upsampling operation to obtain the detail feature map.

[0045] Optionally, the processing module is specifically used for:

[0046] The convolutional feature map is input into the attention residual module of the first residual convolution processing module in the first structural block to obtain the initial residual feature map.

[0047] The initial residual feature map is input into the attention residual module in the residual convolution processing module after the first residual convolution processing module, and the attention residual modules in each residual convolution processing module after the first residual convolution processing module perform residual processing in sequence to obtain the residual feature map output by each attention residual module.

[0048] The initial residual feature map is input into the compressed convolution module of the first residual convolution processing module to obtain the initial structure block convolution feature map. The compressed convolution modules in each residual convolution processing module after the first residual convolution processing module perform convolution processing on the residual feature maps output by each attention residual module to obtain the structure block convolution feature maps output by each compressed convolution module.

[0049] The initial convolutional feature map of the structural blocks and the convolutional feature maps of the structural blocks output by each compressed convolution module are input into the total compressed convolution module for convolution processing to obtain the processed feature map.

[0050] Optionally, the processing module is specifically used for:

[0051] The convolutional feature map is input into the attention residual module of the first residual convolution processing module in the first structural block. The attention residual module determines the channel variance and channel mean of the convolutional feature map. The channel mean and channel variance are summed. The sum is then subjected to erosion convolution, dilation convolution and normalization processing in sequence to obtain the normalized feature map.

[0052] The convolutional feature map is then subjected to convolution processing to obtain a new convolutional feature map;

[0053] The convolutional feature map, the normalized feature map, and the new convolutional feature map are fused to obtain the initial residual feature map.

[0054] Optionally, the lightweight neural network model further includes an upsampling layer and a downsampling layer, wherein the input of the downsampling layer is connected to the output of the upsampling layer, and the output of the downsampling layer is connected to the input of the convolutional layer.

[0055] The processing module is specifically used for:

[0056] The image to be processed is input into the up-resampled layer and the down-resampled layer, and the up-resampled layer and the down-resampled layer perform sampling processing in sequence to obtain the sampled feature map;

[0057] The sampled feature map is input into the convolutional layer, and the detailed feature map is obtained through processing by the convolutional layer, each of the structural blocks, and the upsampling layer.

[0058] Optionally, the processing module is specifically used for:

[0059] If the image to be processed is a frame from a video, then the detail feature map is stored in the cache pool;

[0060] After receiving the next frame of the image to be processed, the next frame is input into the lightweight neural network model. The lightweight neural network model extracts detail features from the next frame based on the detail feature maps stored in the cache pool. Based on the extracted detail feature maps of the next frame and the next frame, the super-resolution image corresponding to the next frame is obtained, and the detail feature maps of the next frame are stored in the cache pool.

[0061] Optionally, the processing module is specifically used for:

[0062] The next frame image is input to the up-resampled layer, and the convolutional feature map is obtained through the up-resampled layer, the down-resampled layer, and the convolutional layer.

[0063] The detailed feature map stored in the cache pool is input into the first video compression convolutional layer of the lightweight neural network model to obtain the first video compression convolutional feature map;

[0064] The convolutional feature map and the first video convolutional feature map are input into the second video compression convolutional layer of the lightweight neural network model for convolution processing to obtain the second video convolutional feature map.

[0065] The second video convolutional feature map is input into the first structural block of the lightweight neural network model, and is processed sequentially by each residual convolution processing module and the total compression convolution module in the first structural block to obtain the processed feature map.

[0066] The processed feature map is processed sequentially by each subsequent structure block to obtain the feature map to be sampled;

[0067] The feature map to be sampled is input into the upsampling layer for upsampling processing to obtain the detail feature map of the next frame image.

[0068] Thirdly, embodiments of this application also provide an electronic device, including: a processor, a storage medium, and a bus, wherein the storage medium stores program instructions executable by the processor, and when the application runs, the processor communicates with the storage medium via the bus, and the processor executes the program instructions to perform the steps of the image super-resolution processing method described in the first aspect above.

[0069] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which is read and executes the steps of the image super-resolution processing method described in the first aspect.

[0070] The beneficial effects of this application are:

[0071] This application provides an image super-resolution processing method, apparatus, electronic device, and storage medium. By extracting detailed features from the image to be processed using a lightweight neural network in a terminal device, and fusing the extracted detailed feature map with the image to be processed to obtain the corresponding super-resolution image, users can directly process image data online on the terminal device. Furthermore, the lightweight neural network composed of convolutional layers, multiple structural blocks, and upsampling layers can greatly improve the speed of image processing, avoid the time consumption of network transmission, and achieve the purpose of real-time image processing.

[0072] Secondly, by upsampling and downsampling the image to be processed, it is possible to process images of any size, adapting to more application scenarios and flexibly meeting user needs.

[0073] In addition, by adding a cache pool of detail feature maps to the video images, a lightweight neural network model can extract detail features from the images based on the detail feature maps in the cache pool, which can make the extracted video image feature maps more detailed and the visual effect more delicate. Attached Figure Description

[0074] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0075] Figure 1 is a flowchart illustrating an image super-resolution processing method provided in an embodiment of this application;

[0076] Figure 2 is a schematic diagram of the structure of a lightweight neural network model provided in an embodiment of this application;

[0077] Figure 3 is a structural schematic diagram of a structural block provided in an embodiment of this application;

[0078] Figure 4 is a flowchart illustrating another image super-resolution processing method provided in an embodiment of this application;

[0079] Figure 5 is a flowchart illustrating another image super-resolution processing method provided in an embodiment of this application;

[0080] Figure 6 is a schematic diagram of an attention residual module provided in an embodiment of this application, as shown in Figure 6;

[0081] Figure 7 is a flowchart illustrating another image super-resolution processing method provided in an embodiment of this application;

[0082] Figure 8 is a schematic diagram of another lightweight neural network structure provided in an embodiment of this application;

[0083] Figure 9 is a flowchart illustrating a video super-resolution processing method provided in an embodiment of this application;

[0084] Figure 10 is a schematic diagram of the structure of another lightweight neural network model provided in an embodiment of this application;

[0085] Figure 11 is a flowchart illustrating another video super-resolution processing method provided in an embodiment of this application;

[0086] Figure 12 is a schematic diagram of an apparatus for an image super-resolution processing method provided in an embodiment of this application;

[0087] Figure 13 is a structural block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0088] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0089] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0090] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0091] During the process of saving, editing, transmitting, and sharing images, cameras may reduce image resolution due to the large amount of space and bandwidth consumed. Existing super-resolution technologies are based on two techniques: traditional algorithms and deep learning. Traditional algorithms lack the ability to generate image details, resulting in low-resolution images and poor performance. Existing deep learning technologies use large convolutional neural networks, which require cloud processing. This not only results in long processing times but also incurs network transmission time costs, making online image data processing impossible.

[0092] In addition, existing super-resolution technologies only support fixed magnification, such as 2x, 4x or 8x, and cannot support arbitrary magnification. In other words, they cannot support any image data being magnified to other arbitrary sizes of super-resolution images.

[0093] The method in this application embodiment is applied to a terminal device, which may be a mobile phone, laptop, tablet computer, PDA, desktop computer, or other terminal device with a display screen. This terminal device can acquire image data input by the user and process the received image data stream using the method of this application embodiment, enabling the processing of image data while simultaneously displaying it to the user, thus achieving real-time image data processing.

[0094] Figure 1 is a flowchart illustrating an image super-resolution processing method provided in an embodiment of this application. The execution subject of this method is the aforementioned terminal device. As shown in Figure 1, the method includes:

[0095] S101, Receive the image to be processed input by the user.

[0096] The image to be processed can be a grayscale image or a color image, and its resolution can be any size, such as a 720P resolution image, a 1080P resolution image, a 4K resolution image, or image data of other sizes.

[0097] Optionally, the number of channels in the image to be processed can be 1 channel or 3 channels, which in RGB color mode refers to a single red channel, green channel or blue channel.

[0098] Optionally, users can take a picture using the camera on their terminal device. When a new picture is taken, the image data in the gallery will be updated, and the terminal device can also obtain the newly taken image data in real time and use it as an image to be processed.

[0099] S102. Input the image to be processed into a lightweight neural network model. The lightweight neural network extracts detail features from the image to be processed and fuses the extracted detail feature map with the image to be processed to obtain the super-resolution image corresponding to the image to be processed.

[0100] The lightweight neural network model includes sequentially connected convolutional layers, multiple structural blocks, and upsampling layers, as shown in Figure 2. Figure 2 is a schematic diagram of the structure of a lightweight neural network model provided in this embodiment. The number of channels in the feature map output by the convolutional layer can be represented by F, and the number of multiple structural blocks can be represented by N. F and N are parameters in the lightweight neural network model. The larger the value, the better the effect, but the slower the speed. As an example, when F is set to 48 and N is set to 4, the lightweight neural network model can achieve real-time image processing and obtain a high-resolution image.

[0101] Each structural block includes multiple residual convolution processing modules and a total compressed convolution module connected in sequence. Each residual convolution processing module includes an attention residual module and a compressed convolution module. The output of the compressed convolution module of each residual convolution processing module is connected to the total compressed convolution module, as shown in Figure 3. Figure 3 is a schematic diagram of the structure of a structural block provided in an embodiment of this application. The number of input channels of each compressed convolution module is F, which is the number of channels of the feature map output by the convolutional layer. The number of output channels of each compressed convolution module is F / 2. The number of input channels of the total compressed convolution module is the sum of the number of output channels of each compressed convolution module, and the number of output channels of the total compressed convolution module is F. For example, if there are 4 compressed convolution modules, the number of input channels of the total compressed convolution module is 2F. As an example, the multiple residual convolution processing modules can be selected as 4 layers, then each structural block can include 4 layers of residual convolution processing modules connected in sequence.

[0102] In this embodiment, by extracting detailed features from the image to be processed using a lightweight neural network in the terminal device, and fusing the extracted detailed feature map with the image to be processed to obtain the corresponding super-resolution image, users can directly process image data online on the terminal device. Furthermore, the lightweight neural network composed of convolutional layers, multiple structural blocks, and upsampling layers can greatly improve the speed of image processing, avoid the time consumption of network transmission, and achieve the goal of real-time image processing.

[0103] Figure 4 is a flowchart illustrating another image super-resolution processing method provided in this application embodiment. As shown in Figure 4, in step S102 above, the image to be processed is input into a lightweight neural network model, and the lightweight neural network extracts detailed features from the image to be processed. This may include:

[0104] S201. Output the image to be processed to the convolutional layer for convolution operation to obtain the convolutional feature map.

[0105] The convolutional kernel of the convolutional layer can be 3×3, and the number of channels of the feature map after convolution is F. Then, for an input 1-channel or 3-channel image to be processed, after the convolution operation is performed by the convolutional layer, the resulting convolutional feature map is an F-channel convolutional feature map, for example, a 48-channel convolutional feature map.

[0106] S202. Input the convolutional feature map into the first structural block, and process it sequentially by the residual convolution processing modules and the total compression convolution module in the first structural block to obtain the processed feature map.

[0107] In this structure block, the residual convolution processing modules are connected sequentially, and the output of the compressed convolution module of each residual convolution processing module is connected to the total compressed convolution module. After receiving the convolutional feature map in the first structure block, the feature maps output by each residual convolution processing module can be obtained using a preset method based on the sequentially connected residual convolution processing modules. These feature maps are the outputs of the compressed convolution modules of each residual convolution processing module. The feature maps output by the compressed convolution modules of each residual convolution processing module are then input into the total compressed convolution module. After processing by the total compressed convolution module, the final processed feature map is obtained. This processed feature map is the output of the first structure block.

[0108] S203. The processed feature map is processed sequentially by each structure block after the first structure block to obtain the feature map to be sampled.

[0109] The processing method of the input feature map input to the first structural block for each structural block after the first structural block is the same as the processing method of the convolutional feature map for the first structural block in step S202 above. The only difference is that the input feature maps of each structural block connected in sequence after the first structural block are different. The input feature map of each structural block after the first structural block is the output feature map of the previous structural block. The structure of each structural block after the first structural block is the same as the structure of the first structural block.

[0110] For example, if there are four structural blocks, such as the first structural block, the second structural block, the third structural block, and the fourth structural block; for the first structural block, the input feature map is the convolutional feature map, and the output feature map is the processed feature map in S202 above; for the second structural block, the input feature map is the processed feature map of the first structural block, and the output feature map is the processed feature map of the second structural block; for the third structural block, the input feature map is the processed feature map of the second structural block, and the output feature map is the processed feature map of the third structural block; for the fourth structural block, the input feature map is the processed feature map of the third structural block, and the output feature map is the processed feature map of the fourth structural block. For a larger number of structural blocks, the input feature map and output feature map of each structural block can be obtained in the same way.

[0111] Optionally, the processed feature map output by the last structural block can be used as the feature map to be sampled. For example, in the above four structural blocks, the feature map to be sampled is the processed feature map of the fourth structural block.

[0112] S204. Input the feature map to be sampled into the upsampling layer to perform upsampling operation and obtain the detail feature map.

[0113] The upsampling layer can use the pixel shuuffle method, the deconvolution sampling method, the upsampling + convolution method, or other upsampling methods. This embodiment does not limit the method.

[0114] Optionally, the feature map to be sampled obtained in S203 above can be input into the upsampling layer. The upsampling layer can use any of the upsampling methods mentioned above to perform upsampling operations on the feature map to be sampled, so that the feature map to be sampled can be magnified by any factor, and the upsampled feature map can be output as a detail feature map.

[0115] In this embodiment, feature extraction of the image to be processed is performed through convolutional layers, multiple sequentially connected structural layers, and upsampling layers, which makes the details in the extracted detail feature map more accurate.

[0116] Figure 5 is a flowchart illustrating another image super-resolution processing method provided in this application embodiment. As shown in Figure 5, in step S202 above, the residual convolution processing modules and the total compression convolution module in the first structural block process the data sequentially to obtain the processed feature map, which may include:

[0117] S301. Input the convolutional feature map into the attention residual module of the first residual convolution processing module in the first structural block to obtain the initial residual feature map.

[0118] The initial residual post-feature map is the output feature map of the attention residual module of the first residual convolution processing module.

[0119] S302. Input the initial residual feature map into the attention residual module in the residual convolution processing module after the first residual convolution processing module, and have the attention residual modules in each residual convolution processing module after the first residual convolution processing module perform residual processing in sequence to obtain the residual feature map output by each attention residual module.

[0120] Optionally, each residual convolutional processing module includes an attention residual module and a compression convolutional module. The feature map output by the attention residual module in each residual convolutional processing module is input to the compression convolutional module in that residual convolutional processing module and the attention residual module of the next residual convolutional processing module in that residual convolutional processing module.

[0121] For example, if the structure block includes four residual convolution processing modules, such as a first residual convolution processing module, a second residual convolution processing module, a third residual convolution processing module, and a fourth residual convolution processing module. The input feature map for the first residual convolution processing module is the convolutional feature map in S301 described above. After residual processing by the attention residual module in the first residual convolution processing module, an initial residual feature map is obtained. This initial residual feature map is then input to the attention residual module in the second residual convolution processing module, and after residual processing, a second residual feature map is obtained, output by the attention residual module in the second residual convolution processing module. The second residual feature map is then input to the attention residual module in the third residual convolution processing module, and after residual processing, a third residual feature map is obtained, output by the attention residual module in the third residual convolution processing module. The third residual feature map is then input to the attention residual module in the fourth residual convolution processing module, and after residual processing, a fourth residual feature map is obtained, output by the attention residual module in the fourth residual convolution processing module. For more residual convolution processing modules, the same principle can be applied to obtain the residual feature maps output by each attention residual module in each residual convolution processing module.

[0122] S303. Input the initial residual feature map into the compressed convolution module of the first residual convolution processing module to obtain the initial structure block convolution feature map. Then, the compressed convolution modules in each residual convolution processing module after the first residual convolution processing module perform convolution processing on the residual feature maps output by each attention residual module to obtain the structure block convolution feature maps output by each compressed convolution module.

[0123] For example, continuing with the four residual convolution processing modules mentioned above, for the first residual convolution processing module, the input feature map of the compressed convolution module in this module is the initial residual feature map output by the attention residual module of this module. After processing by the compressed convolution module, the initial structural block convolution feature map output by the compressed convolution module is obtained. For the second residual convolution processing module, the second residual feature map output by the attention residual module in the second residual convolution processing module is input to the compressed convolution module in the second residual convolution processing module. In the convolution module, the convolutional feature map of the second structure block is obtained; for the third residual convolution processing module, the third residual feature map output by the attention residual module in the third residual convolution processing block is input into the compression convolution module in the third residual convolution processing block to obtain the convolutional feature map of the third structure block; for the fourth residual convolution processing module, the fourth residual feature map output by the attention residual module in the fourth residual convolution processing block is input into the compression convolution module in the fourth residual convolution processing block to obtain the convolutional feature map of the fourth structure block.

[0124] S304. Input the initial convolutional feature map of the structure blocks and the convolutional feature maps of the structure blocks output by each compressed convolution module into the total compressed convolution module for convolution processing to obtain the processed feature map.

[0125] For example, continuing with the four residual convolution processing modules mentioned above, the feature maps after convolution of the initial structure block, the second structure block, the third structure block, and the fourth structure block can all be input into the total compression convolution module for compression convolution processing to obtain the processed feature map. This processed feature map refers to the processed feature map output by the first structure block.

[0126] It is worth noting that the processing of the feature maps input to each structure block after the first structure block is the same as the process in S301-S304 above. The only difference is that the input feature map of other structure blocks after the first structure block is the output feature map of the previous structure block. Each structure block can output the processed feature map corresponding to that structure block, and the processed feature map output by the last structure block is used as the feature map to be sampled.

[0127] In this embodiment, the attention residual module and the compression convolution module in each residual convolution processing module of each structural block can make the feature extraction of the input feature map more refined and obtain more detailed feature maps.

[0128] Figure 6 is a schematic diagram of the structure of an attention residual module provided in an embodiment of this application. As shown in Figure 6, the attention residual module includes a calculation of the variance and mean of the input feature map channels, an erosion convolutional layer, a dilation convolutional layer, a normalization layer, and a convolutional layer.

[0129] In step S301 above, the convolutional feature map is input into the attention residual module of the first residual convolution processing module in the first structural block to obtain the initial residual feature map, which may include:

[0130] Optionally, the convolutional feature map is input into the attention residual module of the first residual convolution processing module in the first structural block. The attention residual module determines the channel variance and channel mean of the convolutional feature map. The channel variance and channel mean are summed, and the sum is sequentially subjected to erosion convolution, dilation convolution, and normalization to obtain the normalized feature map.

[0131] For example, the kernels of erosive convolution and dilation convolution can be 1×1. The number of input channels of erosive convolution is F and the number of output channels is H. The number of input channels of dilation convolution is H and the number of output channels is F. The number of channels F is the same as the number of channels F of the image to be processed mentioned above.

[0132] Optionally, the attention residual module can calculate the channel variance and channel mean of the feature map based on the channels of the convolutional feature map, sum the channel variance and channel mean and input them into the erosion convolution and dilation convolution to convolve the convolutional feature map, and then use normalization to normalize the feature map after dilation convolution to generate a normalized feature map.

[0133] Optionally, the convolutional feature map can be further convolved to obtain a new convolutional feature map.

[0134] Optionally, the convolutional feature map, the normalized feature map, and the new convolutional feature map can be fused to obtain the initial residual feature map. Specifically, the result of the new convolutional feature map and the normalized feature map can be multiplied, and then added to the convolutional feature map to obtain the initial residual feature map.

[0135] It is worth noting that the processing of the input feature map of each residual convolution processing module by the attention residual module is the same as the process of obtaining the initial residual feature map by the attention residual module of the first residual convolution processing module in the first structural block mentioned above, which will not be elaborated here.

[0136] Figure 7 is a flowchart illustrating another image super-resolution processing method provided in an embodiment of this application. As shown in Figure 7, in step S102 above, the image to be processed is input into a lightweight neural network model, and the lightweight neural network extracts detailed features from the image to be processed. This may include:

[0137] Optionally, the lightweight neural network module may further include an upsampling layer and a downsampling layer, wherein the input of the downsampling layer is connected to the output of the upsampling layer, and the output of the downsampling layer is connected to the input of the convolutional layer, as shown in Figure 8. Figure 8 is a schematic diagram of another lightweight neural network structure provided in an embodiment of this application.

[0138] S401. Input the image to be processed into the up-resampled layer and the down-resampled layer, and perform sampling processing by the up-resampled layer and the down-resampled layer in sequence to obtain the sampled feature map.

[0139] Optionally, for the input image to be processed, it is first upsampled through an upsampling layer to obtain an upsampled feature map. The upsampled feature map is then input into a downsampling layer for downsampling to obtain a sampled feature map. The sampled feature map meets the size requirements of the convolutional layer.

[0140] Upsampling can be performed using a sampling method of amplification + convolution. Amplification can be performed using interpolation methods such as nearest neighbor, bilinear, or bicubic, and the amplification factor can be any factor. Upsampling can also be performed using other sampling methods, which are not limited in this application embodiment.

[0141] Downsampling can be performed using methods such as stride convolution or shrink + convolution, and this application does not impose any restrictions on this method.

[0142] S402. Input the sampled feature map into the convolutional layer, and process it through the convolutional layer, each structural block and the upsampling layer to obtain the detailed feature map.

[0143] Optionally, the sampled feature map is processed by convolutional layers, each structural block, and upsampling layers to obtain a detailed feature map. This process is consistent with the method steps in the specific embodiments described above, and will not be repeated here.

[0144] In this embodiment, by upsampling and downsampling the image to be processed, the size of the feature map input to the convolutional layer is made to meet the requirements of the convolutional layer. This enables the processing of images of any size, adapting to more application scenarios and flexibly meeting user needs.

[0145] Figure 9 is a flowchart illustrating a video super-resolution processing method provided in an embodiment of this application. As shown in Figure 9, after the image to be processed is input into a lightweight neural network model in step S102, and the lightweight neural network extracts detailed features from the image to be processed, the method may further include:

[0146] S501. If the image to be processed is a frame from a video, the detail feature map is stored in the cache pool.

[0147] Optionally, a video is composed of consecutive image frames. If the video image is to be processed, the images in the video can be converted into multiple images according to the frame rate information. For the first frame of the video, the lightweight neural network model mentioned above can be used to extract detailed features, and the extracted detailed feature map is stored in the cache pool.

[0148] S502. After receiving the next frame of the image to be processed, the next frame is input into the lightweight neural network model. The lightweight neural network model extracts the details of the next frame based on the detail feature map stored in the cache pool. Based on the extracted detail feature map of the next frame and the next frame, the super-resolution image corresponding to the next frame is obtained, and the detail feature map of the next frame is stored in the cache pool.

[0149] Optionally, when processing the current frame image, the detail feature map in the cache pool is obtained. The lightweight neural network model can extract detail features of the current frame based on the current frame image and the detail feature map obtained in the cache pool. The detail feature map stored in the cache pool is the detail feature map of each frame image before the current frame.

[0150] For example, for the first frame image, if the cache pool does not contain detail feature maps, then the detail feature maps of the first frame image can be obtained according to the above steps S401-S402, and stored in the cache pool, so the cache pool contains one detail feature map. For the second frame image, since the cache pool contains the detail feature maps of the first frame image, the detail feature maps in the cache pool are obtained. The lightweight neural network model can extract detail features from the current second frame image based on the detail feature maps of the first frame image obtained from the cache pool, and obtain the detail feature map of the second frame image, which is then stored in the cache pool. For the third frame image, since the cache pool contains two detail feature maps, namely the detail feature maps of the first frame image and the second frame image, the two detail feature maps in the cache pool are obtained. The lightweight neural network model can extract detail features from the current third frame image based on the two detail feature maps obtained from the cache pool, and obtain the detail feature map of the third frame image, which is then stored in the cache pool. This process continues for the Nth frame image.

[0151] In this embodiment, when processing video images, a lightweight neural network model extracts detail features from the next frame image based on the detail feature maps in the cache pool, which can make the extracted image feature maps more detailed.

[0152] Figure 10 is a schematic diagram of the structure of another lightweight neural network model provided in the embodiment of this application. As shown in Figure 10, the video lightweight neural network model also includes a buffer pool, a first video compression convolutional layer and a second video compression convolutional layer, wherein the convolution kernels of the first video compression convolutional layer and the second video compression convolutional layer can be 1×1.

[0153] Figure 11 is a flowchart illustrating another video super-resolution processing method provided in this application embodiment. As shown in Figure 11, the step S502 above, in which a lightweight neural network model extracts detail features from the next frame image based on detail feature maps stored in the cache pool, may include:

[0154] S601. Input the next frame image into the up-resampling layer, and obtain the convolutional feature map through the up-resampling layer, down-resampling layer and convolutional layer.

[0155] Optionally, the process in this step has been described in detail in the above specific embodiments, and will not be repeated here.

[0156] S602. Input the detail feature map stored in the cache pool into the first video compression convolutional layer of the lightweight neural network model to obtain the first video convolutional feature map.

[0157] Optionally, if the next frame is the (K+1)th frame in the video, then K detail feature maps can be obtained from the cache pool. These K detail feature maps are detail feature maps obtained after detail feature extraction from the previous K frames.

[0158] Optionally, K detail feature maps can be obtained from the cache pool, channel-by-channel concatenation can be performed, and the data can be input into the first video compression convolutional layer for compression convolution processing to obtain the first video convolutional feature map.

[0159] S603. Input the convolutional feature map and the first video convolutional feature map into the second video compression convolutional layer of the lightweight neural network model for convolution processing to obtain the second video convolutional feature map.

[0160] Optionally, the convolutional feature map obtained in S601 and the first video convolutional feature map obtained in S602 are input into the second video compression convolutional layer for processing to obtain the second video convolutional feature map.

[0161] S604. Input the second video convolutional feature map into the first structural block of the lightweight neural network. The residual convolution processing module and the total compression convolution module in the first structural block process the feature map in sequence to obtain the processed feature map.

[0162] The process of this step has been described in detail in step S202 above, and will not be repeated here.

[0163] S605. The processed feature map is processed sequentially by each structure block after the first structure block to obtain the feature map to be sampled.

[0164] The process of this step has been described in detail in step S203 above, and will not be repeated here.

[0165] S606. Input the feature map to be sampled into the upsampling layer for upsampling processing to obtain the detail feature map of the next frame image.

[0166] The process of this step has been described in detail in step S204 above, and will not be repeated here.

[0167] Figure 12 is a schematic diagram of an apparatus for an image super-resolution processing method provided in an embodiment of this application. As shown in Figure 12, the apparatus includes:

[0168] The receiving module 701 is used to receive the image to be processed input by the user;

[0169] The processing module 702 is used to input the image to be processed into a lightweight neural network model, whereby the lightweight neural network extracts detail features from the image to be processed and fuses the extracted detail feature map with the image to be processed to obtain a super-resolution image corresponding to the image to be processed. The lightweight neural network model includes convolutional layers, multiple structural blocks, and upsampling layers connected in sequence. Each structural block includes multiple residual convolution processing modules and a total compression convolution module connected in sequence. Each residual convolution processing module includes an attention residual module and a compression convolution module. The output of the compression convolution module of each residual convolution processing module is connected to the total compression convolution module.

[0170] Optionally, the processing module 702 is specifically used for:

[0171] The image to be processed is input into the convolutional layer for convolution operation to obtain the convolutional feature map;

[0172] The convolutional feature map is input into the first structural block, and is processed sequentially by each residual convolution processing module and the total compression convolution module in the first structural block to obtain the processed feature map.

[0173] The processed feature map is processed sequentially by each subsequent structure block to obtain the feature map to be sampled;

[0174] The feature map to be sampled is input into the upsampling layer for upsampling operation to obtain the detail feature map.

[0175] Optionally, the processing module 702 is specifically used for:

[0176] The convolutional feature map is input into the attention residual module of the first residual convolution processing module in the first structural block to obtain the initial residual feature map.

[0177] The initial residual feature map is input into the attention residual module in the residual convolution processing module after the first residual convolution processing module, and the attention residual modules in each residual convolution processing module after the first residual convolution processing module perform residual processing in sequence to obtain the residual feature map output by each attention residual module.

[0178] The initial residual feature map is input into the compressed convolution module of the first residual convolution processing module to obtain the initial structure block convolution feature map. The compressed convolution modules in each residual convolution processing module after the first residual convolution processing module perform convolution processing on the residual feature maps output by each attention residual module to obtain the structure block convolution feature maps output by each compressed convolution module.

[0179] The initial convolutional feature map of the structural blocks and the convolutional feature maps of the structural blocks output by each compressed convolution module are input into the total compressed convolution module for convolution processing to obtain the processed feature map.

[0180] Optionally, the processing module 702 is specifically used for:

[0181] The convolutional feature map is input into the attention residual module of the first residual convolution processing module in the first structural block. The attention residual module determines the channel variance and channel mean of the convolutional feature map. The channel mean and channel variance are summed. The sum is then subjected to erosion convolution, dilation convolution and normalization processing in sequence to obtain the normalized feature map.

[0182] The convolutional feature map is then subjected to convolution processing to obtain a new convolutional feature map;

[0183] The convolutional feature map, the normalized feature map, and the new convolutional feature map are fused to obtain the initial residual feature map.

[0184] Optionally, the lightweight neural network model further includes an upsampling layer and a downsampling layer, wherein the input of the downsampling layer is connected to the output of the upsampling layer, and the output of the downsampling layer is connected to the input of the convolutional layer.

[0185] Processing module 702 is specifically used for:

[0186] The image to be processed is input into the up-resampled layer and the down-resampled layer, and the up-resampled layer and the down-resampled layer perform sampling processing in sequence to obtain the sampled feature map;

[0187] The sampled feature map is input into the convolutional layer, and the detailed feature map is obtained through processing by the convolutional layer, each of the structural blocks, and the upsampling layer.

[0188] Optionally, the processing module 702 is specifically used for:

[0189] If the image to be processed is a frame from a video, then the detail feature map is stored in the cache pool;

[0190] After receiving the next frame of the image to be processed, the next frame is input into the lightweight neural network model. The lightweight neural network model extracts detail features from the next frame based on the detail feature maps stored in the cache pool. Based on the extracted detail feature maps of the next frame and the next frame, the super-resolution image corresponding to the next frame is obtained, and the detail feature maps of the next frame are stored in the cache pool.

[0191] Optionally, the processing module 702 is specifically used for:

[0192] The next frame image is input to the up-resampled layer, and the convolutional feature map is obtained through the up-resampled layer, the down-resampled layer, and the convolutional layer.

[0193] The detailed feature map stored in the cache pool is input into the first video compression convolutional layer of the lightweight neural network model to obtain the first video compression convolutional feature map;

[0194] The convolutional feature map and the first video convolutional feature map are input into the second video compression convolutional layer of the lightweight neural network model for convolution processing to obtain the second video convolutional feature map.

[0195] The second video convolutional feature map is input into the first structural block of the lightweight neural network model, and is processed sequentially by each residual convolution processing module and the total compression convolution module in the first structural block to obtain the processed feature map.

[0196] The processed feature map is processed sequentially by each subsequent structure block to obtain the feature map to be sampled;

[0197] The feature map to be sampled is input into the upsampling layer for upsampling processing to obtain the detail feature map of the next frame image.

[0198] Figure 13 is a structural block diagram of an electronic device 800 provided in an embodiment of this application. As shown in Figure 13, the electronic device may include: a processor 801 and a memory 802.

[0199] Optionally, a bus 803 may also be included, wherein the memory 802 is used to store machine-readable instructions executable by the processor 801. When the electronic device 800 is running, the processor 801 and the memory 802 communicate via the bus 803. When the machine-readable instructions are executed by the processor 801, the method steps in the above method embodiments are performed.

[0200] This application also provides a computer-readable storage medium storing a computer program, which, when run by a processor, executes the method steps described in the above-described image super-resolution processing method embodiments.

[0201] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the method embodiments, and will not be repeated here. In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection can be through some communication interfaces; the indirect coupling or communication connection of devices or modules can be electrical, mechanical, or other forms.

[0202] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. If the functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program code.

[0203] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. An image super-resolution processing method, characterized in that, The method, applied to a terminal device, includes: receiving an image to be processed input by a user; inputting the image to be processed into a lightweight neural network model, whereby the lightweight neural network extracts detail features from the image to be processed, and fuses the extracted detail feature maps with the image to be processed to obtain a super-resolution image corresponding to the image to be processed. The lightweight neural network model includes sequentially connected convolutional layers, multiple structural blocks, and upsampling layers. Each structural block includes sequentially connected multiple residual convolution processing modules and a total compression convolution module. Each residual convolution processing module includes an attention residual module and a compression convolution module. The output of each residual convolution processing module's compressed convolution module is connected to the overall compressed convolution module. The step of inputting the image to be processed into a lightweight neural network model, where the lightweight neural network model extracts detailed features from the image, includes: inputting the image to be processed into the convolutional layer for convolution processing to obtain a convolutional feature map; inputting the convolutional feature map into the first structural block, where each residual convolution processing module and the overall compressed convolution module in the first structural block process it sequentially to obtain a processed feature map; and processing the processed feature map sequentially through subsequent structural blocks to obtain a feature map to be sampled. The feature map to be sampled is input into the upsampling layer for upsampling processing to obtain the detail feature map. The convolutional feature map is then input into the first structural block, where it is processed sequentially by each residual convolution processing module and the overall compression convolution module to obtain the processed feature map. This process includes: inputting the convolutional feature map into the attention residual module of the first residual convolution processing module in the first structural block to obtain the initial residual feature map; and inputting the initial residual feature map into the attention residual modules of the residual convolution processing modules following the first residual convolution processing module, where it is processed sequentially by the attention residual modules of the subsequent residual convolution processing modules. The attention residual modules sequentially perform residual processing to obtain the residual feature maps output by each attention residual module. The initial residual feature maps are then input into the compression convolution module of the first residual convolution processing module to obtain the initial structure block convolution feature maps. The compression convolution modules in each residual convolution processing module after the first residual convolution processing module perform convolution processing on the residual feature maps output by each attention residual module to obtain the structure block convolution feature maps output by each compression convolution module. The initial structure block convolution feature maps and the structure block convolution feature maps output by each compression convolution module are then input into the overall compression convolution module for convolution processing to obtain the processed feature maps.

2. The image super-resolution processing method according to claim 1, characterized in that, The step of inputting the convolutional feature map into the attention residual module of the first residual convolution processing module in the first structural block to obtain the initial residual feature map includes: inputting the convolutional feature map into the attention residual module of the first residual convolution processing module in the first structural block; the attention residual module determines the channel variance and channel mean of the convolutional feature map; summing the channel mean and channel variance; sequentially performing erosion convolution, dilation convolution, and normalization on the sum to obtain a normalized feature map; performing convolution processing on the convolutional feature map to obtain a new convolutional feature map; and fusing the convolutional feature map, the normalized feature map, and the new convolutional feature map to obtain the initial residual feature map.

3. The image super-resolution processing method according to claim 1, characterized in that, The lightweight neural network model further includes an upsampling layer and a downsampling layer, wherein the input of the downsampling layer is connected to the output of the upsampling layer, and the output of the downsampling layer is connected to the input of the convolutional layer. The image to be processed is input into the lightweight neural network model, and the lightweight neural network model performs detail feature extraction on the image to be processed, including: inputting the image to be processed into the upsampling layer and the downsampling layer, and sequentially performing sampling processing by the upsampling layer and the downsampling layer to obtain a sampled feature map; inputting the sampled feature map into the convolutional layer, and processing it through the convolutional layer, each of the structural blocks, and the upsampling layer to obtain the detail feature map.

4. The image super-resolution processing method according to claim 1, characterized in that, The image to be processed is input into a lightweight neural network model. After the lightweight neural network model extracts detail features from the image to be processed, the process further includes: if the image to be processed is a frame in a video, the detail feature map is stored in a cache pool; after receiving the next frame of the image to be processed, the next frame is input into the lightweight neural network model, and the lightweight neural network model extracts detail features from the next frame based on the detail feature map stored in the cache pool. Based on the extracted detail feature map of the next frame and the next frame, a super-resolution image corresponding to the next frame is obtained, and the detail feature map of the next frame is stored in the cache pool.

5. The image super-resolution processing method according to claim 4, characterized in that, The lightweight neural network model extracts detailed features from the subsequent frame image based on the detailed features of the image to be processed stored in the cache pool. This includes: inputting the subsequent frame image into an upsampling layer, passing it through an upsampling layer, a downsampling layer, and the convolutional layer to obtain a convolutional feature map; inputting the detailed feature map stored in the cache pool into the first video compression convolutional layer of the lightweight neural network model to obtain a first video convolutional feature map; inputting the convolutional feature map and the first video convolutional feature map into the second video compression convolutional layer of the lightweight neural network model for convolution processing to obtain a second video convolutional feature map; inputting the second video convolutional feature map into the first structural block of the lightweight neural network model, where it is processed sequentially by each residual convolution processing module and the overall compression convolution module to obtain a processed feature map; processing the processed feature map sequentially through each structural block after the first structural block to obtain a feature map to be sampled; and inputting the feature map to be sampled into the upsampling layer for upsampling processing to obtain the detailed feature map of the subsequent frame image.

6. An image super-resolution processing apparatus, characterized in that, include: The receiving module is used to receive the image to be processed input by the user; A processing module is used to input the image to be processed into a lightweight neural network model. The lightweight neural network extracts detail features from the image to be processed and fuses the extracted detail feature maps with the image to be processed to obtain a super-resolution image corresponding to the image to be processed. The lightweight neural network model includes sequentially connected convolutional layers, multiple structural blocks, and upsampling layers. Each structural block includes sequentially connected residual convolution processing modules and a total compression convolution module. Each residual convolution processing module includes an attention residual module and a compression convolution module. The compression convolution of each residual convolution processing module... The outputs of all modules are connected to the total compression convolution module; the step of inputting the image to be processed into the lightweight neural network model, and having the lightweight neural network model extract detailed features from the image to be processed, includes: inputting the image to be processed into the convolutional layer for convolution processing to obtain a convolutional feature map; inputting the convolutional feature map into the first structural block, where it is processed sequentially by each residual convolution processing module and the total compression convolution module in the first structural block to obtain a processed feature map; processing the processed feature map sequentially through each structural block after the first structural block to obtain a feature map to be sampled; and inputting the feature map to be sampled into... The upsampling layer performs upsampling processing to obtain the detailed feature map. The convolutional feature map is then input into the first structural block, where it is processed sequentially by each residual convolution processing module and the overall compression convolution module to obtain a processed feature map. This process includes: inputting the convolutional feature map into the attention residual module of the first residual convolution processing module in the first structural block to obtain an initial residual feature map; and inputting the initial residual feature map into the attention residual modules of subsequent residual convolution processing modules, which are then processed by the attention residual modules of subsequent residual convolution processing modules. The residuals are processed sequentially to obtain the post-residual feature maps output by each attention residual module. The initial post-residual feature maps are then input into the compression convolution module of the first residual convolution processing module to obtain the initial post-convolution feature maps of the structure blocks. The compression convolution modules in each residual convolution processing module after the first residual convolution processing module perform convolution processing on the post-residual feature maps output by each attention residual module to obtain the post-convolution feature maps of the structure blocks output by each compression convolution module. The initial post-convolution feature maps of the structure blocks and the post-convolution feature maps of the structure blocks output by each compression convolution module are then input into the overall compression convolution module for convolution processing to obtain the processed feature maps.

7. An electronic device, characterized in that, The method includes a memory and a processor, wherein the memory stores a computer program executable by the processor, and the processor executes the computer program to implement the steps of the image super-resolution processing method according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the image super-resolution processing method as described in any one of claims 1-5.

Images