Image super-resolution processing method and device, equipment and medium

Abstract

The invention discloses an image super-resolution processing method and device, equipment and a medium, and relates to the technical field of image processing. Comprising the following steps: acquiring target image data, and performing shallow feature extraction operation on the target image data; the shallow feature extraction processing is an operation of extracting shallow feature data of the target image data through convolution calculation; performing deep feature extraction operation on the target image data based on the shallow feature data through a deep feature extraction end to obtain deep feature data of the target image data; the deep feature extraction end is composed of a plurality of target attention modules; the target attention module is constructed based on local feature extraction and a lightweight grouping attention module; and performing image reconstruction operation based on the deep feature data of the target image data to obtain the target image data after super-resolution processing. Therefore, the calculation amount and the parameter amount can be reduced in the image super-resolution processing process, and the influence of abnormal values is reduced.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image super-resolution processing method, device, equipment and medium. Background Art

[0002] In the modern banking environment, image super-resolution technology has a variety of application scenarios, which can significantly improve the bank's operational efficiency and service quality. First, in document processing, banks need to process a large number of customer applications and identity verification documents. Through super-resolution technology, image quality can be effectively improved, making text clearer, thereby improving the accuracy of optical character recognition systems and accelerating the document review process. Secondly, image super-resolution also plays an important role in the field of surveillance and security. Banks' surveillance videos are often blurry due to insufficient light or low camera resolution. Super-resolution technology can enhance the details of the surveillance images and help identify suspicious people or events, thereby improving security prevention capabilities. In addition, image super-resolution can also improve the accuracy of customer identity verification. When performing biometrics, clear images are essential to ensure recognition accuracy. By improving the quality of photos or videos submitted by customers, banks can better authenticate identities and reduce the risk of misidentification. At the same time, this technology can also help banks optimize marketing and customer service. Using high-resolution images can enhance the visual effects of promotional materials, attract more customer participation, and enhance brand image. Finally, in risk management, image super-resolution can be used to analyze image data of collateral and improve assessment accuracy. Through these applications, banks can improve customer experience and operational efficiency while ensuring security.

[0003] SwinIR is an image super-resolution algorithm based on Swin Transformer (a deep learning model for computer vision). SwinIR takes advantage of the Swin Transformer and achieves efficient image super-resolution through techniques such as Adaptive Group Convolution and Cross-Stage Connection. However, in the deep feature extraction module of the SwinIR method, self-attention affects its long-distance modeling ability when calculated within a small window, and continuous Swin Transformer blocks contain many redundant components. Restormer (an efficient Transformer model) uses channel space to calculate self-attention, but calculating self-attention in channel space sacrifices useful spatial information such as texture and structural information, thus affecting downstream tasks.

[0004] From the above, it can be seen that how to reduce the amount of calculation and the number of parameters in the image super-resolution processing and reduce the impact of outliers is an urgent problem to be solved. Summary of the invention

[0005] In view of this, the purpose of the present invention is to provide an image super-resolution processing method, device, equipment and medium, which can reduce the amount of calculation and the amount of parameters in the image super-resolution processing process and reduce the impact of abnormal values. The specific scheme is as follows:

[0006] In a first aspect, the present application provides an image super-resolution processing method, comprising:

[0007] Acquire target image data, and perform shallow feature extraction on the target image data; the shallow feature extraction process is an operation of extracting shallow feature data of the target image data through convolution calculation;

[0008] A deep feature extraction end performs a deep feature extraction operation on the target image data based on the shallow feature data to obtain deep feature data of the target image data; the deep feature extraction end is composed of a plurality of target attention modules; the target attention module is constructed based on local feature extraction and a lightweight grouping attention module;

[0009] An image reconstruction operation is performed based on the deep feature data of the target image data to obtain the target image data after super-resolution processing.

[0010] Optionally, the target attention module is a module consisting of two shift convolutions and a lightweight grouping attention module cascaded.

[0011] Optionally, performing a deep feature extraction operation on the target image data based on the shallow feature data by a deep feature extraction end to obtain deep feature data of the target image data includes:

[0012] Extracting local feature data of the shallow feature data by the shift convolution;

[0013] The lightweight grouping attention module performs attention calculation on the local feature data to obtain deep feature data of the target image data.

[0014] Optionally, extracting the local feature data of the shallow feature data by the shift convolution includes:

[0015] The shallow feature data is divided into a plurality of input matrices by the shift convolution, and the input matrices are shifted along different spatial dimensions to obtain input matrix information after the shift;

[0016] Convolution calculation is performed on the input matrix information to extract local feature data of the shallow feature data.

[0017] Optionally, performing attention calculation on the local feature data by the lightweight grouping attention module to obtain deep feature data of the target image data includes:

[0018] Dividing the local feature data into a plurality of feature groups, and performing attention calculations on the plurality of feature groups respectively based on a cross attention mechanism;

[0019] Integrating the attention calculation results of the plurality of feature groups to obtain global attention information;

[0020] Connecting the local feature data with the global attention information through a residual connection to obtain deep feature data of the target image data;

[0021] Among them, the cross-attention mechanism is a mechanism in which the feature group calculates itself and interacts with other groups to obtain corresponding attention calculation results.

[0022] Optionally, the image super-resolution processing method further includes:

[0023] The deep feature data of the target image data is optimized based on the smooth L1 loss function and the edge-aware loss function.

[0024] Optionally, optimizing the deep feature data of the target image data based on an edge-aware loss function includes:

[0025] Extracting edge information of the target image data using an edge detector;

[0026] A weighted operation is performed on the edge information so as to optimize the deep feature data of the target image data.

[0027] In a second aspect, the present application provides an image super-resolution processing device, comprising:

[0028] A shallow feature data extraction module is used to obtain target image data and perform a shallow feature extraction operation on the target image data; the shallow feature extraction process is an operation of extracting shallow feature data of the target image data through convolution calculation;

[0029] A deep feature data extraction module, used for performing a deep feature extraction operation on the target image data based on the shallow feature data through a deep feature extraction end to obtain deep feature data of the target image data; the deep feature extraction end is composed of a plurality of target attention modules; the target attention module is constructed based on local feature extraction and a lightweight grouping attention module;

[0030] The image super-resolution processing module is used to perform an image reconstruction operation based on the deep feature data of the target image data to obtain the target image data after super-resolution processing.

[0031] In a third aspect, the present application provides an electronic device, including:

[0032] Memory, used to store computer programs;

[0033] The processor is used to execute the computer program to implement the above-mentioned image super-resolution processing method.

[0034] In a fourth aspect, the present application provides a computer-readable storage medium for storing a computer program, wherein the computer program implements the aforementioned image super-resolution processing method when executed by a processor.

[0035] The present application provides an image super-resolution processing method, which first acquires target image data and performs a shallow feature extraction operation on the target image data; the shallow feature extraction processing is an operation of extracting shallow feature data of the target image data through convolution calculation; then a deep feature extraction operation is performed on the target image data based on the shallow feature data through a deep feature extraction port to obtain deep feature data of the target image data; the deep feature extraction port is composed of a plurality of target attention modules; the target attention module is constructed based on local feature extraction and a lightweight grouping attention module; finally, an image reconstruction operation is performed based on the deep feature data of the target image data to obtain the target image data after super-resolution processing.

[0036] As can be seen from the above, the present application extracts shallow features of the low-resolution target image data through convolution calculation, and further extracts deep features of the target image data based on the shallow feature data through several target attention modules constructed based on local feature extraction and lightweight grouping attention modules to generate high-resolution target image data. That is, by calculating self-attention through the target attention module, it is possible to avoid sacrificing useful spatial information such as texture and structural information, thereby reducing the amount of calculation and parameters in the image super-resolution processing process and reducing the impact of outliers. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0038] Figure 1 This is a flow chart of an image super-resolution processing method disclosed in this application;

[0039] Figure 2 A schematic diagram of a high-performance attention block (EAB) structure disclosed in this application;

[0040] Figure 3 A specific shift convolution schematic diagram disclosed in this application;

[0041] Figure 4 A schematic diagram of a specific lightweight grouping attention module structure disclosed in this application;

[0042] Figure 5 A schematic diagram of a specific image super-resolution processing network architecture disclosed in this application;

[0043] Figure 6 A flowchart of a specific image super-resolution processing method disclosed in this application;

[0044] Figure 7 A schematic diagram of an image super-resolution processing device disclosed in this application;

[0045] Figure 8 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] In the modern banking environment, image super-resolution technology has a variety of application scenarios, which can significantly improve the bank's operational efficiency and service quality. SwinIR is an image super-resolution algorithm based on Swin Transformer. SwinIR takes advantage of the Swin Transformer and achieves efficient image super-resolution through technologies such as adaptive group convolution and cross-stage connection. However, in the deep feature extraction module of the SwinIR method, self-attention will affect its long-distance modeling ability when calculated within a small window, and continuous Swin Transformer blocks contain many redundant components. Restormer uses channel space to calculate self-attention, but calculating self-attention in channel space will sacrifice useful spatial information such as texture and structural information, thereby affecting downstream tasks. To this end, the present application provides an image super-resolution processing solution that can reduce the amount of calculation and parameters in the image super-resolution processing process and reduce the impact of outliers.

[0048] See also Figure 1 As shown, the embodiment of the present application discloses an image super-resolution processing method, comprising:

[0049] Step S11, acquiring target image data, and performing shallow feature extraction operations on the target image data.

[0050] In this embodiment, target image data is obtained, and a shallow feature extraction operation is performed on the target image data. The shallow feature extraction process is an operation of extracting shallow feature data of the target image data through convolution calculation. It can be understood that the specific type of the convolution can be determined according to the actual application scenario. For example, in a specific implementation, a 3×3 convolution can be selected to calculate the target image data. Specifically, the shallow feature extraction operation on the target image data can include: performing a shallow feature extraction operation on the target image data through a 3×3 convolution.

[0051] Step S12: performing a deep feature extraction operation on the target image data based on the shallow feature data by the deep feature extraction end to obtain deep feature data of the target image data.

[0052] In this embodiment, a deep feature extraction end performs a deep feature extraction operation on the target image data based on the shallow feature data to obtain the deep feature data of the target image data. The deep feature extraction end is composed of a plurality of target attention modules.

[0053] See also Figure 2As shown, in a specific implementation, a high-performance attention block (EAB) is selected as the target attention module, and the high-performance attention block is composed of local feature extraction and a lightweight grouped self-attention module (LGAM). First, shift convolution is used to extract local structural information of the image, and the complexity is kept at the same order of magnitude as the 1×1 convolution, and then the self-attention is calculated on non-overlapping feature groups using different window sizes through the lightweight grouped self-attention module. Specifically, the target attention module is a module composed of two shift convolutions and a lightweight grouped attention module cascaded. Specifically, the deep feature extraction end performs a deep feature extraction operation on the target image data based on the shallow feature data to obtain the deep feature data of the target image data, which may include: extracting the local structural information of the shallow feature data through the shift convolution; and performing attention calculation on the local structural information through the lightweight grouped attention module to obtain the deep feature data of the target image data. It should be noted that the efficiency of local feature data extraction can be improved by adding a ReLU (Rectified Linear Unit, an artificial neural network activation function) activation function between two shift convolutions.

[0054] In this embodiment, each convolution kernel of the shift convolution has only one element that is 1, and all others are 0. The shift convolution calculation is to translate the input matrix in a certain direction. By setting shift convolution kernels in different directions, the input matrix is ​​translated from different directions in different channels, and then 1×1 convolution is used to fuse the information between channels to extract spatial domain and channel domain information. The learnable parameters of shift convolution are only concentrated in the 1×1 convolution operation. Therefore, combined with shift convolution, only 1×1 convolution is used to extract structured and hierarchical spatial domain information, which can reduce the number of parameters and calculations in the convolution network design.

[0055] See also Figure 3As shown, the shift convolution consists of a shift operation and a 1×1 convolution. First, the shift convolution divides the input features into five groups on average, wherein any four groups of features are shifted along different spatial dimensions, and the remaining group remains unchanged, and then the information of adjacent elements after the shift is obtained through a 1×1 convolution. It is possible to avoid introducing additional learnable parameters while maintaining a complexity similar to that of a 1×1 convolution. Specifically, the extraction of local feature data of the shallow feature data through the shift convolution may include: dividing the shallow feature data into five groups of input feature matrices through the shift convolution, and shifting any four groups of input feature matrices along different spatial dimensions to obtain input matrix information after the shift; performing a 1×1 convolution calculation on the input feature matrix information to obtain adjacent element information after the shift, and extracting local feature data of the shallow feature data based on the adjacent element information.

[0056] In this embodiment, the lightweight group attention module is mainly composed of two parts: channel attention and spatial attention. The lightweight group attention module is an organic combination of local attention and global attention, in which channel attention and spatial attention are used to capture local and global features of the image, respectively. Channel attention and spatial attention perform calculations within the group, and each group extracts relevant features within a certain range. This processing method belongs to local attention calculation (CA, Cross Attention). LGAM mines the information of local areas by grouping feature maps. After the features of all groups are calculated locally, LGAM further integrates the information between all groups through the global self-attention module (SA, Self Attention), so that the global features can be connected to form global attention. This mechanism ensures the global consistency of features, thereby improving the ability of super-resolution processing to understand the overall structure of the image.

[0057] See also Figure 4As shown, the input feature map is divided into several groups (4 groups in the embodiment of the present application: X1, X2, X3, X4). It should be noted that these groups can be non-overlapping feature blocks or different channels or regions of the feature map. In each group, a local cross-attention mechanism is performed. That is, the features of each group will not only be calculated with itself, but also interact with other groups. The cross arrows in the figure show the information interaction between features in different groups, which can improve the model's ability to capture local information. After cross-attention calculation, the output of each group is recorded as Y1, Y2, Y3, and Y4, representing the features after local and cross-attention mechanisms. The above output features are integrated through a global self-attention module to ensure the connectivity and consistency of global features. Finally, the input is directly jumped to the output of the self-attention module through a residual connection (see the addition symbol in the figure), which can alleviate the gradient vanishing problem and maintain the information of the input features in the deep network. Specifically, the performing attention calculation on the local feature data by the lightweight grouping attention module to obtain the deep feature data of the target image data may include: dividing the local feature data into several feature groups, and performing attention calculation on the several feature groups respectively based on the cross-attention mechanism; integrating the attention calculation results of the several feature groups to obtain global attention information; connecting the local feature data with the global attention information through residual connection to obtain the deep feature data of the target image data; wherein the cross-attention mechanism is a mechanism by which the feature group calculates itself and interacts with other groups to obtain corresponding attention calculation results.

[0058] Step S13: performing an image reconstruction operation based on the deep feature data of the target image data to obtain the target image data after super-resolution processing.

[0059] See also Figure 5 As shown, in this embodiment, before performing an image reconstruction operation based on the deep feature data of the target image data, it is also necessary to obtain shallow feature data of the target image data through cross-stage connection, and obtain the target image data after super-resolution processing through convolution calculation and sampling operation. Specifically, performing an image reconstruction operation based on the deep feature data of the target image data to obtain the target image data after super-resolution processing may include: performing an image reconstruction operation based on the deep feature data of the target image data and obtaining shallow feature data of the target image data through cross-stage connection through an image reconstruction module to obtain the target image data after super-resolution processing; wherein, the image reconstruction module is composed of a convolution module and a sampling module.

[0060] As can be seen from the above, the embodiment of the present application extracts shallow features of the low-resolution target image data through convolution calculation, and further extracts deep features of the target image data based on the shallow feature data through several target attention modules constructed based on local feature extraction and lightweight grouping attention modules to generate high-resolution target image data. That is, firstly, shift convolution is used to extract local structural information of the image, and the complexity is kept at the same order of magnitude as that of 1×1 convolution, and then self-attention is calculated on non-overlapping feature groups using different window sizes through lightweight grouping self-attention modules, which can avoid sacrificing useful spatial information such as texture and structural information, thereby reducing the amount of calculation and the amount of parameters in the image super-resolution processing process.

[0061] For further information, see Figure 6 As shown, in order to reduce the influence of outliers and enhance the reconstruction capability of image edge features, the embodiment of the present application provides a specific image super-resolution processing method, including:

[0062] Step S21: Acquire target image data, and perform shallow feature extraction operations on the target image data.

[0063] Step S22: performing a deep feature extraction operation on the target image data based on the shallow feature data by the deep feature extraction end to obtain deep feature data of the target image data.

[0064] Step S23: optimizing the deep feature data of the target image data based on the smooth L1 loss function and the edge-aware loss function.

[0065] In this embodiment, a loss function is introduced in the training phase of the image super-resolution network to optimize the deep feature data of the target image data. Specifically, the deep feature data of the target image data is optimized based on the Smooth L1 loss, which may include: and real high-resolution images Compare each pixel of the predicted image and calculate the Smooth L1 loss of the whole image to measure the global error; To input a low-resolution image into the high-resolution image output by the super-resolution network, the real high-resolution image is the high-resolution label image corresponding to the low-resolution input image; the smooth L1 loss function formula is as follows:

[0066] ;

[0067] in, is an estimate; compared to the Least Absolute Deviations (LAD), it is differentiable with respect to zero, and the closer it is to zero, the smaller the derivative is, and it is easy to converge; compared to the Least Square Estimate (LSE), the maximum outlier derivative is 1, and the gradient will not explode. In other words, the smooth L1 loss function can reduce the impact of outliers by balancing accuracy and robustness.

[0068] Furthermore, since high-resolution images are often extracted from jagged edges, and edge-aware loss enhances the ability to reconstruct image edge features, it is possible to ensure that the edges in complex images are clearer and more natural. An edge detector is used to extract the edge information of the image, and then the detected edge areas are weighted so that the network pays more attention to these areas during the learning process. It is understandable that the edge detector can be determined based on the actual application. For example, in a specific embodiment, the Laplace transform can be used as an edge detector. Specifically, optimizing the deep feature data of the target image data based on the edge-aware loss function can include: extracting the edge information of the target image data using the Laplace transform; based on the edge information, weighting the prediction error of the edge feature to enhance the network's attention to the edge. Among them, in the real image In the detector, the edge map is obtained ,exist Binary mask value at The calculation formula is as follows:

[0069] ;

[0070] in is a preset threshold; the edge perception loss function expression is as follows:

[0071] ;

[0072] in, To predict image results, is a constant, " is element-wise multiplication, is the Charbonnier loss function, and its function expression is as follows:

[0073] ;

[0074] in, Is a constant.

[0075] Furthermore, the deep feature data of the target image data is optimized by calculating the gradient and updating the model parameters using a back-propagation algorithm, and the loss function optimization runs through each iteration of the training to guide the network to continuously adjust the parameters to generate high-quality super-resolution images.

[0076] Step S24: performing an image reconstruction operation based on the deep feature data of the target image data to obtain target image data after super-resolution processing.

[0077] Among them, the specific implementation process of the above steps S21, S22 and S24 can refer to the corresponding content in the previous embodiment, and will not be repeated here.

[0078] As can be seen from the above, the embodiment of the present application extracts shallow features of the low-resolution target image data through convolution calculation, and further extracts deep features of the target image data based on the shallow feature data through several target attention modules constructed based on local feature extraction and lightweight grouping attention modules to generate high-resolution target image data. And by introducing smooth L1 loss and edge-aware loss to balance accuracy and robustness, reduce the impact of outliers, strengthen the ability to reconstruct image edge features, and ensure that the edges in complex images are clearer and more natural. Thereby, the amount of calculation and the amount of parameters can be reduced in the image super-resolution processing process, and the impact of outliers can be reduced.

[0079] Accordingly, see Figure 7 As shown, the embodiment of the present application provides an image super-resolution processing device, comprising:

[0080] A shallow feature data extraction module 11 is used to obtain target image data and perform a shallow feature extraction operation on the target image data; the shallow feature extraction process is an operation of extracting shallow feature data of the target image data through convolution calculation;

[0081] A deep feature data extraction module 12, configured to perform a deep feature extraction operation on the target image data based on the shallow feature data through a deep feature extraction end to obtain deep feature data of the target image data; the deep feature extraction end is composed of a plurality of target attention modules; the target attention module is constructed based on local feature extraction and a lightweight grouping attention module;

[0082] The image super-resolution processing module 13 is used to perform an image reconstruction operation based on the deep feature data of the target image data to obtain the target image data after super-resolution processing.

[0083] As can be seen from the above, the embodiment of the present application extracts shallow features of the low-resolution target image data through convolution calculation, and further extracts deep features of the target image data based on the shallow feature data through several target attention modules constructed based on local feature extraction and lightweight grouping attention modules to generate high-resolution target image data. That is, by calculating self-attention through the target attention module, it is possible to avoid sacrificing useful spatial information such as texture and structural information, thereby reducing the amount of calculation and parameters in the image super-resolution processing process and reducing the impact of outliers.

[0084] In some specific implementations, the deep feature data extraction module 12 may specifically include:

[0085] A local feature data extraction submodule, used to extract local feature data of the shallow feature data through the shift convolution;

[0086] The deep feature data extraction submodule is used to perform attention calculation on the local feature data through the lightweight grouping attention module to obtain the deep feature data of the target image data.

[0087] Furthermore, in some specific implementations, the local feature data extraction submodule may specifically include:

[0088] An input matrix information acquisition unit, used for dividing the shallow feature data into a plurality of input matrices through the shift convolution, and shifting the input matrices along different spatial dimensions to obtain shifted input matrix information;

[0089] The local feature data acquisition unit is used to perform convolution calculation on the input matrix information to extract local feature data of the shallow feature data.

[0090] Furthermore, in some specific implementations, the deep feature data extraction submodule may specifically include:

[0091] an attention calculation unit, used for dividing the local feature data into a plurality of feature groups, and performing attention calculations on the plurality of feature groups respectively based on a cross attention mechanism;

[0092] A global attention information acquisition unit, used for integrating the attention calculation results of the plurality of feature groups to obtain global attention information;

[0093] A deep feature data acquisition unit, used to connect the local feature data with the global attention information through a residual connection to obtain deep feature data of the target image data;

[0094] Among them, the cross-attention mechanism is a mechanism in which the feature group calculates itself and interacts with other groups to obtain corresponding attention calculation results.

[0095] In some specific implementations, the image super-resolution processing device may further include:

[0096] The loss function optimization module is used to optimize the deep feature data of the target image data based on the smooth L1 loss function and the edge-aware loss function.

[0097] Furthermore, in some specific implementations, the loss function optimization module may specifically include:

[0098] The smooth L1 loss function optimization submodule is used to optimize the deep feature data of the target image data based on the smooth L1 loss function.

[0099] The edge-aware loss function optimization submodule is used to optimize the deep feature data of the target image data based on the edge-aware loss function.

[0100] Furthermore, in some specific implementations, the edge-aware loss function optimization submodule may specifically include:

[0101] An edge information extraction unit, used for extracting edge information of the target image data using an edge detector;

[0102] The target image data optimization unit is used to perform a weighted operation on the edge information so as to optimize the deep feature data of the target image data.

[0103] Furthermore, the present application also discloses an electronic device. Figure 8 2 is a structural diagram of an electronic device 20 according to an exemplary embodiment, and the content in the figure cannot be regarded as any limitation on the scope of use of this application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the image super-resolution processing method disclosed in any of the aforementioned embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0104] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0105] In addition, the memory 22, as a carrier for storing resources, can be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0106] The operating system 221 is used to manage and control the hardware devices and computer program 222 on the electronic device 20, and can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program that can be used to complete the image super-resolution processing method performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include a computer program that can be used to complete other specific tasks.

[0107] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the image super-resolution processing method disclosed above is implemented. For the specific steps of the method, reference may be made to the corresponding contents disclosed in the above embodiments, and no further description will be given here.

[0108] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0109] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0110] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0111] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0112] The technical solution provided by the present application is introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technicians in this field, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for image super-resolution processing, characterized in that: include: Acquire target image data, and perform shallow feature extraction on the target image data; the shallow feature extraction process is an operation of extracting shallow feature data of the target image data through convolution calculation; A deep feature extraction end performs a deep feature extraction operation on the target image data based on the shallow feature data to obtain deep feature data of the target image data; the deep feature extraction end is composed of a plurality of target attention modules; the target attention module is constructed based on local feature extraction and a lightweight grouping attention module; An image reconstruction operation is performed based on the deep feature data of the target image data to obtain the target image data after super-resolution processing.

2. The image super-resolution processing method according to claim 1, characterized in that: The target attention module is a module composed of two shift convolutions and a lightweight grouping attention module cascaded.

3. The image super-resolution processing method according to claim 2, characterized in that: The deep feature extraction end performs a deep feature extraction operation on the target image data based on the shallow feature data to obtain the deep feature data of the target image data, including: Extracting local feature data of the shallow feature data by the shift convolution; The lightweight grouping attention module performs attention calculation on the local feature data to obtain deep feature data of the target image data.

4. The image super-resolution processing method according to claim 3, characterized in that: The extracting the local feature data of the shallow feature data by the shift convolution comprises: The shallow feature data is divided into a plurality of input matrices by the shift convolution, and the input matrices are shifted along different spatial dimensions to obtain input matrix information after the shift; Convolution calculation is performed on the input matrix information to extract local feature data of the shallow feature data.

5. The image super-resolution processing method according to claim 3, characterized in that: The step of performing attention calculation on the local feature data by the lightweight grouping attention module to obtain deep feature data of the target image data includes: Dividing the local feature data into a plurality of feature groups, and performing attention calculations on the plurality of feature groups respectively based on a cross attention mechanism; Integrating the attention calculation results of the plurality of feature groups to obtain global attention information; Connecting the local feature data with the global attention information through a residual connection to obtain deep feature data of the target image data; Among them, the cross-attention mechanism is a mechanism in which the feature group calculates itself and interacts with other groups to obtain corresponding attention calculation results.

6. The image super-resolution processing method according to any one of claims 1 to 5, characterized in that: Also includes: The deep feature data of the target image data is optimized based on the smooth L1 loss function and the edge-aware loss function.

7. The image super-resolution processing method according to claim 6, characterized in that: Optimizing the deep feature data of the target image data based on the edge-aware loss function includes: Extracting edge information of the target image data using an edge detector; A weighted operation is performed on the edge information so as to optimize the deep feature data of the target image data.

8. An image super-resolution processing device, characterized in that: include: A shallow feature data extraction module is used to obtain target image data and perform a shallow feature extraction operation on the target image data; the shallow feature extraction process is an operation of extracting shallow feature data of the target image data through convolution calculation; A deep feature data extraction module, used for performing a deep feature extraction operation on the target image data based on the shallow feature data through a deep feature extraction end to obtain deep feature data of the target image data; the deep feature extraction end is composed of a plurality of target attention modules; the target attention module is constructed based on local feature extraction and a lightweight grouping attention module; The image super-resolution processing module is used to perform an image reconstruction operation based on the deep feature data of the target image data to obtain the target image data after super-resolution processing.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the image super-resolution processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein when the computer program is executed by a processor, the image super-resolution processing method according to any one of claims 1 to 7 is implemented.

Images