A method, apparatus, device and medium for image super-resolution processing

Abstract

This application discloses an image super-resolution processing method, apparatus, device, and medium, relating to the field of image processing technology. It includes: acquiring target image data and performing shallow feature extraction on the target image data; the shallow feature extraction process involves extracting shallow feature data from the target image data through convolution calculation; performing deep feature extraction on the target image data based on the shallow feature data through a deep feature extraction terminal to obtain deep feature data of the target image data; the deep feature extraction terminal consists of several target attention modules; the target attention modules are constructed based on local feature extraction and lightweight group attention modules; and performing image reconstruction based on the deep feature data of the target image data to obtain the super-resolution processed target image data. This reduces the computational load and parameter count during image super-resolution processing and minimizes the impact of outliers.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image super-resolution processing method, apparatus, device, and medium. Background Technology

[0002] In the modern banking environment, image super-resolution technology has diverse applications, significantly improving operational efficiency and service quality. Firstly, in document processing, banks handle a large volume of customer applications and identity verification documents. Super-resolution technology effectively enhances image quality, making text clearer and improving the accuracy of optical character recognition systems, thus accelerating the document review process. Secondly, image super-resolution plays a crucial role in surveillance and security. Bank surveillance videos are often blurry due to insufficient lighting or low camera resolution. Super-resolution technology enhances the details of surveillance footage, helping to identify suspicious individuals or events, thereby improving security capabilities. Furthermore, image super-resolution can improve the accuracy of customer identity verification. Clear images are essential for ensuring accuracy in biometric identification. By improving the quality of photos or videos submitted by customers, banks can better verify identities and reduce the risk of misidentification. Simultaneously, this technology can help banks optimize marketing and customer service. Using high-resolution images enhances the visual appeal of promotional materials, attracting more customers and improving brand image. Finally, in risk management, image super-resolution can be used to analyze image data of collateral, improving assessment accuracy. Through these applications, banks can improve customer experience and operational efficiency while ensuring security.

[0003] SwinIR is an image super-resolution algorithm based on the Swin Transformer (a deep learning model for computer vision). SwinIR leverages the advantages of the Swin Transformer, achieving efficient image super-resolution through techniques such as Adaptive Group Convolution and Cross-Stage Connection. However, in the deep feature extraction module of the SwinIR method, self-attention, when computed within a small window, affects its ability to model long distances, and consecutive Swin Transformer blocks contain many redundant components. Restormer (an efficient Transformer model) uses channel-space self-attention, but computing self-attention in channel space sacrifices useful spatial information such as texture and structural information, thus affecting downstream tasks.

[0004] As can be seen from the above, how to reduce the amount of computation and parameters and minimize the impact of outliers in the process of image super-resolution processing is an urgent problem to be solved. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide an image super-resolution processing method, apparatus, device, and medium that can reduce the computational load and parameter count, and minimize the impact of outliers during image super-resolution processing. The specific solution is as follows:

[0006] In a first aspect, this application provides an image super-resolution processing method, including:

[0007] Acquire target image data and perform shallow feature extraction on the target image data; the shallow feature extraction process is an operation to extract shallow feature data of the target image data through convolution calculation;

[0008] The deep feature extraction terminal performs deep feature extraction on the target image data based on the shallow feature data to obtain the deep feature data of the target image data; the deep feature extraction terminal is composed of several target attention modules; the target attention modules are constructed based on local feature extraction and lightweight group attention modules;

[0009] Image reconstruction is performed based on the deep feature data of the target image data to obtain the target image data after super-resolution processing.

[0010] Optionally, the target attention module is a module composed of two cascaded shift convolutions and a lightweight group attention module.

[0011] Optionally, the step of performing deep feature extraction on the target image data based on the shallow feature data using a deep feature extraction terminal to obtain deep feature data of the target image data includes:

[0012] Local feature data of the shallow feature data are extracted through the shift convolution;

[0013] The lightweight group attention module performs attention calculations on the local feature data to obtain the deep feature data of the target image data.

[0014] Optionally, the step of extracting local feature data from the shallow feature data through the shift convolution includes:

[0015] The shallow feature data is divided into several input matrices by the shift convolution, and the input matrices are shifted along different spatial dimensions to obtain the shifted input matrix information.

[0016] The input matrix information is subjected to convolution calculation to extract local feature data of the shallow feature data.

[0017] Optionally, the step of performing attention calculation on the local feature data through the lightweight group attention module to obtain the deep feature data of the target image data includes:

[0018] The local feature data is divided into several feature groups, and attention is calculated for each of the feature groups based on a cross-attention mechanism.

[0019] The attention calculation results of the aforementioned feature groups are integrated to obtain global attention information;

[0020] The local feature data is connected to the global attention information through residual connection to obtain the deep feature data of the target image data;

[0021] The cross-attention mechanism is a mechanism in which the feature group calculates itself and interacts with other groups to obtain the corresponding attention calculation result.

[0022] Optionally, the image super-resolution processing method further includes:

[0023] The deep feature data of the target image data are optimized based on the smooth L1 loss function and the edge-aware loss function.

[0024] Optionally, the deep feature data of the target image data is optimized based on an edge-aware loss function, including:

[0025] Edge information of the target image data is extracted using an edge detector;

[0026] The edge information is weighted to optimize the deep feature data of the target image data.

[0027] Secondly, this application provides an image super-resolution processing apparatus, comprising:

[0028] A shallow feature data extraction module is used to acquire target image data and perform shallow feature extraction on the target image data; the shallow feature extraction process is an operation of extracting shallow feature data of the target image data through convolution calculation;

[0029] A deep feature data extraction module is used to perform deep feature extraction on the target image data based on the shallow feature data through a deep feature extraction terminal to obtain the deep feature data of the target image data; the deep feature extraction terminal is composed of several target attention modules; the target attention modules are constructed based on local feature extraction and lightweight group attention modules;

[0030] The image super-resolution processing module is used to perform image reconstruction operations based on the deep feature data of the target image data to obtain the target image data after super-resolution processing.

[0031] Thirdly, this application provides an electronic device, comprising:

[0032] Memory, used to store computer programs;

[0033] A processor is used to execute the computer program to implement the aforementioned image super-resolution processing method.

[0034] Fourthly, this application provides a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the aforementioned image super-resolution processing method.

[0035] This application provides an image super-resolution processing method. First, target image data is acquired, and shallow feature extraction is performed on the target image data. The shallow feature extraction process involves extracting shallow feature data from the target image data through convolution calculation. Then, a deep feature extraction port is used to perform deep feature extraction on the target image data based on the shallow feature data to obtain deep feature data of the target image data. The deep feature extraction port consists of several target attention modules. These target attention modules are constructed based on local feature extraction and lightweight group attention modules. Finally, an image reconstruction operation is performed based on the deep feature data of the target image data to obtain the super-resolution processed target image data.

[0036] As can be seen from the above, this application extracts shallow features from the low-resolution target image data through convolutional computation, and further extracts deep features from the target image data based on the shallow feature data through a target attention module constructed based on local feature extraction and lightweight group attention modules, thereby generating high-resolution target image data. That is, by calculating self-attention through the target attention module, useful spatial information such as texture and structural information can be avoided, thus reducing the computational load and parameter count during image super-resolution processing and minimizing the impact of outliers. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0038] Figure 1 This is a flowchart of an image super-resolution processing method disclosed in this application;

[0039] Figure 2 This is a schematic diagram of a high-performance attention block (EAB) structure disclosed in this application;

[0040] Figure 3 This is a schematic diagram of a specific shift convolution disclosed in this application;

[0041] Figure 4 This is a schematic diagram of a specific lightweight group attention module structure disclosed in this application;

[0042] Figure 5 This is a schematic diagram of a specific image super-resolution processing network architecture disclosed in this application;

[0043] Figure 6 This is a flowchart of a specific image super-resolution processing method disclosed in this application;

[0044] Figure 7 This is a schematic diagram of an image super-resolution processing device disclosed in this application;

[0045] Figure 8 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0046] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0047] In modern banking environments, image super-resolution technology has diverse applications, significantly improving operational efficiency and service quality. SwinIR is an image super-resolution algorithm based on the Swing Transformer. SwinIR leverages the advantages of the Swing Transformer, achieving efficient image super-resolution through adaptive grouped convolution and cross-stage connections. However, in the deep feature extraction module of the SwinIR method, self-attention, when computed within a small window, affects its long-distance modeling ability, and consecutive Swing Transformer blocks contain many redundant components. Restormer computes self-attention in the channel space, but this sacrifices useful spatial information such as texture and structural information, thus impacting downstream tasks. Therefore, this application provides an image super-resolution processing scheme that reduces computational cost and parameter count, and minimizes the impact of outliers during image super-resolution processing.

[0048] See Figure 1 As shown in the figure, this application discloses an image super-resolution processing method, including:

[0049] Step S11: Obtain target image data and perform shallow feature extraction on the target image data.

[0050] In this embodiment, target image data is acquired, and shallow feature extraction is performed on the target image data. The shallow feature extraction process involves extracting shallow feature data from the target image data through convolution calculation. It is understood that the specific type of convolution can be determined according to the actual application scenario. For example, in one specific implementation, a 3×3 convolution can be used to calculate the target image data. Specifically, the shallow feature extraction operation on the target image data may include performing shallow feature extraction on the target image data through a 3×3 convolution.

[0051] Step S12: Perform deep feature extraction on the target image data based on the shallow feature data using the deep feature extraction terminal to obtain the deep feature data of the target image data.

[0052] In this embodiment, a deep feature extraction end performs deep feature extraction on the target image data based on the shallow feature data to obtain the deep feature data of the target image data. The deep feature extraction end consists of several target attention modules.

[0053] See Figure 2As shown, in one specific implementation, a high-performance attention block (EAB) is selected as the target attention module. The high-performance attention block consists of local feature extraction and a lightweight grouped self-attention module (LGAM). First, shift convolution is used to extract local structural information of the image, maintaining the same order of magnitude complexity as a 1×1 convolution. Then, the lightweight grouped self-attention module calculates self-attention on non-overlapping feature groups using different window sizes. Specifically, the target attention module is a module composed of two cascaded shift convolutions and a lightweight grouped attention module. Specifically, the step of performing deep feature extraction on the target image data based on the shallow feature data to obtain the deep feature data of the target image data can include: extracting local structural information of the shallow feature data through the shift convolution; and performing attention calculation on the local structural information through the lightweight grouped attention module to obtain the deep feature data of the target image data. It should be noted that the efficiency of local feature data extraction can be improved by adding a ReLU (Rectified Linear Unit, an artificial neural network activation function) activation function between the two shift convolutions.

[0054] In this embodiment, each kernel of the shift convolution has only one element set to 1, while all others are 0. The shift convolution calculation essentially translates the input matrix in a specific direction. By setting shift convolution kernels in different directions, the input matrix is ​​translated in different channels, and then 1×1 convolutions are used to fuse information between channels, thereby extracting spatial and channel domain information. The learnable parameters of the shift convolution are concentrated in the 1×1 convolution operation. Therefore, by combining shift convolution with only 1×1 convolutions, structured and hierarchical spatial domain information can be extracted, reducing the number of parameters and computational cost in convolutional network design.

[0055] See Figure 3As shown, the shift convolution consists of a shift operation and a 1×1 convolution. First, the shift convolution divides the input features into five groups, where any four groups are shifted along different spatial dimensions, while the remaining group remains unchanged. Then, a 1×1 convolution is used to obtain the information of neighboring elements after the shift. This approach maintains a similar complexity to 1×1 convolution while avoiding the introduction of additional learnable parameters. Specifically, extracting local feature data from the shallow feature data using the shift convolution can include: dividing the shallow feature data into five input feature matrices using the shift convolution, shifting any four of these input feature matrices along different spatial dimensions to obtain the shifted input matrix information; performing a 1×1 convolution on the input feature matrix information to obtain the information of neighboring elements after the shift; and extracting local feature data from the shallow feature data based on the neighboring element information.

[0056] In this embodiment, the lightweight group attention module mainly consists of two parts: channel attention and spatial attention. The lightweight group attention module is an organic combination of local and global attention, where channel attention and spatial attention are used to capture local and global features of the image, respectively. Channel attention and spatial attention perform calculations within groups, with each group extracting relevant features within a certain range. This processing method belongs to local attention computation (CA, Cross Attention). LGAM mines information from local regions by grouping feature maps. After all group features have undergone local computation, LGAM further integrates the information between all groups through a global self-attention module (SA, Self Attention), enabling global features to be connected and forming global attention. This mechanism ensures the global consistency of features, thereby improving the super-resolution processing's ability to understand the overall structure of the image.

[0057] See Figure 4As shown, the input feature map is divided into several groups (four groups in this embodiment: X1, X2, X3, X4). It's important to note that these groups can be non-overlapping feature blocks, or different channels or regions of the feature map. Within each group, a local cross-attention mechanism is executed. That is, the features of each group not only perform calculations with themselves but also interact with other groups. The cross arrows in the figure illustrate the information interaction between features in different groups, which improves the model's ability to capture local information. After cross-attention calculation, the output of each group is denoted as Y1, Y2, Y3, and Y4, representing the features after the local and cross-attention mechanisms. The above output features are integrated through a global self-attention module to ensure the connectivity and consistency of global features. Finally, the input is directly connected to the output of the self-attention module via residual connections (see the addition symbol in the figure), which mitigates the gradient vanishing problem and preserves the information of the input features in deep networks. Specifically, the step of performing attention calculations on the local feature data using the lightweight grouped attention module to obtain deep feature data of the target image data may include: dividing the local feature data into several feature groups, and performing attention calculations on each of the several feature groups based on a cross-attention mechanism; integrating the attention calculation results of the several feature groups to obtain global attention information; and connecting the local feature data and the global attention information through residual connections to obtain deep feature data of the target image data; wherein, the cross-attention mechanism is a mechanism in which the feature group performs its own calculations and interacts with other groups to obtain corresponding attention calculation results.

[0058] Step S13: Perform image reconstruction based on the deep feature data of the target image data to obtain the target image data after super-resolution processing.

[0059] See Figure 5 As shown, in this embodiment, before performing image reconstruction based on the deep feature data of the target image data, it is necessary to obtain the shallow feature data of the target image data through cross-stage connections, and then obtain the super-resolution processed target image data through convolution calculation and sampling operations. Specifically, the image reconstruction operation based on the deep feature data of the target image data to obtain the super-resolution processed target image data may include: performing image reconstruction based on the deep feature data of the target image data and the shallow feature data of the target image data obtained through cross-stage connections using an image reconstruction module to obtain the super-resolution processed target image data; wherein, the image reconstruction module consists of a convolution module and a sampling module.

[0060] As can be seen from the above, the embodiments of this application extract shallow features of the low-resolution target image data through convolution calculation, and further extract deep features of the target image data based on the shallow feature data through a target attention module constructed based on local feature extraction and a lightweight grouped attention module, so as to generate high-resolution target image data. That is, firstly, shift convolution is used to extract local structural information of the image, and the complexity is kept at the same order of magnitude as that of 1×1 convolution. Then, the lightweight grouped self-attention module uses different window sizes to calculate self-attention on non-overlapping feature groups, which can avoid sacrificing useful spatial information such as texture and structural information, thereby reducing the amount of computation and parameters in the image super-resolution processing.

[0061] Further, see Figure 6 As shown, in order to reduce the impact of outliers and enhance the ability to reconstruct image edge features, this application provides a specific image super-resolution processing method, including:

[0062] Step S21: Obtain target image data and perform shallow feature extraction on the target image data.

[0063] Step S22: Perform deep feature extraction on the target image data based on the shallow feature data using the deep feature extraction terminal to obtain the deep feature data of the target image data.

[0064] Step S23: Optimize the deep feature data of the target image data based on the smooth L1 loss function and the edge-aware loss function.

[0065] In this embodiment, a loss function is introduced during the training phase of the image super-resolution network to optimize the deep feature data of the target image data. Specifically, optimizing the deep feature data of the target image data based on Smooth L1 loss may include: optimizing the predicted image... and true high-resolution images Each pixel is compared, and the Smooth L1 loss of the entire image is calculated to measure the global error; where, the predicted image To input a low-resolution image into a high-resolution image output by a super-resolution network, the true high-resolution image is... The high-resolution label image corresponds to the low-resolution input image; the smooth L1 loss function formula is as follows:

[0066] ;

[0067] in, The L1 loss function provides an estimate; compared to Least Absolute Deviations (LAD), which is differentiable with respect to zero and whose derivative decreases as it approaches zero, making it easier to converge; and compared to Least Square Estimate (LSE), where the derivative at outliers is at most 1, the gradient will not explode. In other words, the smooth L1 loss function balances accuracy and robustness, reducing the impact of outliers.

[0068] Furthermore, since high-resolution images are often extracted from jagged edges, and edge-aware loss enhances the ability to reconstruct image edge features, it ensures that edges in complex images are clearer and more natural. Edge detectors are used to extract edge information from images, and then the detected edge regions are weighted, making the network pay more attention to these regions during the learning process. It is understood that the edge detector can be determined according to the actual application. For example, in one specific embodiment, the Laplace transform can be used as the edge detector. Specifically, optimizing the deep feature data of the target image data based on the edge-aware loss function can include: extracting edge information from the target image data using the Laplace transform; and weighting the prediction error of edge features based on the edge information to enhance the network's attention to edges. In real images... In the process, edge mapping is obtained using a detector. ,exist binary mask value at the location The calculation formula is as follows:

[0069] ;

[0070] in It is a preset threshold; the expression for the edge-aware loss function is as follows:

[0071] ;

[0072] in, To predict the image results, As a constant, "For element-wise multiplication, The Charbonnier loss function is expressed as follows:

[0073] ;

[0074] in, It is a constant.

[0075] Furthermore, the deep feature data of the target image data is optimized by using the backpropagation algorithm to calculate gradients and update model parameters. The loss function optimization is carried out throughout each training iteration to guide the network to continuously adjust parameters in order to generate high-quality super-resolution images.

[0076] Step S24: Perform image reconstruction based on the deep feature data of the target image data to obtain the target image data after super-resolution processing.

[0077] The specific implementation process of steps S21, S22 and S24 can be referred to the corresponding content in the foregoing embodiments, and will not be repeated here.

[0078] As can be seen from the above, this embodiment extracts shallow features from the low-resolution target image data through convolution calculation, and further extracts deep features from the target image data based on the shallow feature data through a target attention module constructed based on several local feature extraction and lightweight group attention modules, thereby generating high-resolution target image data. Furthermore, by introducing smooth L1 loss and edge-aware loss to balance accuracy and robustness, the impact of outliers is reduced, and the ability to reconstruct image edge features is enhanced, ensuring that edges in complex images are clearer and more natural. This reduces the computational load and parameter count during image super-resolution processing, and minimizes the impact of outliers.

[0079] Accordingly, see Figure 7 As shown, this application provides an image super-resolution processing apparatus, including:

[0080] The shallow feature data extraction module 11 is used to acquire target image data and perform shallow feature extraction on the target image data; the shallow feature extraction process is an operation of extracting shallow feature data of the target image data through convolution calculation;

[0081] The deep feature data extraction module 12 is used to perform deep feature extraction on the target image data based on the shallow feature data through the deep feature extraction end, so as to obtain the deep feature data of the target image data; the deep feature extraction end is composed of several target attention modules; the target attention modules are constructed based on local feature extraction and lightweight group attention modules;

[0082] The image super-resolution processing module 13 is used to perform image reconstruction operations based on the deep feature data of the target image data to obtain the target image data after super-resolution processing.

[0083] As can be seen from the above, the embodiments of this application extract shallow features from the low-resolution target image data through convolution calculation, and further extract deep features from the target image data based on the shallow feature data through a target attention module constructed based on local feature extraction and lightweight group attention modules, so as to generate high-resolution target image data. That is, by calculating self-attention through the target attention module, useful spatial information such as texture and structural information can be avoided, thereby reducing the amount of computation and parameters in the image super-resolution processing and reducing the impact of outliers.

[0084] In some specific embodiments, the deep feature data extraction module 12 may specifically include:

[0085] The local feature data extraction submodule is used to extract local feature data of the shallow feature data through the shift convolution.

[0086] The deep feature data extraction submodule is used to perform attention calculation on the local feature data through the lightweight group attention module to obtain the deep feature data of the target image data.

[0087] Furthermore, in some specific embodiments, the local feature data extraction submodule may specifically include:

[0088] The input matrix information acquisition unit is used to divide the shallow feature data into several input matrices through the shift convolution, and to shift the input matrices along different spatial dimensions to obtain the shifted input matrix information;

[0089] The local feature data acquisition unit is used to perform convolution calculations on the input matrix information to extract local feature data from the shallow feature data.

[0090] Furthermore, in some specific embodiments, the deep feature data extraction submodule may specifically include:

[0091] An attention calculation unit is used to divide the local feature data into several feature groups and perform attention calculation on the several feature groups respectively based on a cross-attention mechanism;

[0092] A global attention information acquisition unit is used to integrate the attention calculation results of the several feature groups to obtain global attention information;

[0093] A deep feature data acquisition unit is used to connect the local feature data with the global attention information through residual connection to obtain the deep feature data of the target image data;

[0094] The cross-attention mechanism is a mechanism in which the feature group calculates itself and interacts with other groups to obtain the corresponding attention calculation result.

[0095] In some specific embodiments, the image super-resolution processing apparatus may further include:

[0096] The loss function optimization module is used to optimize the deep feature data of the target image data based on the smooth L1 loss function and the edge-aware loss function.

[0097] Furthermore, in some specific embodiments, the loss function optimization module may specifically include:

[0098] The smooth L1 loss function optimization submodule is used to optimize the deep feature data of the target image data based on the smooth L1 loss function.

[0099] The edge-aware loss function optimization submodule is used to optimize the deep feature data of the target image data based on the edge-aware loss function.

[0100] Furthermore, in some specific embodiments, the edge-aware loss function optimization submodule may specifically include:

[0101] An edge information extraction unit is used to extract edge information of the target image data using an edge detector;

[0102] The target image data optimization unit is used to perform weighted operations on the edge information in order to optimize the deep feature data of the target image data.

[0103] Furthermore, embodiments of this application also disclose an electronic device, Figure 8 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the image super-resolution processing method disclosed in any of the foregoing embodiments. Furthermore, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0104] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0105] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0106] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the image super-resolution processing method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include a computer program capable of performing other specific tasks.

[0107] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned image super-resolution processing method. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0108] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0109] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0110] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0111] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0112] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An image super-resolution processing method, characterized in that, include: Acquire target image data and perform shallow feature extraction on the target image data; the shallow feature extraction operation is an operation to extract shallow feature data of the target image data through convolution calculation; The deep feature extraction terminal performs deep feature extraction on the target image data based on the shallow feature data to obtain the deep feature data of the target image data; the deep feature extraction terminal is composed of several target attention modules; the target attention modules are constructed based on local feature extraction and lightweight group attention modules; Image reconstruction is performed based on the deep feature data of the target image data to obtain the target image data after super-resolution processing. The target attention module is a module composed of two cascaded shift convolutions and a lightweight group attention module; Accordingly, the step of performing deep feature extraction on the target image data based on the shallow feature data through the deep feature extraction end to obtain the deep feature data of the target image data includes: Local feature data of the shallow feature data are extracted through the shift convolution; The lightweight grouped attention module performs attention calculations on the local feature data to obtain the deep feature data of the target image data. Accordingly, the extraction of local feature data from the shallow feature data through the shift convolution includes: The shallow feature data is divided into several input matrices by the shift convolution, and the input matrices are shifted along different spatial dimensions to obtain the shifted input matrix information. Convolution calculations are performed on the input matrix information to extract local feature data from the shallow feature data; Accordingly, the step of performing attention calculations on the local feature data through the lightweight grouping attention module to obtain the deep feature data of the target image data includes: The local feature data is divided into several feature groups, and attention is calculated for each of the feature groups based on a cross-attention mechanism. The attention calculation results of the aforementioned feature groups are integrated to obtain global attention information; The local feature data is connected with the global attention information through residual connection to obtain the deep feature data of the target image data; the cross-attention mechanism is a mechanism in which the feature group calculates for itself and interacts with other groups to obtain the corresponding attention calculation result.

2. The image super-resolution processing method according to claim 1, characterized in that, Also includes: The deep feature data of the target image data are optimized based on the smooth L1 loss function and the edge-aware loss function.

3. The image super-resolution processing method according to claim 2, characterized in that, Optimize the deep feature data of the target image data based on the edge-aware loss function, including: Edge information of the target image data is extracted using an edge detector; The edge information is weighted to optimize the deep feature data of the target image data.

4. An image super-resolution processing apparatus, used to implement the image super-resolution processing method as described in any one of claims 1 to 3, characterized in that, include: A shallow feature data extraction module is used to acquire target image data and perform shallow feature extraction on the target image data; the shallow feature extraction operation is an operation to extract shallow feature data of the target image data through convolution calculation; A deep feature data extraction module is used to perform deep feature extraction on the target image data based on the shallow feature data through a deep feature extraction terminal to obtain the deep feature data of the target image data; the deep feature extraction terminal is composed of several target attention modules; the target attention modules are constructed based on local feature extraction and lightweight group attention modules; The image super-resolution processing module is used to perform image reconstruction based on the deep feature data of the target image data to obtain the target image data after super-resolution processing. The target attention module is a module composed of two cascaded shift convolutions and a lightweight group attention module; Accordingly, the deep feature data extraction module includes: The local feature data extraction submodule is used to extract local feature data of the shallow feature data through the shift convolution. The deep feature data extraction submodule is used to perform attention calculation on the local feature data through the lightweight group attention module to obtain the deep feature data of the target image data. Accordingly, the local feature data extraction submodule includes: The input matrix information acquisition unit is used to divide the shallow feature data into several input matrices through the shift convolution, and to shift the input matrices along different spatial dimensions to obtain the shifted input matrix information; A local feature data acquisition unit is used to perform convolution calculations on the input matrix information to extract local feature data from the shallow feature data; Accordingly, the deep feature data extraction submodule includes: An attention calculation unit is used to divide the local feature data into several feature groups and perform attention calculation on the several feature groups respectively based on a cross-attention mechanism; A global attention information acquisition unit is used to integrate the attention calculation results of the several feature groups to obtain global attention information; The deep feature data acquisition unit is used to connect the local feature data with the global attention information through residual connection to obtain the deep feature data of the target image data; the cross-attention mechanism is a mechanism in which the feature group calculates for itself and interacts with other groups to obtain the corresponding attention calculation result.

5. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the image super-resolution processing method as described in any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that, Used to store a computer program, wherein the computer program, when executed by a processor, implements the image super-resolution processing method as described in any one of claims 1 to 3.

Images