Image resolution improving method and device, computer equipment and storage medium
By using shallow feature extraction, edge information learning and attention feature fusion in the image resolution enhancement method, the image resolution enhancement is directly implemented, which solves the problem of high calculation cost of existing super-resolution image processing methods, and realizes the resource High-efficiency image resolution improvements on restricted devices.
Patent Information
- Application Number
- CN202411237791.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-04
- Publication Date
- 2024-12-06
AI Technical Summary
The existing super-resolution image processing methods have high computational cost and are difficult to apply on resource-constrained devices, and there are problems of increased memory consumption and operational overhead.
Provide an image resolution enhancement method, which directly uses shallow feature extraction, edge information learning and attention feature fusion to directly enhance the resolution to simplify the calculation process to reduce the calculation process Calculate the cost.
It achieves improved image resolution in a simpler and more efficient way, reduces computing costs, and improves processing efficiency, and is suitable for resource-constrained devices.
Smart Images

Figure CN119090721A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology, and in particular to an image resolution enhancement method, device, computer equipment and storage medium. Background Art
[0002] Single image super-resolution is the task of restoring a high-resolution image from a low-resolution image. In the prior art, because low-resolution images can be generated from many different high-resolution images, the reconstructed high-resolution images often appear distorted and blurred. To address these problems, early solutions were mainly based on interpolation methods. Although interpolation-based methods are simple and have low computational complexity, they cannot restore lost image details in low-resolution (LR) images. Sparse dictionary learning-based methods improve interpolation-based methods. This type of method mainly learns the mapping function between low-resolution images and high-resolution images through training images. However, the current mainstream method for studying single image super-resolution technology is convolutional neural network technology, which can directly learn the mapping function between low-resolution and high-resolution images end-to-end, achieving better performance.
[0003] However, most existing super-resolution methods have high computational costs and often cannot be applied on resource-constrained devices. In order to deploy efficient super-resolution models, more and more networks use recursive methods, parameter sharing, and information distillation strategies to reduce model parameters, but they inevitably have problems with frequent access to previous states and additional nodes, which leads to increased memory consumption and operating overhead. Therefore, how to achieve a simpler and more efficient image resolution enhancement method has become an urgent problem to be solved. Summary of the invention
[0004] The main purpose of this application is to provide a method, device, computer equipment and storage medium for improving image resolution, so as to realize a simpler and more efficient method for improving image resolution.
[0005] In a first aspect, the present application provides a method for improving image resolution, the method comprising the following steps:
[0006] Acquire an initial image whose resolution is to be improved; input initial data corresponding to the initial image into a resolution enhancement model, perform resolution processing on the initial image in the resolution enhancement model, and obtain enhanced data after the image processing by the resolution enhancement model; obtain a high-resolution target image based on the enhanced data of the resolution enhancement model.
[0007] In a second aspect, the present application further provides an image super-resolution determination device, the image super-resolution determination device comprising:
[0008] An initial image acquisition module, used to acquire an initial image whose resolution is to be improved;
[0009] An enhanced data determination module, used for inputting the initial data corresponding to the initial image into a resolution enhancement model, performing resolution processing on the initial image in the resolution enhancement model, and obtaining enhanced data after the image processing by the resolution enhancement model;
[0010] The target image determination module is used to obtain a high-resolution target image according to the enhanced data of the resolution enhancement model.
[0011] In a third aspect, the present application also provides a computer device, comprising a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the image resolution enhancement method as described above is implemented.
[0012] In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, wherein when the computer program is executed by a processor, the image resolution enhancement method as described above is implemented.
[0013] The present application provides an image resolution enhancement method, apparatus, computer equipment and storage medium, the method comprising: obtaining an initial image whose resolution is to be enhanced; inputting initial data corresponding to the initial image into a resolution enhancement model, performing resolution processing on the initial data in the resolution enhancement model, and obtaining enhanced data after the image is processed by the resolution enhancement model; obtaining a high-resolution target image based on the enhanced data of the resolution enhancement model, thereby reducing the computational cost of improving the image resolution and improving the image resolution in a simpler and more efficient manner. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0015] Figure 1 A flowchart of a method for improving image resolution provided in one embodiment of the present application.
[0016] Figure 2 A schematic flowchart of determining target text information provided by an embodiment of the present application.
[0017] Figure 3A schematic flowchart of determining edge information provided by an embodiment of the present application.
[0018] Figure 4 A schematic diagram of a process for determining attention features provided in accordance with an embodiment of the present application.
[0019] Figure 5 A flowchart of another method for improving image resolution provided in an embodiment of the present application.
[0020] Figure 6 A schematic diagram of an image resolution enhancement device provided in one embodiment of the present application.
[0021] Figure 7 A schematic block diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION
[0022] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0023] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.
[0024] The embodiments of the present application provide a method, apparatus, computer device and storage medium for improving image resolution.
[0025] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.
[0026] Please refer to Figure 1 , Figure 1A flowchart of an image resolution enhancement method provided in one embodiment of the present application. The image resolution enhancement method can be used in a terminal or a server to simplify the computing cost and implement a simpler and more efficient image resolution enhancement method. The terminal can be an electronic device such as a mobile phone, a tablet computer, a laptop computer, a desktop computer, a personal digital assistant device, etc. The server can be an independent server, or a server cluster, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), as well as big data and artificial intelligence platforms. Figure 1 As shown, the image resolution enhancement method specifically includes steps S101 to S103.
[0027] S101: Acquire an initial image whose resolution is to be improved.
[0028] The image type of the initial image may be: low-resolution screenshots, old photos, satellite or drone images, medical images, surveillance video screenshots, scientific research images, etc.
[0029] Exemplarily, the initial image whose resolution is to be improved may be acquired based on digital photography technology, sensor data acquisition technology, scanning technology, or other technical means.
[0030] It should be noted that the above-listed techniques for obtaining the initial image are only for illustrative purposes. In practical applications, any other suitable techniques may be selected as needed, and this application does not impose any limitation on this.
[0031] S102: Inputting initial data corresponding to the initial image into a resolution enhancement model, performing resolution processing on the initial image in the resolution enhancement model, and obtaining enhanced data after the image processing by the resolution enhancement model.
[0032] Specifically, the initial data corresponding to the initial image can be input into the resolution enhancement model based on an image transmission method such as a command line interface, a graphical user interface, an application programming interface (API) call, etc. For example, in an image transmission method based on a command line interface, a programming language can be used to pass the path of the initial image to the resolution enhancement model by inputting appropriate command line parameters, wherein the programming language can be Python, C language, Java, etc. In actual applications, an appropriate programming language can be selected according to actual needs, and this application does not limit this.
[0033] For example, before transmitting the initial image to the resolution enhancement model, in order to facilitate the transmission of the initial image and ensure that the initial image does not lose information during the transmission process, the initial image may be appropriately compressed based on image compression technology, wherein the image compression technology may include: lossless compression, lossy compression, vector compression, etc.
[0034] It should be noted that the image compression technologies listed above are only used for illustrative purposes. In actual applications, appropriate image compression technologies can be selected according to specific application scenarios, and this application does not impose any restrictions on this.
[0035] As another example, the data type of the initial image may be different from the data type that the resolution enhancement model can accept as input. Before the initial image is input into the resolution enhancement model, the initial image needs to be converted into a corresponding data type and its format adjusted according to the image data specification compatible with the model to ensure that the resolution enhancement model can fully accept and effectively process the input initial image.
[0036] In addition, before the initial image is input into the resolution enhancement model, the initial image can be appropriately adjusted according to the requirements of the resolution enhancement model for the image size, data type, etc. of the input initial image, so that the adjusted initial image meets the input requirements of the resolution enhancement model, thereby avoiding the situation where the initial image cannot be input into the resolution enhancement model due to factors such as image size or image data type, thereby improving the working efficiency of the resolution enhancement model.
[0037] In addition, before the initial image is input into the resolution enhancement model, a series of standardized preprocessing operations are performed on the initial image to meet the specific requirements of the resolution enhancement model for the input initial image size and data type. These operations ensure the format consistency of the data input into the initial image, thereby avoiding input errors of the resolution enhancement model due to initial image size mismatch or data type incompatibility, thereby improving the processing efficiency of the resolution enhancement model. Specifically, it includes but is not limited to the following steps: adjusting the image size to the input specifications specified by the resolution enhancement model, performing data type conversion to meet the numerical accuracy requirements of the resolution enhancement model, and making necessary channel order adjustments. Through these measures, it is ensured that the resolution enhancement model can efficiently and accurately process the input initial image and optimize the overall calculation process.
[0038] S103: Obtain a high-resolution target image according to the enhanced data of the resolution enhancement model.
[0039] Specifically, the enhanced data may include: a multidimensional array of high-resolution pixel values, the width and height of the target image, etc.
[0040] For example, the multidimensional array of high-resolution pixel values can be combined based on array concatenation technology to obtain a high-resolution target image, wherein the array concatenation technology includes: array slicing, concatenate function, np.vstack function or np.hstack function and other keywords in programming languages.
[0041] The image resolution enhancement method provided in the embodiment of the present application includes: obtaining an initial image whose resolution is to be enhanced; inputting initial data corresponding to the initial image into a resolution enhancement model, performing resolution processing on the initial image in the resolution enhancement model, and obtaining enhanced data after the image processing by the resolution enhancement model; obtaining a high-resolution target image based on the enhanced data of the resolution enhancement model, and obtaining enhanced data by directly performing resolution enhancement processing on the initial image in the resolution enhancement model, thereby simplifying the calculation cost, and then obtaining a high-resolution target image based on the enhanced data, and directly improving the resolution of the image in a simpler way based on the enhancement processing of the resolution enhancement model.
[0042] See also Figure 2 , Figure 2 A schematic diagram of a process for determining a target image according to an embodiment of the present application. Figure 2 As shown, the following steps S201 to S205 are included.
[0043] S201. Input the initial image into the shallow feature extraction module to obtain shallow features output by the shallow feature extraction module.
[0044] Exemplarily, after the initial image is input into the shallow feature extraction block (Shallow Feature Extraction Block, SFEB) to obtain the shallow features output by the shallow feature extraction block, the shallow features can be represented by f.
[0045] Wherein, the shallow feature extraction module is located in the resolution enhancement model, and the resolution enhancement model also includes: an edge information learning module (Edge Structure study Unit, ESTU) and an image reconstruction module (Upsampling Module, Upsamp le).
[0046] Exemplarily, the shallow feature extraction module includes Blueprint Separable Convolutions (BSConv).
[0047] Specifically, when the initial image is input into the shallow feature extraction module, the shallow feature extraction module will perform a series of processing on the initial image to extract the shallow features of the initial image. Among them, these shallow features include the basic elements of the initial image such as edges, corners, textures, etc. The shallow features are crucial for subsequent image analysis, recognition, and classification tasks.
[0048] It should be noted that the above-mentioned shallow features include the edges, corners, textures and other basic elements of the initial image, which are only used for illustrative purposes. The shallow features may also include other elements, which are not limited in this application.
[0049] In some embodiments, the edge information learning module consists of an edge operator module and an attention module.
[0050] Specifically, after obtaining the shallow feature f of the initial image, the edge operator module is used to receive the shallow feature f and learn the edge information of the shallow feature f in the edge operator module. The attention module (Enhanced Saliency Attention with Cross-scale Context Aggregation, ESA+CCA) is used to receive the edge information output by the edge operator module and assign weights to the edge information.
[0051] The specific implementation steps of the edge operator module and the attention module are described in detail in the subsequent S202 and S203, and will not be described again in this application.
[0052] S202: Input the shallow features into the edge operator module to obtain edge information output by the edge operator module.
[0053] Specifically, the edge operator module includes edge algorithms and edge operators, and edge information in shallow features is extracted based on edge algorithms and edge operators. Among them, edge algorithms include: Canny edge detector, Smallest Univ ale Segment Assimilating Nucleus (SUSAN) edge detector, etc., and edge operators include: Sobel operator, Prewitt operator, Roberts operator, etc.
[0054] When the shallow features are input to the edge operator module, at least one of the above edge algorithms and edge operators can be called in the edge operator module to extract edge information from the shallow features. The edge information can be expressed in the form of edge position, edge direction, and edge strength, and subsequent image analysis, object detection, shape recognition, and other tasks can be accurately implemented based on the edge information.
[0055] It should be noted that the above-mentioned edge information may also be represented in any other suitable form. In practical applications, any suitable one or more edge information representation forms may be selected according to the specific application, and this application does not limit this.
[0056] In some embodiments, the inputting of the shallow features into the edge operator module to obtain the edge information output by the edge operator module includes: inputting the shallow features into a first edge detection operator, a second edge detection operator and a second-order edge detection operator in parallel to obtain a first detection result output by the first edge detection operator, a second detection result output by the second edge detection operator and a second-order detection result output by the second-order edge detection operator, respectively, wherein the first edge detection operator and the second edge detection operator are first-order edge detection operators, and the edge operator module includes a first-order edge detection operator and a second-order edge detection operator; adding the first detection result, the second detection result and the second-order detection result and outputting them to the edge operator module to obtain the edge information.
[0057] Exemplarily, the first edge detection operator may be represented by Sx, the second edge detection operator may be represented by Sy, and the second-order edge detection operator may be represented by Lap.
[0058] Specifically, Sx is the edge detection operator of the Sobel operator in the horizontal direction; Sy is the edge detection operator of the Sobel operator in the vertical direction. Lap is the Laplace edge detection operator. The above Sx, Sy, and Lap process the edge part of the initial image from different dimensions.
[0059] The specific calculation process can be found in Figure 3 . Figure 3 The left part in the middle represents the structure of the shallow feature f of the initial image in the training stage. The left part of the structure is re-parameterized to obtain the right structure, which is applied in the reasoning stage of the target image.
[0060] Preferably, only one shallow feature extraction operation is performed in the resolution enhancement model, followed by four consecutive edge information learning modules, so that the resolution enhancement model can achieve a better balance between performance and parameter quantity.
[0061] S203: Input the edge information into the attention module to obtain the attention features output by the attention module.
[0062] Specifically, based on the weight of the information category in the edge preprocessing information, the attention feature corresponding to the edge information is determined.
[0063] S204: Fusing the shallow features and the attention features to obtain fused features.
[0064] Specifically, before feature fusion, if the shallow features and the attention features have different sizes or channel numbers, it is necessary to perform feature alignment operations on the shallow features and the attention features, where the alignment operations include: upsampling, downsampling, and channel number adjustment.
[0065] It should be noted that the above alignment operation is only used for illustrative purposes. In practical applications, any other suitable alignment operation may be selected according to actual needs, and this application does not impose any limitation on this.
[0066] Exemplarily, feature fusion includes: weighted summation, concatenation, product, etc., to combine shallow features and attention features. Among them, weighted summation involves assigning different weights to shallow features and attention features, concatenation directly merges the feature maps corresponding to shallow features and attention features, and product multiplies the corresponding elements of the two feature maps in shallow features and attention features.
[0067] It should be noted that the specific method of feature fusion mentioned above is only for illustrative purposes. The specific feature fusion method can be reasonably selected according to actual needs, and this application does not impose any restrictions on this.
[0068] S205: Input the fusion feature into the image reconstruction module, and obtain the target image according to the enhanced data of the image reconstruction module.
[0069] Among them, the enhanced data includes: image features, hyperparameters, etc.
[0070] Among them, image reconstruction technology includes image super-resolution, image denoising, image generation, etc.
[0071] Exemplarily, according to the enhanced data of the image reconstruction module, the enhanced data may be processed or integrated based on an image processor to obtain the target image output by the image processor. Alternatively, the enhanced data may be input into image generation software, and the target image may be obtained by processing the enhanced data based on the image generation software.
[0072] In some embodiments, after obtaining a high-resolution target image according to the enhanced data of the resolution enhancement model, the method further includes: acquiring a first set of target image collections and a second set of target image collections; determining a preset standard test set; based on the first set of target image collections, determining a peak signal-to-noise ratio and a structural similarity index between the initial image and the target image on the preset standard test set; testing the resolution enhancement model based on the peak signal-to-noise ratio and the structural similarity index; and verifying the resolution enhancement model based on the second set of target image collections.
[0073] Exemplarily, the preset standard test sets may include: one or more of Set5, Set14, B100, Urban100, and Manga109.
[0074] Exemplarily, assuming that a total of 810 target images are obtained, the first 800 target images can be used as the first group of target images, and the remaining 801 to 810 images can be used as the second group of target images. The peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) are calculated based on the above-mentioned preset standard test set to evaluate the performance of the resolution enhancement model; and the 801 to 810 target images are used to verify the feasibility of the resolution enhancement model.
[0075] As another example, in the brightness (Y) channel of the YCbCr space, the resolution enhancement model may be evaluated using the following formula corresponding to the peak signal-to-noise ratio and the formula corresponding to the structural similarity index:
[0076] The formula corresponding to the peak signal-to-noise ratio is:
[0077]
[0078] Among them, PSNR represents the peak signal-to-noise ratio, MAX I It represents the maximum value of the pixels in the target image, and MSE represents the mean square error;
[0079] The formula corresponding to the structural similarity index is:
[0080]
[0081] Among them, μ x represents the mean brightness of the initial image, μ y represents the mean brightness of the target image, σ x represents the brightness standard deviation of the initial image, σ yRepresents the brightness standard deviation of the target image, σ xy represents the brightness covariance, and C1 and C2 are both constants.
[0082] Among them, the larger the PSNR value, the higher the resolution of the target image, and the larger the SSIM value, the more similar the target image is to the initial image.
[0083] The method for determining the target image provided by the embodiment of the present application includes: inputting the initial image into the shallow feature extraction module to obtain the shallow features output by the shallow feature extraction module; inputting the shallow features into the edge operator module to obtain the edge information output by the edge operator module; inputting the edge information into the attention module to obtain the attention features output by the attention module; performing feature fusion on the shallow features and the attention features to obtain the fused features after the shallow features and the attention features are fused; inputting the fused features into the image reconstruction module to obtain the target image according to the enhanced data of the image reconstruction module. In the above method, the initial image is input into the shallow feature extraction module to obtain the shallow features of the initial image, and the shallow features of the initial image are extracted in the shallow feature extraction module to improve the accuracy of the extracted shallow features, and the shallow features are input into the edge operator module to obtain at least one edge information processed by the edge operator module, and the above edge information can provide an information basis for obtaining a more accurate target image in the subsequent process. Then, the shallow features and the attention features are fused to obtain fused features. The image reconstruction module obtains the target image based on the fused features, thereby improving the resolution of the target image.
[0084] See also Figure 4 , Figure 4 A flowchart of determining attention features is provided in one embodiment of the present application. Figure 4 As shown, the process includes the following steps S401 to S405.
[0085] S401: Preprocess the edge information to obtain edge preprocessing information corresponding to the edge information.
[0086] The preprocessing includes one or more of: image grayscale, noise elimination, edge enhancement, edge refinement, etc.
[0087] Exemplarily, according to the actual situation of the edge information, one or more of the above preprocessing methods are selected to process the edge information.
[0088] It should be noted that the preprocessing methods listed above are only for illustrative purposes. In practical applications, any suitable preprocessing method can be selected according to actual needs, and this application does not impose any limitation on this.
[0089] S402: Determine the information category of the edge preprocessing information.
[0090] The information categories include: a first category, a second category, a third category, ..., and an Nth category. Specifically, each information category in the edge preprocessing information corresponds to a portion of the edge preprocessing information.
[0091] It should be noted that the number of specific information categories into which the edge preprocessing information is specifically divided can be reasonably divided according to actual applications, and this application does not impose any restrictions on this.
[0092] S403: Determine the weight of each information category in the edge preprocessing information based on a preset attention mechanism algorithm.
[0093] Among them, the preset attention mechanism algorithm includes: feature aggregation, importance scoring, weight normalization, weight adjustment and other steps.
[0094] Exemplarily, feature aggregation includes: aggregating the features of each information category to form a feature set. Importance scoring is to score each information category in the edge preprocessing information to evaluate their importance to the overall task, which can be implemented through a learnable weight function, such as using a fully connected layer. Weight normalization is to normalize the calculated importance scores to ensure that all weights add up to 1 (or a fixed value) to ensure that the weights are comparable. Use the normalized weights to weight the original features to achieve weight adjustment.
[0095] It should be noted that the steps of the above-mentioned preset attention mechanism algorithm can be reasonably increased or deleted according to the specific application, and this application does not impose any restrictions on this.
[0096] S404: Perform weighted processing on each information category according to the weight of each information category to obtain a weighted result for each information category.
[0097] Specifically, the weighted processing may be to multiply the weight of each information category by the corresponding information category to obtain a weighted result of each information category.
[0098] S405: Taking the maximum value of all the weighted results as the attention feature.
[0099] Specifically, all weighted results are summarized, all weighted results are traversed, the maximum value among all weighted results is found, and the maximum value among the weighted results is determined as the attention feature.
[0100] Exemplarily, the method of traversing all weighted results may be: loop traversal, keyword function based on programming language, sorting method, etc.
[0101] It should be noted that the above method of obtaining the maximum value of the weighted result is only for illustrative purposes. In actual applications, a suitable method can be selected according to specific needs to obtain the maximum value of the weighted result, and this application will not go into details here.
[0102] The embodiment of the present application provides a method for determining an attention feature, in which the information category of the edge preprocessing information is determined in turn, the weight of each of the information categories is determined, and each of the information categories is weighted based on the weight, and the maximum value of the weighted result after the weighted processing is used as the attention feature. Based on the above steps, the accuracy of obtaining the attention feature is improved, and the correct attention feature data is provided for the subsequent high-resolution and accurate target image based on the attention feature.
[0103] Figure 5 This is a flow chart of another method for improving image resolution provided by an embodiment of the present application. The specific implementation steps are similar to those in the above method, and the present application will not repeat them here.
[0104] See also Figure 6 , Figure 6 This is a schematic diagram of an image resolution enhancement device provided in an embodiment of the present application. The image resolution enhancement device can be configured in a server or a terminal to execute the aforementioned image resolution enhancement method.
[0105] like Figure 6 As shown, the image resolution enhancement device 600 includes: an initial image acquisition module 601, an enhanced data determination module 602, and a target image determination module 603.
[0106] The initial image acquisition module 601 is used to acquire the initial image whose resolution is to be improved.
[0107] The enhanced data determination module 602 is used to input the initial data corresponding to the initial image into the resolution enhancement model, perform resolution processing on the initial image in the resolution enhancement model, and obtain enhanced data after the image is processed by the resolution enhancement model.
[0108] The target image determination module 603 is used to obtain a high-resolution target image according to the enhanced data of the resolution enhancement model.
[0109] In some embodiments, the initial image acquisition module 601 is specifically used to:
[0110] Get the initial image whose resolution is to be improved.
[0111] In some embodiments, the enhanced data determination module 602 is specifically configured to:
[0112] The initial data corresponding to the initial image is input into a resolution enhancement model, and the resolution of the initial image is processed in the resolution enhancement model to obtain enhanced data after the image is processed by the resolution enhancement model.
[0113] In some embodiments, the target image determination module 603 is specifically configured to:
[0114] The initial image is input into the shallow feature extraction module to obtain the shallow features output by the shallow feature extraction module; the shallow features are input into the edge operator module to obtain the edge information output by the edge operator module; the edge information is input into the attention module to obtain the attention features output by the attention module; the shallow features are fused with the attention features to obtain the fused features after the shallow features and the attention features are fused; the fused features are input into the image reconstruction module to obtain the target image according to the enhanced data of the image reconstruction module.
[0115] In some embodiments, the target image determination module 603 is further configured to:
[0116] The shallow features are input into the first edge detection operator, the second edge detection operator and the second-order edge detection operator in parallel to obtain a first detection result output by the first edge detection operator, a second detection result output by the second edge detection operator and a second-order detection result output by the second-order edge detection operator respectively, wherein the first edge detection operator and the second edge detection operator are first-order edge detection operators, and the edge operator module includes a first-order edge detection operator and a second-order edge detection operator; the first detection result, the second detection result and the second-order detection result are added and output to the edge operator module to obtain the edge information.
[0117] In some embodiments, the target image determination module 603 is further configured to:
[0118] Preprocessing the edge information to obtain edge preprocessing information corresponding to the edge information;
[0119] Determine the information category of the edge preprocessing information; determine the weight of each information category in the edge preprocessing information based on a preset attention mechanism algorithm; perform weighted processing on each information category according to the weight of each information category to obtain a weighted result for each information category; and use the maximum value of all the weighted results as the attention feature.
[0120] In some embodiments, after obtaining a high-resolution target image according to the enhanced data of the resolution enhancement model, the target image determination module 603 is further configured to:
[0121] Acquire a first set of target image sets and a second set of target image sets; determine a preset standard test set; based on the first set of target image sets, determine the peak signal-to-noise ratio and the structural similarity index between the initial image and the target image on the preset standard test set; based on the peak signal-to-noise ratio and the structural similarity index, test the resolution enhancement model; based on the second set of target image sets, verify the resolution enhancement model.
[0122] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described device and each module can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.
[0123] The above-mentioned device can be implemented in the form of a computer program. Figure 7 Runs on the computer device shown.
[0124] See also Figure 7 , Figure 7 1 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device may be a server.
[0125] See also Figure 7 The computer device includes a processor, a memory and a network interface connected through a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.
[0126] The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute any image resolution enhancement method.
[0127] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.
[0128] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any image resolution enhancement method.
[0129] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that Figure 7The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0130] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.
[0131] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps:
[0132] Acquire an initial image whose resolution is to be improved; input initial data corresponding to the initial image into a resolution enhancement model, perform resolution processing on the initial data in the resolution enhancement model, and obtain enhanced data after the image is processed by the resolution enhancement model; obtain a high-resolution target image based on the enhanced data of the resolution enhancement model.
[0133] In one embodiment, the resolution enhancement model includes a shallow feature extraction module, an edge information learning module, and an image reconstruction module.
[0134] In one embodiment, the edge information learning module consists of an edge operator module and an attention module.
[0135] In one embodiment, the inputting of the initial data corresponding to the initial image into the resolution enhancement model and obtaining a high-resolution target image according to the enhanced data of the resolution enhancement model is used to achieve:
[0136] The initial image is input into the shallow feature extraction module to obtain the shallow features output by the shallow feature extraction module; the shallow features are input into the edge operator module to obtain the edge information output by the edge operator module; the edge information is input into the attention module to obtain the attention features output by the attention module; the shallow features are fused with the attention features to obtain the fused features after the shallow features and the attention features are fused; the fused features are input into the image reconstruction module to obtain the target image according to the enhanced data of the image reconstruction module.
[0137] In one embodiment, when the shallow feature is input into the edge operator module to obtain the edge information output by the edge operator module, it is used to implement:
[0138] The shallow features are input into the first edge detection operator, the second edge detection operator and the second-order edge detection operator in parallel to obtain a first detection result output by the first edge detection operator, a second detection result output by the second edge detection operator and a second-order detection result output by the second-order edge detection operator respectively, wherein the first edge detection operator and the second edge detection operator are first-order edge detection operators, and the edge operator module includes a first-order edge detection operator and a second-order edge detection operator; the first detection result, the second detection result and the second-order detection result are added and output to the edge operator module to obtain the edge information.
[0139] In one embodiment, when the edge information is input into the attention module to obtain the attention feature output by the attention module, it is used to implement:
[0140] Preprocess the edge information to obtain edge preprocessing information corresponding to the edge information; determine the information category of the edge preprocessing information; determine the weight of each information category in the edge preprocessing information based on a preset attention mechanism algorithm; perform weighted processing on each information category according to the weight of each information category to obtain a weighted result for each information category; and use the maximum value of all the weighted results as the attention feature.
[0141] In one embodiment, after obtaining a high-resolution target image according to the enhanced data of the resolution enhancement model, the method is used to implement:
[0142] Acquire a first set of target image sets and a second set of target image sets; determine a preset standard test set; based on the first set of target image sets, determine the peak signal-to-noise ratio and the structural similarity index between the initial image and the target image on the preset standard test set; based on the peak signal-to-noise ratio and the structural similarity index, test the resolution enhancement model; based on the second set of target image sets, verify the resolution enhancement model.
[0143] A computer-readable storage medium is also provided in an embodiment of the present application. The computer-readable storage medium stores a computer program. The computer program includes program instructions. The processor executes the program instructions to implement any image resolution enhancement method provided in the embodiments of the present application.
[0144] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (Smart Medi a Card, SMC), a secure digital (Secure Digital, SD) card, a flash memory card (Flash Card), etc., equipped on the computer device.
[0145] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.
Claims
1. A method for improving image resolution, characterized in that: include: Obtaining an initial image whose resolution is to be improved; Inputting initial data corresponding to the initial image into a resolution enhancement model, performing resolution processing on the initial data in the resolution enhancement model, and obtaining enhanced data after the resolution enhancement model image processing; A high-resolution target image is obtained according to the enhanced data of the resolution enhancement model.
2. The image resolution enhancement method according to claim 1, characterized in that: The resolution enhancement model includes a shallow feature extraction module, an edge information learning module, and an image reconstruction module.
3. The image resolution enhancement method according to claim 2, characterized in that: The edge information learning module consists of an edge operator module and an attention module.
4. The method for improving image resolution according to claim 3, characterized in that: The step of inputting initial data corresponding to the initial image into a resolution enhancement model, and obtaining a high-resolution target image according to the enhanced data of the resolution enhancement model, comprises: Inputting the initial image into the shallow feature extraction module to obtain shallow features output by the shallow feature extraction module; Inputting the shallow features into the edge operator module to obtain edge information output by the edge operator module; Inputting the edge information into the attention module to obtain the attention feature output by the attention module; Performing feature fusion on the shallow feature and the attention feature to obtain a fusion feature after the shallow feature and the attention feature are fused; The fusion features are input into the image reconstruction module, and the target image is obtained according to the enhanced data of the image reconstruction module.
5. The method for improving image resolution according to claim 4, characterized in that: The step of inputting the shallow features into the edge operator module to obtain edge information output by the edge operator module includes: Inputting the shallow features into a first edge detection operator, a second edge detection operator and a second-order edge detection operator in parallel, respectively obtaining a first detection result output by the first edge detection operator, a second detection result output by the second edge detection operator and a second-order detection result output by the second-order edge detection operator, wherein the first edge detection operator and the second edge detection operator are first-order edge detection operators, and the edge operator module includes a first-order edge detection operator and a second-order edge detection operator; The first detection result, the second detection result and the second-order detection result are added and output to the edge operator module to obtain the edge information.
6. The method for improving image resolution according to claim 4, characterized in that: The step of inputting the edge information into the attention module to obtain the attention feature output by the attention module includes: Preprocessing the edge information to obtain edge preprocessing information corresponding to the edge information; Determining an information category of the edge preprocessing information; Determine the weight of each information category in the edge preprocessing information based on a preset attention mechanism algorithm; According to the weight of each information category, weighted processing is performed on each information category to obtain a weighted result of each information category; The maximum value of all the weighted results is taken as the attention feature.
7. The method for improving image resolution according to claim 1, characterized in that: After obtaining a high-resolution target image according to the enhanced data of the resolution enhancement model, the method further comprises: Acquire a first set of target image sets and a second set of target image sets; Determine the preset standard test set; Based on the first set of target images, determining a peak signal-to-noise ratio and a structural similarity index between the initial image and the target image on the preset standard test set; Testing the resolution enhancement model based on the peak signal-to-noise ratio and the structural similarity index; The resolution enhancement model is verified based on the second set of target images.
8. An image resolution enhancement device, characterized in that: include: An initial image acquisition module, used to acquire an initial image whose resolution is to be improved; An enhanced data determination module, used for inputting the initial data corresponding to the initial image into a resolution enhancement model, performing resolution processing on the initial image in the resolution enhancement model, and obtaining enhanced data after the image processing by the resolution enhancement model; The target image determination module is used to obtain a high-resolution target image according to the enhanced data of the resolution enhancement model.
9. A computer device, characterized in that: The computer device includes a memory and a processor; The memory is used to store computer programs; The processor is used to execute the computer program and implement the image resolution improvement method according to any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the image resolution enhancement method according to any one of claims 1 to 7.
