Super-resolution calculation method and system and electronic equipment
By precalculating the mapping relationship between image pixel features and filters, selecting the filter in real time for super resolution calculation, solving the problems of image amplification loss and large calculation volume in the prior art, improving the operating stability of mid- and low-end mobile phones and User experience.
Patent Information
- Application Number
- CN202410778912.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-17
- Publication Date
- 2024-11-15
AI Technical Summary
The existing super resolution algorithms have high losses during image amplification and large calculation volume, resulting in poor user experience, especially when running on mid- and low-end mobile phones for a long time, there will be problems with excessive processor temperature and frame drop.
By precalculating the mapping relationship between the feature information of the image pixel and the filter, selecting the corresponding filter in real time based on the characteristics of the image pixels for super resolution calculation, which is divided into two parts: precalculation and real-time calculation, reducing the data processing amount in real-time calculation , improve computing accuracy and user experience.
It reduces the loss during image amplification, reduces the processor load of mid- and low-end mobile phones, avoids processor heating and frame reduction problems, and improves the user experience.
Smart Images

Figure CN118967440A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology, and in particular to a super-resolution calculation method, system and electronic device. Background Art
[0002] Super resolution is an image processing technique that aims to generate a high-resolution image by processing a low-resolution image. This technique is widely used in the fields of image processing, computer vision, and digital image processing, especially in improving image quality and enhancing image details.
[0003] Existing super-resolution algorithms include methods based on deep learning and temporal anti-aliasing upscaling (TAAU) based on upsampling. The existing technology has a high loss after image enlargement and cannot meet the user's demand for lossless image enlargement. In addition, the existing technology has a large amount of calculation. When running for a long time on low-, medium- and high-end mobile phones, the processor temperature will be too high, resulting in frame drops, affecting the user experience.
[0004] The existing technology has the problem of poor user experience due to high loss and large amount of calculation when enlarging the image. Summary of the invention
[0005] The embodiments of the present application provide a super-resolution calculation method, system and electronic device to at least solve the problem of poor user experience caused by high loss and large calculation amount when enlarging the image in the related art.
[0006] In a first aspect, an embodiment of the present application provides a super-resolution calculation method, comprising:
[0007] Obtain the mapping relationship between the feature information of image pixels and the filter;
[0008] Calculating target feature information of pixels in the target image, and determining a target filter according to the target feature information according to the mapping relationship;
[0009] A super-resolution image is determined according to the pixels of the target image and the target filter corresponding to each pixel.
[0010] In one embodiment, the step of acquiring a mapping relationship between feature information of image pixels and filters includes:
[0011] Acquire a corresponding low-definition image data set according to the high-definition image data set, and enlarge the images in the low-definition image data set to the resolution of the high-definition image through an interpolation filter to obtain the image data set to be calculated;
[0012] Calculating feature information of each low-definition pixel in the image data set to be calculated;
[0013] A filter corresponding to the feature information is calculated according to the low-definition pixel and the high-definition pixel corresponding to the low-definition pixel.
[0014] In one embodiment, the obtaining of the mapping relationship between the feature information of the image pixels and the filter further includes:
[0015] Grouping the feature information of each low-definition pixel;
[0016] The filter corresponding to each group is calculated according to the low-definition pixels in each group and the high-definition pixels corresponding to the low-definition pixels.
[0017] In one embodiment, calculating the feature information of each low-definition pixel in the image data set to be calculated includes:
[0018] Obtain a pixel block of a set size centered on a low-definition pixel, and decompose the pixel block to obtain an eigenvalue and an eigenvector;
[0019] Taking the largest eigenvalue among the eigenvalues as the intensity in the feature information;
[0020] The angle between the eigenvectors corresponding to the two largest eigenvalues among the eigenvalues is calculated, and the angle is used as the angle in the feature information.
[0021] In one embodiment, the calculating the filter corresponding to the feature information includes:
[0022] The filter corresponding to the feature information is calculated by a least squares method or a deep learning method.
[0023] In one embodiment, the target image is configured to be acquired by:
[0024] Acquire an original image, render the original image to a target resolution, and obtain an intermediate image, wherein the resolution of the target image is lower than the resolution of the original image;
[0025] The target image is obtained by enlarging the intermediate image to the resolution of the original image through an interpolation filter; or
[0026] An initial image is acquired, and the initial image is enlarged to a target resolution by an interpolation method to obtain a target image, wherein the resolution of the target image is higher than the resolution of the initial image.
[0027] In a second aspect, an embodiment of the present application provides a super-resolution computing system, including:
[0028] Mapping module: used to obtain the mapping relationship between the feature information of image pixels and the filter;
[0029] A selection module: used for calculating target feature information of pixels in a target image, and determining a target filter according to the target feature information according to the mapping relationship;
[0030] Determination module: used to determine the super-resolution image according to the pixels of the target image and the target filter corresponding to each pixel.
[0031] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the super-resolution calculation method described in the first aspect when executing the computer program.
[0032] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the super-resolution calculation method described in the first aspect.
[0033] A super-resolution calculation method, system and electronic device provided by the embodiments of the present application have at least the following technical effects.
[0034] This application pre-calculates the mapping relationship between the feature information of image pixels and multiple filters. During real-time calculation, the corresponding filter is selected according to the pixel features of the real-time image for super-resolution calculation. The accuracy of super-resolution calculation is improved by using multiple filters that match the pixel features. At the same time, this application divides the calculation into two parts: pre-calculation and real-time calculation. The mapping relationship between feature information and filters is determined in advance. During real-time calculation, only the corresponding filter needs to be selected, which reduces the amount of data processing during real-time calculation, is conducive to the long-term operation of mid- and low-end mobile phones, and improves the user experience.
[0035] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0037] Figure 1 is a flowchart of a super-resolution calculation method according to an exemplary embodiment;
[0038] Figure 2 is a structural block diagram of a super-resolution computing system according to an exemplary embodiment;
[0039] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0040] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments provided in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.
[0041] Obviously, the drawings described below are only some examples or embodiments of the present application. For ordinary technicians in this field, the present application can also be applied to other similar scenarios based on these drawings without creative work. In addition, it can also be understood that although the efforts made in this development process may be complicated and lengthy, for ordinary technicians in this field related to the content disclosed in this application, some changes in design, manufacturing or production based on the technical content disclosed in this application are just conventional technical means, and should not be understood as insufficient content disclosed in this application.
[0042] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in this application may be combined with other embodiments without conflict.
[0043] Unless otherwise defined, the technical terms or scientific terms involved in this application should be understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the" and the like involved in this application do not indicate a quantitative limitation, and may represent the singular or plural. The terms "include", "comprise", "have" and any of their variations involved in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.
[0044] Existing methods for calculating image resolution include those based on deep learning. Currently, the best results are achieved by deep learning methods, such as those based on generative adversarial networks (GANs). However, the amount of computation required to run deep learning on mobile phones is too large, and it cannot reach 16.66ms per frame. For example, NVIDIA has developed Deep Learning Super Sampling, an image enhancement technology based on deep learning, but this method can only be used on desktop N cards.
[0045] Existing image resolution calculation methods also include temporal anti-aliasing (TAA) and upscaling (TAAU). TAA can smooth edges by mixing the images of the current frame and the previous frame. In dynamic scenes, TAA adapts to scene changes by calculating the motion vector and reprojection of objects. The format of the motion vector is R16G16, and the historical frames also occupy bandwidth. In addition, the depth buffer will cause the bandwidth occupied during operation to be larger. Therefore, the operation on the mobile phone will be greatly affected by the bandwidth, and multiple sampling must be enabled, which requires a large amount of calculation. TAAU is a method that combines TAA and Upscaling. Upscaling uses the Catmull-Rom filter. TAAU is commonly used on computers and is greatly affected by bandwidth on mobile phones.
[0046] Existing image resolution calculation methods also include super-resolution sharpening technology (AMD FidelityFX SuperResolution, FSR), which is based on mixing the current frame and the historical frame. It has the same disadvantages as TAAU, and requires anti-aliasing, which is computationally intensive. This method can run normally on high-end mobile phones, but the frame rate on low-end mobile phones (such as Xiaomi 6) is only 30FPS.
[0047] Based on the above situation, the embodiments of the present application provide a super-resolution calculation method, system and electronic device.
[0048] In a first aspect, an embodiment of the present application provides a super-resolution calculation method. Figure 1 is a flowchart of a super-resolution calculation method according to an exemplary embodiment. Figure 1 As shown, the super-resolution calculation method includes:
[0049] Step S101, obtaining a mapping relationship between feature information of image pixels and filters.
[0050] Optionally, step S101 is a pre-calculation step, in which a corresponding filter is calculated according to feature information of image pixels, and a mapping relationship between the feature information and the filter is stored. The storage method includes but is not limited to an array and a list. Preferably, the mapping relationship is stored in a two-dimensional array, and the array elements are matrix linked lists.
[0051] In this way, multiple filters corresponding to the feature information of the pixel are calculated in advance, and the mapping relationship between the feature information and the filter is saved. In the subsequent real-time calculation, only the corresponding filter needs to be selected, which reduces the amount of data processing in the real-time calculation. In addition, multiple filters corresponding to the features are provided to improve the accuracy of the super-resolution calculation method and reduce the loss in the image enlargement process.
[0052] In one example, step S101 includes:
[0053] Step S201 , obtaining a corresponding low-definition image data set according to a high-definition image data set, and magnifying the images in the low-definition image data set to the resolution of a high-definition image by an interpolation filter to obtain an image data set to be calculated.
[0054] Optionally, the present application is specifically applied to the field of games, where multiple pictures are captured from the game as a high-definition image data set, and the images in the high-definition image data set are reduced to obtain a low-resolution image data set. The images in the low-definition image data set are enlarged back to the resolution of the images in the high-definition image data set by linear interpolation, nearest neighbor interpolation, bilinear interpolation, or Lanczos interpolation to obtain the image data set to be calculated. Preferably, the Lanczos interpolation method is used in the present application for preliminary enlargement of the image.
[0055] Step S202, calculating feature information of each low-definition pixel in the image data set to be calculated.
[0056] Optionally, the feature information of each low-definition pixel in the image data set to be calculated is calculated by principal component analysis (PCA), singular value decomposition (SVD), etc. Preferably, the present application calculates the feature information by SVD.
[0057] In one example, step S202 includes:
[0058] Step S2021, obtaining a pixel block of a set size centered on a low-definition pixel, and decomposing the pixel block to obtain eigenvalues and eigenvectors.
[0059] Step S2022: taking the largest eigenvalue among the eigenvalues as the intensity in the feature information.
[0060] Step S2023, calculating the angle between the eigenvectors corresponding to the two largest eigenvalues in the eigenvalues, and using the angle as the angle in the feature information.
[0061] Optionally, the feature information includes intensity and angle. The specific calculation process includes: dividing the low-definition pixels of each image in the image data set to be calculated into 3×3 pixel blocks, decomposing the pixel blocks by SVD method, obtaining 3 eigenvalues and 3 eigenvectors, and taking the largest eigenvalue as the intensity; finding the two eigenvectors with the largest eigenvalues, calculating their angle, and taking the angle as the angle. The 3X3 pixel block is stored in a two-dimensional array according to its angle and intensity. Among them, the size of the pixel block can be set according to actual needs, and is not limited to the 3X3 set in this application.
[0062] Step S203: Calculate a filter corresponding to the feature information according to the low-definition pixel and the high-definition pixel corresponding to the low-definition pixel.
[0063] Optionally, the low-definition pixels and the high-definition pixels corresponding to the low-definition pixels stored in each element of the two-dimensional array are traversed, and the filter corresponding to each element in the array is calculated by a deep learning method or a least squares method. The feature information of the pixel and the corresponding filter are stored in an array or a list.
[0064] In one example, step S203 includes: calculating a filter corresponding to the feature information by a least squares method or a deep learning method.
[0065] Optionally, when the least square method is used, the specific formula is as follows:
[0066]
[0067] Among them, h is the filter, L is the number of pixels stored in each element of the array, i represents the i-th pixel in the element, LR is the low-definition pixel, and HR is the high-definition pixel corresponding to the low-definition pixel. In this way, the "shortest distance" from the low-definition pixel stored in the same element to the corresponding high-definition pixel is calculated, so as to adapt the corresponding pixels, which is convenient for improving the accuracy of image magnification in subsequent real-time calculations.
[0068] In one example, step S101 includes:
[0069] Step S204, grouping the feature information of each low-definition pixel.
[0070] Step S205 , calculating a filter corresponding to each group according to the low-definition pixels in each group and the high-definition pixels corresponding to the low-definition pixels.
[0071] Optionally, after calculating the feature information of low-definition pixels, when storing pixel blocks in an array, the size of the array can be adjusted to group the feature information. For example, if only 100 angles and 2 intensities are saved, all angles are first divided into 100 intervals, and the angles in this interval are placed in the same element of the array. That is, similar features (angle and intensity) are placed together.
[0072] In this example, the feature information is grouped to facilitate the subsequent calculation of the filter corresponding to the feature and improve the accuracy of image magnification. At the same time, adjusting the array size and reducing the number of intervals for feature division can improve calculation efficiency and be compatible with low-end machines.
[0073] Step S102, calculating target feature information of pixels in the target image, and determining a target filter according to the target feature information based on a mapping relationship.
[0074] Optionally, the target feature information of each pixel is calculated by a pixel shader, and the target filter is determined from the mapping relationship according to the image feature information. When the image feature information is stored in groups, the group corresponding to the target feature information is first determined, and then the corresponding filter is determined according to the group.
[0075] In one example, the target image in step S102 is configured to be acquired in the following manner: acquiring an original image, rendering the original image to a target resolution, obtaining an intermediate image, wherein the resolution of the target image is lower than the resolution of the original image; and enlarging the intermediate image to the resolution of the original image through an interpolation filter to obtain the target image.
[0076] Optionally, the resolution of the target image is lower than the original image. When used in the game in the present application, the image is rendered to 1 / 4 of the native resolution, that is, the length and width are divided by 2. The rendered image is enlarged back to the native resolution by linear interpolation, nearest neighbor interpolation, bilinear interpolation and Lanczos interpolation. Preferably, Lanczos interpolation is used in the present application for preliminary enlargement of the image. In this way, the resolution of the image is reduced, and during the game, the target image can be calculated instead of directly calculating the original image, reducing the amount of calculation and improving the calculation efficiency. At the same time, the requirements for the processor during the game are reduced, so that low-end and mid-end mobile phones can run normally for a long time, avoiding the problem of frame drop caused by heating caused by long-term high-load operation of the processor.
[0077] In one example, acquiring a target image includes acquiring an initial image, and enlarging the initial image to a target resolution through an interpolation filter to obtain a target image, wherein the resolution of the target image is higher than the resolution of the initial image. Optionally, a low-resolution image is directly acquired, and the rendered image is enlarged to the target resolution through linear interpolation, nearest neighbor interpolation, bilinear interpolation, Lanczos interpolation, and the like. Preferably, Lanczos interpolation is used in the present application to perform preliminary enlargement of the image. This facilitates subsequent processing of the image through a corresponding filter to improve the accuracy of image enlargement.
[0078] Step S103: determining a super-resolution image according to pixels of the target image and a target filter corresponding to each pixel.
[0079] Optionally, the target pixel is multiplied by the target filter to obtain a super-resolution image. In this way, the image is enlarged by the filter corresponding to the target image pixel, which improves the progress of the super-resolution calculation method and reduces the loss in the image enlargement process.
[0080] In summary, the present application pre-calculates the mapping relationship between the feature information of image pixels and multiple filters. During real-time calculation, the corresponding filter is selected according to the pixel features of the real-time image to perform image amplification. The accuracy of super-resolution calculation is improved by using a filter that matches the pixel features. At the same time, the present application divides the calculation into two parts: pre-calculation and real-time calculation. The mapping relationship between the feature information and the filter is determined in advance. During real-time calculation, only the corresponding filter needs to be selected, which reduces the amount of data processing during real-time calculation, is conducive to the long-term operation of mid- and low-end mobile phones, and improves the user experience. Through the super-resolution calculation method of the present application, during the game, the low-resolution target image can be calculated instead of directly calculating the original image, reducing the amount of calculation and improving the calculation efficiency. At the same time, the requirements for the processor during the game are reduced, so that the mid- and low-end mobile phones can run normally for a long time, avoiding the problem of heating caused by the long-term high-load operation of the processor, thereby causing frame drop.
[0081] In a second aspect, an embodiment of the present application provides a super-resolution computing system. Figure 2 is a structural block diagram of a super-resolution computing system according to an exemplary embodiment. Figure 2 As shown, the system includes:
[0082] Mapping module 100: used to obtain the mapping relationship between feature information of image pixels and filters.
[0083] Selection module 200: is used to calculate target feature information of pixels in a target image, and determine a target filter according to the target feature information based on a mapping relationship.
[0084] Determination module 300: is used to determine a super-resolution image according to pixels of a target image and a target filter corresponding to each pixel.
[0085] In one example, the mapping module 100 includes:
[0086] Feature unit: used to obtain a corresponding low-definition image data set according to a high-definition image data set, and to enlarge the images in the low-definition image data set to the resolution of a high-definition image through an interpolation filter to obtain an image data set to be calculated.
[0087] Calculate the feature information of each low-definition pixel in the image dataset to be calculated.
[0088] Filter unit: used to calculate the filter corresponding to the feature information based on the low-definition pixels and the high-definition pixels corresponding to the low-definition pixels.
[0089] In one example, the mapping module 100 further includes:
[0090] Grouping unit: used to group the feature information of each low-definition pixel.
[0091] Group filtering unit: used to calculate the filter corresponding to each group according to the low-definition pixels in each group and the high-definition pixels corresponding to the low-definition pixels.
[0092] In one example, the feature unit of the mapping module 100 includes:
[0093] Decomposition subunit: used to obtain a pixel block of a set size centered on a low-definition pixel, and decompose the pixel block to obtain eigenvalues and eigenvectors.
[0094] Strength subunit: used to take the largest eigenvalue among the eigenvalues as the strength in the feature information.
[0095] Angle subunit: used to calculate the angle between the eigenvectors corresponding to the two largest eigenvalues in the eigenvalues, and use the angle as the angle in the feature information.
[0096] In one example, the filter unit of the mapping module 100 includes: calculating the filter corresponding to the feature information by a least square method or a deep learning method.
[0097] In one example, the target image in the selection module 200 is configured to be acquired in the following manner: acquiring an original image, rendering the original image to a target resolution, and obtaining an intermediate image, wherein the resolution of the target image is lower than the resolution of the original image; and enlarging the intermediate image to the resolution of the original image through an interpolation filter to obtain the target image.
[0098] In one example, the target image in the selection module 200 is configured to be acquired in the following manner: acquiring an initial image, and enlarging the initial image to a target resolution by interpolation to obtain the target image, wherein the resolution of the target image is higher than the resolution of the initial image.
[0099] In summary, the present application obtains the mapping relationship between the feature information of the pre-calculated image pixels and multiple filters through the mapping module 100. During real-time calculation, the corresponding filter is selected according to the pixel features of the real-time image to perform image amplification. The filter matching the pixel features improves the accuracy of super-resolution calculation. At the same time, the present application divides the calculation into two parts: pre-calculation and real-time calculation, and predetermines the mapping relationship between the feature information and the filter. During real-time calculation, only the corresponding filter needs to be selected, which reduces the amount of data processing during real-time calculation, is conducive to the long-term operation of mid- and low-end mobile phones, and improves the user experience. Through the super-resolution calculation method of the present application, during the game process, the low-resolution target image can be calculated instead of directly calculating the original image, reducing the amount of calculation and improving the calculation efficiency. At the same time, the requirements for the processor during the game are reduced, so that the mid- and low-end mobile phones can run normally for a long time, avoiding the problem of heating caused by the long-term high-load operation of the processor, thereby causing frame drop.
[0100] In a third aspect, an embodiment of the present application provides an electronic device, Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the super-resolution calculation method provided in the first aspect when executing the program. Figure 3 The electronic device 60 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.
[0101] The electronic device 60 may be in the form of a general-purpose computing device, for example, it may be a server device. The components of the electronic device 60 may include but are not limited to: at least one processor 61, at least one memory 62, and a bus 63 connecting different system components (including the memory 62 and the processor 61).
[0102] The bus 63 includes a data bus, an address bus, and a control bus.
[0103] The memory 62 may include a volatile memory, such as a random access memory (RAM) 621 and / or a cache memory 622 , and may further include a read-only memory (ROM) 623 .
[0104] The memory 62 may also include a program / utility 625 having a set (at least one) of program modules 624, such program modules 624 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.
[0105] The processor 61 executes various functional applications and data processing by running the computer programs stored in the memory 62, such as the super-resolution calculation method provided in the first aspect of the present application.
[0106] The electronic device 60 may also communicate with one or more external devices 64 (e.g., keyboards, pointing devices, etc.). Such communication may be performed via an input / output (I / O) interface 65. Furthermore, the model-generated device 60 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 66. As shown, the network adapter 66 communicates with other modules of the model-generated device 60 via a bus 63. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the model-generated device 60, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (RAID) systems, tape drives, and data backup storage systems.
[0107] It should be noted that although several units / modules or sub-units / modules of the electronic device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to an embodiment of the present invention, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided into multiple units / modules to be embodied.
[0108] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a program stored thereon, and when the program is executed by a processor, the super-resolution calculation method provided in the first aspect is implemented.
[0109] The readable storage medium may include but is not limited to: a portable disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical storage device, a magnetic storage device or any suitable combination of the above.
[0110] In a possible implementation, the present invention can also be implemented in the form of a program product, which includes program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps of the super-resolution calculation method provided in the first aspect.
[0111] The program code for executing the present invention may be written in any combination of one or more programming languages, and may be executed entirely on a user device, partially on a user device, as an independent software package, partially on a user device and partially on a remote device, or entirely on a remote device.
[0112] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0113] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.
Claims
1. A super-resolution calculation method, It is characterized in that include: Obtain the mapping relationship between the feature information of image pixels and the filter; Calculating target feature information of pixels in the target image, and determining a target filter according to the target feature information according to the mapping relationship; A super-resolution image is determined according to the pixels of the target image and the target filter corresponding to each pixel.
2. The super-resolution calculation method according to claim 1, It is characterized in that The obtaining of the mapping relationship between the feature information of the image pixels and the filter includes: Acquire a corresponding low-definition image data set according to the high-definition image data set, and enlarge the images in the low-definition image data set to the resolution of the high-definition image through an interpolation filter to obtain the image data set to be calculated; Calculating feature information of each low-definition pixel in the image data set to be calculated; A filter corresponding to the feature information is calculated according to the low-definition pixel and the high-definition pixel corresponding to the low-definition pixel.
3. The super-resolution calculation method according to claim 2, It is characterized in that The step of obtaining the mapping relationship between the feature information of the image pixels and the filter further includes: Grouping the feature information of each low-definition pixel; The filter corresponding to each group is calculated according to the low-definition pixels in each group and the high-definition pixels corresponding to the low-definition pixels.
4. The super-resolution calculation method according to claim 2, It is characterized in that The calculating feature information of each low-definition pixel in the image data set to be calculated includes: Obtain a pixel block of a set size centered on a low-definition pixel, and decompose the pixel block to obtain an eigenvalue and an eigenvector; Taking the largest eigenvalue among the eigenvalues as the intensity in the feature information; The angle between the eigenvectors corresponding to the two largest eigenvalues among the eigenvalues is calculated, and the angle is used as the angle in the feature information.
5. The super-resolution calculation method according to claim 2, It is characterized in that The calculating the filter corresponding to the feature information includes: The filter corresponding to the feature information is calculated by a least squares method or a deep learning method.
6. The super-resolution calculation method according to claim 1, It is characterized in that The target image is configured to be acquired by: Acquire an original image, render the original image to a target resolution, and obtain an intermediate image, wherein the resolution of the target image is lower than the resolution of the original image; The target image is obtained by enlarging the intermediate image to the resolution of the original image through an interpolation filter; or An initial image is acquired, and the initial image is enlarged to a target resolution by an interpolation method to obtain a target image, wherein the resolution of the target image is higher than the resolution of the initial image.
7. A super-resolution computing system, It is characterized in that include: Mapping module: used to obtain the mapping relationship between the feature information of image pixels and the filter; A selection module: used for calculating target feature information of pixels in a target image, and determining a target filter according to the target feature information according to the mapping relationship; Determination module: used to determine the super-resolution image according to the pixels of the target image and the target filter corresponding to each pixel.
8. An electronic device, It is characterized in that include Memory, processor, and A computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the super-resolution calculation method according to any one of claims 1 to 6 is implemented.
9. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the program is executed by a processor, the super-resolution calculation method according to any one of claims 1 to 6 is implemented.
