Super-resolution method and device, equipment and storage medium

Abstract

The invention discloses a super-resolution method. The method comprises the following steps: acquiring current system information; wherein the current system information at least comprises one or more of the following items: system temperature information, system electric quantity information, system load information and foreground program information; determining one or more super-resolution programs corresponding to the current system information; obtaining an original image frame, and performing super-resolution processing on the original image frame based on one or more super-resolution programs to obtain a processed image frame corresponding to the original image frame; wherein the resolution of the original image frame is smaller than that of the processed image frame.

Description

[0001] This application claims priority to U.S. Application No. 63 / 650,764, filed May 22, 2024, entitled “ADAPTIVE AND EFFICIENT GRAPHICSPIPELINE FOR REAL-TIME SUPER RESOLUTION ON ANDROID SYSTEM”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This invention relates to the field of image processing technology, and in particular to a super-resolution method, apparatus, device, and storage medium. Background Technology

[0003] Super-resolution (SR) of images or videos refers to the process of recovering high-resolution (HR) images or videos based on low-resolution (LR) inputs. It is an important super-resolution technique in computer vision and is widely used in games, movies, imaging and many other fields.

[0004] However, when performing image super-resolution processing, the relevant technologies cannot simultaneously achieve real-time performance, low power consumption, and high performance, and have poor compatibility, which in turn reduces the processing performance and effect of super-resolution. Summary of the Invention

[0005] This application provides a super-resolution method, apparatus, device, and storage medium that can balance real-time performance, low power consumption, and high performance, and has good compatibility, thereby improving the processing performance and effect of super-resolution.

[0006] The technical solution of this application embodiment is implemented as follows:

[0007] In a first aspect, embodiments of this application provide a super-resolution method, the method comprising:

[0008] Obtain current system information; the current system information includes at least one or more of the following: system temperature information, system power information, system load information, and foreground program information;

[0009] Identify one or more super-resolution programs corresponding to the current system information;

[0010] The original image frame is acquired, and super-resolution processing is performed on the original image frame based on one or more super-resolution programs to obtain the processed image frame corresponding to the original image frame; wherein the resolution of the original image frame is smaller than the resolution of the processed image frame.

[0011] Secondly, embodiments of this application provide a super-resolution device, which includes:

[0012] The acquisition unit is used to acquire current system information; wherein, the current system information includes at least one or more of the following: system temperature information, system power information, system load information, and foreground program information;

[0013] The determination unit is used to determine one or more super-resolution programs corresponding to the current system information;

[0014] The acquisition unit is also used to acquire the original image frame and perform super-resolution processing on the original image frame based on one or more super-resolution programs to obtain the processed image frame corresponding to the original image frame; wherein the resolution of the original image frame is smaller than the resolution of the processed image frame.

[0015] Thirdly, embodiments of this application provide an electronic device, which includes a processor and a memory storing processor-executable instructions, wherein when the instructions are executed by the processor, the method as described in the first aspect is implemented.

[0016] Fourthly, embodiments of this application provide a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the method described in the first aspect above.

[0017] This application provides a super-resolution method, apparatus, device, and storage medium for acquiring current system information. The current system information includes at least one or more of the following: system temperature information, system power information, system load information, and foreground program information. One or more super-resolution programs corresponding to the current system information are determined. An original image frame is acquired, and super-resolution processing is performed on the original image frame based on the one or more super-resolution programs to obtain a processed image frame corresponding to the original image frame. The resolution of the original image frame is less than the resolution of the processed image frame. Therefore, the super-resolution method proposed in this application includes an adaptive graphics pipeline. Based on the acquired current system information, one or more super-resolution programs are adaptively selected for super-resolution processing to intelligently adjust the graphics pipeline and use a more suitable super-resolution algorithm to achieve super-resolution. This method balances real-time performance, low power consumption, and high performance, and has good compatibility, thereby improving the processing performance and effect of super-resolution. Attached Figure Description

[0018] Figure 1 This is a schematic diagram illustrating the implementation process of the super-resolution method proposed in this application.

[0019] Figure 2 This is a schematic diagram illustrating the implementation process of the super-resolution method proposed in this application.

[0020] Figure 3 This is a schematic diagram illustrating the implementation of super-resolution proposed in an embodiment of this application;

[0021] Figure 4 This is a schematic diagram illustrating the implementation of super-resolution proposed in an embodiment of this application;

[0022] Figure 5 This is a schematic diagram of the composition structure of the super-resolution device proposed in the embodiments of this application;

[0023] Figure 6 This is a schematic diagram of the composition structure of the electronic device proposed in the embodiments of this application. Detailed Implementation

[0024] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only for explaining the differences between the applications and are not intended to limit the application. Furthermore, it should be noted that, for ease of description, only the parts that differ from the relevant applications are shown in the accompanying drawings.

[0025] Image or video super-resolution (SR) refers to the process of recovering high-resolution (HR) images or videos based on low-resolution (LR) inputs. It is an important super-resolution technique in computer vision and is widely used in games, movies, imaging, and many other fields.

[0026] SR pipeline solutions can generally be divided into learning-based solutions and non-learning-based solutions.

[0027] Learning-based SR methods involve using various network architectures, such as Convolutional Neural Networks (CNNs) and Generative Adversarial Networks (GANs), to train models on massive datasets of high-resolution and corresponding low-resolution image pairs. These networks process images through multiple layers, progressively extracting features and learning how to map low-resolution features to corresponding high-resolution paired features using the learned network weights.

[0028] On the other hand, non-learning-based methods can be further divided into initial sketches, gradient contours, and field of expert (FoE) methods. These methods reconstruct HR images from LR images using rule-based algorithms. In more advanced SR processing pipelines, multiple LR images are used as inputs to obtain temporal information and combined to generate a final HR image.

[0029] The biggest drawback of learning-based graphics pipelines is the difficulty in achieving real-time performance on Android platforms like smartphone chips, as well as the challenges in resolving compatibility issues and achieving significant leaps in power consumption. Furthermore, using deep learning (DL)-based pipelines makes the quality of SR highly dependent on the quality of the training data and the scenario. Achieving a balanced performance across various scenarios in different games can be difficult, and it increases the complexity of integration and maintenance; for example, adding a completely new game requires retraining the model using a new dataset.

[0030] Non-network-based graphics pipelines are typically designed to support only one corresponding algorithm and are not optimized for all input sources. For example, an SR algorithm might work for one game, but produce artifacts for another.

[0031] In other words, the relevant technologies cannot simultaneously achieve real-time performance, low power consumption, and high performance when performing image super-resolution processing, and they also have poor compatibility, which reduces the processing performance and effect of super-resolution.

[0032] To address the aforementioned deficiencies, this application provides a super-resolution method, apparatus, device, and storage medium, which acquires current system information. The current system information includes at least one or more of the following: system temperature information, system power information, system load information, and foreground program information. One or more super-resolution programs corresponding to the current system information are determined. An original image frame is acquired, and super-resolution processing is performed on the original image frame based on the one or more super-resolution programs to obtain a processed image frame corresponding to the original image frame. The resolution of the original image frame is less than the resolution of the processed image frame. Therefore, the super-resolution method proposed in this application includes an adaptive graphics pipeline. Based on the acquired current system information, one or more super-resolution programs are adaptively selected for super-resolution processing to intelligently adjust the graphics pipeline and use a more suitable super-resolution algorithm to achieve super-resolution. This method balances real-time performance, low power consumption, and high performance, and has good compatibility, thereby improving the processing performance and effect of super-resolution.

[0033] One embodiment of this application provides a super-resolution method that can be applied to a super-resolution device or electronic device, and can also be applied to any terminal that includes a super-resolution device or electronic device.

[0034] It is understood that the super-resolution method proposed in the embodiments of this application may include an image super-resolution scheme and / or a video super-resolution scheme.

[0035] The super-resolution method proposed in this application can be applied to scenarios such as games, movies, and imaging, and this application does not impose any specific limitations.

[0036] The super-resolution method proposed in this application will be illustrated below using an electronic device as an example.

[0037] In the embodiments of this application, Figure 1 This is a schematic diagram illustrating the implementation process of the super-resolution method proposed in this application. Figure 1 As shown, the super-resolution method may include the following steps:

[0038] Step 101: Obtain current system information; where the current system information includes at least one or more of the following: system temperature information, system power information, system load information, and foreground program information.

[0039] In the embodiments of this application, current system information can be obtained first. This current system information can be used to determine one or more of the following: the current system's operating status, load status, and program operating status.

[0040] In the embodiments of this application, the current system information includes at least one or more of the following: system temperature information, system power information, system load information, and foreground program information.

[0041] System temperature information can be used to determine the real-time temperature of electronic devices. Higher real-time temperatures result in greater limitations on the performance and computing power of the electronic devices; conversely, lower real-time temperatures result in less limitation on the performance and computing power of the electronic devices.

[0042] System power information can be used to determine the real-time available power of electronic devices. The higher the real-time available power, the greater the limitation on the performance and computing power of the electronic devices; conversely, the lower the real-time available power, the less the limitation on the performance and computing power of the electronic devices.

[0043] System load information can include the real-time load of the central processing unit (CPU) and / or the graphics processing unit (GPU) of an electronic device. A higher real-time load indicates greater performance and computing power requirements of the electronic device; conversely, a lower real-time load indicates less performance and computing power requirements.

[0044] Foreground program information can be used to determine the program type and content of programs running in the foreground. The performance and computing power requirements of electronic devices differ depending on the program type and / or content. For example, game programs have significantly higher performance and computing power requirements than document programs.

[0045] In the embodiments of this application, the acquired current system information may also include one or more of device information, scene information, and shooting information. This application does not specifically limit the specific content included in the current system information. For example, the current system information may include real-time information of the system hardware or real-time information of the system software.

[0046] Step 102: Determine one or more super-resolution programs corresponding to the current system information.

[0047] In the embodiments of this application, after obtaining the current system information, one or more super-resolution programs corresponding to the current system information can be further determined.

[0048] In the embodiments of this application, one or more super-resolution programs are selected and determined based on the obtained current system information. In this way, using one or more super-resolution programs that match the current system information for subsequent super-resolution processing can yield better super-resolution processing results.

[0049] In the embodiments of this application, a super-resolution program determined based on the current system information can be any type of super-resolution program, or in other words, it can be a super-resolution program used to perform any type of image processing.

[0050] In the embodiments of this application, the multiple super-resolution programs determined based on the current system information may include any type of super-resolution program, or in other words, may include super-resolution programs for performing any type of image processing.

[0051] In embodiments of this application, one or more super-resolution procedures include at least one or more of the following: one or more sharpening procedures; one or more magnification levels; and one or more post-processing procedures. The sharpening procedure can be used to perform image sharpening processing, thereby improving image clarity; the magnification procedure can be used to adjust the image size, i.e., adjust the image pixels, thereby improving image resolution; and the post-processing procedure can be used to perform high dynamic range (HDR) enhancement, cinematic tone mapping, image flipping, and other processing on the image, thereby improving the image quality.

[0052] Of course, the program type of one or more super-resolution programs determined based on the current system information is not limited to sharpening, magnification and post-processing, but may also include other types of super-resolution programs. This application does not make specific limitations.

[0053] For example, in some embodiments, one or more super-resolution programs corresponding to the current system information can be selected from a pre-built algorithm library based on the current system information. The pre-built algorithm library includes different types of candidate programs, such as those shown in Table 1, including multiple candidate sharpening programs, multiple candidate magnification programs, multiple candidate post-processing programs, and multiple candidate enhancement programs.

[0054] Table 1

[0055] Sharpening Program 1 Amplification Program 1 Post-processing 1 Enhancement Program 1 …… Sharpening Program 2 Amplification Program 2 Post-processing 2 Enhancement Program 2 …… Sharpening Program 3 Amplification Program 3 Post-processing 3 Enhancement Program 3 …… …… …… …… …… ……

[0056] In the embodiments of this application, when determining one or more super-resolution programs corresponding to the current system information, the current system performance parameters and / or current system load parameters can be determined based on the current system information, and one or more super-resolution programs can be determined based on the current system performance parameters and / or current system load parameters.

[0057] In the embodiments of this application, the current system performance parameters can be used to determine the real-time hardware capabilities and / or real-time software capabilities of an electronic device. The current system performance parameters can directly affect the super-resolution processing capability; for example, the current system performance parameters are proportional to the super-resolution processing capability.

[0058] In the embodiments of this application, the current system load parameter can be used to determine the real-time load of the electronic device. The current system load parameter can directly affect the super-resolution processing capability. For example, the current system load parameter is inversely proportional to the super-resolution processing capability.

[0059] In some embodiments, the current system performance parameters and / or current system load parameters can be determined first based on the current system information, and one or more super-resolution programs corresponding to the current system information can be determined based on the current system performance parameters and / or current system load parameters.

[0060] In the embodiments of this application, when determining one or more super-resolution programs based on current system performance parameters and / or current system load parameters, the magnitude parameters of the super-resolution are determined based on the current system performance parameters and / or current system load parameters; one or more super-resolution programs are determined based on the magnitude parameters.

[0061] In the embodiments of this application, the magnitude parameter of super-resolution can be used to determine parameters such as capability, computing power and power consumption required for super-resolution processing, and then the type and number of programs used to perform super-resolution processing can be selected based on the magnitude parameter of super-resolution.

[0062] For example, in some embodiments, if the current system load parameter determined based on the system load information is high, then a smaller super-resolution order of magnitude parameter can be determined based on the current system load parameter, that is, a lightweight SR algorithm can be selected to ensure smoothness and frame rate (Frames Per Second, FPS); if the current system load parameter determined based on the system load information is low, then a larger super-resolution order of magnitude parameter can be determined based on the current system load parameter, that is, an algorithm that enables the foreground APP to obtain the best SR quality can be selected.

[0063] Of course, in the embodiments of this application, one or more super-resolution procedures can also be determined directly based on the current system information, skipping the determination and use of the magnitude parameters of super-resolution.

[0064] In the embodiments of this application, when determining one or more super-resolution programs corresponding to the current system information, one or more sharpening programs can be determined from one or more candidate sharpening programs based on the current system information; and / or, one or more magnification programs can be determined from one or more candidate magnification programs based on the current system information; and / or, one or more post-processing programs can be determined from one or more candidate post-processing programs based on the current system information.

[0065] For example, in some embodiments, based on the determined current system information, one or more super-resolution programs corresponding to the current system information can be directly selected from a pre-built algorithm library. The pre-built algorithm library includes different types of programs, such as those shown in Table 2, including multiple candidate sharpening programs, multiple candidate magnification programs, and multiple candidate post-processing programs.

[0066] Table 2

[0067] Sharpening Program 1 Amplification Program 1 Post-processing 1 Sharpening Program 2 Amplification Program 2 Post-processing 2 Sharpening Program 3 Amplification Program 3 Post-processing 3 …… …… ……

[0068] In the embodiments of this application, when determining one or more super-resolution programs corresponding to the current system information, the current system performance parameters and / or current system load parameters can be determined based on the current system information, and one or more super-resolution programs can be determined based on the current system performance parameters and / or current system load parameters.

[0069] For example, in some embodiments, it is assumed that the current system information includes system load information. If the current system load is large as determined by the system load information, then a lightweight super-resolution algorithm needs to be selected to balance the system load. Therefore, a super-resolution program with low complexity, low computational cost, and small number can be selected.

[0070] For example, in some embodiments, it is assumed that the current system information includes system power information. If the current system power determined based on the system power information is large, it can support better super-resolution processing. Therefore, a super-resolution program with better processing effect can be selected.

[0071] For example, in some embodiments, it is assumed that the current system information includes system temperature information. If the current system temperature is determined to be high based on the system temperature information, then a lightweight super-resolution algorithm needs to be selected to avoid the continuous rise in system temperature. Therefore, a super-resolution program with low complexity, low computational load, and few programs can be selected.

[0072] For example, in some embodiments, it is assumed that the current system information includes foreground program information. Based on the foreground program information, it is determined that the program running in the foreground has high requirements for the detail information of the image or video and requires better super-resolution processing. Therefore, a super-resolution program with better processing effect can be selected.

[0073] In embodiments of this application, when adaptively selecting one or more super-resolution programs, in addition to information such as current system information as a reference, the adaptive selection of super-resolution programs can also be performed through context-aware and / or configurable rendering options.

[0074] In the embodiments of this application, context awareness relies on analyzing factors such as device functionality, scene complexity, and user preferences to select the most suitable SR algorithm in real time. This ensures a balance between achieving high visual quality and maintaining smooth performance across different mobile hardware.

[0075] In embodiments of this application, configurable rendering options refer to the control over various rendering parameters. This allows for fine-tuning of aspects such as Levels of Detail (LOD), texture filtering, and shader complexity based on the needs of different Android apps. This flexibility enables optimization of different types of graphics rendering processes (e.g., prioritizing the performance of fast-paced action games or the visual effects of graphics-intensive subtitles).

[0076] Step 103: Obtain the original image frame and perform super-resolution processing on the original image frame based on one or more super-resolution programs to obtain the processed image frame corresponding to the original image frame; wherein, the resolution of the original image frame is less than the resolution of the processed image frame.

[0077] In the embodiments of this application, after determining one or more super-resolution programs corresponding to the current system information, the acquired original image frame can be further super-resolution processed based on one or more super-resolution programs to obtain the processed image frame corresponding to the original image frame; wherein, the resolution of the original image frame is smaller than the resolution of the processed image frame.

[0078] In the embodiments of this application, the original image frame is the image frame before super-resolution processing, i.e., the LR frame, while the processed image frame is the image frame after super-resolution processing, i.e., the HR frame. Therefore, the resolution of the original image frame is smaller than the resolution of the processed image frame.

[0079] For example, in some embodiments, through super-resolution processing, the original image frame can be expanded to the screen resolution, for example, the resolution of the processed image frame is typically expanded to the range of 1.3 to 2 times the resolution of the original image frame.

[0080] In the embodiments of this application, the original image frame can be obtained first. The original image frame can be the image frame and / or video frame to be super-resolution performed on the current image, and the original image frame can be understood as an LR frame.

[0081] For example, in some embodiments, LR frames can be obtained from SurfaceFlinger as raw image frames. For instance, LR frames rendered by a game application can be obtained from SurfaceFlinger.

[0082] SurfaceFlinger is a key component of the Android operating system, primarily responsible for screen rendering and image compositing. It receives the Surfaces of all windows as input, calculates the position of each Surface in the final composite image based on parameters such as transparency, size, and position, and then generates the final display buffer using the CPU or GPU, displaying it on the specific display device. The display buffer is a dedicated memory area for temporarily storing graphics data; its core function is to optimize the display output process and reduce screen rendering latency. Between the display device (such as the screen) and the processor, the display buffer acts as a data relay station, storing pixel information or character encoding to be rendered, ensuring smooth screen updates.

[0083] In the embodiments of this application, SurfaceFlinger operates through the following steps:

[0084] 1. Receive input: SurfaceFlinger receives the Surface of all windows as input.

[0085] 2. Image synthesis: Calculate the position of each Surface in the final synthesized image based on parameters such as ZOrder, transparency, size, and position.

[0086] 3. Generate the display buffer: Use OpenGL or Hardware Composer to composite a set of Surfaces and generate the final display buffer.

[0087] 4. Display Output: Display the generated display buffer on a specific display device.

[0088] OpenGL directly calls GPU hardware acceleration through native C / C++ interfaces, making it suitable for 3D applications and game development requiring high frame rate rendering. C++'s pointer manipulation and memory management capabilities allow for precise control of graphics data flow, enabling efficient rendering pipeline optimization. C++'s class mechanism can encapsulate OpenGL functional modules (such as Shader, Mesh, and Texture), improving code reusability and maintainability. For example, the Shader class can manage shader compilation, or the Camera class can implement viewpoint control. OpenGL supports systems such as Windows, Linux, and macOS, and combined with C++'s cross-platform compilation capabilities, it allows for the construction of a unified graphics application framework.

[0089] In some embodiments, SurfaceFlinger is a standalone Service, with its code located in the frameworks / native / services / surfaceflinger directory of the Android source code. Its main implementation file is main_surfaceflinger.cpp, which defines the entry point and initialization process of SurfaceFlinger. The startup process of SurfaceFlinger includes setting the thread pool, priority, and scheduling policy, etc.34

[0090] In some embodiments, since SurfaceFlinger is responsible for layer compositing and display, its performance directly affects the system's display effect. Optimizing SurfaceFlinger's performance can be approached from the following aspects:

[0091] 1. Reduce the number of layers: Reducing unnecessary layers can lighten the load on SurfaceFlinger.

[0092] 2. Optimize layer properties: Properly set the layer's transparency, size, and position to reduce unnecessary calculations and rendering.

[0093] 3. Use hardware acceleration: Utilizing the GPU for layer compositing and rendering can significantly improve efficiency.

[0094] 4. Adjust priority and scheduling strategy: Optimize process priority and scheduling strategy to ensure SurfaceFlinger can run efficiently.

[0095] In the embodiments of this application, when performing super-resolution processing on the original image frame based on one or more super-resolution programs to obtain the processed image frame corresponding to the original image frame, the initial frame can be magnified based on one or more magnification programs to obtain the magnified image frame; the magnified image frame can be sharpened based on one or more sharpening programs to obtain the sharpened image frame; and the sharpened image frame can be post-processed based on one or more post-processing programs to obtain the processed image frame.

[0096] In the embodiments of this application, image post-processing includes at least one or more of the following: high dynamic range enhancement processing; cinematic tone mapping processing.

[0097] In the embodiments of this application, High Dynamic Range Enhancement (HDR) aims to expand the brightness range of an image, making it closer to the real world as perceived by the human eye, thereby preserving more detail in scenes with strong contrast between light and dark. Traditional Standard Dynamic Range (SDR) images often suffer from overexposure in highlights or loss of detail in shadows due to their limited brightness range, while HDR solves this problem through multi-frame synthesis and algorithm optimization.

[0098] In the embodiments of this application, film tone mapping adapts the wide brightness information of high dynamic range (HDR) to standard display devices through dynamic range compression and color optimization, while simulating film texture and cinematic visual style. Its core objective is to balance the details of highlights (such as areas directly exposed to sunlight) and shadows (such as shadows), avoiding overexposure or underexposure; at the same time, it enhances the sense of depth and artistic expression of the image by simulating the imaging characteristics of film through non-linear tone curves.

[0099] In the embodiments of this application, if one or more super-resolution procedures determined based on the current system information include a sharpening procedure, an enlargement procedure, and a post-processing procedure, then one can choose to first enlarge the original image using one or more enlargement procedures, then sharpen the enlarged image using one or more sharpening procedures, and finally post-process the sharpened image using one or more post-processing procedures to obtain the processed image frame of the original image after super-resolution.

[0100] In the embodiments of this application, the execution order of different types of procedures in the image super-resolution processing process is not specifically limited. For example, image magnification, image sharpening, and image post-processing can be performed sequentially, or image sharpening, image magnification, and image post-processing can be performed sequentially.

[0101] In the embodiments of this application, if image magnification is performed first, followed by image sharpening, the details are clearer after magnification, the sharpening effect is more accurate, and the already sharpened edges are not blurred during the magnification process. Furthermore, modern super-resolution algorithms (such as AI models) already include detail enhancement, and sharpening, as a post-processing step, can be further optimized. However, if the magnification algorithm is of poor quality, sharpening may amplify noise or artifacts, and the computational load is large (requiring processing of high-resolution images).

[0102] In the embodiments of this application, if image sharpening is performed first and then image magnification is performed, the sharpening calculation speed for small images is fast, which is suitable for traditional interpolation magnification (such as bicubic interpolation). However, high-frequency noise after sharpening may be amplified after magnification, and the sharpening edges may be blurred during the magnification process, resulting in a loss of effect. Sharpening marks (such as the Halo effect) are more obvious after magnification.

[0103] As can be seen, in the embodiments of this application, for one or more determined super-resolution programs, the execution order of different types of programs can be selected based on the different application scenarios.

[0104] For example, in some embodiments, for applications involving high-quality super-resolution (such as ESRGAN, Real-ESRGAN), printing output, and large-screen display, image magnification can be performed first, followed by image sharpening, to achieve a more accurate result.

[0105] For example, in some embodiments, for applications that require fast processing of low-resolution preview images and traditional interpolation methods (such as Bicubic), image sharpening can be performed first, followed by image magnification, to achieve faster processing speed.

[0106] For example, in some embodiments, if one or more super-resolution procedures determined based on current system information include a sharpening procedure, an upscaling procedure, and a post-processing procedure, then one can choose to first perform image sharpening processing on the original image through one or more sharpening procedures, then perform image upscaling processing on the sharpened image through one or more upscaling procedures, and finally perform image post-processing on the upscaled image through one or more post-processing procedures to finally obtain the processed image frame of the original image after super-resolution.

[0107] For example, in some embodiments, if one or more super-resolution procedures determined based on current system information include a sharpening procedure and a post-processing procedure, then one can choose to first perform image sharpening processing on the original image through one or more sharpening procedures, and then perform image post-processing on the sharpened image through one or more post-processing procedures, and finally obtain the processed image frame of the original image after super-resolution.

[0108] For example, in some embodiments, if one or more super-resolution procedures determined based on current system information include a sharpening procedure and a post-processing procedure, then one can choose to first perform image sharpening processing on the original image through one or more sharpening procedures, and then perform image post-processing on the sharpened image through one or more post-processing procedures, and finally obtain the processed image frame of the original image after super-resolution.

[0109] For example, in some embodiments, if one or more super-resolution procedures determined based on current system information include an upscaling procedure and a post-processing procedure, then one can choose to first perform image upscaling processing on the original image through one or more upscaling procedures, and then perform image post-processing on the upscaled image through one or more post-processing procedures, and finally obtain the processed image frame of the original image after super-resolution.

[0110] In some embodiments, for low-quality images (containing noise / compression artifacts), it is possible to perform image upscaling first, followed by image sharpening (using AI super-resolution to suppress noise).

[0111] In some embodiments, for high-quality images (which require slight magnification), image sharpening can be performed first, followed by image upscaling (traditional interpolation), but the sharpening intensity must be limited (avoiding Radius > 1px).

[0112] In some embodiments, for extreme cases (such as animation / vector graphics), you can choose to first sharpen the image edges and then perform image magnification, for example, by using an edge-preserving magnification algorithm (such as Waifu2x), but you need to customize the sharpening mask.

[0113] In the embodiments of this application, when performing super-resolution processing on the original image frame based on one or more super-resolution programs to obtain the processed image frame corresponding to the original image frame, the one or more super-resolution programs are assembled to determine the graphics shader corresponding to the current system information; the original image frame is then subjected to super-resolution processing based on the graphics shader to obtain the processed image frame.

[0114] In other words, in the embodiments of this application, for one or more super-resolution programs corresponding to the current system information, the one or more super-resolution programs can be further assembled to determine the graphics shader corresponding to the current system information, and the graphics shader can be used to perform super-resolution processing on the original image frame to obtain the corresponding processed image frame.

[0115] A graphics shader is a dedicated program that runs on the GPU to control specific processing stages in the graphics rendering pipeline. It implements core rendering functions such as geometric transformations, lighting calculations, and texture mapping in a programmable manner. Essentially, it decouples the traditional fixed rendering pipeline into modular program units, giving developers fine-grained control over the graphics processing.

[0116] Graphics shaders can include, but are not limited to, various types such as vertex shaders, fragment shaders, geometry shaders, tessellation shaders, and compute shaders.

[0117] The vertex shader processes model vertex data (such as position, normals, and texture coordinates) and performs coordinate space transformations (model → world → view → projection space). Vertex shaders can be applied to skeletal animation deformation and particle system motion trajectory calculation.

[0118] The fragment shader calculates the final color value for each pixel, including operations such as texture sampling, lighting models (e.g., Phong, PBR), and alpha blending. The fragment shader can achieve complex material representations such as water refraction and flame particle effects.

[0119] The geometry shader generates new geometry based on primitives (triangles, line segments) and supports dynamic level of detail (LOD) adjustment and instantiated rendering. The geometry shader can be applied to batch generation of grassland vegetation and real-time subdivision of hair.

[0120] The tessellation shader dynamically subdivides low-poly models into high-precision meshes, improving surface smoothness. The tessellation shader can be applied to terrain rendering and enhancing character model details.

[0121] Computation shaders perform general-purpose parallel computations (non-graphics tasks) on the GPU, such as physics simulations and image processing. Computation shaders can also be applied to fluid dynamics simulations and accelerate deep learning inference.

[0122] For example, in some embodiments, one or more super-resolution programs corresponding to the current system information are selected from a pre-built algorithm library, and then the one or more super-resolution programs are assembled to obtain a graphics shader. During the program assembly process, adaptive adjustments and changes can be made according to the needs of single-channel or multi-channel programs.

[0123] In the embodiments of this application, the original image frame is transmitted to the graphics shader through a unified buffer object; super-resolution processing is performed by the graphics shader to obtain the processed image frame.

[0124] In the embodiments of this application, a Uniform Buffer Object (UBO) is a mechanism for efficiently managing uniform variables in a shader. It allows multiple uniform variables to be packaged into a structured buffer and shared among multiple shader programs, thereby reducing application programming interface (API) call overhead and improving performance. Specifically, UBOs can perform batch data transfers, storing multiple uniform variables in a single buffer and passing them to the shader all at once. UBOs can be shared across shaders, meaning the same UBO can be reused by multiple shader programs, avoiding the duplication of setting the same data. UBOs can optimize performance by reducing the overhead of frequent function calls, making them suitable for dynamic data (such as transformation matrices and lighting parameters).

[0125] In the embodiments of this application, Figure 2 This is a schematic diagram illustrating the implementation process of the super-resolution method proposed in this application. Figure 2 As shown, the super-resolution method may include the following steps:

[0126] Step 104: Generate display data based on the processed image frames, and display the image based on the display data.

[0127] In the embodiments of this application, after performing super-resolution processing on the original image frame based on one or more super-resolution programs to obtain the processed image frame corresponding to the original image frame, display data can be further generated based on the processed image frame, so that the display can be performed based on the display data.

[0128] In the embodiments of this application, after the original image frame is super-resolution processed based on one or more super-resolution programs determined in real time, the corresponding original image frame can be replaced by the obtained processed image frame corresponding to the original image frame, and the display data to be displayed can be output. Since the resolution of the original image frame is smaller than the resolution of the processed image frame, the display data including the processed image frame can be displayed in the end, which can improve the display effect.

[0129] In embodiments of this application, the display data may include, but is not limited to, image display data and / or video display data.

[0130] For example, in some embodiments, LR frames are obtained from SurfaceFlinger as raw image frames, and HR frames after super-resolution processing are sent back to SurfaceFlinger as processed image frames, and synthesized into a final buffer, which is then displayed on the display.

[0131] In embodiments of this application, the electronic device may be equipped with a display, and after generating display data based on the processed image frame, the display data can be directly displayed on the display.

[0132] In embodiments of this application, after generating display data, the electronic device may also send the display data to other devices configured with displays so that the other devices can display the display data.

[0133] In summary, the super-resolution method proposed in this application includes an adaptive graphics processing flow that enhances the effects of images and / or videos through super-resolution (SR).

[0134] For example, in some embodiments, the original image frame to be processed can be obtained from SurfaceFlinger, and the processed image frame can be sent back to SurfaceFlinger for display, thereby enabling the graphics processing flow proposed in this application to be seamlessly integrated with the rendering workflow without affecting the core rendering pipeline.

[0135] For example, in some embodiments, for game applications, the original image frame can be a raw low-resolution (LR) frame rendered by the game engine, and the image frame after super-resolution processing can be a high-resolution (HR) SR frame that matches the display resolution. This can greatly improve the image clarity without introducing artifacts, thus allowing for the display of finer details in textures, character models, and environmental features. This results in a more realistic and visually immersive experience.

[0136] For example, in some embodiments, LR frames often exhibit aliasing, which manifests as jagged edges on objects. Higher resolution significantly reduces aliasing, resulting in smoother and more visually pleasing images.

[0137] For example, in some embodiments, for gaming applications, directly rendering higher resolution frames by the game engine typically uses more of the GPU's processing power and increases power consumption, potentially leading to lower frame rates and negatively impacting gameplay smoothness. However, embodiments of this application, through adaptive management of shader computations in the graphics pipeline, can maintain the same frame rate while performing super-resolution without significant power consumption.

[0138] Of course, the super-resolution method proposed in this application can not only perform super-resolution of images and / or videos for game applications, but also be applied to other super-resolution scenarios. By implementing the current system information obtained, the super-resolution program is adaptively determined, thereby realizing super-resolution processing of images and / or videos adapted to the current environment. The specific application scenarios of this application are not limited.

[0139] Therefore, embodiments of this application propose an adaptive graphics pipeline for multi-platform super-resolution (SR). The graphics rendering pipeline receives a buffer of relatively low-resolution (LR) frames (raw image frames) provided by a previous rendering process, and then adaptively processes the frame buffer by using the graphics shader-managed device state to upscale the frames and construct higher-resolution, more detailed output frames (processed image frames). In embodiments of this application, the invention of the graphics pipeline can be seamlessly integrated without interfering with the original rendering process.

[0140] It is understood that the super-resolution method proposed in this application can be implemented on the Android platform and can be applied to scenarios that improve the real-time game experience. However, the super-resolution method proposed in this application also supports cross-platform integration and can be applied to any scenario. This application does not impose any specific limitations.

[0141] It is understood that, in the embodiments of this application, benefiting from the parallel computing advantages of graphics shaders, the adaptive SR graphics pipeline method proposed in the embodiments of this application can render any required graphics scene at a significantly improved resolution without sacrificing frame rate or generating huge power consumption, which ensures a smooth and consistent user experience.

[0142] This application proposes a super-resolution method to acquire current system information, which includes at least one or more of the following: system temperature information, system power information, system load information, and foreground program information; determine one or more super-resolution programs corresponding to the current system information; acquire an original image frame, and perform super-resolution processing on the original image frame based on one or more super-resolution programs to obtain a processed image frame corresponding to the original image frame; wherein the resolution of the original image frame is less than the resolution of the processed image frame. Therefore, the super-resolution method proposed in this application includes an adaptive graphics pipeline, wherein one or more super-resolution programs are adaptively selected based on the acquired current system information to intelligently adjust the graphics pipeline and use a more suitable super-resolution algorithm to achieve super-resolution. This method can balance real-time performance, low power consumption, and high performance, and has good compatibility, thereby improving the processing performance and effect of super-resolution.

[0143] Based on the above embodiments, this application proposes a super-resolution method, including a general framework for super-resolution (SR). This framework provides compatibility with a range of SR algorithms, including single-channel techniques for real-time performance and multi-channel methods for superior detail extraction. Furthermore, the framework integrates seamlessly with post-processing techniques such as high dynamic range enhancement and cinematic tone mapping.

[0144] In the embodiments of this application, the general framework of SR proposed in the embodiments of this application can be applied to any operating system device, such as, but not limited to, Android devices.

[0145] This application proposes a super-resolution method that enables an adaptive graphics pipeline by adaptively selecting a super-resolution (SR) algorithm for images and / or videos. Specifically, in this application's embodiments, current system information, including but not limited to content type, temperature, battery level, and GPU / CPU load, is acquired to intelligently determine and modify the graphics pipeline, thereby supporting the optimal SR algorithm for the SR task.

[0146] This application proposes a super-resolution method, an adaptive graphics pipeline for multi-platform super-resolution (SR). The graphics rendering pipeline receives a relatively low-resolution (LR) frame buffer provided by a previous rendering process, and then adaptively processes the frame buffer by using the graphics shader to manage device state to upscale the frames and construct a more detailed output frame at a higher resolution. Our invention involving a graphics pipeline can be seamlessly integrated without interfering with the original rendering process. While our invention is intended for implementation on the Android platform and is specifically crafted to enhance the real-time gaming experience, our invention supports cross-platform integration and is scene-independent.

[0147] In the embodiments of this application, benefiting from the parallel computing advantages of graphics shaders, our invention involving an adaptive SR graphics pipeline can render any required graphics scene at a significantly improved resolution without sacrificing frame rate or incurring huge power consumption. This ensures a smooth and consistent user experience.

[0148] In embodiments of this application, even when the input source is blurry and lacks contrast, users will have a better visual experience of content on the Android platform through improved clarity, realism, and immersion, much like generating content at a higher resolution.

[0149] In the embodiments of this application, Figure 3 This is a schematic diagram illustrating the implementation of super-resolution proposed in an embodiment of this application, as shown below. Figure 3 As shown, this application proposes a super-resolution method that enables real-time super-resolution, possesses the advantages of graphics shaders, and relies solely on the current frame information. The graphics pipeline acquires a single frame from the SurfaceFlinger, such as frame T (e.g., frame T is an LR frame rendered by a game application). Then, based on the system state (current system information), the intelligent SR algorithm selection unit in the graphics pipeline intelligently selects an SR algorithm from a prepared algorithm library. The algorithm can use single-channel or multi-channel shaders and different input formats. In the embodiments of this application, the graphics pipeline can be dynamically assembled to adapt to the algorithm's needs. After graphics shader processing, the input frame (original image frame) can be expanded to the screen resolution, typically by 1.3 to 2 times the original frame resolution, ultimately outputting a processed image frame, such as SR frame T, which is then sent back to the SurfaceFlinger.

[0150] It is understood that by utilizing the adaptability and compatibility of the super-resolution method proposed in the embodiments of this application, the graphics pipeline can generate detailed and continuous SR frames without artifacts or occlusions, while achieving low computational cost.

[0151] In the embodiments of this application, taking the Android system as an example, on the one hand, OpenGL and C++ are chosen because they have good cross-platform capabilities and computational efficiency. On the other hand, since image data is being processed, GPGPU programming is used in the graphics pipeline to use multiple graphics chips for parallel graphics processing.

[0152] In the embodiments of this application, Figure 4 This is a schematic diagram illustrating the implementation of super-resolution proposed in an embodiment of this application, as shown below. Figure 4As shown, during initialization, the adaptive graphics pipeline queries system status information (system information, such as Android system properties), including system temperature, CPU / GPU load, foreground application (APP), and application type. Based on this system information, the adaptive graphics pipeline finds a suitable SR algorithm to balance power consumption and system performance.

[0153] For example, when the load is high, the system can select a lightweight SR algorithm to ensure smoothness and FPS; when the load is low, the system can select an algorithm that gives the foreground APP the best SR quality.

[0154] In the embodiments of this application, after selecting the SR algorithm, the adaptive graphics pipeline assembles the graphics shader according to the candidate algorithms (one or more super-resolution programs) in the algorithm library, i.e., it performs program assembly, making appropriate changes according to the needs of single-channel or multi-channel programs. Next, the system obtains the current frame as the input frame (raw image frame) from the background buffer in the rendering pipeline (Android graphics pipeline), utilizes the parallel processing of the GPU, and performs upscaling, sharpening, and post-processing (i.e., shader calculation) on the input image through the selected algorithm to obtain the final output frame (processed image frame). Then, the output frame is sent back to SurfaceFlinger to be composited into the final buffer and sent to the display.

[0155] In embodiments of this application, a Unified Buffer Object (UBO) can be utilized to ensure efficient data management and maintain smooth performance. UBOs facilitate the efficient transfer of relevant data to the graphics shader, eliminating the overhead associated with setting up multiple independent unifiers. This optimization translates into a faster processing pipeline, ultimately contributing to better SR performance.

[0156] In embodiments of this application, when making adaptive selection of one or more super-resolution programs, in addition to adaptive graphics pipelines of system properties such as temperature and CPU / GPU load, adaptive selection of algorithms can also be made through context-aware and / or configurable rendering options.

[0157] In the embodiments of this application, context awareness relies on analyzing factors such as device functionality, scene complexity, and user preferences to select the most suitable SR algorithm in real time. This ensures a balance between achieving high visual quality and maintaining smooth performance across different mobile hardware.

[0158] In embodiments of this application, configurable rendering options refer to the control over various rendering parameters. This allows for fine-tuning of aspects such as Level of Detail (LOD), texture filtering, and shader complexity based on the needs of different Android apps. This flexibility enables optimization of different types of graphics rendering processes (e.g., prioritizing the performance of fast-paced action games or the visual effects of graphics-intensive subtitles).

[0159] In the embodiments of this application, the potential for achieving a superior SR experience across different mobile game genres is unlocked through a parameter optimization process. This involves selecting parameters such as sharpening and post-processing techniques (HDR, tone mapping) to meet the visual requirements of different game genres. These parameters can also be continuously refined by obtaining user feedback and image quality assessments. This optimization allows the system to adapt to user preferences and the unique visual attributes of each game genre.

[0160] It is understood that the super-resolution method proposed in this application can ensure that all supported Android games have a consistent high-quality, customized SR experience.

[0161] In summary, the super-resolution method proposed in this application includes an adaptive graphics pipeline that employs super-resolution (SR) to enhance the visual fidelity of Android games. This adaptive graphics pipeline can be seamlessly integrated with the rendering workflow, using high-resolution (HR) SR frames that match the display resolution to replace the original low-resolution (LR) frames rendered by the game engine. This significantly improves image clarity without introducing artifacts or affecting the core rendering pipeline, allowing for the display of finer details in textures, character models, and environmental features, resulting in a more realistic and visually immersive experience.

[0162] In particular, LR frames often exhibit aliasing, which manifests as jagged edges on objects. Higher resolutions significantly reduce aliasing, resulting in smoother graphics and a more visually pleasing experience.

[0163] Rendering higher-resolution frames for games typically utilizes more GPU processing power and increases power consumption. This could potentially lead to a decrease in frame rate, negatively impacting gameplay smoothness. Conversely, adaptive management of shader computation through the graphics pipeline can maintain the same frame rate while reducing screen refresh rate (SR) without significant power consumption.

[0164] The super-resolution method proposed in this application adaptively represents the first SR graphics pipeline. In contrast, other SR graphics pipelines may not be able to adjust based on system state and only support a specific shader algorithm. The adaptive graphics pipeline proposed in this application, however, offers greater flexibility and can be optimized for different types, while the aforementioned other SR graphics pipelines lack flexibility and are not optimized for different types. If the SR pipeline does not function as expected, the aforementioned other SR graphics pipelines may introduce circuit breakers or timeout mechanisms to limit performance or even stop working, resulting in a negative user experience.

[0165] This application proposes a super-resolution method, including an adaptive graphics pipeline, wherein one or more super-resolution programs are adaptively selected for super-resolution processing based on the acquired current system information, so as to intelligently adjust the graphics pipeline and use a more suitable super-resolution algorithm to achieve super-resolution. This method can balance real-time performance, low power consumption, and high performance, and has good compatibility, thereby improving the processing performance and effect of super-resolution.

[0166] Based on the above embodiments, this application proposes a super-resolution method that can recover high-resolution (HR) images or videos from low-resolution (LR) inputs, and can be applied to fields such as games, movies, and imaging.

[0167] This application proposes a super-resolution method that can solve the following problems in common SR graphics pipelines: learning-based methods are difficult to achieve real-time performance and have compatibility and power consumption issues; non-learning-based methods usually only support one algorithm and are not optimized for all input sources.

[0168] The super-resolution method proposed in this application includes an adaptive graphics pipeline that can intelligently select and adjust the graphics pipeline based on the current hardware status (such as content type, temperature, battery level, GPU / CPU load, etc.) to support the SR algorithm best suited for the current task.

[0169] The super-resolution method proposed in this application can achieve seamless integration without interfering with the original rendering process; it supports cross-platform integration and scene independence; and it leverages the parallel computing advantages of graphics shaders to significantly improve resolution without sacrificing frame rate or significantly increasing power consumption.

[0170] For example, in some embodiments, a single frame (such as an LR frame rendered by a game application) can be obtained from SurfaceFlinger, the SR algorithm can be intelligently selected based on the system state, and the graphics pipeline can be dynamically assembled to adapt to the algorithm requirements. After processing, the input frame is expanded to the mobile phone screen resolution.

[0171] The super-resolution method proposed in this application uses OpenGL and C++ and has good cross-platform capability and computational efficiency.

[0172] The super-resolution method proposed in this application utilizes GPGPU programming to process graphics in parallel using multiple graphics chips.

[0173] The super-resolution method proposed in this application queries the system status at startup and selects a suitable SR algorithm based on the current environment to balance power consumption and system performance.

[0174] The super-resolution method proposed in this application is compatible with various SR algorithms, including single-channel techniques for real-time performance and multi-channel methods for detail extraction.

[0175] The super-resolution method proposed in this application can achieve adaptive SR algorithm selection: the most suitable SR algorithm is selected in real time based on device capabilities, scene complexity and user preferences.

[0176] The super-resolution method proposed in this application provides configurable rendering options, allowing adjustment of parameters such as level of detail (LOD), texture filtering, and shader complexity to optimize the graphics rendering process of different types of Android applications.

[0177] The super-resolution method proposed in this application can achieve game-based parameter adjustment: it provides a superior SR experience for different mobile game types through a parameter optimization process, including the selection and adjustment of parameters such as sharpness and post-processing techniques (HDR, tone mapping).

[0178] This application proposes a super-resolution method that significantly improves the visual fidelity of Android games by replacing the original LR frames rendered by the game engine with HR SR frames that match the display resolution. This improves image clarity, reduces jagged edges, and enhances graphics smoothness and visual appeal. By adaptively managing shader calculations, SR display is achieved while maintaining the same frame rate without significantly increasing power consumption.

[0179] This application proposes a super-resolution method, including an adaptive graphics pipeline, wherein one or more super-resolution programs are adaptively selected for super-resolution processing based on the acquired current system information, so as to intelligently adjust the graphics pipeline and use a more suitable super-resolution algorithm to achieve super-resolution. This method can balance real-time performance, low power consumption, and high performance, and has good compatibility, thereby improving the processing performance and effect of super-resolution.

[0180] Based on the above embodiments, in another embodiment of this application... Figure 5This is a schematic diagram of the composition structure of the super-resolution device proposed in the embodiments of this application, as shown below. Figure 5 As shown, the super-resolution device 110 proposed in this application embodiment may include:

[0181] The acquisition unit 1101 is used to acquire current system information; wherein, the current system information includes at least one or more of the following: system temperature information, system power information, system load information, and foreground program information;

[0182] The determining unit 1102 is used to determine one or more super-resolution programs corresponding to the current system information;

[0183] The acquisition unit 1101 is further configured to acquire the original image frame and perform super-resolution processing on the original image frame based on one or more super-resolution programs to obtain the processed image frame corresponding to the original image frame; wherein the resolution of the original image frame is less than the resolution of the processed image frame.

[0184] In the embodiments of this application, further, Figure 6 This is a schematic diagram of the composition structure of the electronic device proposed in the embodiments of this application, such as... Figure 6 As shown, the electronic device 120 proposed in this application embodiment may include a processor 1201, a memory 1202, a communication interface 1203, and a bus 1204 for connecting the processor 1201, the memory 1202 and the communication interface 1203.

[0185] In the embodiments of this application, the processor 1201 can be at least one of the following: Application-Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field-Programmable Gate Array (FPGA), Central Processing Unit (CPU), Controller, Microcontroller, and Microprocessor. It is understood that for different devices, the electronic device used to implement the above-mentioned processor function can also be other types, and this application embodiment does not specifically limit this. The electronic device 120 may also include a memory 1202, which can be connected to the processor 1201. The memory 1202 is used to store executable program code, which includes computer operation instructions. The memory 1202 may include high-speed RAM memory and may also include non-volatile memory, such as at least two disk drives.

[0186] In embodiments of this application, bus 1204 is used to connect communication interface 1203, processor 1201, and memory 1202, as well as the mutual communication between these devices.

[0187] In practical applications, the aforementioned memory 1202 can be volatile memory, such as random-access memory (RAM); or non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD); or a combination of the above types of memory, and provide instructions and data to the processor 1201.

[0188] In an embodiment of this application, the processor 1201 is configured to: acquire current system information; wherein the current system information includes at least one or more of the following: system temperature information, system power information, system load information, and foreground program information; determine one or more super-resolution programs corresponding to the current system information; acquire an original image frame, and perform super-resolution processing on the original image frame based on one or more super-resolution programs to obtain a processed image frame corresponding to the original image frame; wherein the resolution of the original image frame is less than the resolution of the processed image frame.

[0189] Furthermore, in this embodiment, the functional modules can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional module.

[0190] If the integrated unit is implemented as a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the method of this embodiment. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0191] This application provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the super-resolution method described above.

[0192] Specifically, the program instructions corresponding to a super-resolution method in this embodiment can be stored on storage media such as optical discs, hard disks, and USB flash drives. When the program instructions corresponding to a super-resolution method in the storage media are read or executed by an electronic device, the following steps are included:

[0193] Obtain current system information; the current system information includes at least one or more of the following: system temperature information, system power information, system load information, and foreground program information;

[0194] Identify one or more super-resolution programs corresponding to the current system information;

[0195] The original image frame is acquired, and super-resolution processing is performed on the original image frame based on one or more super-resolution programs to obtain the processed image frame corresponding to the original image frame; wherein the resolution of the original image frame is smaller than the resolution of the processed image frame.

[0196] This application also provides a computer program product.

[0197] In some embodiments, the computer program product may include a computer program or instructions.

[0198] In some embodiments, the computer program product can be applied to the computer device in the embodiments of this application, and the computer program instructions cause the computer to execute the corresponding processes implemented by the computer device in the various methods of the embodiments of this application. For the sake of brevity, they will not be described in detail here.

[0199] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0200] This application is described with reference to schematic and / or block diagrams of implementations of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the schematic and / or block diagrams can be implemented by computer program instructions, and combinations of blocks in the schematic and / or block diagrams can be implemented. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the schematic and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0201] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in the implementation flow diagram. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0202] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0203] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application.

Claims

1. A super-resolution method, characterized in that, The method includes: Obtain current system information; wherein, the current system information includes at least one or more of the following: system temperature information, system power information, system load information, and foreground program information; Determine one or more super-resolution programs corresponding to the current system information; An original image frame is acquired, and the original image frame is subjected to super-resolution processing based on one or more of the super-resolution programs to obtain a processed image frame corresponding to the original image frame; wherein the resolution of the original image frame is smaller than the resolution of the processed image frame.

2. The method according to claim 1, characterized in that, The process of determining one or more super-resolution procedures corresponding to the current system information includes: Determine the current system performance parameters and / or current system load parameters based on the current system information; One or more of the super-resolution programs are determined based on the current system performance parameters and / or the current system load parameters.

3. The method according to claim 2, characterized in that, The step of determining one or more super-resolution programs based on the current system performance parameters and / or the current system load parameters includes: The order of magnitude of the super-resolution is determined based on the current system performance parameters and / or the current system load parameters; One or more super-resolution procedures are determined based on the magnitude parameters.

4. The method according to any one of claims 1 to 3, characterized in that, One or more of the super-resolution programs include at least one or more of the following: One or more sharpening programs; One or more degrees of magnification; One or more post-processors; The process of determining one or more super-resolution procedures corresponding to the current system information includes: Based on the current system information, determine one or more sharpening programs from one or more candidate sharpening programs; and / or, Based on the current system information, determine one or more amplification procedures from one or more candidate amplification procedures; and / or, Based on the current system information, one or more post-processors are determined from one or more candidate post-processors.

5. The method according to claim 4, characterized in that, The step of performing super-resolution processing on the original image frame based on one or more of the super-resolution procedures to obtain the processed image frame corresponding to the original image frame includes: The initial frame is magnified using one or more of the magnification procedures to obtain a magnified image frame. The magnified image frame is sharpened using one or more of the aforementioned sharpening procedures to obtain a sharpened image frame. The sharpened image frame is post-processed based on one or more of the post-processing procedures to obtain the processed image frame; The image post-processing includes at least one or more of the following: High dynamic range enhancement processing; Film tone mapping processing.

6. The method according to any one of claims 1 to 3, characterized in that, The step of performing super-resolution processing on the original image frame based on one or more of the super-resolution procedures to obtain the processed image frame corresponding to the original image frame includes: The super-resolution program is assembled to determine the graphics shader corresponding to the current system information; The original image frame is subjected to super-resolution processing based on the graphics shader to obtain the processed image frame.

7. The method according to claim 6, characterized in that, The method further includes: The original image frame is transmitted to the graphics shader via a unified buffer object; The super-resolution processing is performed by the graphics shader to obtain the processed image frame.

8. The method according to any one of claims 1 to 3, 4, 5, and 7, characterized in that, The method further includes: Display data is generated based on the processed image frame, and then displayed based on the display data.

9. A super-resolution device, characterized in that, The super-resolution device includes: The acquisition unit is used to acquire current system information; wherein the current system information includes at least one or more of the following: system temperature information, system power information, system load information, and foreground program information; A determining unit is used to determine one or more super-resolution programs corresponding to the current system information; The acquisition unit is further configured to acquire an original image frame and perform super-resolution processing on the original image frame based on one or more of the super-resolution programs to obtain a processed image frame corresponding to the original image frame; wherein the resolution of the original image frame is smaller than the resolution of the processed image frame.

10. An electronic device, characterized in that, The electronic device includes a processor and a memory storing processor-executable instructions, which, when executed by the processor, implement the method as described in any one of claims 1 to 8.

11. A computer-readable storage medium having a program stored thereon, characterized in that, When the program is executed by a processor, it implements the method as described in any one of claims 1 to 8.

Images