Method of a computing device for processing an image and computing device

Abstract

Some aspects of this application provide a scene detection module in a computing device that determines a quality score based on a graphics command output of a first image. Depending on the quality score, the scene detection module can output a quality-perceived scaling factor, which can be used to reduce the pixel resolution of a second image before displaying it to the user. Therefore, the computing device can be improved by saving system resources (including memory bandwidth, application or instance processing power) without negatively impacting the user's visual perception of the image.

Description

TECHNICAL FIELD

[0001] The present application relates generally to systems and methods for displaying graphics in a computing device, and in particular, to systems and methods for displaying graphics in a portable or mobile device. BACKGROUND

[0002] Modern mobile devices, such as smartphones, tablets, or other handheld or wearable computing devices, provide a variety of functions for their human users, such as communication, productivity, and entertainment, which are enabled in part by software including the mobile device operating system and a multitude of available mobile applications (apps) that can be installed on top of the operating system. Some mobile devices are equipped with large, high-resolution, high-refresh-rate displays that can provide an enhanced visual experience for the user, such as when playing a mobile gaming application on the mobile device. SUMMARY

[0003] Some embodiments are directed to a method for processing an image by a computing device, the computing device comprising a graphics command module and an image detection module. The method comprises receiving, by the image detection module, a first graphics command output representing a first image from the graphics command module; determining, by the image detection module, a quality score of the first image based on the first graphics command output, and determining a scale factor based on the quality score.

[0004] Some embodiments are directed to a computing device comprising: a graphics command module configured to provide a graphics command output; an image detection module coupled to the graphics command module and configured to: receive a first graphics command output representing a first image from the graphics command module; determine a quality score of the first image based on the first graphics command output, and determine a scale factor based on the quality score.

[0005] Some embodiments are directed to an apparatus comprising at least one display interface, at least one computer-readable storage device storing executable instructions, and at least one processor executing the executable instructions to perform a method comprising: receiving a first graphics command output configured to cause a first image to be displayed by the at least one display interface; determining a quality score of the first image based on the first graphics command output, and determining a scale factor based on the quality score. BRIEF DESCRIPTION OF DRAWINGS

[0006] Various aspects and embodiments will be described with reference to the following figures. It should be appreciated that the figures are not necessarily drawn to scale. In the figures, each identical, or nearly identical, component that is illustrated in various figures is represented with a like numeral. For purposes of clarity, not every component is labelled in every figure.

[0007] Figure 1 is a high-level block diagram illustrating an example computing device embodying some embodiments;

[0008] Figure 2 is a schematic diagram illustrating an example computing device with an image detection module according to some embodiments of the application;

[0009] Figure 3 is a schematic diagram illustrating an example process for applying a scale factor output by an image detection module to a subsequent frame according to some embodiments;

[0010] Figure 4 is a schematic flow diagram illustrating an example process for determining a quality score and a scale factor using an image detection module according to some embodiments;

[0011] Figure 5A is a schematic flow diagram illustrating an example process for generating a quality score and an output scale factor using an image detection module according to some embodiments;

[0012] Figure 5B is a schematic flow diagram illustrating a second example of using an image detection module with a dynamic quality threshold according to some embodiments;

[0013] Figure 5C is a schematic flow diagram illustrating a third example of using an image detection module with a feedback loop for adjusting a scale factor according to some embodiments;

[0014] Figure 6 is a schematic flow diagram illustrating an example process for applying a scale factor to a graphics command output using a resolution controller according to some embodiments;

[0015] Figure 7 is a schematic flow diagram illustrating an example of outputting multiple scale factors according to some embodiments. DETAILED DESCRIPTION

[0016] Aspects of the present application relate to a graphics image detection technique that provides adaptive scale factors based on image quality, such that certain images can be scaled down before being displayed to save system resources without significantly impacting image quality for the user.

[0017] Certain mobile applications, such as mobile gaming applications, command large amounts of system resources, such as graphics processing power, memory storage, and power consumption, in part due to the amount of computation used to render images, each of which has a large number of pixels displayed to the user at a high refresh rate. Large data bandwidth and computing power are used when the application is displaying a high resolution game while attempting to maintain a high framerate per second (FPS), requiring large amounts of system resources, especially when the system resources become limited. When system resources become limited, the gaming application can no longer be able to maintain the FPS, resulting in dropped frames, glitches, or other artifacts that can negatively impact the user's experience playing the mobile device game.

[0018] The inventors have recognized and appreciated that certain images can be presented to a user at a lower resolution without being perceived as lower image quality. Some aspects of the present application provide a ghosting detection module in a computing device that determines a quality score using a graphics command output for a first image. Depending on the quality score, the ghosting detection module can output a quality-aware scale factor that can be used to reduce the pixel resolution of a second image before displaying the second image to the user. Thus, the computing device can be improved by saving system resources, including memory bandwidth, processing power of the application or instance, without negatively impacting the user's perception of ghosting visual artifacts.

[0019] In some embodiments, the ghosting detection module can use a perceptual image quality metric to determine a quality score for an image. A predetermined threshold can be provided such that when the quality score exceeds the threshold, the image is considered to have a high enough quality that displaying a down-scaled version of the image with a lower pixel resolution will not be perceived by the user as low quality, and thus a scale factor can be selected for the down-scaled image. In some embodiments where a scale factor has already been applied, when it is determined that the quality score exceeds the predetermined threshold, the ghosting detection module can select an even lower, more aggressive scale factor to further reduce system resource consumption. The predetermined threshold need not be a constant value for all images, and in some embodiments, the predetermined threshold can be dynamically adjusted during operation of the methods disclosed herein according to one or more criteria.

[0020] In some embodiments, the quality score can be determined by comparing the input image to a rescaled image. The rescaled image can be generated by first scaling down the input image by a scale factor to obtain an intermediate image. The intermediate image can have a lower pixel resolution than the input image. Subsequently, the intermediate image is scaled up to the same pixel resolution as the original input image as the rescaled image. Because information in certain pixels is lost during the scaling down step to obtain the intermediate image, the rescaled image will require fewer system resources to render compared to the original image. If the rescaled image is perceived to be close enough to the original input image such that the quality score exceeds a predetermined threshold, the aliasing detection module can output the scale factor to be applied to the next image. If the quality score does not exceed the threshold, the aliasing detection module can revert to a previous, less aggressive scale factor as the output.

[0021] In some embodiments, the quality score can be determined by computing a peak signal to noise ratio (PSNR) function representing a quality difference between the input image and a rescaled version of the input image, although any suitable image quality metric can be used to determine the quality score.

[0022] Some aspects of the application are directed to how a computing device can utilize an aliasing detection module to obtain a second graphics command output based on a scale factor output by a first graphics command output. In some embodiments, a resolution controller is provided that receives the output scale factor from the aliasing detection module. The resolution controller can be coupled to a graphics command module that generates the graphics command output. The resolution controller can provide additional commands to the graphics command module such that the second graphics command output is scaled down before being displayed to the user. In one example, the resolution controller can change the size of a frame buffer of a second image represented by the second graphics command output according to the scale factor.

[0023] The aspects and embodiments described above, as well as additional aspects and embodiments, will be further described below. These aspects and / or embodiments can be used individually or together, in any combination, and the application is not limited in this regard.

[0024] Figure 1 is a high-level block diagram illustrating an example computing device in which some embodiments can be implemented. In Figure 1In particular embodiments, computing device 10 can be a portable, hand-held, or wearable electronic device. In some embodiments, computing device 10 can be a smart phone, a personal digital assistant (PDA), a tablet computer, a smart watch. Computing device 10 can be powered by a battery, such as a rechargeable battery. Aspects of the application are not limited to portable devices or battery-powered devices, and computing device 10 can also be a general purpose computer.

[0025] Computing device 10 includes a central processing unit (CPU) 12 having one or more processors, a graphics processing unit (GPU) 14 having one or more graphics processors, and a non-transitory computer-readable storage medium 16, which can include, for example, volatile and / or non-volatile memory. Memory 16 can store one or more instructions that, when executed by CPU 12 and / or GPU 14, perform any of the functions described herein. Computing device 10 can have one or more input devices and / or output devices, such as user input interface 18 and output interface 17, as shown. Figure 1 Examples of output interfaces that can be used to provide a user interface include a printer or display screen to render visual output, a speaker or other sound producing device to render audible output, and a vibrator or mechanical motion generator to render tactile output. Examples of input interfaces that can be used in a user interface include a keyboard and pointing device, such as a mouse, touch pad, or digitizer for pen, stylus, or finger touch input. As another example, input interface 18 can include one or more microphones to capture audio signals, one or more cameras and light sensors to capture visual signals, and output interface 17 can include a display interface, such as a display screen, to visually render images or text to user 30 of computing device 10 and / or a speaker to aurally render images or text.

[0026] As Figure 1As shown, computing device 10 can include one or more network interfaces 19 to enable communication via various networks (e.g., network 20). Examples of networks include a local area network or a wide area network, such as an enterprise network or the Internet. Such networks can be based on any suitable technology and can operate according to any suitable protocol and can include wireless networks, wired networks, or fiber optic networks. Examples of network interfaces include Wi-Fi, WiMAX, 3G, 4G, 5G NR, white space, 802.11x, satellite, Bluetooth, near field communication (NFC), LTE, GSM / WCDMA / HSPA, CDMA lx / Evolution Data Only (EVDO), Dedicated Short-Range Communication (DSRC), GPS, etc. Although not shown, computing device 10 can additionally include one or more high-speed data buses that interconnect the various components and a power supply subsystem that provides power to the components, as is Figure 1 shown.

[0027] In some embodiments, computing device 10 can be a mobile device and output interface 17 can include a high-pixel resolution display. Pixel resolution can refer to the count of pixels along two axes of a rectangular display surface, such as a vertical and horizontal axis, or along width and height, or along rows and columns. The display can have a pixel resolution of 720p or 1280x720, 1080p or 1920x1080, 2K or at least 2048 pixels along one axis, Ultra High Definition (UHD) or 3840x2160, 4K or at least 4096 pixels along one axis, 8K or at least 7680 pixels along one axis, 10K or at least 10240 pixels along one axis, or any suitable number of pixels higher or lower than the listed pixel counts along either axis. The display can support a frame rate of at least 24 FPS, at least 30 FPS, at least 48 FPS, at least 60 FPS, at least 120 FPS, at least 240 FPS, between 30 FPS and 240 FPS, or any suitable frame rate, i.e., frequency of displaying successive images or frames.

[0028] Some embodiments, as described below, are directed to methods that can be performed by CPU 12 and / or GPU 14 based on instructions stored on non-transitory storage medium 16. Such instructions can be software, program code, and / or firmware. Certain embodiments can be embodied in one or more modules. A module can be a software module, which is embodied in instructions stored in a non-transitory machine-readable medium. A module can also, or instead, be a hardware-implemented module, which is a tangible unit capable of performing certain operations and can be configured or arranged in a certain manner. In some embodiments, one or more components of computing system 10, including one or more processors, can be configured by software (e.g., an application or application portion) as hardware-implemented modules that operate to perform certain operations as described herein.

[0029] Figure 2 A schematic diagram of an example computing device with an image detection module is shown in accordance with some embodiments. In Figure 2In this embodiment, computing device 100 includes a graphics command module 110 that generates output graphics command output 120 to represent frames or images of an image. Graphics command output 120 can be any suitable graphics command format for rendering images, such as, but not limited to, OpenGL, DirectX, or Vulkan. Graphics command output 120 can represent an input image, and image detection module 130 receives graphics command output 120, analyzes the input image based on graphics command output 120 to determine a quality score 132 of the input image. Image detection module 130 further determines and outputs a scaling factor 134 based on the quality score. Image detection module 130 can use any suitable image quality perception algorithm to determine the quality score 132 of the input image based on graphics command output 120. In some embodiments, if the quality score 132 indicates that the image can be further reduced without affecting user perception, the scaling factor 134 can be selected to be less than 1x for further reduction. Other examples of determining the quality score and scaling factor will be discussed below. In some embodiments, the graphics command module 110 reduces the buffer size of the current image using a previous scaling factor obtained from a previous image to obtain a graphics command output, or reduces the buffer size of the current image using a previous scaling factor obtained from a previous image and then enlarges it to obtain a graphics command output, and inputs the graphics command output to the image detection module. Depending on the quality score 132, the image detection module 130 can determine that the scaling factor 134 can be lower than the previous scaling factor, higher than the previous scaling factor, or remain the same as the previous scaling factor. Specifically, reducing the buffer size of the current image using a previous scaling factor obtained from a previous image and then enlarging it to obtain a graphics command output includes: the graphics command module 110 reduces the buffer size of the current image using a previous scaling factor, then executes a drawing command on the reduced buffer of the current image, and then enlarges the reduced buffer back to the original buffer size to obtain the graphics command output.

[0030] Figure 3 A schematic diagram illustrates an exemplary process, according to some embodiments, for applying a scaling factor output by an image detection module to subsequent frames. Figure 3In some embodiments, the graphics command module 110 generates a first graphics command output 121 representing a first image or current frame 181, while the graphics command module 110 further generates a second graphics command output 122 representing a second image (a subsequent image of the first image) or next frame 182. The image detection module 130 receives the first graphics command output 121 of the current frame 181, determines a quality score and outputs a scale factor 134. The graphics command module 110 then applies the scale factor 134 when generating the second graphics command output 122, such that the second image is rendered at a rescaled size for presentation to the display 170. The rescaled second image can be displayed to the user by the display 170 using any suitable protocol and hardware. System resources can be conserved during the rendering of the second image 182 due to the application of the scale factor.

[0031] With further reference to Figure 3 A resolution controller 140 can be provided to receive the output scale factor 134, which provides an additional command 142 to the graphics command module 110, causing the graphics command module 110 to generate the second graphics command output.

[0032] In this embodiment, the first graphics command output 121 representing the first image is provided to the display 170 for display by the display and to the image detection module 130. The image detection module 130 receives the first graphics command output 121, determines a quality score and outputs a scale factor 134. The graphics command module 110 then applies the scale factor 134 when generating the second graphics command output 122. In this way, there is no delay in the display of the first graphics command output 121, and the scale factor used by the graphics command module 110 to generate the next graphics command output (the second graphics command output 122) is also generated synchronously.

[0033] Figure 4 A schematic flow diagram illustrating an exemplary process for determining a quality score and a scale factor using an image detection module is shown in accordance with some embodiments.

[0034] Figure 4At the beginning of the process, the graphics command module 210 is shown generating a graphics command output 221 representing an input image 237. The image detection module 230 receives the graphics command output 221 and, at block 231, downscales the input image using a next scale factor 236 to obtain an intermediate image. Still at block 231, the intermediate image is then upscaled back to the same pixel resolution as the input image 237 as a rescaled image or resampled image 233. At block 239, a PSNR function is computed using the input image 237 and the resampled input image 235 as parameters, although it should be understood that any function that can represent a comparison of perceived image quality between two images can be used in place of the PSNR function. The result of the execution of block 239 is to obtain a quality score 232, which is received at block 238, from which an output scale factor 234 is determined.

[0035] Figure 4 The resolution controller 240 is also shown receiving the output scale factor 234 and providing additional commands 242 to the graphics command module 210 to perform scaling on one or more subsequent images of the input image 237. Thus, a loop is formed including the graphics command module 210, the image detection module 230, and the resolution controller 240, such that the image detection module 230 evaluates the quality of images output from the graphics command module 210, and the resulting scale factors are applied back to the graphics command module 210 via the resolution controller. Thus, the images can be downscaled without affecting the perceived image quality of the user, and system resources used to render the images can be conserved.

[0036] where the next scale factor can be the next scale factor of the scale factor that the image detection module output to the graphics command module the previous time.

[0037] Figure 5A A schematic flowchart showing a first example of using an image detection module to generate a quality score and an output scale factor based on some embodiments is shown.

[0038] In some embodiments, the image detection module 230 can be implemented as a software module that is executed by a processor of a computer system. In some embodiments, the image detection module 230 can be implemented as a hardware module that is integrated into a computer system. Figure 5AIn this non-limiting example, the next scale factor 236 with a value of 0.5x is used for illustrative purposes, and at act 331 A, the input image 337 is scaled down by 0.5x to result in an intermediate image 331 having 0.5x the number of pixels in each axis compared to the input image. The intermediate image 331 thus requires a smaller amount of memory and processing power to process. At act 331B, the intermediate image 331 is scaled up to be a rescaled image 335 using any suitable scaling up algorithm, including but not limited to interpolation such as nearest-neighbor interpolation, bilinear interpolation, trilinear interpolation (mipmapping), and Lanczos resampling, as well as scaling up based on Fast Fourier Transform (FFT) / Discrete Fourier Transform (DFT), machine learning, or using a rescaling hardware accelerator. Because scaling up and down are used to assess the loss of quality during the image downscaling operation, in some embodiments the scaling up algorithm is implemented in a manner similar to the actual scaling up algorithm of the GPU or rescaling hardware accelerator used to perform the image processing.

[0039] At act 339, the original input image 337 and the rescaled image 335 are compared using a PSNR function. The result or PSNR value of the PSNR function representing the difference in quality between the input image 337 and the rescaled image 335 is provided as a quality score 332. Generally, a high PSNR value is associated with good image quality. At block 338, the PSNR value is compared to a predetermined threshold to determine the output scale factor. In the illustrated example, if the PSNR value is greater than the threshold, the output scale factor is selected to be the next scale factor 0.5x applied during the act of scaling down 331A, because the quality score indicates that the scaled image is acceptable to the user, and thus the next scale factor is acceptable. On the other hand, if the PSNR value is equal to or less than the threshold, the comparison result indicates that the rescaled image 335 after scaling by the next scale factor at step 331A is not acceptable. Thus, a different scale factor will be selected as the output. As Figure 5A As shown in the example in FIG. 3, the selected different scale factor can be a previous scale factor, such as another scale factor used by the impact detection module previously, which does not scale down the image as aggressively as the “next scale factor” applied during the act of scaling down 331A.

[0040] In some embodiments, the image detection module retains a list of previously used scaling factors, such as 1x, 0.9x, 0.8x, 0.7x, and 0.6x. When it is determined that the next scaling factor, 0.5x, does not provide a satisfactory quality score, Figure 5A The process 500A shown will revert to the previous scaling factor of 0.6x as the output scaling factor. In the next iteration of process 500A, if it is determined that the scaling factor of 0.6x is too aggressive, process 500A will revert to the previous scaling factor of 0.7x from the previous scaling factor of 0.6x, and so on, up to a scaling factor of 1x (if necessary) to maintain a satisfactory quality score. Here, the previous scaling factor is the one that was used earlier.

[0041] Depending on the comparison between the quality score and a predetermined threshold, the image detection module can select the output scaling factor in any of a variety of ways. In some embodiments, it is not necessary to choose between two options for the output scaling factor, and in some embodiments, the output scaling factor can be a function of the quality score. For example, if the quality score indicates that the rescaled image has higher quality compared to the original input image, a more aggressive scaling factor can be selected for the output. The output scaling factor is also not necessarily a scaling factor that represents a reduction. For example, sometimes it can be determined that no scaling factor is applied at all, or that a scaling factor of 1x will be used to maintain the user's perceived image quality for certain images.

[0042] The threshold used for comparison with the quality score at block 338 can be determined in several ways. For example, the threshold can be predetermined by calibrating with an image of known quality specifications to a typical user. The predetermined threshold can be stored in memory 16 during the manufacture of the computing device. In other examples, the predetermined threshold does not need to be constant and can be updated after the computing device is shipped to the user, for example, to accommodate the user's personal preferences.

[0043] Figure 5B A schematic flowchart of a second example of using an image detection module with a dynamic quality threshold according to some embodiments is shown.

[0044] exist Figure 5B As shown, in many ways process 500B and Figure 5AThe process 500B is similar to process 500A, with similar components indicated by the same reference numerals. Process 500B differs from process 500A in that it provides a dynamic thresholding module 341, which receives an input scaling factor 345 and determines a threshold 343 for comparison with the output of the PSNR function in block 338. The input scaling factor 345 may be a "next" scaling factor determined based on the output of a previous stage of the image detection module. The dynamic thresholding module 341 can use any suitable command or algorithm to adjust the threshold 343 based on system information to provide different strategies. In some embodiments, the threshold can be adjusted based on the input scaling factor. As an example of an algorithm that can be used by the quality thresholding module 341, the threshold 343 may be equal to the ratio of a base threshold to the input scaling factor; for example, the threshold 343 is equal to the base threshold divided by the input scaling factor. When the input scaling factor is 0.5x, such as... Figure 5B As shown, the threshold 343 can be twice the base threshold. For example, if the base threshold is chosen to be 20 dB, then in Figure 5B In block 338, threshold 343 will be determined as 40 dB for use by block 338. In block 338, if the PSNR value is greater than threshold 343, the output scaling factor is selected as the next scaling factor of 0.5x. On the other hand, if the PSNR value is less than or equal to threshold 343, in this example, the output scaling factor is selected as the previous scaling factor of 0.6x, thus applying a less aggressive scaling.

[0045] In this embodiment, the threshold is dynamically set; the smaller the input scaling factor, the larger the threshold, thereby enabling less aggressive scaling of the application.

[0046] Figure 5C A schematic flowchart is shown for a third example of using an image detection module with a feedback loop for adjusting a scaling factor, according to some embodiments.

[0047] exist Figure 5CIn this process, process 500C begins with an input scaling factor (445) of 0.7x, while the previous scaling factor (447) was 0.8x at the start of the process. Initially, at block 439, the PSNR value is determined by scaling down / up the input image using the input scaling factor 445 as described above. At block 438, the PSNR value is compared with a predetermined threshold. If the comparison at block 438 is true, it means that the scaled image quality is acceptable (“accept”), and the input scaling factor 445 is passed to logic table 450 to determine the next scaling factor 436, as described below. If the comparison at block 438 is false, i.e., the PSNR value is less than or equal to the threshold 343, it means that the scaled image quality is unacceptable (“reject”). In each iteration of process 500C, the next scaling factor 436 is determined using logic table 450 as shown in Table 1 below. Depending on the comparison result at block 438, the input to logic table 450 can be either the input scaling factor 445 or the previous scaling factor 447. For each iteration of process 500C, the value of the next scaling factor 436 can be used to calculate the PSNR score at block 439.

[0048] Table 1 is a logic table used to determine the next scaling factor.

[0049]

[0050] As shown in Table 1, during each iteration, logic table 450 is called, and the next scaling factor is adjusted in increments of 0.1x based on the comparison result of the current iteration in block 438 and the comparison result of the previous iteration in block 438. For example, if both the current and previous iteration comparison results are "rejected," the scaling factor is increased by 0.1x to apply a less aggressive downsizing to the input image to maintain image quality. The feedback loop in process 500C can continue iterating until the final scaling factor returns to 1x.

[0051] Figure 6 A schematic flowchart illustrating an exemplary process of applying a scaling factor to the output of a graphics command using a resolution controller, according to some embodiments, is shown. Figure 6 In the middle, process 600 can be... Figure 4The resolution controller 240 is used for implementation, and at block 641, the resolution controller adds a command to the graphics command module 210 to change the frame buffer size to a reduced frame buffer size 651. For example, if the resolution controller 240 receives an output scaling factor of 0.5x, the frame buffer size can be reduced to 1 / 4 of the input image. At block 642, the resolution controller 240 sends an additional command to the graphics command module 210 to render image 652 with the reduced frame buffer size. In block 642, the graphics command module 210 uses drawing commands 1, 2, ... N-1, N to render the corresponding objects in image 652. After rendering the objects in image 652, the buffers are swapped. After rendering image 652, the graphics command module 210 can render an enlarged image 653 with the final frame buffer size for display to the user. Optionally, in other embodiments, in block 642, the graphics command module 210 uses drawing commands 1, 2, ... N-1, N to render the corresponding objects in the image 652, and outputs the rendered image 652 to the display, which then enlarges and displays the image.

[0052] Figure 7 A schematic flowchart illustrating an example of outputting multiple scaling factors according to some embodiments is shown. Figure 7 Process 700 is illustrated, in which the application has two frame buffers 751 and 752, and the input images for the two frame buffers 751 and 752 are provided with their respective scaling factors. Initially, at the start of process 700, input scaling factors C = 0.7x and D = 0.8x are provided. Resolution controller 740 sends graphics command 720 to control graphics command module to apply scaling factor C to draw frame buffer A (751) and apply scaling factor D to draw frame buffer B (752), using frame buffer A as a texture when drawing frame buffer B (752). Image detector 730 evaluates the images in the two frame buffers using any of the above methods, and in an exemplary scenario, image detector 730 may generate the following results: after scaling factor C is rejected, an adjusted scaling factor C' = 0.8x is provided; after scaling factor D is accepted, an adjusted scaling factor D' = 0.7x is provided; however, other scenarios are also possible. The image detector 730 provides scaling factors C' and D' to the resolution controller 740, which applies the scaling factors to frame buffer A and frame buffer B respectively for displaying the image.

[0053] Several aspects of at least one embodiment of the present invention have been described, and it should be understood that various changes, modifications and improvements will be readily apparent to those skilled in the art.

[0054] Such changes, modifications, and improvements are intended to be part of and fall within the scope of this invention. Furthermore, while advantages of the invention have been pointed out, it should be understood that not every embodiment of the technology described herein will include every described advantage. Some embodiments may not implement any features described herein as advantageous, and in some cases, one or more of the described features may be implemented to achieve further embodiments. Therefore, the foregoing description and figures are merely illustrative.

[0055] Various aspects of the present invention can be used individually, in combination, or in various arrangements not specifically discussed in the foregoing embodiments. Therefore, the application of the present invention is not limited to the details and arrangements of the components set forth in the foregoing description or shown in the drawings. For example, an aspect described in one embodiment can be combined in any way with aspects described in other embodiments.

[0056] Furthermore, the present invention can be embodied as a method, examples of which have been provided. The actions performed as part of this method can be ordered in any suitable manner. Therefore, embodiments can be constructed that perform actions in a different order than those illustrated, which may include performing some actions simultaneously, even those shown as being performed sequentially in the illustrative embodiments.

[0057] The use of sequential terms such as “first,” “second,” and “third” in claims to modify claim elements does not imply any priority, precedence, or temporal order of actions of one claim element relative to another claim element, but is merely a label to distinguish one claim element with a specific name from another element with the same name (but using ordinal terms) to differentiate claim elements.

[0058] Furthermore, the wording and terminology used herein are for descriptive purposes and should not be considered limiting. The use of “including,” “contains,” or “has,” “includes,” “involves,” and variations thereof in this document is intended to cover the items listed thereafter and their equivalents, as well as additional items.

Claims

1. A method for a computing device to process an image, the method comprising: The computing device includes a graphics command module and an image detection module, the method comprising: The graphics command module scales down a buffer size of a first image using a scale factor determined from a quality score of a previous image to obtain a first graphics command output, or scales down a buffer size of a first image and then scales up to obtain a first graphics command output; the graphics command module provides the first graphics command output to a display and to the image detection module; The image detection module receives the first graphics command output representing the first image from the graphics command module; and The image detection module determines a quality score of the first image from the first graphics command output and determines a scale factor for scaling down a buffer size of a second image in generating a second graphics command output for the second image from the quality score.

2. The method of claim 1, wherein, Further comprising: applying at least one scaling to a buffer size of a second image according to the scale factor to obtain the second graphics command output.

3. The method of claim 1, wherein, Further comprising: comparing the first image and the rescaled first image.

4. The method of claim 3, wherein, Comparing the first image and the rescaled first image includes: scaling down the first image by a first scale factor to obtain a first intermediate image; scaling up the first intermediate image to obtain a rescaled first image having a same pixel resolution as the first image; and computing the quality score by comparing the first image and the rescaled first image.

5. The method of claim 4, wherein, Where computing the quality score includes: computing a peak signal-to-noise ratio (PSNR) representing a quality difference between the first image and the rescaled first image.

6. The method of claim 4, wherein, Determining the scale factor from the quality score includes: comparing the quality score to a predetermined threshold; and setting the scale factor to a value equal to the first scale factor or different from the first scale factor according to a result of the comparison of the quality score to the predetermined threshold.

7. The method of claim 2, wherein applying at least one scaling to a buffer size of a second image includes: scaling down a buffer size of a second image by the scale factor prior to displaying the second image.

8. The method of claim 2, wherein, applying at least one scaling to a buffer size of a second image to obtain a second graphics command output includes: scaling down a buffer size of a second image by the scale factor prior to displaying the second image to obtain the second graphics command output; or scaling down a buffer size of a second image by the scale factor prior to displaying the second image, scaling up the scaled down buffer size to obtain the second graphics command output. Further comprising:

9. The method of claim 2, wherein, the graphics command module generates the first graphics command output and the second graphics command output; a resolution controller receives the scale factor from the image detection module; and the resolution controller provides one or more commands to the graphics command module to cause the graphics command module to apply at least one scaling to a buffer size of the second image according to the scale factor. Further comprising: the image detection module receives the second graphics command output representing the second image from the graphics command module; 10. The method of claim 2, wherein, ​ ​ scaling down the second image by a next scale factor of the scale factor, the next scale factor of the scale factor being smaller than the scale factor by a predetermined step size, obtaining a second intermediate image; scaling up the second intermediate image, obtaining a rescaled second image having the same pixel resolution as the second image; and calculating a quality score by comparing the second image and the rescaled second image, determining the scale factor for the buffer size of the third image based on the quality score.

11. The method of claim 6, wherein, the value different from the first scale factor is a previous scale factor of the first scale factor, the previous scale factor of the first scale factor being larger than the first scale factor by a predetermined step size.

12. A computing device, comprising: comprising: a graphics command module configured to scale down a buffer size of a first image using a scale factor determined based on a quality score of a previous image, obtaining a first graphics command output, or scale down a buffer size of a first image and then scale up, obtaining a first graphics command output; the graphics command module providing the first graphics command output to a display and an image detection module; the image detection module coupled to the graphics command module, configured to: receive a first graphics command output representing the first image from the graphics command module; and determine a quality score of the first image based on the first graphics command output, and determine a scale factor for scaling down a buffer size of a second image when generating a second graphics command output of the second image based on the quality score.

13. The computing device of claim 12, wherein, the image detection module is further configured to: scale down the first image by a first scale factor, obtaining a first intermediate image; scale up the first intermediate image, obtaining a rescaled first image having the same pixel resolution as the first image; and calculate the quality score by comparing the first image and the rescaled first image.

14. The computing device of claim 13, wherein, the image detection module is further configured to: calculate the quality score by calculating a peak signal to noise ratio (PSNR) representing a quality difference between the first image and the rescaled first image.

15. The computing device of claim 14, wherein, the image detection module is further configured to: compare the quality score with a predetermined threshold; and set the scale factor to be equal to the first scale factor or a value different from the first scale factor based on a comparison result of the quality score and the predetermined threshold.

16. The computing device of claim 12, wherein, further comprising: a resolution controller coupled to the graphics command module and the image detection module, configured to receive the scale factor from the image detection module; and provide one or more commands to the graphics command module, so that the graphics command module applies at least one scaling to a buffer size of a second image based on the scale factor, obtaining a second graphics command output.

17. The computing device of claim 16, wherein, applying at least one scaling to a buffer size of a second image based on the scale factor, obtaining a second graphics command output, comprises: scaling down a buffer size of a second image by the scale factor, obtaining the second graphics command output, before displaying the second image; or, Before displaying the second image, scaling down a buffer size of the second image according to the scaling factor, and scaling up the scaled down buffer size to obtain the second graphics command output.

18. The computing device of claim 12, wherein, Further comprising: a display interface configured to display the first image according to the first graphics command output.

19. A computing device, comprising: Comprising: at least one display interface; at least one computer-readable storage device storing executable instructions; and at least one processor executing the executable instructions to perform a method comprising: receiving a first graphics command output configured to cause a first image to be displayed by the at least one display interface; wherein the first graphics command output is obtained by scaling down a buffer size of the first image using a scaling factor determined according to a quality score of a previous image, or by scaling down and then scaling up a buffer size of the first image using a scaling factor determined according to a quality score of a previous image; determining a quality score of the first image according to the first graphics command output, and determining a scaling factor for scaling down a buffer size of a second image when generating a second graphics command output of the second image according to the quality score.

20. The computing device of claim 19, wherein, Further comprising: applying at least one scaling to a buffer size of the second image according to the scaling factor to obtain the second graphics command output configured to cause the second image to be displayed by the at least one display interface.

21. The computing device of claim 19, wherein the act of determining the quality score comprises: scaling down the first image according to a first scaling factor to obtain a first intermediate image; scaling up the first intermediate image to obtain a rescaled first image having a same pixel resolution as the first image; and computing the quality score by comparing the first image and the rescaled first image.

22. The computing device of claim 21, wherein, the act of computing the quality score comprises: computing a peak signal-to-noise ratio (PSNR) representing a quality difference between the first image and the rescaled first image. the act of computing the quality score comprises: computing a peak signal-to-noise ratio (PSNR) representing a quality difference between the first image and the rescaled first image.

Images