Method and apparatus for controlling image quality enhancement

Abstract

The application discloses a control method and device for image quality enhancement, and the method comprises the following steps: monitoring whether the output resolution of a target image changes relative to a current image; the current image is a previous frame image of the target image; in response to the change of the output resolution, obtaining a quality enhancement parameter of the target image; performing quality enhancement on the target image according to the quality enhancement parameter of the target image, and the quality enhancement parameter of the target image makes the time consumption of performing quality enhancement on the target image less than or equal to a target threshold; wherein the quality enhancement parameter comprises a quality enhancement mode used by each image sub-region contained in the target image, the quality enhancement mode comprises a first enhancement mode or a second enhancement mode, the time consumption of the quality enhancement of the first enhancement mode is greater than that of the second enhancement mode, and the quality enhancement effect of the first enhancement mode is better than that of the second enhancement mode.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a control method and apparatus for enhancing image quality. Background Technology

[0002] When playing a video, the player can enhance the image quality as the video is decoded, allowing users to experience the viewing effect of ultra-high-definition video even with a low-definition video stream.

[0003] Currently, machine learning models are commonly used to enhance image quality. However, this method requires a large amount of computation. If the video itself has a high resolution, the increased computational load for image enhancement may cause video stuttering. Summary of the Invention

[0004] In view of the above problems, this application provides an image quality enhancement control method and apparatus to avoid image output stuttering, thereby improving the smoothness of image output. The specific solution is as follows:

[0005] A first aspect of this application provides a method for controlling image quality enhancement, the method comprising:

[0006] Monitor whether the output resolution of the target image has changed relative to the current image; the current image is the previous frame of the target image.

[0007] In response to the change in output resolution, image quality enhancement parameters of the target image are obtained;

[0008] Based on the image quality enhancement parameters of the target image, the image quality of the target image is enhanced, wherein the image quality enhancement parameters of the target image make the time spent enhancing the image quality of the target image less than or equal to the target threshold.

[0009] The image quality enhancement parameters include the image quality enhancement method used for each image sub-region contained in the target image. The image quality enhancement method includes a first enhancement method or a second enhancement method. The image quality enhancement time of the first enhancement method is greater than that of the second enhancement method, and the image quality enhancement effect of the first enhancement method is better than that of the second enhancement method.

[0010] In one possible implementation, obtaining the image quality enhancement parameters of the target image includes:

[0011] Determine whether image quality enhancement should be performed on the current image;

[0012] When image quality enhancement is performed on the current image, the image quality enhancement parameters of the target image are obtained in a first acquisition method;

[0013] If no image quality enhancement is performed on the current image, the image quality enhancement parameters of the target image are obtained using a second acquisition method.

[0014] The first acquisition method and the second acquisition method are different.

[0015] In one possible implementation, the image quality enhancement parameters of the target image are obtained in a first acquisition method, including:

[0016] Based on the relationship between the output resolution of the target image and the output resolution of the current image, the image quality enhancement parameters of the current image are adjusted to obtain the image quality enhancement parameters of the target image.

[0017] In one possible implementation, the image quality enhancement parameters of the current image are adjusted based on the relationship between the output resolution of the target image and the output resolution of the current image to obtain the image quality enhancement parameters of the target image, including:

[0018] Determine whether the output resolution of the target image is greater than the output resolution of the current image;

[0019] If the output resolution of the target image is greater than the output resolution of the current image, at least one first sub-region in the image quality enhancement parameters of the current image is sequentially adjusted from the first enhancement method to the second enhancement method until a candidate enhancement parameter that meets the preferred conditions is obtained;

[0020] If the output resolution of the target image is less than the output resolution of the current image, at least one second sub-region in the image quality enhancement parameters of the current image is sequentially adjusted from the second enhancement method to the first enhancement method until a candidate enhancement parameter that meets the preferred conditions is obtained;

[0021] The candidate enhancement parameters that meet the preferred conditions are determined as the image quality enhancement parameters of the target image.

[0022] In one possible implementation, the area of ​​the first sub-region that was previously adjusted is smaller than the area of ​​the first sub-region that was subsequently adjusted.

[0023] The area of ​​the second sub-region that was adjusted in the previous adjustment is larger than the area of ​​the second sub-region that was adjusted in the subsequent adjustment.

[0024] In one possible implementation, the image quality enhancement parameters of the target image are obtained in a second acquisition method, including:

[0025] When the output resolution of the target image is less than the output resolution of the current image, an enhancement judgment result is obtained, wherein the enhancement judgment result indicates whether to perform image quality enhancement on the target image;

[0026] If the enhancement judgment result indicates that image quality enhancement is performed on the target image, the initial enhancement parameters are adjusted to obtain the image quality enhancement parameters of the target image;

[0027] In the initial enhancement parameters, each of the image sub-regions uses the second enhancement method.

[0028] In one possible implementation, the initial enhancement parameters are adjusted to obtain the image quality enhancement parameters of the target image, including:

[0029] At least one third sub-region in the initial enhancement parameters is sequentially adjusted from the second enhancement method to the first enhancement method until candidate enhancement parameters that meet the preferred conditions are obtained;

[0030] The candidate enhancement parameters that meet the preferred conditions are determined as the image quality enhancement parameters of the target image.

[0031] In one possible implementation, the area of ​​the third sub-region that was previously adjusted is smaller than the area of ​​the third sub-region that was subsequently adjusted.

[0032] In one possible implementation, the preferred condition includes: the image quality enhancement parameter makes the image quality enhancement time of the target image less than or equal to the target threshold, and the image quality enhancement parameter makes the image quality enhancement time of the target image the largest among all the candidate enhancement parameters.

[0033] A second aspect of this application provides an image quality enhancement control device, comprising:

[0034] A resolution monitoring unit is used to monitor whether the output resolution of the target image has changed relative to the current image; the current image is the previous frame of the target image.

[0035] The parameter acquisition unit is used to obtain the image quality enhancement parameters of the target image in response to a change in the output resolution;

[0036] An enhanced execution unit is configured to enhance the image quality of the target image according to the image quality enhancement parameters of the target image, wherein the image quality enhancement parameters of the target image make the time spent enhancing the image quality of the target image less than or equal to a target threshold.

[0037] The image quality enhancement parameters include the image quality enhancement method used for each image sub-region contained in the target image. The image quality enhancement method includes a first enhancement method or a second enhancement method. The image quality enhancement time of the first enhancement method is greater than that of the second enhancement method, and the image quality enhancement effect of the first enhancement method is better than that of the second enhancement method.

[0038] A third aspect of this application provides a computer program product including computer-readable instructions that, when executed on an electronic device, cause the electronic device to implement the image quality enhancement control method described in the first aspect or any implementation thereof.

[0039] A fourth aspect of this application provides an electronic device, including at least one processor and a memory connected to the processor, wherein:

[0040] The memory is used to store computer programs;

[0041] The processor is used to execute the computer program so that the electronic device can implement the image quality enhancement control method of the first aspect or any implementation thereof.

[0042] The fifth aspect of this application provides a computer storage medium carrying one or more computer programs, which, when executed by an electronic device, enable the electronic device to control the image quality enhancement method described in the first aspect or any implementation thereof.

[0043] By employing the above technical solution, the image quality enhancement control method and apparatus provided in this application obtain image quality enhancement parameters for the target image when a change in the output resolution of the target image relative to the previous frame image is detected. These parameters include image quality enhancement methods applicable to each sub-region of the target image, such as a first enhancement method and a second enhancement method with different enhancement times and effects. These parameters ensure that the time required to enhance the target image is less than or equal to a target threshold. Therefore, in response to changes in output resolution, this application obtains image quality enhancement parameters that ensure the time required to enhance the target image does not exceed the target threshold. This means that whenever the output resolution changes, image quality enhancement is performed using these parameters with a time limit not exceeding the target threshold, thereby avoiding image output stuttering caused by the enhancement time exceeding the target threshold and improving the smoothness of image output. Attached Figure Description

[0044] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 A flowchart illustrating an image quality enhancement control method provided in this application embodiment;

[0046] Figure 2 This is a partial flowchart of an image quality enhancement control method provided in an embodiment of this application;

[0047] Figure 3 This is an example diagram of the image enhancement control for the player in the embodiments of this application;

[0048] Figure 4 This is another part of the flowchart of an image quality enhancement control method provided in the embodiments of this application;

[0049] Figure 5 This is another part of the flowchart of an image quality enhancement control method provided in the embodiments of this application;

[0050] Figure 6 This is a schematic diagram of the structure of an image quality enhancement control device provided in an embodiment of this application;

[0051] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0052] Figure 8 This is a flowchart illustrating the overall process of enhancing image quality in scenarios where this application is applicable to video playback on mobile devices;

[0053] Figure 9 This is a flowchart illustrating the process of performing image quality enhancement in scenarios where this application applies to video playback on mobile devices;

[0054] Figure 10 This is a flowchart illustrating the image quality enhancement feedback update estimation value in a scenario applicable to video playback on a mobile phone player, as described in this application.

[0055] Figure 11 This is a flowchart illustrating the calculation of the total time required for current frame quality enhancement estimation in scenarios applicable to mobile phone video playback in this application;

[0056] Figure 12 This is a flowchart illustrating the logic for readjusting the optimal image quality enhancement strategy in scenarios where this application is applicable to video playback on mobile devices. Detailed Implementation

[0057] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is for explaining specific embodiments only and is not intended to limit the scope of this application.

[0058] The embodiments of this application will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.

[0059] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0060] This application can be applied to the field of image processing. The technical solution of this application will be introduced below using the scenario of a terminal outputting video through a player as an example.

[0061] First, when the user's local network is poor, or when the user chooses to play the video in standard definition or ultra-high definition to save data, or when the video source is not clear to begin with, the terminal can enhance the picture quality through the player when the video is decoded. This allows the user to experience the viewing effect of ultra-high definition video with a low-definition video stream.

[0062] However, the super-resolution model (machine learning model) used for image enhancement is computationally intensive. When enhancing image quality in feature regions (visual features of human figures, subtitles, station logos, etc.), the image enhancement effect cannot be optimal due to the large differences in terminal performance. Moreover, the computational load of the super-resolution model increases significantly with the area as the resolution increases, making it difficult to fully utilize the terminal performance under multiple resolutions. As a result, the image enhancement function at high resolution has to be turned off, or the image enhancement function at high resolution will cause video output stuttering.

[0063] To address the aforementioned problems, this application provides an image quality enhancement control method. The image quality enhancement control method of this application embodiment will be described in detail below with reference to the accompanying drawings.

[0064] Reference Figure 1This is a flowchart illustrating an image quality enhancement control method provided in this application embodiment. This method is applicable to electronic devices capable of image processing, such as mobile phones and tablets with media players. The technical solution in this embodiment primarily aims to avoid image output stuttering caused by the time consumed during image quality enhancement exceeding a target threshold, thereby improving the smoothness of image output.

[0065] Specifically, the method in this embodiment may include the following steps:

[0066] Step 101: Monitor whether the output resolution of the target image has changed relative to the current image. If the output resolution has not changed, proceed to step 102. If the output resolution has changed, proceed to step 103.

[0067] Here, the current image is the previous frame of the target image. For example, taking a video player on a mobile phone outputting a variety show video as an example, the current image is the image being output in the variety show video, and the target image is the next frame of the variety show video that is to be rendered and output.

[0068] Step 102: Use the image quality enhancement parameters of the current image as the image quality enhancement parameters of the target image.

[0069] In other words, during the video output process, if the output resolution remains unchanged when enhancing the image quality of each frame, then the image enhancement parameters used in the previous frame will continue to be used to enhance the subsequent images. If the output resolution changes, then the image enhancement parameters for the subsequent images need to be redefined.

[0070] Step 103: Obtain the image quality enhancement parameters of the target image.

[0071] Among them, the image quality enhancement parameters of the target image ensure that the time required to enhance the image quality of the target image is less than or equal to the target threshold.

[0072] It should be noted that if the image enhancement parameters that take less than or equal to the target threshold are not obtained, then the image enhancement function for the target image can be turned off. For example, the image enhancement function of the video player can be turned off.

[0073] Step 104: Enhance the image quality of the target image according to the image quality enhancement parameters of the target image.

[0074] The target image comprises multiple image sub-regions and a remaining sub-region. Each image sub-region requires image quality enhancement, while the remaining sub-regions do not. Therefore, the image quality enhancement parameters include the image quality enhancement method used for each image sub-region of the target image. Different image quality enhancement parameters may use different enhancement methods for one or more image sub-regions.

[0075] It should be noted that the image enhancement methods include the first enhancement method or the second enhancement method. The first enhancement method takes longer to enhance the image quality than the second enhancement method, and the image enhancement effect of the first enhancement method is better than that of the second enhancement method.

[0076] For example, the first enhancement method is to use a super-resolution model for image quality enhancement, and the second enhancement method is to use Unsharp Masking (USM) for image quality enhancement. Although the USM method is faster, its image quality enhancement effect is not as good as that of the super-resolution model method. Taking a variety show video output by a mobile phone video player as an example, the video frame of the variety show video contains sub-regions of visible human features, sub-regions of subtitles, sub-regions of station logos, and remaining sub-regions. Each sub-region is enhanced using a corresponding image quality enhancement method.

[0077] It should be noted that the time taken to enhance the image quality of the target image can also be called the image quality enhancement time of the target image. The image quality enhancement time of the target image includes at least the total time taken to enhance the image quality of each sub-region of the target image, or it can be the maximum value among the times taken to enhance the image quality of each sub-region of the target image. The image quality enhancement time of a sub-region can be obtained by multiplying the statistical value of the time taken per unit area of ​​the sub-region by the area of ​​the sub-region. The statistical value of the time taken per unit area of ​​the sub-region refers to the statistical time taken to enhance the image quality of the sub-region within a unit area. The statistical value of the time taken per unit area of ​​the sub-region can be the initial statistical value, or it can be the average historical unit time taken by image regions of the same region type as the sub-region within a unit area for image quality enhancement. The historical unit time refers to the historical time taken by image regions of the same region type as the sub-region within a unit area for image quality enhancement.

[0078] Different image enhancement methods have different time unit statistics for enhancing image sub-regions. For example, the time unit statistics for enhancing a certain image sub-region using the first enhancement method is greater than that using the second enhancement method. However, the image enhancement effect of enhancing a certain image sub-region using the first enhancement method is better than that using the second enhancement method.

[0079] Based on the above technical solution, in the image quality enhancement control method provided by this application embodiment, when a change in the output resolution of the target image relative to the previous frame image is detected, image quality enhancement parameters of the target image are obtained. These parameters include image quality enhancement methods applicable to each image sub-region contained in the target image, such as a first enhancement method and a second enhancement method with different enhancement times and effects. These parameters ensure that the time required to enhance the target image is less than or equal to a target threshold. Therefore, in this embodiment, in response to changes in output resolution, image quality enhancement parameters are obtained that ensure the time required to enhance the target image does not exceed the target threshold. Thus, whenever the output resolution changes, image quality enhancement is performed using these parameters with a time not exceeding the target threshold, thereby avoiding image output stuttering caused by the enhancement time exceeding the target threshold, and improving the smoothness of image output.

[0080] In one implementation, when obtaining the image quality enhancement parameters of the target image in step 103, it can be achieved in the following way, such as... Figure 2 As shown:

[0081] Step 201: Determine whether image quality enhancement is performed on the current image. If image quality enhancement is performed on the current image, proceed to step 202. If image quality enhancement is not performed on the current image, proceed to step 203.

[0082] Specifically, in this embodiment, determining whether the current image has undergone image quality enhancement can be done by: determining whether the player that outputs the current image has enabled the image quality enhancement function.

[0083] For example, taking a mobile phone media player as an example, the player has image quality enhancement controls, such as... Figure 3 As shown, when the image enhancement control is turned on, image enhancement is performed on the current image; when the image enhancement control is turned off, image enhancement is not performed on the current image.

[0084] Step 202: Obtain the image quality enhancement parameters of the target image using the first acquisition method.

[0085] Step 203: Obtain the image quality enhancement parameters of the target image using the second acquisition method.

[0086] The first acquisition method differs from the second acquisition method.

[0087] For example, the first acquisition method is to obtain the image quality enhancement parameters of the target image based on the image quality enhancement parameters of the current image, while the second acquisition method is different from the first acquisition method.

[0088] In one implementation, the first acquisition method in step 202 is: adjusting the image quality enhancement parameters of the current image according to the size relationship between the output resolution of the target image and the output resolution of the current image, so as to obtain the image quality enhancement parameters of the target image.

[0089] Specifically, when the output resolution of the target image and the output resolution of the current image are different, the image quality enhancement parameters of the current image are adjusted in different ways to obtain the image quality enhancement parameters of the target image.

[0090] Specifically, step 202 can be achieved in the following ways, such as... Figure 4 As shown:

[0091] Step 401: Determine whether the output resolution of the target image is greater than the output resolution of the current image. If the output resolution of the target image is greater than the output resolution of the current image, proceed to step 402. If the output resolution of the target image is less than the output resolution of the current image, proceed to step 403.

[0092] Step 402: Sequentially adjust at least one first sub-region of the current image quality enhancement parameters from the first enhancement mode to the second enhancement mode until a candidate enhancement parameter that meets the preferred conditions is obtained.

[0093] In cases where the output resolution increases, if the target image is further enhanced using the current image enhancement parameters, the enhancement time may exceed the target threshold. Therefore, the first sub-region in the current image enhancement parameters can be changed from the first enhancement method to the second enhancement method as a candidate enhancement parameter to reduce the enhancement time.

[0094] Specifically, in step 402, the image quality enhancement time of the target image can be calculated first using the image quality enhancement parameters of the current image. If the image quality enhancement time of the target image is less than or equal to the target threshold, the image quality enhancement parameters of the current image are used as the image quality enhancement parameters of the target image. If the image quality enhancement time of the target image is greater than the target threshold, then at least one first sub-region in the image quality enhancement parameters of the current image is sequentially adjusted from the first enhancement mode to the second enhancement mode until a candidate enhancement parameter that meets the preferred condition is obtained. If the preferred condition is still not met when all image sub-regions in the image quality enhancement parameters of the current image are in the second enhancement mode, then there is no candidate enhancement parameter that meets the preferred condition.

[0095] It should be noted that the first sub-region was changed from the first enhancement method to the second enhancement method, which can reduce the time required for image enhancement by sacrificing the image quality enhancement effect.

[0096] The preferred conditions may include: the image quality enhancement parameters make the image quality enhancement time of the target image less than or equal to the target threshold, and the image quality enhancement parameters make the image quality enhancement time of the target image the largest among all the candidate enhancement parameters.

[0097] Furthermore, the preferred conditions may also include: the image enhancement parameters make the area of ​​the sub-region in the target image that is enhanced in the first enhancement method the largest among all the candidate enhancement parameters.

[0098] Based on this, in this embodiment, the image quality enhancement parameters with the best image quality enhancement effect and the largest image quality enhancement time but less than or equal to the target threshold are obtained based on the image quality enhancement parameters of the current image, and are used as candidate enhancement parameters that meet the preferred conditions.

[0099] In one implementation, in step 402, the area of ​​the first sub-region that was previously adjusted is smaller than the area of ​​the first sub-region that is subsequently adjusted.

[0100] In other words, in step 402, regarding the image quality enhancement parameters of the current image, starting with the smallest image sub-region using the first enhancement method, its image quality enhancement method is adjusted to the second enhancement method, which has a lower time consumption. If the adjusted image quality enhancement parameter makes the image quality enhancement time of the target image less than or equal to the target threshold, then the adjusted image quality enhancement parameter is determined as a candidate enhancement parameter that meets the preferred conditions. If the adjusted image quality enhancement parameter makes the image quality enhancement time of the target image greater than the target threshold, then only the image quality enhancement method of the second smallest image sub-region using the first enhancement method is adjusted to the second enhancement method, which has a lower time consumption. If the adjusted image quality enhancement parameter makes the image quality enhancement time of the target image less than or equal to the target threshold, then the adjusted image quality enhancement parameter is determined as a candidate enhancement parameter that meets the preferred conditions. If the adjusted image quality enhancement parameter makes the image quality enhancement time of the target image greater than the target threshold, then only the image quality enhancement method of the larger image sub-region using the first enhancement method is adjusted to the second enhancement method, and so on. If only the image quality enhancement method of the largest image sub-region using the first enhancement method is adjusted... If the second enhancement method, which has a lower time consumption, still results in the image enhancement time of the target image exceeding the target threshold, then starting with the image sub-region with the smallest area using the first enhancement method, the image enhancement methods of the two smallest and second smallest image sub-regions using the first enhancement method are adjusted to the second enhancement method, which has a lower time consumption. If the adjusted image enhancement parameters result in the image enhancement time of the target image being less than or equal to the target threshold, then the adjusted image enhancement parameters are determined as candidate enhancement parameters that meet the preferred conditions. If the adjusted image enhancement parameters result in the image enhancement time of the target image exceeding the target threshold, then the image enhancement methods of the two smallest and largest image sub-regions using the first enhancement method are adjusted to the second enhancement method, which has a lower time consumption, and so on, until the adjusted image enhancement parameters result in the image enhancement time of the target image being less than or equal to the target threshold. At this point, the adjusted image enhancement parameters are determined as candidate enhancement parameters that meet the preferred conditions. Alternatively, if the image enhancement methods of all image sub-regions are adjusted to the second enhancement method and still cannot result in the image enhancement time of the target image being less than or equal to the target threshold, then there are no candidate enhancement parameters that meet the preferred conditions.

[0101] The image enhancement time of the target image can be obtained in the following way: according to the image enhancement method used by each image sub-region in the current image, for each image sub-region in the target image, obtain the product of the area of ​​the image sub-region and the statistical value of the time unit corresponding to the image enhancement method used by the image sub-region of the same type in the current image, and then obtain the sum of the products to obtain the image enhancement time of the target image.

[0102] Step 403: Sequentially adjust at least one second sub-region of the current image quality enhancement parameters from the second enhancement mode to the first enhancement mode until a candidate enhancement parameter that meets the preferred conditions is obtained.

[0103] In the case of a smaller output resolution, if the target image is further enhanced using the current image enhancement parameters, the enhancement time of the target image will be less than or equal to the target threshold, but the enhancement effect may be poor. Therefore, the second sub-region in the current image enhancement parameters can be changed from the second enhancement method to the first enhancement method as the candidate enhancement parameter to optimize the enhancement effect.

[0104] Specifically, in step 403, the image quality enhancement time of the target image can be calculated using the image quality enhancement parameters of the current image. If the image quality enhancement time of the target image is less than or equal to the target threshold, then at least one second sub-region in the image quality enhancement parameters of the current image is sequentially adjusted from the second enhancement method to the first enhancement method until candidate enhancement parameters that meet the preferred conditions are obtained. Wherein, if the preferred conditions are still met when all image sub-regions in the image quality enhancement parameters of the current image are in the first enhancement method, then the image quality enhancement parameters in which all image sub-regions are in the first enhancement method are taken as candidate enhancement parameters.

[0105] It should be noted that the second sub-region has been changed from the second enhancement method to the first enhancement method, which can optimize the image quality enhancement effect by sacrificing image quality and increasing processing time.

[0106] The preferred conditions may include: the image quality enhancement parameters make the image quality enhancement time of the target image less than or equal to the target threshold, and the image quality enhancement parameters make the image quality enhancement time of the target image the largest among all the candidate enhancement parameters.

[0107] Furthermore, the preferred conditions may also include: the image enhancement parameters make the area of ​​the sub-region in the target image that is enhanced in the first enhancement method the largest among all the candidate enhancement parameters.

[0108] Based on this, in this embodiment, the image quality enhancement parameters with the best image quality enhancement effect and the largest image quality enhancement time but less than or equal to the target threshold are obtained based on the image quality enhancement parameters of the current image, and are used as candidate enhancement parameters that meet the preferred conditions.

[0109] In one implementation, in step 403, the area of ​​the second sub-region that was previously adjusted is greater than the area of ​​the second sub-region that was subsequently adjusted.

[0110] In other words, in step 403, regarding the image quality enhancement parameters of the current image, starting with the image sub-region with the largest area using the second enhancement method, the image quality enhancement method is adjusted to the more effective first enhancement method. If the adjusted image quality enhancement parameters cause the image quality enhancement time of the target image to exceed the target threshold, then the image quality enhancement parameters before adjustment are determined as candidate enhancement parameters that meet the preferred conditions. If the adjusted image quality enhancement parameters still cause the image quality enhancement time of the target image to be less than or equal to the target threshold, then the image quality enhancement method of the second largest area image sub-region using the second enhancement method is adjusted to the more effective first enhancement method. If the adjusted image quality enhancement parameters cause the image quality enhancement time of the target image to exceed the target threshold, then the image quality enhancement parameters before adjustment are determined as... To determine the optimal enhancement parameters, if the adjusted image enhancement parameters still result in the image enhancement time of the target image being less than or equal to the target threshold, then the image enhancement method for smaller image sub-regions using the second enhancement method is adjusted to the more effective first enhancement method. This process continues until the adjusted image enhancement parameters result in the image enhancement time of the target image being greater than the target threshold. At this point, the image enhancement parameters before adjustment are determined as the optimal enhancement parameters. Alternatively, if adjusting the image enhancement methods of all image sub-regions to the first enhancement method fails to result in the image enhancement time of the target image being greater than the target threshold, then the image enhancement parameters for all image sub-regions that have been adjusted to the first enhancement method are determined as the optimal enhancement parameters.

[0111] Step 404: Select the candidate enhancement parameters that meet the preferred conditions as the image quality enhancement parameters of the target image.

[0112] It should be noted that if there are no candidate enhancement parameters that meet the preferred criteria, the image quality enhancement function for the target image can be turned off.

[0113] In one implementation, the second acquisition method in step 203 can be: obtaining the image quality enhancement parameters of the target image based on the initial enhancement parameters.

[0114] Specifically, in step 203, it can first be determined whether the output resolution of the target image is greater than the output resolution of the current image, wherein:

[0115] If the output resolution of the target image is greater than the output resolution of the current image, do not perform image quality enhancement on the target image. In other words, if the output resolution of the target image is greater than the output resolution of the current image, it means that performing image quality enhancement when outputting a target image with a higher output resolution will consume more device resources and will definitely cause image stuttering. Therefore, do not perform image quality enhancement on the target image in this case.

[0116] When the output resolution of the target image is less than the output resolution of the current image, an enhancement judgment result is first obtained. This result indicates whether to perform image quality enhancement on the target image. For example, the time required for image quality enhancement of the target image can be calculated using initial enhancement parameters. If the initial enhancement parameters cause the time required for image quality enhancement of the target image to be greater than a target threshold, then an enhancement judgment result indicating that image quality enhancement will not be performed on the target image is obtained. If the initial enhancement parameters cause the time required for image quality enhancement of the target image to be less than or equal to the target threshold, then an enhancement judgment result indicating that image quality enhancement will be performed on the target image is obtained.

[0117] Based on this, if the enhancement judgment result indicates that image quality enhancement is performed on the target image, then the initial enhancement parameters can be adjusted to obtain the image quality enhancement parameters of the target image.

[0118] In the initial enhancement parameters, each image sub-region uses the second enhancement method.

[0119] Specifically, in step 203, when adjusting the initial enhancement parameters to obtain the image quality enhancement parameters of the target image, this can be achieved in the following ways: Figure 5 As shown:

[0120] Step 501: Sequentially adjust at least one third sub-region in the initial enhancement parameters from the second enhancement mode to the first enhancement mode until candidate enhancement parameters that meet the preferred conditions are obtained.

[0121] If the target image is enhanced using the initial enhancement parameters, the enhancement time will definitely be less than or equal to the target threshold, but the enhancement effect will be poor. Therefore, the third sub-region in the initial enhancement parameters can be changed from the second enhancement method to the first enhancement method as the candidate enhancement parameter to optimize the enhancement effect.

[0122] Specifically, in step 501, the image quality enhancement time of the target image can be calculated using the initial enhancement parameters. If the image quality enhancement time of the target image is less than or equal to the target threshold, then at least one third sub-region in the initial enhancement parameters is sequentially adjusted from the second enhancement method to the first enhancement method until candidate enhancement parameters that meet the preferred conditions are obtained. Wherein, if the preferred conditions are still met when all image sub-regions in the initial enhancement parameters are in the first enhancement method, then the image quality enhancement parameters in which the image quality enhancement method for all image sub-regions is the first enhancement method are taken as candidate enhancement parameters.

[0123] It should be noted that the third sub-region has been changed from the second enhancement method to the first enhancement method, which can optimize the image enhancement effect by sacrificing the image enhancement time.

[0124] The preferred conditions may include: the image quality enhancement parameters make the image quality enhancement time of the target image less than or equal to the target threshold, and the image quality enhancement parameters make the image quality enhancement time of the target image the largest among all the candidate enhancement parameters.

[0125] Furthermore, the preferred conditions may also include: the image enhancement parameters make the area of ​​the sub-region in the target image that is enhanced in the first enhancement method the largest among all the candidate enhancement parameters.

[0126] Based on this, in this embodiment, the image quality enhancement parameters with the best image quality enhancement effect and the largest image quality enhancement time but less than or equal to the target threshold are obtained based on the initial enhancement parameters, and are used as candidate enhancement parameters that meet the preferred conditions.

[0127] In one implementation, in step 501, the area of ​​the previously adjusted third sub-region is smaller than the area of ​​the subsequently adjusted third sub-region.

[0128] Specifically, in step 501, for the initial enhancement parameters, starting with the image sub-region with the smallest area using the second enhancement method, its image quality enhancement method is adjusted to the more effective first enhancement method. If the adjusted image quality enhancement parameters result in the image quality enhancement time of the target image being greater than the target threshold, then the image quality enhancement parameters before adjustment are determined as candidate enhancement parameters that meet the preferred conditions. If the adjusted image quality enhancement parameters result in the image quality enhancement time of the target image still being less than or equal to the target threshold, then the image quality enhancement method of the second smallest area image sub-region using the second enhancement method is adjusted to the more effective first enhancement method. If the adjusted image quality enhancement parameters result in the image quality enhancement time of the target image being greater than the target threshold, then the image quality enhancement parameters before adjustment are determined as candidate enhancement parameters that meet the preferred conditions. If the adjusted image enhancement parameters still result in the image enhancement time of the target image being less than or equal to the target threshold, then the image enhancement method for the larger image sub-regions using the second enhancement method is adjusted to the more effective first enhancement method, and so on, until the adjusted image enhancement parameters result in the image enhancement time of the target image being greater than the target threshold. At this point, the image enhancement parameters before adjustment are determined as candidate enhancement parameters that meet the preferred conditions. Alternatively, if the image enhancement time of the target image is still not greater than the target threshold even after all image sub-regions have been adjusted to the first enhancement method, then the image enhancement parameters for all image sub-regions that have been adjusted to the first enhancement method are determined as candidate enhancement parameters that meet the preferred conditions.

[0129] In another implementation, in step 501, the area of ​​the previously adjusted third sub-region is greater than the area of ​​the subsequently adjusted third sub-region.

[0130] Specifically, in step 501, for the initial enhancement parameters, starting with the image sub-region with the largest area using the second enhancement method, the image enhancement method is adjusted to the more effective first enhancement method. If the adjusted image enhancement parameters result in the image enhancement time of the target image exceeding the target threshold, then the image enhancement parameters before adjustment are determined as candidate enhancement parameters that meet the preferred conditions. If the adjusted image enhancement parameters result in the image enhancement time of the target image still being less than or equal to the target threshold, then the image enhancement method of the second largest area image sub-region using the second enhancement method is adjusted to the more effective first enhancement method. If the adjusted image enhancement parameters result in the image enhancement time of the target image exceeding the target threshold, then the image enhancement parameters before adjustment are determined as candidate enhancement parameters that meet the preferred conditions. If the adjusted image enhancement parameters still result in the image enhancement time of the target image being less than or equal to the target threshold, then the image enhancement method for smaller image sub-regions using the second enhancement method is adjusted to the more effective first enhancement method, and so on, until the adjusted image enhancement parameters result in the image enhancement time of the target image being greater than the target threshold. At this point, the image enhancement parameters before adjustment are determined as candidate enhancement parameters that meet the preferred conditions. Alternatively, if the image enhancement time of the target image is still not greater than the target threshold even after all image sub-regions have been adjusted to the first enhancement method, then the image enhancement parameters for all image sub-regions that have been adjusted to the first enhancement method are determined as candidate enhancement parameters that meet the preferred conditions.

[0131] Step 502: Determine the candidate enhancement parameters that meet the preferred conditions as the image quality enhancement parameters of the target image.

[0132] The above describes an image quality enhancement control method provided by the embodiments of this application. The following will describe the apparatus for performing the above image quality enhancement control method.

[0133] Please see Figure 6 This is a schematic diagram of an image quality enhancement control device provided in an embodiment of this application. The device can be configured in an electronic device capable of image processing, such as a mobile phone or tablet computer with a player. The technical solution in this embodiment is mainly used to avoid image output stuttering caused by the time taken for image quality enhancement exceeding a target threshold, thereby improving the smoothness of image output.

[0134] Specifically, the device in this embodiment may include the following units:

[0135] The resolution monitoring unit 601 is used to monitor whether the output resolution of the target image has changed relative to the current image; the current image is the previous frame image of the target image.

[0136] The parameter acquisition unit 602 is used to obtain the image quality enhancement parameters of the target image in response to the change in the output resolution;

[0137] The enhancement execution unit 603 is used to enhance the image quality of the target image according to the image quality enhancement parameters of the target image, wherein the image quality enhancement parameters of the target image make the time spent enhancing the image quality of the target image less than or equal to a target threshold.

[0138] The image quality enhancement parameters include the image quality enhancement method used for each image sub-region contained in the target image. The image quality enhancement method includes a first enhancement method or a second enhancement method. The image quality enhancement time of the first enhancement method is greater than that of the second enhancement method, and the image quality enhancement effect of the first enhancement method is better than that of the second enhancement method.

[0139] Based on the above technical solution, in the image quality enhancement control device provided in this application embodiment, when a change in the output resolution of the target image relative to the previous frame image is detected, image quality enhancement parameters of the target image are obtained. These parameters include image quality enhancement methods applicable to each image sub-region contained in the target image, such as a first enhancement method and a second enhancement method with different enhancement times and effects. The image quality enhancement parameters ensure that the time required to enhance the target image is less than or equal to a target threshold. Therefore, in this embodiment, in response to changes in output resolution, image quality enhancement parameters are obtained that ensure the time required to enhance the target image does not exceed the target threshold. Thus, whenever the output resolution changes, image quality enhancement is performed using image quality enhancement parameters with a time not exceeding the target threshold. This avoids image output stuttering caused by the enhancement time exceeding the target threshold, thereby improving the smoothness of image output.

[0140] In one implementation, when obtaining the image quality enhancement parameters of the target image, the parameter acquisition unit 602 is specifically used to: determine whether the current image has undergone image quality enhancement; if the current image has undergone image quality enhancement, obtain the image quality enhancement parameters of the target image using a first acquisition method; if the current image has not undergone image quality enhancement, obtain the image quality enhancement parameters of the target image using a second acquisition method; wherein the first acquisition method and the second acquisition method are different.

[0141] In one implementation, when the parameter acquisition unit 602 obtains the image quality enhancement parameters of the target image in a first acquisition method, it is specifically used to: adjust the image quality enhancement parameters of the current image according to the size relationship between the output resolution of the target image and the output resolution of the current image, so as to obtain the image quality enhancement parameters of the target image.

[0142] For example, the parameter acquisition unit 602 determines whether the output resolution of the target image is greater than the output resolution of the current image; if the output resolution of the target image is greater than the output resolution of the current image, at least one first sub-region in the image quality enhancement parameters of the current image is sequentially adjusted from the first enhancement method to the second enhancement method until a candidate enhancement parameter that meets the preferred conditions is obtained; if the output resolution of the target image is less than the output resolution of the current image, at least one second sub-region in the image quality enhancement parameters of the current image is sequentially adjusted from the second enhancement method to the first enhancement method until a candidate enhancement parameter that meets the preferred conditions is obtained; the candidate enhancement parameter that meets the preferred conditions is determined as the image quality enhancement parameter of the target image.

[0143] In a preferred embodiment, the area of ​​the first sub-region that was previously adjusted is smaller than the area of ​​the first sub-region that was subsequently adjusted; and the area of ​​the second sub-region that was previously adjusted is larger than the area of ​​the second sub-region that was subsequently adjusted.

[0144] In one implementation, when the parameter acquisition unit 602 obtains the image quality enhancement parameters of the target image in a second acquisition method, it is specifically used to: obtain an enhancement judgment result when the output resolution of the target image is less than the output resolution of the current image, wherein the enhancement judgment result indicates whether to perform image quality enhancement on the target image; if the enhancement judgment result indicates that image quality enhancement is performed on the target image, adjust the initial enhancement parameters to obtain the image quality enhancement parameters of the target image; wherein each of the image sub-regions in the initial enhancement parameters uses the second enhancement method.

[0145] For example, the parameter acquisition unit 602 sequentially adjusts at least one third sub-region in the initial enhancement parameters from the second enhancement mode to the first enhancement mode until a candidate enhancement parameter that meets the preferred conditions is obtained; the candidate enhancement parameter that meets the preferred conditions is determined as the image quality enhancement parameter of the target image.

[0146] In a preferred embodiment, the area of ​​the third sub-region that was previously adjusted is smaller than the area of ​​the third sub-region that was subsequently adjusted.

[0147] The preferred conditions include: the image quality enhancement parameter makes the image quality enhancement time of the target image less than or equal to the target threshold, and the image quality enhancement parameter makes the image quality enhancement time of the target image the largest among all the candidate enhancement parameters.

[0148] It should be noted that the specific implementation of each unit in this embodiment can be referred to the corresponding content above, and will not be described in detail here.

[0149] This application also provides an electronic device in its embodiments. (See reference...) Figure 7 The diagram illustrates a structural schematic suitable for implementing the electronic device in the embodiments of this application. The electronic device in the embodiments of this application may include, but is not limited to, fixed terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Figure 7 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0150] like Figure 7 As shown, the electronic device may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. When the electronic device is powered on, the RAM 703 also stores various programs and data required for the operation of the electronic device. The processing unit 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0151] Typically, the following devices can be connected to I / O interface 705: input devices 706 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 707 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 708 including, for example, memory cards, hard drives, etc.; and communication devices 709. Communication device 709 allows electronic devices to exchange data via wireless or wired communication with other devices. Although Figure 7 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0152] This application also provides a computer program product including computer-readable instructions, which, when executed on an electronic device, cause the electronic device to implement any of the image quality enhancement control methods provided in this application.

[0153] This application also provides a computer storage medium that carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any of the image quality enhancement control methods provided in this application.

[0154] Taking the scenario of playing videos on a mobile phone media player as an example, the technical solution of this application is illustrated below:

[0155] To improve the quality of videos played by the player, when users are watching standard low-definition videos, the player can use an AI super-resolution model to calculate and enhance the standard low-definition videos to ultra-high-definition videos. This can save users data, allow them to watch higher-quality videos, and reduce the pressure on server bandwidth costs, achieving three benefits in one go.

[0156] However, the computing performance of terminal devices such as mobile phones varies greatly. Moreover, as the image resolution increases, the computing area increases significantly, leading to a surge in computational load. This can result in situations where insufficient performance necessitates disabling the AI ​​super-resolution image enhancement function.

[0157] Therefore, this application proposes a method to dynamically and in real-time calculate the image enhancement time of each image sub-region. It also introduces the traditional USM image enhancement algorithm (which greatly reduces the computational load compared to the AI ​​super-resolution model). As the resolution changes, the optimal calculation strategy, namely the image enhancement parameters mentioned above, is adaptively adjusted to ensure the smooth operation of the image enhancement function of the AI ​​super-resolution model and improve the user's viewing experience.

[0158] The key improvements in the technical solution of this application are as follows:

[0159] (1) Add the traditional USM image enhancement algorithm, which ensures that the image enhancement algorithm of the AI ​​super-resolution model runs smoothly on low-end performance machines. It can also solve the problems of time consumption, power consumption and machine overheating caused by the large amount of AI super-resolution calculation for 720p and above resolution videos. If the requirements are not met, the function is turned off.

[0160] (2) By statistically analyzing the time consumption data in real time, the optimal AI image quality enhancement calculation strategy is adaptively selected. While ensuring the power consumption and heat generation of the machine, the machine's hardware performance is fully utilized to maximize the AI ​​image quality enhancement effect.

[0161] (3) When switching between multiple resolutions, automatically determine whether the target resolution can meet the computation time required for AI image enhancement, and automatically turn the AI ​​image enhancement function on or off.

[0162] like Figure 8The diagram shows the overall process of this application. This includes: before video frame decoding, obtaining the current frame (i.e., the target image), then determining whether the output resolution has changed relative to the previous frame. If so, the optimal image quality enhancement strategy is readjusted, and it is determined whether image quality enhancement is enabled or disabled, and accordingly, whether to change the execution flag (the execution flag indicates whether the image quality enhancement function is enabled). If the resolution has not changed, or after changing the execution flag, it is determined whether to perform image quality enhancement. If not, the current frame is directly rendered and output. If so, image quality enhancement is performed according to the adjusted enhancement strategy. Afterwards, the enhanced current frame is rendered, and the real-time enhancement computation time data is updated. Then, the super-resolution model time statistics and USM computation time statistics are updated based on the real-time execution time to facilitate subsequent adjustments to the enhancement strategy.

[0163] It should be noted that the final set of regions in the current frame can be generated during video encoding on the server side and obtained through a fusion algorithm. Taking variety show videos as an example, video frames are mainly divided into three categories of image sub-regions: the region of visible human features, the subtitle region, and the logo region.

[0164] 1. Video frame decoding to obtain the current frame: The player decodes the video stream sent by the server to obtain continuous video image data and a set of feature interval sequences for the images, and then uses a fusion algorithm to obtain the final set of regions to be super-resolution calculated.

[0165] 2. Has the resolution changed? If it has changed, the calculation strategy will be adjusted accordingly.

[0166] 3. Whether to perform enhancement calculation: Determine whether to perform AI super-resolution image quality enhancement calculation based on the calculation execution label.

[0167] 4. Perform AI super-resolution image enhancement calculation: Perform AI super-resolution calculation based on the set of regions and the corresponding calculation strategy.

[0168] 5. Update estimated values ​​by performing AI super-resolution image enhancement calculations: Update the statistical values ​​of each model or USM calculation based on the actual time consumed by the AI ​​super-resolution image enhancement calculations.

[0169] 6. Enter Rendering: Replace the current frame with the image frame calculated by super-resolution and enter it into the rendering queue for rendering.

[0170] The following nodes require special explanation:

[0171] (1) Super-resolution region description: The final region set of the current frame is generated during the video encoding on the server and obtained through the fusion algorithm (mainly divided into three categories: human portrait visible feature region, subtitle region, and station logo region).

[0172] (2) AI Super-Resolution Image Enhancement Calculation Strategy: The computational workload of the AI ​​super-resolution model and the USM (Unseen Mean Square) is as follows: super-resolution model corresponding to the visible features of the human face > super-resolution model corresponding to the subtitle area > super-resolution model corresponding to the logo area > USM. Therefore, when the resolution increases and the computational workload of the AI ​​model becomes too large to meet real-time requirements, the USM calculation is replaced by the AI ​​super-resolution model calculation to achieve real-time performance requirements. Conversely, when there is excess performance, the AI ​​super-resolution model is used to replace the USM calculation to ensure the image enhancement effect.

[0173] (3) The values ​​that need to be statistically analyzed in real time are: super-resolution model time g1 corresponding to the visible feature area of ​​the human face, super-resolution model time g2 corresponding to the subtitle area, super-resolution model time g3 corresponding to the logo area, and USM calculation time g4 (the unit of time is ms / s, i.e. the time per unit area), which is the statistical value of the time unit mentioned above.

[0174] like Figure 9 The diagram shows the flowchart for image enhancement. First, the current video frame is decoded and fused to obtain three main regions: a set of regions showing prominent human features, a set of subtitle regions, and a set of station logo regions. Based on the strategy (i.e., image enhancement parameters), an appropriate super-resolution model is selected to perform AI super-resolution calculations or Unseen Mean Square (USM) image enhancement calculations on each region. Then, an image fusion algorithm is used to obtain the final image after image enhancement.

[0175] like Figure 10 The diagram shown is a flowchart of the image quality enhancement feedback update estimate. Wherein:

[0176] 1) Update of super-resolution model statistics g1 corresponding to the human portrait visible feature region: Since the AI ​​super-resolution model or the USM image algorithm directly calculates the human portrait visible feature region, when using the super-resolution model for calculation, the update of the super-resolution model statistics g1 corresponding to the human portrait visible feature region is calculated using the formula g1 = (g1*(n-1) + tn / sn) / n, where n is the nth image enhancement, tn is the time taken for this image enhancement, and sn is the area of ​​this image enhancement. At the start of playback, the initial value of g1 is 0. Here, n is the number of times the super-resolution model corresponding to the human portrait visible feature region is calculated (the statistical value may be different for other models or USM calculations). When using USM calculation, USM calculation statistics are performed (that is, the historical average value is calculated).

[0177] 2) Update of super-resolution model statistics for the subtitle area: Similar to the update of super-resolution model statistics for the visible feature area of ​​the human face, the update of the subtitle model statistics g2 is calculated using the formula g2=(g2*(n-1)+tn / sn) / n. The initial value of g2 is 0 when playback begins.

[0178] 3) Update of super-resolution model statistics for the logo area: Similar to the update of super-resolution model statistics for the visible features area of ​​the human face, the update of the logo model statistics g3 is calculated using the formula g3 = (g3 * (n-1) + tn / sn) / n. The initial value of g3 is 0 at the start of playback.

[0179] 4) USM Calculation Statistics Update: Update the USM calculation statistics g4, calculated using the formula g4 = (g4 * (n-1) + tn / sn) / n. The initial value of g4 is 0 at the start of playback. If multiple regions are involved in USM calculations, multiple updates are sufficient.

[0180] like Figure 11 The diagram shown is a flowchart for calculating the total time required to enhance the image quality of the current frame, where:

[0181] When adjusting the optimal calculation strategy, the time required for image quality enhancement in the current frame is estimated. The estimated time for the visible feature area of ​​the portrait is t1, where s1 is the area of ​​the visible feature area of ​​the portrait in the current frame. Therefore, the estimated value of t1 is either s1*g1 or s1*g4. Similarly, the estimated time for subtitles is t2, and the estimated time for the logo is t3. The total time t = t1 + t2 + t3, or the total time is the maximum value among t1, t2, and t3. The remaining area is relatively simple to calculate, and its time is very small compared to t1, t2, and t3, so it can be ignored.

[0182] like Figure 12 The diagram shown is a flowchart of the logic for readjusting the optimal image quality enhancement strategy, in which:

[0183] Step 1201: When the resolution changes, the policy adjustment logic will be triggered. First, determine if the current state is on. If it is off, proceed to step 1202; otherwise, proceed to step 1203.

[0184] Step 1202: Determine whether the change is larger or smaller.

[0185] Wherein, if the resolution is increased, the process jumps to step 1204. If the resolution is in a decreased state, first, unsharp mask (USM) calculation is used for all three regions to estimate the time consumption and obtain a value t1. Then, the USM calculation is replaced with the AI super-resolution model sequentially in the order of the station logo area, the subtitle area, and the human portrait explicit feature area, and the estimated time consumption t2 (the station logo area uses the AI super-resolution model), t3 (the subtitle area uses the AI super-resolution model), t4, etc. are calculated, wherein t1<t2<t3<t4. Then, judgment is performed among t1, t2, t3, and t4: if t1 is greater than a threshold m (i.e., a target threshold, m is the minimum enabling threshold, which is usually 20 ms), the enabling condition for adjustment is not satisfied, and the process jumps to step 1204. If t1 is less than m, t1, t2, t3, and t4 are judged sequentially to find the value closest to but less than the threshold m, adjustment is performed according to the corresponding calculation strategy, and the process jumps to step 1205.

[0186] Step 1203: judge whether the resolution is increased or decreased this time. If the resolution is increased, first calculate a value t0 estimated by the current calculation strategy; if t0 is greater than a threshold M, sequentially replace the USM calculation in the order of (station logo model, subtitle model, human portrait explicit model) and recalculate a new estimated value t until t is less than or equal to the threshold m, then adjust to the current calculation strategy and return to step 1205; if all recalculated estimated values t after USM calculation for all regions are greater than the threshold m, return to step 1204.

[0187] If the resolution is decreased: if the estimated value t0 is less than or equal to the threshold m, sequentially replace the USM calculation in the current strategy in the order of (human portrait explicit model, subtitle model, station logo model) and reobtain a new estimated value t0; if t0 is greater than the threshold m, update according to the last enhancement strategy and return to step 1205; if t0 is less than the threshold m, continue to replace the strategy until the new estimated value t0 calculated after all regions are replaced with the AI super-resolution model is still less than the threshold m, update the calculation strategy to full calculation by the AI super-resolution model and return to step 1205.

[0188] Step 1204: set super-resolution image quality enhancement to an off state, and end the strategy adjustment.

[0189] Step 1205: set super-resolution image quality enhancement to an on state, and end the strategy adjustment.

[0190] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0191] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0192] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0193] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

Claims

1. A method for controlling image quality enhancement, characterized in that, The method includes: Monitor whether the output resolution of the target image has changed relative to the current image; the current image is the previous frame of the target image. In response to the change in output resolution, image quality enhancement parameters for the target image are obtained; Based on the image quality enhancement parameters of the target image, the image quality of the target image is enhanced, wherein the image quality enhancement parameters of the target image make the time spent enhancing the image quality of the target image less than or equal to the target threshold. The image quality enhancement parameters include the image quality enhancement method used by each image sub-region contained in the target image. The image quality enhancement method includes a first enhancement method or a second enhancement method. The image quality enhancement time of the first enhancement method is greater than that of the second enhancement method, and the image quality enhancement effect of the first enhancement method is better than that of the second enhancement method. The process of obtaining the image quality enhancement parameters of the target image includes: Determine whether image quality enhancement should be performed on the current image; When image quality enhancement is performed on the current image, the image quality enhancement parameters of the target image are obtained in a first acquisition method; the first acquisition method for obtaining the image quality enhancement parameters of the target image includes: determining whether the output resolution of the target image is greater than the output resolution of the current image; If the output resolution of the target image is greater than the output resolution of the current image, at least one first sub-region in the image quality enhancement parameters of the current image is sequentially adjusted from the first enhancement method to the second enhancement method until a candidate enhancement parameter that meets the preferred conditions is obtained; If the output resolution of the target image is less than the output resolution of the current image, at least one second sub-region in the image quality enhancement parameters of the current image is sequentially adjusted from the second enhancement method to the first enhancement method until a candidate enhancement parameter that meets the preferred conditions is obtained; The candidate enhancement parameters that meet the preferred conditions are determined as the image quality enhancement parameters of the target image.

2. The control method according to claim 1, characterized in that, Obtaining the image quality enhancement parameters of the target image further includes: If no image quality enhancement is performed on the current image, the image quality enhancement parameters of the target image are obtained using a second acquisition method. The first acquisition method and the second acquisition method are different.

3. The control method according to claim 1, characterized in that, The area of ​​the first sub-region that was adjusted in the previous adjustment is smaller than the area of ​​the first sub-region that was adjusted in the next adjustment. The area of ​​the second sub-region that was adjusted in the previous adjustment is larger than the area of ​​the second sub-region that was adjusted in the subsequent adjustment.

4. The control method according to claim 2, characterized in that, The image quality enhancement parameters of the target image are obtained using a second acquisition method, including: When the output resolution of the target image is less than the output resolution of the current image, an enhancement judgment result is obtained, wherein the enhancement judgment result indicates whether to perform image quality enhancement on the target image; If the enhancement judgment result indicates that image quality enhancement is performed on the target image, the initial enhancement parameters are adjusted to obtain the image quality enhancement parameters of the target image; In the initial enhancement parameters, each of the image sub-regions uses the second enhancement method.

5. The control method according to claim 4, characterized in that, Adjusting the initial enhancement parameters to obtain the image quality enhancement parameters of the target image includes: At least one third sub-region in the initial enhancement parameters is sequentially adjusted from the second enhancement method to the first enhancement method until candidate enhancement parameters that meet the preferred conditions are obtained; The candidate enhancement parameters that meet the preferred conditions are determined as the image quality enhancement parameters of the target image.

6. The control method according to claim 5, characterized in that, The area of ​​the third sub-region that was previously adjusted is smaller than the area of ​​the third sub-region that was subsequently adjusted.

7. The control method according to claim 1 or 5, characterized in that, The preferred conditions include: the image quality enhancement parameters make the image quality enhancement time of the target image less than or equal to the target threshold, and the image quality enhancement parameters make the image quality enhancement time of the target image the largest among all the candidate enhancement parameters.

8. A control device for enhancing image quality, characterized in that, include: A resolution monitoring unit is used to monitor whether the output resolution of the target image has changed relative to the current image; The current image is the previous frame of the target image; The parameter acquisition unit is used to obtain the image quality enhancement parameters of the target image in response to a change in the output resolution; An enhanced execution unit is configured to enhance the image quality of the target image according to the image quality enhancement parameters of the target image, wherein the image quality enhancement parameters of the target image make the time spent enhancing the image quality of the target image less than or equal to a target threshold. The image quality enhancement parameters include the image quality enhancement method used by each image sub-region contained in the target image. The image quality enhancement method includes a first enhancement method or a second enhancement method. The image quality enhancement time of the first enhancement method is greater than that of the second enhancement method, and the image quality enhancement effect of the first enhancement method is better than that of the second enhancement method. The method of obtaining the image quality enhancement parameters of the target image includes: determining whether the current image has undergone image quality enhancement; if the current image has undergone image quality enhancement, obtaining the image quality enhancement parameters of the target image using a first acquisition method; the method of obtaining the image quality enhancement parameters of the target image using the first acquisition method includes: determining whether the output resolution of the target image is greater than the output resolution of the current image; if the output resolution of the target image is greater than the output resolution of the current image, sequentially adjusting at least one first sub-region of the image quality enhancement parameters of the current image from the first enhancement method to the second enhancement method until a candidate enhancement parameter that meets the preferred conditions is obtained; if the output resolution of the target image is less than the output resolution of the current image, sequentially adjusting at least one second sub-region of the image quality enhancement parameters of the current image from the second enhancement method to the first enhancement method until a candidate enhancement parameter that meets the preferred conditions is obtained; and determining the candidate enhancement parameter that meets the preferred conditions as the image quality enhancement parameters of the target image.

Images