Model training method and apparatus, image quality improvement method and apparatus, and display apparatus

Abstract

A model training method and apparatus, an image quality improvement method and apparatus, and a display apparatus. The model training method comprises: generating a training image set corresponding to one or more image quality display modes, the training image set comprising an original training image and a degraded image corresponding to the original training image; and using a similarity loss function and the training image set to train an image quality algorithm selection model and / or an automatic parameter tuning algorithm model, the image quality algorithm selection model being used for outputting a weight vector corresponding to an image quality algorithm combination, the automatic parameter tuning algorithm model being used for outputting an optimal algorithm parameter list corresponding to the image quality algorithm combination, and the image quality algorithm combination comprising one or more image quality algorithms.

Description

Model training methods, image quality enhancement methods and devices, and display devices Technical Field

[0001] This disclosure relates to, but is not limited to, the field of image processing technology, and particularly to a model training method, an image quality enhancement method and apparatus, and a display device. Background Technology

[0002] As is well known, neural network models can be used to improve image quality. These models are highly adaptable to various scenes; after training on large datasets, they can achieve optimal image quality processing for any scene. However, the timeliness and effectiveness of neural network image processing heavily rely on computing power. For example, the computing power required for real-time processing of 4K or even 8K images by a neural network model is enormous.

[0003] Many image processing chips incorporate mature image quality intellectual property (IP) algorithm modules that can be quickly imported and developed. When faced with a new requirement, different algorithm combinations can quickly achieve the desired effect. However, the image quality IP algorithms in image processing chips often have numerous algorithm parameters and applicable scenarios; a single algorithm parameter cannot provide good processing results for all scenarios. In practical use, manually tuning parameters is not only time-consuming and laborious but also rarely achieves optimal algorithm performance. Furthermore, debugging a single algorithm is already time-consuming and laborious; jointly debugging multiple different image quality IP algorithms becomes even more challenging.

[0004] In summary, neural network models have strong scene adaptability but require powerful computing capabilities. Image quality IP algorithms in image quality chips have strong real-time processing capabilities but insufficient scene adaptability. Summary of the Invention

[0005] The following is an overview of the subject matter described in detail herein. This overview is not intended to limit the scope of the claims.

[0006] This disclosure provides a model training method, comprising: generating a training image set corresponding to one or more image quality display modes, the training image set including original training images and degraded images corresponding to the original training images; training an image quality algorithm selection model and / or an automatic parameter tuning algorithm model using a similarity loss function and the training image set, wherein the image quality algorithm selection model is used to output a weight vector corresponding to a combination of image quality algorithms, and the automatic parameter tuning algorithm model is used to output a list of optimal algorithm parameters corresponding to the combination of image quality algorithms, wherein the combination of image quality algorithms includes one or more image quality algorithms.

[0007] This disclosure also provides a model training apparatus, including a memory; and a processor connected to the memory, the memory being used to store instructions, the processor being configured to execute the steps of the model training method described in any embodiment of this disclosure based on the instructions stored in the memory.

[0008] This disclosure also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the model training method described in any embodiment of this disclosure.

[0009] This disclosure also provides a program product including instructions that, when executed by a computer, perform a model training method as described in any embodiment of this disclosure.

[0010] This disclosure also provides an image quality enhancement method, comprising: acquiring an image to be processed; performing downsampling processing on the image to be processed to obtain a first image; inputting the first image into a pre-trained image quality algorithm selection model and / or an automatic parameter tuning algorithm model, wherein the image quality algorithm selection model is used to output a weight vector corresponding to an image quality algorithm combination, and the automatic parameter tuning algorithm model is used to output a list of optimal algorithm parameters corresponding to the image quality algorithm combination, wherein the image quality algorithm combination includes one or more image quality algorithms; inputting the weight vector and / or the list of optimal algorithm parameters corresponding to the image quality algorithm combination into an image quality chip; and inputting the image to be processed into the image quality chip, so that the image quality chip uses the weight vector and / or the list of optimal algorithm parameters corresponding to the image quality algorithm combination to perform image quality enhancement processing on the image to be processed to obtain an image with enhanced image quality.

[0011] This disclosure also provides an image quality enhancement apparatus, including a memory; and a processor connected to the memory, the memory being used to store instructions, the processor being configured to execute the steps of the image quality enhancement method described in any embodiment of this disclosure based on the instructions stored in the memory.

[0012] This disclosure also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image quality enhancement method described in any embodiment of this disclosure.

[0013] This disclosure also provides a program product including instructions that, when executed by a computer, perform the image quality enhancement method as described in any embodiment of this disclosure.

[0014] This disclosure also provides a display device, including a storage module, a data processing module, and an image quality chip, wherein:

[0015] The storage module is configured to store a pre-trained image quality algorithm selection model and / or an automatic parameter tuning algorithm model. The image quality algorithm selection model is used to output a weight vector corresponding to one or more image quality algorithms, and the automatic parameter tuning algorithm model is used to output a list of optimal algorithm parameters corresponding to the one or more image quality algorithms.

[0016] The data processing module is configured to acquire an image to be processed, perform downsampling processing on the image to be processed to obtain a first image, input the first image into a pre-trained image quality algorithm selection model and / or an automatic parameter tuning algorithm model; and input the image to be processed and the weight vectors corresponding to one or more image quality algorithms output by the image quality algorithm selection model and / or the optimal algorithm parameter list output by the automatic parameter tuning algorithm model into the image quality chip.

[0017] The image quality chip is configured to perform image quality enhancement processing on the image to be processed using the weight vectors and / or the optimal algorithm parameter list corresponding to one or more image quality algorithms, so as to obtain an image with enhanced image quality.

[0018] The model training method, image quality enhancement method and apparatus, and display device of this disclosure generate a training image set corresponding to one or more image quality display modes. The training image set includes original training images and degraded images corresponding to the original training images. A similarity loss function is used to train an image quality algorithm selection model and / or an automatic parameter tuning algorithm model with the training image set. The image quality algorithm selection model is used to output a weight vector corresponding to the combination of image quality algorithms. The automatic parameter tuning algorithm model is used to output a list of optimal algorithm parameters corresponding to the combination of image quality algorithms. The combination of image quality algorithms includes one or more image quality algorithms, which combines the advantages of strong scene adaptability of neural network models and strong real-time processing capability of image quality algorithms. The entire system can automatically adjust the image quality of the current display screen according to the user-defined image quality display mode to achieve the best display effect.

[0019] Other features and advantages of this disclosure will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the disclosure. Other advantages of this disclosure may be realized and obtained by means of the methods described in the description and the accompanying drawings. Attached Figure Description

[0020] The accompanying drawings are used to provide an understanding of the technical solutions of this disclosure and form part of the specification. They are used together with the embodiments of this disclosure to explain the technical solutions of this disclosure and do not constitute a limitation on the technical solutions of this disclosure.

[0021] Figure 1 is a flowchart illustrating a model training method provided by an exemplary embodiment of this disclosure.

[0022] Figure 2 is a schematic diagram of the original training images in a training image set provided by an exemplary embodiment of this disclosure.

[0023] Figure 3 is a schematic diagram of a method for degrading original training images provided by an exemplary embodiment of this disclosure.

[0024] Figure 4 is a schematic diagram of the training process of an image quality algorithm selection model provided by an exemplary embodiment of this disclosure in a general manner.

[0025] Figure 5 is a schematic diagram of the training process of an automatic parameter tuning algorithm model provided by an exemplary embodiment of this disclosure in a general manner.

[0026] Figure 6 is a flowchart illustrating an image quality enhancement method provided by an exemplary embodiment of this disclosure.

[0027] Figure 7 is a flowchart illustrating another image quality enhancement method provided by an exemplary embodiment of this disclosure.

[0028] Figure 8 is a schematic diagram of the reasoning process of an image quality algorithm selection model provided by an exemplary embodiment of this disclosure.

[0029] Figure 9 is a schematic diagram of the reasoning process of an automatic parameter tuning algorithm model provided by an exemplary embodiment of this disclosure.

[0030] Figure 10 is a schematic diagram of the structure of a display device provided by an exemplary embodiment of the present disclosure.

[0031] Figure 11 is a schematic diagram of the structure of a model training device provided in an exemplary embodiment of this disclosure.

[0032] Figure 12 is a schematic diagram of the structure of an image quality enhancement device provided by an exemplary embodiment of the present disclosure. Detailed Implementation

[0033] This disclosure describes several embodiments, but these descriptions are exemplary and not limiting, and it will be apparent to those skilled in the art that many more embodiments and implementations are possible within the scope of the embodiments described herein. Although many possible combinations of features are shown in the drawings and discussed in the detailed description, many other combinations of the disclosed features are also possible. Unless specifically limited, any feature or element of any embodiment may be used in combination with, or may replace, any feature or element of any other embodiment.

[0034] This disclosure includes and contemplates combinations of features and elements known to those skilled in the art. The embodiments, features, and elements disclosed in this disclosure may also be combined with any conventional features or elements to form a unique inventive scheme as defined by the claims. Any feature or element of any embodiment may also be combined with features or elements from other inventive schemes to form another unique inventive scheme as defined by the claims. Therefore, it should be understood that any feature shown and / or discussed in this disclosure may be implemented individually or in any suitable combination. Therefore, the embodiments are not limited except by the limitations imposed by the appended claims and their equivalents. Furthermore, various modifications and changes may be made within the scope of the appended claims.

[0035] Furthermore, in describing representative embodiments, the specification may have presented methods and / or processes as a specific sequence of steps. However, the method or process should not be limited to the specific order of steps described herein, to the extent that the method or process does not depend on the specific order of steps described herein. As will be understood by those skilled in the art, other sequences of steps are also possible. Therefore, the specific order of steps set forth in the specification should not be construed as a limitation of the claims. Moreover, the claims relating to the method and / or process should not be limited to the steps performed in the order written, and those skilled in the art will readily understand that these orders can be varied and still remain within the spirit and scope of the embodiments disclosed herein.

[0036] As shown in Figure 1, this embodiment of the present disclosure provides a model training method, including:

[0037] Step 101: Generate a training image set corresponding to one or more image quality display modes. The training image set includes the original training images and the degraded images corresponding to the original training images.

[0038] Step 102: Use the similarity loss function and the training image set to train the image quality algorithm selection model and / or the automatic parameter tuning algorithm model. The image quality algorithm selection model is used to output the weight vector corresponding to the combination of image quality algorithms, and the automatic parameter tuning algorithm model is used to output the optimal algorithm parameter list corresponding to the combination of image quality algorithms. The combination of image quality algorithms includes one or more image quality algorithms.

[0039] The model training method of this disclosure generates a training image set corresponding to one or more image quality display modes. The training image set includes original training images and degraded images corresponding to the original training images. A similarity loss function is used to train an image quality algorithm selection model and / or an automatic parameter tuning algorithm model with the training image set. The image quality algorithm selection model is used to output the weight vector corresponding to the combination of image quality algorithms, and the automatic parameter tuning algorithm model is used to output the optimal algorithm parameter list corresponding to the combination of image quality algorithms. This method combines the advantages of strong scene adaptability of neural network models and strong real-time processing capability of image quality algorithms. The entire system can automatically adjust the image quality of the current display screen according to the user-defined image quality display mode to achieve the best display effect.

[0040] In this embodiment of the disclosure, the images in the training image set can be used for the training process of the image quality algorithm selection model and / or the automatic parameter tuning algorithm model, and can also be used for the testing process and / or verification process of the image quality algorithm selection model and / or the automatic parameter tuning algorithm model. This disclosure does not limit this.

[0041] In this embodiment of the disclosure, the multiple image quality algorithms may include, but are not limited to, the following image processing algorithms:

[0042] (1) Gamma correction algorithm: Adjust the Gamma value of the image to match the non-linear perception of brightness by the human eye.

[0043] (2) Histogram equalization algorithm: Adjust the contrast of the image to make the brightness distribution of the image more uniform.

[0044] (3) Color correction algorithm: Adjust the color balance of the image, including color temperature and color saturation.

[0045] (4) Image sharpening algorithm: enhances the edges of the image and improves the clarity of details.

[0046] (5) Denoising algorithm: Reduce noise in images, especially images taken under low light conditions.

[0047] (6) Detail enhancement algorithm: Enhances the details in the image, making the image look clearer.

[0048] (7) Dynamic contrast enhancement algorithm: dynamically adjusts the contrast of the image to adapt to different viewing environments.

[0049] (8) Backlight control algorithm: For backlit LCD screens, control the brightness of the backlight to improve contrast and save energy.

[0050] (9) Motion estimation and motion compensation (MEMC) algorithm: improves the smoothness of dynamic images and reduces blur and ghosting.

[0051] (10) Super-resolution algorithm: Upgrades low-resolution images to high-resolution images to improve image clarity.

[0052] (11) Local contrast enhancement algorithm: enhances local areas of the image to make the image more vivid.

[0053] (12) Region noise reduction algorithm: Denoising is performed on specific regions of the image, such as the sky or smooth areas.

[0054] (13) Skin color correction algorithm: ensures that skin color remains natural under different lighting conditions.

[0055] (14) Scene detection algorithm: Identify different scenes in the image and apply appropriate image quality enhancement algorithms.

[0056] (15) HDR (High Dynamic Range) processing algorithm: expands the brightness and color range of the image, providing a richer visual experience.

[0057] (16) AI image quality enhancement algorithm: Uses artificial intelligence (AI) algorithms to automatically adjust image quality parameters to adapt to different content and viewing conditions.

[0058] (17) Frame rate conversion algorithm: convert the original frame rate into different output frame rates, such as improving smoothness through frame interpolation technology.

[0059] (18) Polarity reversal algorithm: Reduces LCD screen flicker and improves visual comfort.

[0060] (19) 3D noise reduction algorithm: Reduces image noise in 3D display and improves the viewing experience.

[0061] (20) Screen refresh rate enhancement algorithm: The screen refresh rate is improved by algorithm to make dynamic images smoother.

[0062] In this embodiment, the design and implementation of the image quality chip depends on the requirements of the target application. Manufacturers may choose to integrate some or all of the image quality algorithms into the image quality chip to provide optimal image quality. As technology advances, new image quality algorithms can also be continuously developed and integrated into the image quality chip.

[0063] In this embodiment of the disclosure, the image quality algorithm can be developed using a Field Programmable Gate Array (FPGA).

[0064] Many image processing chips can process images of any resolution in real time because the image processing algorithms within these chips are typically developed using FPGAs. The primary reason for using FPGAs to develop image processing algorithms for image processing chips is that FPGAs offer several advantages: strong parallel processing capabilities, excellent real-time performance, customizable design, high flexibility and scalability, low latency, high throughput, hardware acceleration, cost-effectiveness, optimized power consumption, rapid prototyping, strong adaptability, and high security. Developing image processing algorithms using FPGAs combines the strengths of both hardware and software to provide display systems with high-performance, high-efficiency, and customizable solutions.

[0065] In this embodiment, the image quality display mode may include, but is not limited to, high-definition mode, vivid color mode, low-power mode, etc., and this disclosure does not impose any limitations on this. Users can choose different image quality display modes according to their preferences. The image quality algorithm selection model and / or automatic parameter tuning algorithm model are trained using training image sets corresponding to one or more image quality display modes. The image quality algorithm selection model can output a corresponding combination of image quality algorithms and its corresponding weight vector based on the input image quality display mode and the input image.

[0066] In some exemplary embodiments, the image quality algorithm selection model and / or the automatic parameter tuning algorithm model can both be neural network model structures. For example, the image quality algorithm selection model and / or the automatic parameter tuning algorithm model can be different deep neural network structures such as Convolutional Neural Networks (CNN), Transformer, and Mamba; however, this disclosure is not limited thereto. For example, the image quality algorithm selection model and / or the automatic parameter tuning algorithm model can use image classification model structures, such as the ResNet network model structure.

[0067] In this embodiment of the disclosure, the similarity loss function is a loss function used to measure the similarity between two images. It reflects the perceptual quality of an image better than the traditional mean squared error (MSE) and mean absolute error (MAE).

[0068] In some examples, the Structural Similarity Index (SSIM) may be used as a similarity loss function; however, this disclosure does not limit it.

[0069] For example, the structural similarity index between two images can be calculated based on the following three factors, which are more consistent with human visual perception. The similarity between the two images can be assessed by using the calculated structural similarity index:

[0070] (1) Brightness: Evaluate whether the brightness and darkness features and brightness components of the two images are consistent.

[0071] (2) Contrast: Evaluate the degree of brightness difference between different areas in two images.

[0072] (3) Structure: Evaluate the texture and detail in the two images, as well as the relative positional differences between pixels.

[0073] In some exemplary embodiments, when the image quality algorithm combination includes multiple image quality algorithms, each image quality algorithm corresponds to a weight parameter, and the weight parameters corresponding to multiple image quality algorithms form a weight vector.

[0074] In this embodiment of the disclosure, the sum of the weight parameters corresponding to multiple image quality algorithms can be 1; however, this disclosure does not limit this.

[0075] In this embodiment of the disclosure, the image quality algorithm selection model is used to output a weight vector, which includes weight parameters corresponding to one or more image quality algorithms. The larger the weight parameter corresponding to an image quality algorithm, the greater the role that image quality algorithm plays in the image quality enhancement process; if the weight parameter corresponding to an image quality algorithm is 0, it means that the image quality algorithm does not play a role in the image quality enhancement process.

[0076] For example, suppose a certain image quality algorithm combination includes three image quality algorithms, and the weight vector output by the image quality algorithm selection model is [0.2, 0.5, 0.3], where 0.2, 0.5, and 0.3 are the weight parameters corresponding to image quality algorithm 1, image quality algorithm 2, and image quality algorithm 3, respectively. Then, the processing of the input image by this image quality algorithm combination can be expressed by the following formula: Output image = Normalized {0.2 × Image quality algorithm 1 (input image) + 0.5 × Image quality algorithm 2 (input image) + 0.3 × Image quality algorithm 3 (input image)} 0-255 .

[0077] In this context, image quality algorithm 1 (input image) represents the pixel grayscale value of the output image obtained by processing the input image using image quality algorithm 1, image quality algorithm 2 (input image) represents the pixel grayscale value of the output image obtained by processing the input image using image quality algorithm 2, and image quality algorithm 3 (input image) represents the pixel grayscale value of the output image obtained by processing the input image using image quality algorithm 3. Through normalization processing, the pixel grayscale value of the final output image is stretched to between 0 and 255.

[0078] In this embodiment of the disclosure, normalization refers to linearly mapping the grayscale values ​​of an image to the range of 0 to 255. The specific steps are as follows: First, read the image data to be processed; find the maximum and minimum grayscale values ​​in the image; based on the maximum and minimum values, perform a linear transformation on the grayscale value of each pixel in the image to map it to the range of 0 to 255.

[0079] In this embodiment of the disclosure, the optimal algorithm parameter list is a set of optimal algorithm parameters corresponding to each image quality algorithm in the image quality algorithm combination.

[0080] In this embodiment of the disclosure, the images in the training image set can be divided into multiple different use scenarios, each use scenario includes multiple images, and any two images must satisfy at least one of the following: different colors, different textures, and different brightness.

[0081] To ensure that the final image quality algorithm selection model and / or automatic parameter tuning algorithm model are adaptable to most everyday usage scenarios, the model should be trained using training image sets from different usage scenarios from the initial design stage. The training image set should be small and high-quality, with a small data volume. Too much data is extremely time-consuming during model training. Furthermore, the training images should be carefully selected to cover as many scenarios as possible, and the image data for different usage scenarios should cover different colors, textures, brightness, etc. In addition, since the training image set needs to undergo degradation operations to form training image pairs, the original training images in the training image set need to be high-quality images (high resolution, wide color gamut, low compression ratio, etc.).

[0082] For example, as shown in Figure 2, this disclosure selects seven usage scenarios: people, animals, buildings, landscapes, food, daily life scenes, and documents. Five images are selected for each usage scenario as the original training images. In other examples, other combinations of scenarios and the number of images may be used, and this disclosure does not limit this.

[0083] In some exemplary embodiments, the degraded image is an image obtained by degrading the original training image. The image degradation process may include at least one of the following: reducing color gamut, reducing sharpness, reducing brightness, increasing noise, reducing contrast, image compression, image distortion, image smearing, etc.

[0084] Training the model requires training image pairs, as shown in Figure 3. In this disclosure, the training image pairs are obtained through image degradation. Different image quality display modes typically have different image processing effects. For example, to train an image quality algorithm selection model and / or an automatic parameter tuning algorithm model corresponding to a color-enhanced image quality display mode, the original training image needs to be processed by reducing its color gamut to form a degraded image. Thus, when training the image quality algorithm selection model and / or the automatic parameter tuning algorithm model, the low-color-gamut degraded image is used as the input image, and the high-color-gamut original training image is used as the ground truth image, thereby training the color-enhanced image quality algorithm selection model and / or the automatic parameter tuning algorithm model. When the image quality display mode includes multiple image quality requirements, the corresponding image degradation processing can include multiple corresponding degradation processing operations.

[0085] In some exemplary implementations, as shown in Figure 4, the image quality algorithm selection model is trained using the following method:

[0086] The training process involves multiple iterations, each including: downsampling the original training image and the degraded image to obtain the downsampled original training image and the downsampled degraded image; inputting the downsampled degraded image into the image quality algorithm selection model, constructing an image quality algorithm combination based on the weight vector output by the image quality algorithm selection model, inputting the degraded image from the training image set into the image quality algorithm combination, downsampling the output image of the image quality algorithm combination or the first processed image obtained from the output image of the image quality algorithm combination to obtain the downsampled output image; calculating the similarity loss between the downsampled original training image and the downsampled output image based on the similarity loss function, and adjusting the model parameters of the image quality algorithm selection model based on the calculated similarity loss result.

[0087] In some exemplary embodiments, as shown in Figure 4, at least one round of training further includes: inputting the image quality display mode into the image quality algorithm selection model when inputting the downsampled degraded image into the image quality algorithm selection model. In this way, the trained image quality algorithm selection model can output the weight vector corresponding to the optimal combination of image quality algorithms based on the image quality display mode and the downsampled input image.

[0088] In some exemplary embodiments, as shown in Figure 5, the automatic parameter tuning algorithm model is trained using the following method:

[0089] The training process involves multiple iterations, each including: downsampling the original training image and the degraded image to obtain the downsampled original training image and the downsampled degraded image; inputting the downsampled degraded image into the automatic parameter tuning algorithm model; inputting the parameter list output by the automatic parameter tuning algorithm model and the degraded images in the training image set into the image quality algorithm combination; downsampling the output image of the image quality algorithm combination or the first processed image obtained from the output image of the image quality algorithm combination to obtain the downsampled output image; calculating the similarity loss between the downsampled original training image and the downsampled output image based on the similarity loss function; and adjusting the model parameters of the automatic parameter tuning algorithm model based on the calculated similarity loss result.

[0090] The model training method of this disclosure reduces the computing power requirements of the model training device and also reduces the computing power requirements of the display device that uses the downsampled degraded image as the input image of the image quality algorithm selection model and / or the automatic parameter tuning algorithm model during the model training process, and calculates the similarity loss between the downsampled original training image and the downsampled output image.

[0091] In other examples, when the computing power of the model training device and the display device is large enough, the degraded image can be directly used as the input image of the image quality algorithm selection model and / or the automatic parameter tuning algorithm model. In addition, when calculating the similarity loss, it can be calculated directly on the output image of the combination of the original training image and the image quality algorithm, or on the first processed image obtained by the combination of the original training image and the output image of the image quality algorithm.

[0092] In this embodiment of the disclosure, the first processed image obtained from the output image output by the image quality algorithm combination is specifically: after the output image output by the image quality algorithm combination is displayed on a preset display module, a screen capture image is obtained by using a camera or other image acquisition device to capture the image.

[0093] In this embodiment of the present disclosure, the first processed image obtained from the output image output by the image quality algorithm combination can also be an image obtained by processing the output image output by the image quality algorithm combination in other ways, and the present disclosure does not limit it in this way.

[0094] The image quality algorithm selection model and / or automatic parameter tuning algorithm model trained in the embodiments of this disclosure can be adapted to all display modules (general approach) or specifically adapted to a particular display module (module-specific tuning approach). The difference between the two adaptation methods lies in the different training image sets used when training the image quality algorithm selection model and the automatic parameter tuning algorithm model.

[0095] In the general approach, the original training images in the training image set do not need to be output to a specific display module for display and undergo screen capture processing; the original training images in the general approach are the original images. The general approach is compatible with all display modules. Therefore, in this embodiment, under the general approach, the input image for training the image quality algorithm selection model and / or the automatic parameter tuning algorithm model is a degraded version of the original image, and the corresponding ground truth image is the original image.

[0096] The original training images in the training image set corresponding to the module-based image tuning method are the original screen capture images obtained after the original images are output to a specific display module and then processed by screen capture. The module-based image tuning method refers to adaptation for a specific display module; therefore, the image quality algorithm selection model and / or automatic parameter tuning algorithm model are only trained for that specific display module. Since different display modules have different display effects, and industrial cameras can capture photos of the display effects of specific display modules, the display effects of different display modules can be distinguished from each other. Therefore, in this embodiment of the disclosure, under the module-based image tuning method, the input image for training the image quality algorithm selection model and / or automatic parameter tuning algorithm model is the original screen capture image after degradation processing, and the corresponding ground truth image is the original screen capture image.

[0097] In some exemplary embodiments, an automatic parameter tuning algorithm model is trained using a similarity loss function and a training image set, including:

[0098] Construct a combination of image quality algorithms based on the weight vector output by the image quality algorithm selection model;

[0099] The automatic parameter tuning algorithm model is trained using a similarity loss function, a combination of image quality algorithms, and a training image set.

[0100] In some exemplary embodiments, an automatic parameter tuning algorithm model is trained using a similarity loss function and a training image set, including:

[0101] Determine the image quality algorithm combination corresponding to one or more image quality display modes;

[0102] The automatic parameter tuning algorithm model is trained using a similarity loss function, a combination of image quality algorithms, and a training image set.

[0103] In this embodiment, the display device using an image quality algorithm selection model and / or an automatic parameter tuning algorithm model can include two different usage modes: manufacturer mode and user mode. In manufacturer mode, some functional settings are completed before shipment. For example, in one manufacturer mode, the image quality algorithm combination corresponding to one or more image quality display modes and the corresponding weight vector can be embedded in the image quality chip. The display device can integrate an automatic parameter tuning algorithm model. When a user uses a certain image quality display mode, the display device automatically switches to the corresponding image quality algorithm combination and uses the optimal algorithm parameter list output by the automatic parameter tuning algorithm model to process the current display screen. In this case, the selected image quality algorithm combination and weight vector are the same for all display screens. Alternatively, in another manufacturer mode, the display device may not integrate either an image quality algorithm selection model or an automatic parameter tuning algorithm model. It may only embed the image quality algorithm combination corresponding to one or more image quality display modes, the corresponding weight vector, and the optimal algorithm parameter list in the image quality chip. When a user uses a certain image quality display mode, the display device automatically switches to the corresponding image quality algorithm combination and uses the corresponding optimal algorithm parameter list to process the current display screen. At this point, the selected image quality algorithm combination, weight vector, and optimal algorithm parameter list are the same for all displayed screens. The advantages of using the vendor model are less resource consumption, faster processing speed, and lower cost.

[0104] In user mode, the display device integrates all image quality algorithms, image quality algorithm selection models, and automatic parameter tuning algorithm models. When a user selects a certain image quality display mode, the image quality algorithm selection model and the automatic parameter tuning algorithm model can generate, in real time, the corresponding image quality algorithm combination and weight vector, as well as a list of optimal algorithm parameters corresponding to that combination, based on the current display screen. The advantage of using user mode is that it can automatically adjust processing parameters according to the display screen, thereby achieving the best processing effect.

[0105] As shown in Figure 6, this embodiment of the present disclosure also provides a method for improving image quality, including:

[0106] Step 601: Obtain the image to be processed, and perform downsampling processing on the image to be processed to obtain the first image;

[0107] Step 602: Input the first image into the pre-trained image quality algorithm selection model and / or automatic parameter tuning algorithm model. The image quality algorithm selection model is used to output the weight vectors corresponding to one or more image quality algorithms. The automatic parameter tuning algorithm model is used to output the optimal algorithm parameter list corresponding to the one or more image quality algorithms. Input the weight vectors and / or optimal algorithm parameter list corresponding to one or more image quality algorithms into the image quality chip.

[0108] Step 603: Input the image to be processed into the image quality chip so that the image quality chip uses the weight vectors corresponding to one or more image quality algorithms and / or the optimal algorithm parameter list to perform image quality enhancement processing on the image to be processed, and obtain the image with enhanced image quality.

[0109] The image quality enhancement method of this disclosure reduces the computational load of the image quality algorithm selection model and / or automatic parameter tuning algorithm model by downsampling the image to be processed. The downsampled first image is input into a pre-trained image quality algorithm selection model and / or automatic parameter tuning algorithm model to obtain weight vectors and / or optimal algorithm parameter lists for one or more image quality algorithms. The image to be processed, along with the weight vectors and / or optimal algorithm parameter lists for one or more image quality algorithms, is then input into an image quality chip. This allows the image quality chip to perform image quality enhancement processing on the image to be processed using the weight vectors and / or optimal algorithm parameter lists for one or more image quality algorithms, resulting in an enhanced image. This method combines the advantages of strong scene adaptability of neural network models and strong real-time processing capabilities of image quality algorithms. The entire system can automatically adjust the image quality of the current display screen according to user-defined image quality requirements to achieve the best display effect.

[0110] In some exemplary embodiments, the image quality algorithm selection model and / or the automatic parameter tuning algorithm model can both be neural network model structures. For example, the image quality algorithm selection model and / or the automatic parameter tuning algorithm model can be different deep neural network structures such as convolutional neural network structures, Transformer structures, and Mamba structures; however, this disclosure is not limited thereto. For example, the image quality algorithm selection model and / or the automatic parameter tuning algorithm model can use image classification model structures, such as the ResNet network model structure.

[0111] In some exemplary embodiments, inputting the first image into a pre-trained image quality algorithm selection model includes: determining the current image quality display mode, and inputting the current image quality display mode and the first image into the pre-trained image quality algorithm selection model.

[0112] In this embodiment, the image quality display mode may include, but is not limited to, high-definition mode, vivid color mode, and low-power mode. Users can select different image quality display modes according to their preferences. The image quality algorithm selection model and / or automatic parameter tuning algorithm model are trained using training image sets corresponding to one or more image quality display modes. The image quality algorithm selection model can output a corresponding combination of image quality algorithms and its corresponding weight vector based on the input image quality display mode and the input image.

[0113] In some exemplary embodiments, the method further includes determining the current image quality algorithm combination before inputting the first image into a pre-trained automatic parameter tuning algorithm model.

[0114] In this embodiment of the disclosure, the display device applying the image quality enhancement method may include two different usage modes: manufacturer mode and user mode. In manufacturer mode, some functional settings are completed before leaving the factory. For example, in one manufacturer mode, the image quality algorithm combination corresponding to one or more image quality display modes and the corresponding weight vector can be embedded in the image quality chip, and the display device can integrate an automatic parameter tuning algorithm model. When the user uses a certain image quality display mode, the display device automatically switches to the corresponding image quality algorithm combination and uses the optimal algorithm parameter list output by the automatic parameter tuning algorithm model to process the current display screen. At this time, the selected image quality algorithm combination and weight vector are the same for all display screens. Alternatively, in another manufacturer mode, the display device may not integrate either the image quality algorithm selection model or the automatic parameter tuning algorithm model, but only embed the image quality algorithm combination corresponding to one or more image quality display modes, the corresponding weight vector, and the optimal algorithm parameter list in the image quality chip. When the user uses a certain image quality display mode, the display device automatically switches to the corresponding image quality algorithm combination and uses the corresponding optimal algorithm parameter list to process the current display screen. At this point, the selected image quality algorithm combination, weight vector, and optimal algorithm parameter list are the same for all displayed screens. The advantages of using the vendor model are less resource consumption, faster processing speed, and lower cost.

[0115] In user mode, the display device integrates all image quality algorithms, image quality algorithm selection models, and automatic parameter tuning algorithm models. When a user selects a certain image quality display mode, the image quality algorithm selection model and the automatic parameter tuning algorithm model can generate, in real time, the corresponding image quality algorithm combination and weight vector, as well as a list of optimal algorithm parameters corresponding to that combination, based on the current display screen. The advantage of using user mode is that it can automatically adjust processing parameters according to the display screen, thereby achieving the best processing effect.

[0116] In some exemplary embodiments, when the image quality algorithm combination includes multiple image quality algorithms, each image quality algorithm corresponds to a weight parameter, and the weight parameters corresponding to multiple image quality algorithms form a weight vector.

[0117] In this embodiment of the disclosure, the sum of the weight parameters corresponding to multiple image quality algorithms can be 1; however, this disclosure does not limit this.

[0118] In this embodiment of the disclosure, the image quality algorithm selection model is used to output a weight vector, which includes weight parameters corresponding to one or more image quality algorithms. The larger the weight parameter corresponding to an image quality algorithm, the greater the role that image quality algorithm plays in the image quality enhancement process; if the weight parameter corresponding to an image quality algorithm is 0, it means that the image quality algorithm does not play a role in the image quality enhancement process.

[0119] For example, suppose a certain image quality algorithm combination includes three image quality algorithms, and the weight vector output by the image quality algorithm selection model is [0.2, 0.5, 0.3], where 0.2, 0.5, and 0.3 are the weight parameters corresponding to image quality algorithm 1, image quality algorithm 2, and image quality algorithm 3, respectively. Then, the processing of the input image by this image quality algorithm combination can be expressed by the following formula: Output image = Normalized {0.2 × Image quality algorithm 1 (input image) + 0.5 × Image quality algorithm 2 (input image) + 0.3 × Image quality algorithm 3 (input image)} 0-255 .

[0120] In this context, image quality algorithm 1 (input image) represents the pixel grayscale value of the output image obtained by processing the input image using image quality algorithm 1, image quality algorithm 2 (input image) represents the pixel grayscale value of the output image obtained by processing the input image using image quality algorithm 2, and image quality algorithm 3 (input image) represents the pixel grayscale value of the output image obtained by processing the input image using image quality algorithm 3. Through normalization processing, the pixel grayscale value of the final output image is stretched to between 0 and 255.

[0121] In this embodiment of the disclosure, the optimal algorithm parameter list is a set of optimal algorithm parameters corresponding to each image quality algorithm in the image quality algorithm combination.

[0122] In this embodiment of the disclosure, when the display device only includes an image quality algorithm selection model and does not include an automatic parameter tuning algorithm model, the image quality chip can use one or more image quality algorithms selected by the image quality algorithm selection model that correspond to the current image quality display mode to construct an image quality algorithm combination, and use pre-set standard parameters (or default parameters) for each image quality algorithm in the image quality algorithm combination to perform image quality enhancement processing on the image to be processed, so as to obtain an image with enhanced image quality.

[0123] In this embodiment of the disclosure, when the display device only includes an automatic parameter tuning algorithm model and does not include an image quality algorithm selection model, the display device can pre-store one or more image quality display modes corresponding to image quality algorithm combinations. The image quality chip can obtain the corresponding image quality algorithm combination according to the current image quality display mode, and then import the optimal algorithm parameter list corresponding to one or more image quality algorithms output by the automatic parameter tuning algorithm model to perform image quality enhancement processing on the image to be processed, thereby obtaining the image with enhanced image quality.

[0124] In this embodiment of the disclosure, when the display device includes both an image quality algorithm selection model and an automatic parameter tuning algorithm model, the display device can first use one or more image quality algorithms selected by the image quality algorithm selection model that correspond to the current image quality display mode to construct an image quality algorithm combination, and then import the optimal algorithm parameter list corresponding to one or more image quality algorithms output by the automatic parameter tuning algorithm model to perform image quality enhancement processing on the image to be processed, thereby obtaining an image with enhanced image quality.

[0125] In some exemplary embodiments, as shown in FIG7, when using the image quality enhancement method of the present disclosure to enhance the image quality of a display device, the process is as follows:

[0126] (I) Users can set the corresponding image quality display mode according to their needs.

[0127] (II) Downsample the current image to be processed to obtain the first image. By downsampling the image to be processed, the size of the input model image is reduced, thereby greatly reducing the computational load of the model. Therefore, even a GPU chip with low computing power can meet the requirements, thus enabling real-time processing on the terminal.

[0128] (III) Image quality algorithm selection model and automatic parameter tuning algorithm model: Based on the input first image, the model automatically generates the combination of image quality algorithms in the image quality chip and the list of optimal algorithm parameters for each image quality algorithm in the combination, and imports the selected combination of image quality algorithms and the list of optimal algorithm parameters into the image quality chip.

[0129] (IV) The image quality chip selects the combination of image quality algorithms according to the input parameters, imports the corresponding optimal algorithm parameter list, and then uses the combination of image quality algorithms and the optimal algorithm parameter list to process the current image to be processed, so as to obtain the image with improved image quality.

[0130] Before using the image quality enhancement method of the present disclosure embodiments, an image quality algorithm selection model and / or automatic parameter tuning algorithm model can be trained by an external model training device, and the trained image quality algorithm selection model and / or automatic parameter tuning algorithm model can be deployed to a display device. However, the present disclosure does not limit this.

[0131] In some exemplary implementations, the image quality algorithm selection model can be trained for one or more different image quality display modes.

[0132] In some exemplary embodiments, the automatic parameter tuning algorithm model can be trained for one or more different combinations of image quality algorithms, each combination of image quality algorithms including one or more image quality algorithms.

[0133] In some exemplary embodiments, the method further includes, prior to:

[0134] Generate a training image set corresponding to one or more image quality display modes. The training image set includes the original training images and the degraded images corresponding to the original training images.

[0135] Use a similarity loss function and a training image set to train the image quality algorithm selection model and / or the automatic parameter tuning algorithm model.

[0136] In this embodiment of the disclosure, the images in the training image set can be used in the training process of the image quality algorithm selection model and / or the automatic parameter tuning algorithm model, and can also be used in the testing process and / or verification process of the image quality algorithm selection model and / or the automatic parameter tuning algorithm model. This disclosure does not limit this. The method for generating the training image set can refer to the foregoing description, and will not be repeated here.

[0137] In some exemplary embodiments, the method further includes, prior to:

[0138] Determine the image quality algorithm combination, which includes one or more image quality algorithms;

[0139] Generate a training image set corresponding to one or more image quality display modes. The training image set includes the original training images and the degraded images corresponding to the original training images.

[0140] The automatic parameter tuning algorithm model is trained using a similarity loss function, a combination of image quality algorithms, and a training image set.

[0141] In this embodiment of the disclosure, the image quality algorithm combination used when training the automatic parameter tuning algorithm model can be obtained by the following two methods: (1) The image quality algorithm selection model can be trained according to the image quality display mode, and the image quality algorithm combination can be constructed according to the weight vector output by the image quality algorithm selection model; (2) The corresponding image quality algorithm combination can be directly determined according to the image quality display mode.

[0142] The image quality enhancement in this disclosure can be either adapted to all display modules (a general approach) or specifically adapted to particular display modules (a module-based tuning approach). The difference between the two adaptation methods lies in the different training image sets used when training the image quality algorithm selection model and the automatic parameter tuning algorithm model.

[0143] In the general approach, the original training images in the training image set do not need to be output to a specific display module for display and undergo screen capture processing; the original training images in the general approach are the original images. The general approach is compatible with all display modules. Therefore, in this embodiment, under the general approach, the input image for training the image quality algorithm selection model and / or the automatic parameter tuning algorithm model is a degraded version of the original image, and the corresponding ground truth image is the original image.

[0144] In some exemplary implementations, the image quality algorithm selection model is trained in a general manner using the following method:

[0145] The training process involves multiple iterations, each including: downsampling the original training image and the degraded image to obtain the downsampled original training image and the downsampled degraded image; inputting the downsampled degraded image into the image quality algorithm selection model, constructing an image quality algorithm combination based on the weight vector output by the image quality algorithm selection model, inputting the degraded images from the training image set into the image quality algorithm combination, downsampling the output image of the image quality algorithm combination to obtain the downsampled output image; calculating the similarity loss between the downsampled original training image and the downsampled output image based on the similarity loss function, and adjusting the model parameters of the image quality algorithm selection model based on the calculated similarity loss result.

[0146] The original training images in the training image set corresponding to the module-based image tuning method are the original screen capture images obtained after the original images are output to a specific display module and then processed by screen capture. The module-based image tuning method refers to adaptation for a specific display module; therefore, the image quality algorithm selection model and / or automatic parameter tuning algorithm model are only trained for that specific display module. Since different display modules have different display effects, and industrial cameras can capture photos of the display effects of specific display modules, the display effects of different display modules can be distinguished from each other. Therefore, in this embodiment of the disclosure, under the module-based image tuning method, the input image for training the image quality algorithm selection model and / or automatic parameter tuning algorithm model is the original screen capture image after degradation processing, and the corresponding ground truth image is the original screen capture image.

[0147] In other exemplary embodiments, under the module-based tuning mode, the image quality algorithm selection model is trained using the following method:

[0148] The training process involves multiple iterations, each including: downsampling the original training image and the degraded image to obtain the downsampled original training image and the downsampled degraded image; inputting the downsampled degraded image into the image quality algorithm selection model, constructing an image quality algorithm combination based on the weight vector output by the image quality algorithm selection model, inputting the degraded image from the training image set into the image quality algorithm combination, outputting the image from the image quality algorithm combination to a preset display module to obtain a screen capture image, downsampling the screen capture image to obtain the downsampled screen capture image; calculating the similarity loss between the downsampled original training image and the downsampled screen capture image based on the similarity loss function, and adjusting the model parameters of the image quality algorithm selection model based on the calculated similarity loss result.

[0149] In this embodiment, the first step in model parameter tuning is to select a suitable combination of image quality algorithms. Different image quality algorithms have different functions. For example, histogram equalization is used to enhance image contrast; denoising algorithms such as mean filtering, median filtering, and Gaussian filtering are used to reduce noise in images; and color equalization algorithms are used to adjust the colors of images to achieve color and brightness constancy for the human eye. In actual applications, a specific requirement may not necessarily require the participation of all image quality algorithms; sometimes, only a few simple algorithms or even just one algorithm are needed to meet the specific requirement. Manually determining which combination of image quality algorithms is required is time-consuming and laborious, and requires developers to have a high level of technical expertise. The image quality enhancement method disclosed herein automatically selects the optimal combination of algorithms from a large library of image quality algorithms through an image quality algorithm selection model.

[0150] The selection of algorithm modules depends on specific requirements (i.e., image quality display mode). This embodiment of the disclosure trains an image quality algorithm selection model specifically for these specific requirements. For example, a requirement might be to improve image quality or, without reducing image quality, to have the lowest possible power consumption for screen display. In this case, the ultimate goal needs to be determined as improving image quality while reducing power consumption.

[0151] Figure 4 is a schematic diagram of the training process of the image quality algorithm selection model in a general manner according to an embodiment of this disclosure. As shown in Figure 4, the training process of the image quality algorithm selection model includes:

[0152] (A) All image quality algorithms are listed and initialized using standard parameters. In this embodiment of the disclosure, the standard parameters can be the most commonly used empirical values ​​in the daily use of the algorithm;

[0153] (B) Construct an image quality algorithm pool that can accommodate any number of arbitrary image quality algorithms. The image quality algorithms entering the pool can be combined in any way. The selection of image quality algorithms in the pool and how they are combined are controlled by the image quality algorithm selection model.

[0154] (C) Input the image quality display mode number and the corresponding degraded image (after downsampling) into the image quality algorithm selection model. The model outputs a weight vector of the image quality algorithm combination. All image quality algorithms process the input degraded image, multiply it by the corresponding weight, and then add them together. The result is then normalized to 0-255 to obtain the output image. For example, if there are 3 image quality algorithms in the image quality algorithm pool, and the output of the image quality algorithm selection model is a vector [0.2, 0.5, 0.3], then the process of generating the output image from the degraded image can be represented by the following formula: Output image = Normalized {0.2 × Image quality algorithm 1 (degraded image) + 0.5 × Image quality algorithm 2 (degraded image) 0.3 × Image quality algorithm 3 (degraded image)} 0-255 .

[0155] (D) Both the original training image and the output image corresponding to the degraded image are downsampled, and then the downsampled image is used to calculate the loss function and update the parameters of the image quality algorithm selection model.

[0156] (E) Repeat steps (C)-(D) until the image quality algorithm selection model is trained.

[0157] Figure 8 is a schematic diagram of the reasoning process of the image quality algorithm selection model in this embodiment of the present disclosure. As shown in Figure 8, when using the model, the image quality display mode and the downsampled image to be processed are input into the image quality algorithm selection model. The model will automatically output a weight vector. The image quality algorithms in the image quality algorithm pool construct an image quality algorithm combination based on the weight vector. The constructed image quality algorithm combination is used to process the image to be processed, and finally the output image is obtained.

[0158] In some exemplary embodiments, the automatic parameter tuning algorithm model is trained in a general manner using the following method:

[0159] The training process involves multiple iterations. Each iteration includes: downsampling the original training image and the degraded image to obtain the downsampled original training image and the downsampled degraded image; inputting the downsampled degraded image into the automatic parameter tuning algorithm model; inputting the parameter list output by the automatic parameter tuning algorithm model and the degraded images from the training image set into the image quality algorithm combination; downsampling the output image of the image quality algorithm combination to obtain the downsampled output image; calculating the similarity loss between the downsampled original training image and the downsampled output image based on the similarity loss function; and adjusting the model parameters of the automatic parameter tuning algorithm model based on the calculated similarity loss result.

[0160] After selecting the image quality algorithm combination that meets the current image quality requirements through the image quality algorithm selection model or based on the current image quality display mode, the parameters of all image quality algorithms in the current image quality algorithm combination can be adjusted through the automatic parameter tuning algorithm model to obtain the optimal parameter combination.

[0161] Figure 5 is a schematic diagram of the training process of the automatic parameter tuning algorithm model in the general mode according to the embodiment of this disclosure. As shown in Figure 5, the input to the automatic parameter tuning algorithm model during training is the downsampled image of the degraded image. The automatic parameter tuning algorithm model can output a list of the best algorithm parameters for each image quality algorithm in the corresponding image quality algorithm combination for the current input image. The training process of the automatic parameter tuning algorithm model includes:

[0162] (a) The degraded image is downsampled and then input into the automatic parameter tuning algorithm model;

[0163] (b) The automatic parameter tuning algorithm model generates a list of optimal algorithm parameters based on the current input image. This list of optimal algorithm parameters represents a list of all parameters for the selected image quality algorithm combination.

[0164] (c) Input the list of optimal algorithm parameters into the selected image quality algorithm combination;

[0165] (d) The selected image quality algorithm combination is used to process the original input image to obtain the output image;

[0166] (e) Both the original training image and the output image corresponding to the degraded image are downsampled, and then the downsampled image is used to calculate the loss function and update the parameters of the automatic parameter tuning algorithm model.

[0167] (f) Repeat steps (a)-(e) until the automatic parameter tuning algorithm model is trained.

[0168] Figure 9 is a schematic diagram of the reasoning process of the automatic parameter tuning algorithm model of this disclosure embodiment. As shown in Figure 9, when in use, the downsampled image to be processed is input into the automatic parameter tuning algorithm model, and the automatic parameter tuning algorithm model will automatically output the optimal algorithm parameter list. The selected image quality algorithm combination processes the image to be processed according to the optimal algorithm parameter list given by the model, and finally obtains the output image.

[0169] In other exemplary embodiments, under the module chip tuning mode, the automatic parameter tuning algorithm model is trained by the following method:

[0170] The training process involves multiple iterations, each including: downsampling the original training image and the degraded image to obtain the downsampled original training image and the downsampled degraded image; inputting the downsampled degraded image into the automatic parameter tuning algorithm model; inputting the parameter list output by the automatic parameter tuning algorithm model and the degraded image from the training image set into the image quality algorithm combination; outputting the image quality algorithm combination to the preset display module to obtain the screen capture image; downsampling the screen capture image to obtain the downsampled screen capture image; calculating the similarity loss between the downsampled original training image and the downsampled screen capture image based on the similarity loss function; and adjusting the model parameters of the automatic parameter tuning algorithm model based on the calculated similarity loss result.

[0171] In this embodiment of the disclosure, during the iterative training process of the image quality algorithm selection model and / or the automatic parameter tuning algorithm model, the similarity loss between the downsampled original training image and the first processed image obtained by combining the image quality algorithms can be calculated according to the similarity loss function. The first processed image obtained by combining the image quality algorithms can be an image obtained by processing the output image of the image quality algorithm combination in other ways.

[0172] As shown in Figure 10, this embodiment of the present disclosure also provides a display device, including a storage module 1010, a data processing module 1020, and an image quality chip 1030, wherein:

[0173] The storage module 1010 is configured to store a pre-trained image quality algorithm selection model and / or an automatic parameter tuning algorithm model. The image quality algorithm selection model is used to output the weight vectors corresponding to one or more image quality algorithms, and the automatic parameter tuning algorithm model is used to output the optimal algorithm parameter list corresponding to the one or more image quality algorithms.

[0174] The data processing module 1020 is configured to acquire the image to be processed, perform downsampling processing on the image to be processed to obtain a first image, input the first image into a pre-trained image quality algorithm selection model and / or an automatic parameter tuning algorithm model; and input the image to be processed and the weight vectors corresponding to one or more image quality algorithms output by the image quality algorithm selection model and / or the optimal algorithm parameter list output by the automatic parameter tuning algorithm model into the image quality chip 1030.

[0175] The image quality chip 1030 is configured to perform image quality enhancement processing on the image to be processed using weight vectors corresponding to one or more image quality algorithms and / or a list of optimal algorithm parameters, to obtain an image with enhanced image quality.

[0176] In some exemplary embodiments, the data processing module 1020 may be a graphics processor.

[0177] In this embodiment of the disclosure, the image quality algorithm selection model and / or automatic parameter tuning algorithm model can be trained by an external model training device and then deployed to the display device. However, this disclosure does not limit this.

[0178] This disclosure also provides a model training apparatus, including a memory; and a processor connected to the memory, the memory being used to store instructions, the processor being configured to execute the steps of a model training method as described in any embodiment of this disclosure based on the instructions stored in the memory.

[0179] As shown in Figure 11, in one example, the model training device may include: a first processor 1110, a first memory 1120, a first bus system 1130, and a first transceiver 1140, wherein the first processor 1110, the first memory 1120, and the first transceiver 1140 are connected through the first bus system 1130, the first memory 1120 is used to store instructions, and the first processor 1110 is used to execute the instructions stored in the first memory 1120 to control the first transceiver 1140 to transmit and receive signals. Specifically, the first transceiver 1140 can acquire multiple original training images under the control of the first processor 1110. The first processor 1110 generates degraded images corresponding to the multiple original training images according to the image quality display mode. The original training images and the degraded images constitute a training image set. A similarity loss function is used to train an image quality algorithm selection model and / or an automatic parameter tuning algorithm model with the training image set. The image quality algorithm selection model is used to output the weight vector corresponding to the image quality algorithm combination. The automatic parameter tuning algorithm model is used to output the optimal algorithm parameter list corresponding to the image quality algorithm combination. The image quality algorithm combination includes one or more image quality algorithms.

[0180] It should be understood that the first processor 1110 can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), off-the-shelf programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0181] The first memory 1120 may include read-only memory and random access memory, and provides instructions and data to the first processor 1110. A portion of the first memory 1120 may also include non-volatile random access memory. For example, the first memory 1120 may also store device type information.

[0182] In addition to the data bus, the first bus system 1130 may also include a power bus, a control bus, and a status signal bus. However, for clarity, all buses are labeled as the first bus system 1130 in Figure 11.

[0183] In implementation, the processing performed by the processing device can be accomplished through integrated logic circuits in the hardware of the first processor 1110 or through software instructions. That is, the method steps of this embodiment can be executed by a hardware processor, or by a combination of hardware and software modules within the processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other storage media. This storage medium is located in the first memory 1120. The first processor 1110 reads information from the first memory 1120 and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, further details are omitted here.

[0184] This disclosure also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the model training method as described in any embodiment of this disclosure. The model training method driven by executing executable instructions is essentially the same as the model training method provided in the above embodiments of this disclosure, and will not be described in detail here.

[0185] In some possible implementations, various aspects of the model training method provided in this disclosure may also be implemented as a program product comprising program code that, when run on a computer device, causes the computer device to perform the steps in the model training method according to various exemplary embodiments of this disclosure as described above. For example, the computer device may execute the model training method described in the embodiments of this disclosure.

[0186] This disclosure also provides an image quality enhancement apparatus, including a memory; and a processor connected to the memory, the memory being used to store instructions, the processor being configured to perform the steps of the image quality enhancement method as described in any embodiment of this disclosure based on the instructions stored in the memory.

[0187] As shown in Figure 12, in one example, the image quality enhancement device may include: a second processor 1210, a second memory 1220, a second bus system 1230, and a second transceiver 1240. The second processor 1210, the second memory 1220, and the second transceiver 1240 are connected through the second bus system 1230. The second memory 1220 is used to store instructions, and the second processor 1210 is used to execute the instructions stored in the second memory 1220 to control the second transceiver 1240 to send and receive signals. Specifically, the second transceiver 1240, under the control of the second processor 1210, acquires the image to be processed. The second processor 1210 performs downsampling processing on the image to be processed to obtain a first image. The first image is then input into a pre-trained image quality algorithm selection model and / or an automatic parameter tuning algorithm model. The image quality algorithm selection model is used to output weight vectors corresponding to one or more image quality algorithms, and the automatic parameter tuning algorithm model is used to output a list of optimal algorithm parameters corresponding to the one or more image quality algorithms. The weight vectors corresponding to one or more image quality algorithms and / or the list of optimal algorithm parameters are then input into the image quality chip. Finally, the image to be processed is input into the image quality chip so that the image quality chip uses the weight vectors corresponding to one or more image quality algorithms and / or the list of optimal algorithm parameters to perform image quality enhancement processing on the image to be processed, thereby obtaining an image with enhanced image quality.

[0188] It should be understood that the second processor 1210 can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), off-the-shelf programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0189] The second memory 1220 may include read-only memory and random access memory, and provides instructions and data to the second processor 1210. A portion of the second memory 1220 may also include non-volatile random access memory. For example, the second memory 1220 may also store device type information.

[0190] In addition to the data bus, the second bus system 1230 may also include a power bus, a control bus, and a status signal bus. However, for clarity, all buses are labeled as the second bus system 1230 in Figure 12.

[0191] In implementation, the processing performed by the processing device can be accomplished through integrated logic circuits in the hardware of the second processor 1210 or through software instructions. That is, the method steps of this embodiment can be executed by a hardware processor, or by a combination of hardware and software modules within the processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other storage media. This storage medium is located in the second memory 1220. The second processor 1210 reads information from the second memory 1220 and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, further details are omitted here.

[0192] This disclosure also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the image quality enhancement method as described in any embodiment of this disclosure. The image quality enhancement method driven by executing executable instructions is essentially the same as the image quality enhancement method provided in the above embodiments of this disclosure, and will not be described in detail here.

[0193] In some possible implementations, various aspects of the image quality enhancement method provided in this disclosure can also be implemented as a program product including program code. When the program product is run on a computer device, the program code is used to cause the computer device to perform the steps in the image quality enhancement method according to various exemplary embodiments of this disclosure described above. For example, the computer device can execute the image quality enhancement method described in the embodiments of this disclosure.

[0194] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0195] It will be understood by those skilled in the art that all or some of the steps, systems, or apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all components may be implemented as software executed by a processor, such as a digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0196] It should be noted that the above embodiments or implementation methods are merely exemplary and not restrictive. Therefore, this disclosure is not limited to the content specifically shown and described herein. Various modifications, substitutions, or omissions can be made to the form and details of the implementations without departing from the scope of this disclosure.

Claims

1. A model training method, comprising: Generate a training image set corresponding to one or more image quality display modes, the training image set including original training images and degraded images corresponding to the original training images; The image quality algorithm selection model and / or automatic parameter tuning algorithm model are trained using the similarity loss function and the training image set. The image quality algorithm selection model is used to output the weight vector corresponding to the combination of image quality algorithms, and the automatic parameter tuning algorithm model is used to output the optimal algorithm parameter list corresponding to the combination of image quality algorithms. The combination of image quality algorithms includes one or more image quality algorithms.

2. The method according to claim 1, wherein, The optimal algorithm parameter list is a set of optimal algorithm parameters for each image quality algorithm in the image quality algorithm combination.

3. The method according to claim 1, wherein, The image quality algorithm selection model is trained using the following method: The training process involves multiple iterations, each iteration including: downsampling the original training image and the degraded image to obtain the downsampled original training image and the downsampled degraded image; inputting the downsampled degraded image into the image quality algorithm selection model, constructing the image quality algorithm combination based on the weight vector output by the image quality algorithm selection model, inputting the degraded images from the training image set into the image quality algorithm combination, downsampling the output image of the image quality algorithm combination or the first processed image obtained based on the output image of the image quality algorithm combination to obtain the downsampled output image; calculating the similarity loss between the downsampled original training image and the downsampled output image based on the similarity loss function, and adjusting the model parameters of the image quality algorithm selection model based on the calculated similarity loss result.

4. The method according to claim 3, wherein, The training process in at least one round further includes: when the downsampled degraded image is input into the image quality algorithm selection model, the image quality display mode is input into the image quality algorithm selection model.

5. The method according to claim 1, wherein, The automatic parameter tuning algorithm model is trained using the following method: The training process involves multiple iterations, each iteration including: downsampling the original training image and the degraded image to obtain downsampled original training images and downsampled degraded images; inputting the downsampled degraded images into the automatic parameter tuning algorithm model, inputting the parameter list output by the automatic parameter tuning algorithm model and the degraded images in the training image set into the image quality algorithm combination, downsampling the output image of the image quality algorithm combination or the first processed image obtained based on the output image of the image quality algorithm combination to obtain a downsampled output image; calculating the similarity loss between the downsampled original training image and the downsampled output image according to the similarity loss function, and adjusting the model parameters of the automatic parameter tuning algorithm model based on the calculated similarity loss result.

6. The method according to claim 3 or 5, wherein, The first processed image obtained by combining the output image based on the image quality algorithm is specifically: after outputting the output image based on the image quality algorithm to a preset display module for display, a screen capture image is obtained by using an image acquisition device to capture the image.

7. The method according to claim 1, wherein, The step of training the automatic parameter tuning algorithm model using the similarity loss function and the training image set includes: The image quality algorithm combination is constructed based on the weight vector output by the image quality algorithm selection model. The automatic parameter tuning algorithm model is trained using the similarity loss function, the image quality algorithm combination, and the training image set.

8. The method according to claim 1, wherein, The step of training the automatic parameter tuning algorithm model using the similarity loss function and the training image set includes: Determine one or more image quality algorithm combinations corresponding to the image quality display modes; The automatic parameter tuning algorithm model is trained using the similarity loss function, the image quality algorithm combination, and the training image set.

9. A model training apparatus, comprising a memory; and a processor connected to the memory, the memory for storing instructions, the processor being configured to perform the steps of the model training method as claimed in any one of claims 1 to 8 based on the instructions stored in the memory.

10. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the model training method as described in any one of claims 1 to 8.

11. A computer program product comprising instructions that, when executed by a computer, perform the model training method as described in any one of claims 1 to 8.

12. A method for improving image quality, comprising: Obtain the image to be processed, and perform downsampling processing on the image to be processed to obtain the first image; The first image is input into a pre-trained image quality algorithm selection model and / or an automatic parameter tuning algorithm model. The image quality algorithm selection model is used to output the weight vector corresponding to the image quality algorithm combination, and the automatic parameter tuning algorithm model is used to output the optimal algorithm parameter list corresponding to the image quality algorithm combination. The image quality algorithm combination includes one or more image quality algorithms. Input the weight vector and / or the optimal algorithm parameter list corresponding to the image quality algorithm combination into the image quality chip; The image to be processed is input into the image quality chip, so that the image quality chip uses the weight vector corresponding to the image quality algorithm combination and / or the optimal algorithm parameter list to perform image quality enhancement processing on the image to be processed, and obtain an image with enhanced image quality.

13. The method according to claim 12, wherein, The image quality algorithm selection model is trained for one or more image quality display modes; the automatic parameter tuning algorithm model is trained for one or more combinations of image quality algorithms.

14. The method according to claim 12, wherein, The step of inputting the first image into the pre-trained image quality algorithm selection model includes: determining the current image quality display mode, and inputting the current image quality display mode and the first image into the pre-trained image quality algorithm selection model.

15. The method according to claim 12, further comprising, before inputting the first image into the pre-trained automatic parameter tuning algorithm model: The current image quality algorithm combination is determined. The image quality algorithm combination is either a pre-set image quality algorithm combination or an image quality algorithm combination constructed based on the weight vector output by the image quality algorithm selection model.

16. The method according to claim 12, wherein, When the image quality algorithm combination includes multiple image quality algorithms, each image quality algorithm corresponds to a weight parameter, the weight parameters corresponding to multiple image quality algorithms form the weight vector, and the sum of the weight parameters corresponding to multiple image quality algorithms is 1.

17. An image quality enhancement apparatus, comprising a memory; and a processor connected to the memory, the memory for storing instructions, the processor being configured to perform the steps of the image quality enhancement method as claimed in any one of claims 12 to 16 based on the instructions stored in the memory.

18. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image quality enhancement method as described in any one of claims 12 to 16.

19. A computer program product comprising instructions that, when executed by a computer, perform the image quality enhancement method as described in any one of claims 12 to 16.

20. A display device, comprising a storage module, a data processing module, and an image processing chip, wherein: The storage module is configured to store a pre-trained image quality algorithm selection model and / or an automatic parameter tuning algorithm model. The image quality algorithm selection model is used to output a weight vector corresponding to one or more image quality algorithms, and the automatic parameter tuning algorithm model is used to output a list of optimal algorithm parameters corresponding to the one or more image quality algorithms. The data processing module is configured to acquire an image to be processed, perform downsampling processing on the image to be processed to obtain a first image, input the first image into a pre-trained image quality algorithm selection model and / or an automatic parameter tuning algorithm model; and input the image to be processed and the weight vectors corresponding to one or more image quality algorithms output by the image quality algorithm selection model and / or the optimal algorithm parameter list output by the automatic parameter tuning algorithm model into the image quality chip. The image quality chip is configured to perform image quality enhancement processing on the image to be processed using the weight vectors and / or the optimal algorithm parameter list corresponding to one or more image quality algorithms, so as to obtain an image with enhanced image quality.

21. The display device according to claim 20, wherein, The data processing module is a graphics processor.

Images