Image quality enhancement method and device, equipment and storage medium

Abstract

The invention discloses an image quality enhancement method and device, equipment and a storage medium, and relates to the technical field of computers. The method comprises the following steps: acquiring an input image; an image quality score of the input image is obtained through an image quality evaluation model according to the input image and the reference resolution, the reference resolution is the display resolution of the reference display device, and the image quality score is used for measuring the display quality of the input image on the reference display device; and under the condition that the image quality score belongs to the first score interval, performing image quality enhancement on the input image through an image quality enhancement model to obtain an output image, the output image being used for being displayed on the target display device. According to the image quality enhancement method and device, image quality enhancement is not carried out on the input image of which the image quality score does not belong to the first score interval, so that the situation that the input image of which the image quality score is relatively low cannot achieve a relatively good display effect after being subjected to image quality enhancement is avoided, computing resources are saved, and the overall display quality of the image is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an image quality enhancement method, device, equipment and storage medium. Background Art

[0002] Super-resolution enhancement technology can enhance images, improve their resolution and clarity, and make them more delicate and clear.

[0003] Related technologies use Vision Transformer (ViT) to perform super-resolution on images, apply the Transformer architecture to visual recognition tasks, capture feature relationships in images through the self-attention mechanism, and enhance the model's ability to parse visual information. This brings new model designs and stronger representation capabilities to computer vision, for example, reducing computational costs through local attention mechanisms and improving efficiency through structural reparameterization.

[0004] However, the above method has no requirements on the quality of the original image, which makes the super-resolution image prone to defects. Summary of the invention

[0005] The embodiments of the present application provide a method, device, equipment and storage medium for enhancing image quality. The technical solutions provided by the embodiments of the present application are as follows:

[0006] According to one aspect of an embodiment of the present application, a method for enhancing image quality is provided, the method comprising:

[0007] Get the input image;

[0008] Obtaining, by means of a picture quality evaluation model, a picture quality score of the input image according to the input image and a reference resolution, wherein the reference resolution is a display resolution of a reference display device, and the picture quality score is used to measure a display quality of the input image on the reference display device;

[0009] When the image quality score belongs to the first score range, the image quality of the input image is enhanced by using an image quality enhancement model to obtain an output image, and the output image is used for display on a target display device.

[0010] According to one aspect of an embodiment of the present application, there is provided an image quality enhancement device, the device comprising:

[0011] An image acquisition module, used for acquiring an input image;

[0012] A picture quality scoring module, used to obtain a picture quality score of the input image according to the input image and a reference resolution through a picture quality evaluation model, wherein the reference resolution is a display resolution of a reference display device, and the picture quality score is used to measure the display quality of the input image on the reference display device;

[0013] The image quality enhancement module is used to enhance the image quality of the input image through the image quality enhancement model when the image quality score belongs to the first score range, so as to obtain an output image, and the output image is used to be displayed on the target display device.

[0014] According to one aspect of an embodiment of the present application, a computer device is provided, comprising a processor and a memory, wherein a computer program is stored in the memory, and the computer program is loaded and executed by the processor to implement the above-mentioned image quality enhancement method.

[0015] According to one aspect of an embodiment of the present application, a computer-readable storage medium is provided, in which a computer program is stored. The computer program is loaded and executed by a processor to implement the above-mentioned image quality enhancement method.

[0016] According to one aspect of an embodiment of the present application, a computer program product is provided. The computer program product includes a computer program. The computer program is loaded and executed by a processor to implement the above-mentioned image quality enhancement method.

[0017] The technical solution provided in the embodiments of the present application can bring the following beneficial effects:

[0018] By adopting a picture quality evaluation model to score the display quality of the input image on the reference display device, the display effect of the input image on the target display device can be preliminarily judged based on the picture quality score, and the score is scored based on the reference resolution of the reference display device, thereby improving the accuracy of the picture quality score. Thus, the picture quality of the input image whose picture quality score belongs to the first score range is enhanced. Compared with the method of enhancing the picture quality of the input image regardless of the picture quality of the input image in the related art, the technical solution provided by the present application does not enhance the picture quality of the input image whose picture quality score does not belong to the first score range, thereby avoiding the situation that the input image with a low picture quality score still cannot achieve a good display effect after the picture quality enhancement, saving computing resources, improving the efficiency of image quality enhancement, and improving the overall display quality of the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a schematic diagram of an implementation environment of a solution provided by an embodiment of the present application;

[0020] Figure 2is a flow chart of an image quality enhancement method provided by an embodiment of the present application;

[0021] Figure 3 is a schematic diagram of an image whose image quality score does not belong to the first score range provided by an embodiment of the present application;

[0022] Figure 4 is a schematic diagram of the image quality enhancement effect of an image quality enhancement model provided by an embodiment of the present application on an input image;

[0023] Figure 5 is a schematic diagram of an input image including a person's face provided by an embodiment of the present application;

[0024] Figure 6 is a schematic diagram of an image quality enhancement process provided by an embodiment of the present application;

[0025] Figure 7 is a block diagram of an image quality enhancement device provided by an embodiment of the present application;

[0026] Figure 8 It is a structural block diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0028] Please refer to Figure 1 , which shows a schematic diagram of a solution implementation environment provided by an embodiment of the present application. The solution implementation environment can be implemented as an image quality enhancement system. The solution implementation environment may include: a terminal device 10 and a server 20.

[0029] The number of terminal devices 10 can be one or more. The terminal device 10 can be an electronic device such as a mobile phone, a tablet computer, a laptop computer, a desktop computer, a game console, an e-book reader, a multimedia player, a wearable device, an intelligent voice interaction device, a smart home appliance, a vehicle terminal, an aircraft, etc.

[0030] The terminal device 10 may be installed with a client of a target application, and the target application has the function of displaying images, and the user can view the images in the target application. The present application does not limit the type of the target application, including but not limited to music applications, video applications, reading applications, social applications, web applications, etc. Optionally, the target application may be an application that needs to be downloaded and installed, or an application that can be used instantly, and the present application does not limit this.

[0031] The server 20 is used to provide background services for the client of the target application installed and running in the terminal device 10. For example, the server 20 can be the background server of the above application. The server 20 can be an independent physical server, or a server cluster composed of multiple servers, or a cloud computing service center. Optionally, the server 20 provides background services for the applications in multiple terminal devices 10 at the same time. The terminal device 10 and the server 20 can communicate with each other through the network.

[0032] In an embodiment of the present application, a computer device obtains an input image, first scores the image quality of the input image through an image quality evaluation model to obtain an image quality score of the input image, and when the image quality score belongs to a first score range, the image quality of the input image is enhanced through an image quality enhancement model to obtain an output image, and the output image is used to be displayed on a target application. When the image quality score is greater than the upper limit of the first score range, it is not necessary to enhance the image quality of the input image, that is, the input image can be displayed on the target application at the second exposure rate. When the image quality score is less than the lower limit of the first score range, it is not necessary to enhance the image quality of the input image, and the input image can be displayed on the target application at the first exposure rate, and the first exposure rate is less than the second exposure rate.

[0033] Please refer to Figure 2 , which shows a flow chart of an image quality enhancement method provided by an embodiment of the present application. The execution subject of each step of the method may be a computer device. The method may include at least one of the following steps 210 to 230:

[0034] Step 210, obtaining an input image.

[0035] The input image may be any image. Optionally, the input image may be an image obtained by shooting based on an actual person or scene. For example, the input image may be an image actually shot by a camera. The input image may also be a screen image in a film or TV series, used to indicate the person or scene in the film or TV series. Optionally, the input image may also be an image obtained based on a virtual person or scene. For example, the input image may be a screen image in an animation, used to indicate a virtual character and a virtual scene in the animation. The input image may also be a screen image in a game, used to indicate a game character and a game scene in the game. The input image may also be an image drawn by the creator, including a virtual character drawn by the creator, and a virtual scene drawn by the creator.

[0036] In some embodiments, the input image involved in the present application is an image obtained by photographing an actual person, or an image obtained by photographing a virtual person, which means that the input image contains an actual person or a virtual character.

[0037] Step 220, obtaining a picture quality score of the input image based on the input image and the reference resolution through the picture quality evaluation model, wherein the reference resolution is the display resolution of the reference display device, and the picture quality score is used to measure the display quality of the input image on the reference display device.

[0038] The image quality evaluation model is a neural network model, which is used to score the image quality of the input image in combination with the image features and reference resolution of the input image, and obtain the image quality score of the input image. The reference resolution is the display resolution of the reference display device. The reference display device is the terminal device on which the input image is to be displayed based on the prediction of the technician based on big data, and is used to indicate the display scene of the input image. The reference resolution is universal and is used to ensure that the display quality of the input image after image quality enhancement on the target display device is not too low. The display quality refers to the clarity and resolution of the image display. The present application does not limit the specific value of the reference resolution. For example, the reference resolution can be 1080p (1080×1980). In the case where the display resolution of the target display device is lower than the reference resolution, the output image has a good display quality on the target display device. In the case where the display resolution of the target display device is higher than the reference resolution, for example, the display resolution of the target display device is 1440p (1440×2560), it is also possible to try to ensure that the display quality of the output image on the target display device is not too low.

[0039] In some embodiments, step 220 includes at least one of sub-steps 221 to 224 (not shown).

[0040] Sub-step 221, obtaining basic features of the input image.

[0041] The basic features of the input image are used to indicate the basic feature information contained in the input image, and the basic features are scattered and independent feature information.

[0042] In some embodiments, the basic features of the input image include at least one of the following: noise, resolution, brightness, sharpness, contrast, exposure, and motion blur.

[0043] Noise can also be called image noise. The noise of the input image is the signal interference caused by the mixture of useful signals (light signals) and useless signals (impurity signals) received by the input image during the process of photosensitization and imaging. It is an undesirable byproduct of the image capture process. The resolution of the input image is used to indicate the clarity of the input image. The brightness of the input image refers to the brightness of the input image. The sharpness of the input image is used to indicate the degree of enhancement of the edge contour of the object. The contrast of the input image refers to the measurement of the different brightness levels between the brightest white and the darkest black in the light and dark areas of the input image, that is, the grayscale contrast of the input image. The larger the difference range, the greater the contrast, and the smaller the difference range, the smaller the contrast. The exposure of the input image refers to the light perceived during the imaging process of the input image. The motion blur of the input image refers to the blur effect caused by the change of light during the exposure time when the objects in the shooting scene move relative to each other.

[0044] Sub-step 222, obtaining the image features of the input image according to the input image and the basic features of the input image through the first sub-network of the image quality assessment model.

[0045] The image quality evaluation model includes multiple sub-networks, each of which is used to implement different functions, and each sub-network includes at least one neural network layer. The first sub-network of the image quality evaluation model is used to extract features of the input image based on the input image and the basic features of the input image to obtain the image features of the input image.

[0046] The image features of the input image are obtained by linearly transforming the input image and the basic features of the input image and performing nonlinear activation functions on the input image and the basic features of the input image respectively by multiple hidden layers in the first sub-network, and the feature representation obtained by processing the output layer in the first sub-network is used to serialize or matrix the features of the input image.

[0047] Sub-step 223, obtaining a basic score of the input image according to the image features of the input image through the second sub-network of the image quality assessment model.

[0048] The second sub-network of the image quality evaluation model is used to score the image quality of the input image according to the image features of the input image to obtain a basic score of the input image. The basic score of the input image is used to measure the image quality of the input image.

[0049] Sub-step 224, obtaining the image quality score of the input image according to the basic score and reference resolution of the input image through the third sub-network of the image quality assessment model.

[0050] The third sub-network of the image quality evaluation model is used to score the display quality of the input image on the reference display device according to the basic score and reference resolution of the input image, and obtain the image quality score of the input image. The image quality score of the input image is used to measure the display quality of the input image on the reference display device.

[0051] By adopting the image quality evaluation model to score the display quality of the input image on the reference display device, the display effect of the input image on the target display device can be preliminarily judged, so that the image quality of the input image that does not meet the preset display effect can be enhanced to improve the display quality of the image. By filtering low-quality images and high-quality images, the input images with better display effects do not need to be enhanced and can be directly displayed on the target display device. The input images with poor display effects are refused to be enhanced and displayed in a manner of reducing the exposure rate, saving computing resources and avoiding the situation where the image after image quality enhancement still does not meet the preset display effect.

[0052] In some embodiments, the image quality evaluation model is a neural network model trained by the first training sample, and the training process of the image quality evaluation model is as follows: obtaining at least one first training sample of the untrained image quality evaluation model, each first training sample includes a first sample image, a reference resolution and a sample image quality score; obtaining the basic features of the first sample image; obtaining the image features of the first sample image according to the first sample image and the basic features of the first sample image through the first sub-network of the untrained image quality evaluation model; obtaining the basic score of the first sample image according to the image features of the first sample image through the second sub-network of the untrained image quality evaluation model; obtaining the predicted image quality score of the first sample image according to the basic score of the first sample image and the reference resolution through the third sub-network of the untrained image quality evaluation model; calculating the loss function value according to the difference between the predicted image quality score and the sample image quality score, adjusting the parameters of the untrained image quality evaluation model according to the loss function value, and obtaining the trained image quality evaluation model. That is, the image quality evaluation model used in the above step 220 is obtained.

[0053] Step 230, when the image quality score belongs to the first score range, the image quality of the input image is enhanced by the image quality enhancement model to obtain an output image, and the output image is used for display on the target display device.

[0054] The first score interval is set by the technician according to the scoring criteria of the image quality evaluation model and the image quality enhancement requirements, and this application does not limit it. For example, if the score range of the image quality evaluation model is 0 to 5, the first score interval can be 1.5 to 4.5. If the score range of the image quality evaluation model is 0 to 100, the first score interval can be 20 to 80.

[0055] For example, the display quality of an image whose image quality score is less than the lower limit of the first score range can refer to Figure 3 As shown in Figure (1) in the figure, the display effect of the image is poor. The display quality of the image with a quality score greater than the upper limit of the first score range can be referred to Figure 3 In Figure (2), the image display effect is higher.

[0056] For example, the image quality enhancement effect of the image quality enhancement model on the input image can be referred to Figure 4 As shown, Figure 4 Figure (1) in is the input image. Figure 4 Figure (2) in the figure is the output image, from Figure 4 Figure (1) in Figure 4 From Figure (2) in Figure 1, it can be clearly seen that the display quality of the output image has been improved.

[0057] The target display device is the terminal device in the solution implementation environment.

[0058] The image quality enhancement model is used to enhance the image quality of the input image to obtain the input image after image quality enhancement, that is, to obtain the output image. There is no restriction on the type of the input image, and the output image can be directly obtained by the image quality enhancement model.

[0059] The image quality enhancement model is a neural network model trained by the second training sample. The training process of the image quality enhancement model is as follows: obtaining at least one second training sample of an untrained image quality enhancement model, each second training sample includes a second sample input image and a second sample output image; performing image quality enhancement on the second sample input image through the untrained image quality enhancement model to obtain a second predicted output image; taking reducing the difference between the second predicted output image and the second sample output image as the training goal, adjusting the parameters of the untrained image quality enhancement model to obtain a trained image quality enhancement model. That is, the image quality enhancement model used in the above step 230 is obtained.

[0060] Exemplarily, the image quality enhancement model may be a super-resolution enhancement model, which is used to improve the resolution of an image.

[0061] In some embodiments, when the input image is an image obtained by photographing an actual person, or an image obtained by photographing a virtual person, the image quality enhancement model may include a virtual image quality enhancement model and a realistic image quality enhancement model. The virtual image quality enhancement model is used to enhance the image quality of an image containing a virtual character, and the realistic image quality enhancement model is used to enhance the image quality of an image that does not contain a virtual character, that is, to enhance the image quality of an image containing an actual person.

[0062] In some embodiments, when an input image includes a virtual character, the input image is enhanced by a virtual image quality enhancement model to obtain an output image. The virtual image quality enhancement model is a image quality enhancement model used to enhance the image quality of an image including a virtual character.

[0063] When the input image includes a virtual character, the input image is an image obtained based on a virtual character. The virtual image quality enhancement model is mainly used to improve the contrast of the image and the balance of the image color to enhance the vividness of the virtual character in the input image.

[0064] The virtual image quality enhancement model is a neural network model trained by the third training sample. The training process of the virtual image quality enhancement model is as follows: obtaining at least one third training sample of an untrained virtual image quality enhancement model, each third training sample includes a third sample input image and a third sample output image, and the third sample input image is an image containing a virtual character; enhancing the image quality of the third sample input image by the untrained virtual image quality enhancement model to obtain a third predicted output image; taking reducing the difference between the third predicted output image and the third sample output image as the training goal, adjusting the parameters of the untrained virtual image quality enhancement model to obtain a trained virtual image quality enhancement model. That is, the virtual image quality enhancement model used in the above steps is obtained.

[0065] Exemplarily, the virtual image quality enhancement model may be a virtual super-resolution enhancement model.

[0066] In some embodiments, when the input image does not contain a virtual character, the input image is enhanced by a realistic image quality enhancement model to obtain an output image. The realistic super-resolution enhancement model is a quality enhancement model used to enhance the image quality of images that do not contain virtual characters.

[0067] When the input image does not contain a virtual character, the input image is an image taken based on an actual person. The realistic image quality enhancement model is mainly used to enhance the image quality of the face area of ​​the actual person contained in the image, for example, to improve the resolution, contrast and sharpness of the face area of ​​the actual person.

[0068] The realistic image quality enhancement model is a neural network model trained by the fourth training sample. The training process of the realistic image quality enhancement model is as follows: obtaining at least one fourth training sample of an untrained realistic image quality enhancement model, each fourth training sample includes a fourth sample input image and a fourth sample output image, and the fourth sample input image is an image that does not contain a virtual character; enhancing the image quality of the fourth sample input image by the untrained realistic image quality enhancement model to obtain a fourth predicted output image; taking reducing the difference between the fourth predicted output image and the fourth sample output image as the training goal, adjusting the parameters of the untrained realistic image quality enhancement model to obtain a trained realistic image quality enhancement model. That is, the realistic image quality enhancement model used in the above steps is obtained.

[0069] Exemplarily, the realistic image quality enhancement model may be a realistic super-resolution enhancement model.

[0070] By adopting different image quality enhancement models to enhance the image quality of images containing virtual characters and images not containing virtual characters respectively, the image quality of different images can be improved in a targeted manner, so that different images can obtain better display effects on target display devices.

[0071] In some embodiments, when the input image is an image obtained by photographing a real person or scene, the input image is enhanced by a virtual image quality enhancement model to obtain an output image. When the input image is an image obtained by photographing a virtual person or scene, the input image is enhanced by a realistic image quality enhancement model to obtain an output image.

[0072] The technical solution provided in the embodiment of the present application uses a picture quality evaluation model to score the display quality of the input image on the reference display device, so that the display effect of the input image on the target display device can be preliminarily judged based on the picture quality score, and the score is scored based on the reference resolution of the reference display device, thereby improving the accuracy of the picture quality score. Thus, the picture quality of the input image whose picture quality score belongs to the first score interval is enhanced. Compared with the method of enhancing the picture quality regardless of the picture quality of the input image in the related art, the technical solution provided by the present application does not enhance the picture quality of the input image whose picture quality score does not belong to the first score interval, thereby avoiding the situation that the input image with a low picture quality score still cannot achieve a good display effect after the picture quality enhancement, saving computing resources, improving the efficiency of image quality enhancement, and improving the overall display quality of the image.

[0073] In some embodiments, when the input image is an image obtained by photographing an actual person, or an image obtained by photographing a virtual person, the above-mentioned image quality enhancement method also includes step 240, and step 240 includes at least one step of sub-steps 241 to 242.

[0074] Sub-step 241, when the image quality score belongs to the first score range, determine the number of faces contained in the input image through the face detection model according to the input image.

[0075] The face detection model is used to identify the faces of people in the input image and obtain the number of faces contained in the input image. The number of faces includes the number of faces of actual people and the number of faces of virtually existing people.

[0076] The face detection model is a neural network model trained by the fifth training sample, and the training process of the face detection model is as follows: obtaining at least one fifth training sample of an untrained face detection model, each fifth training sample including a fifth sample input image and the number of sample faces; performing face recognition on the fifth sample input image through the untrained face detection model to obtain the predicted number of faces; calculating the loss function value according to the difference between the predicted number of faces and the number of sample faces, adjusting the parameters of the untrained face detection model according to the loss function value, and obtaining the trained face detection model. That is, the face detection model used in the above step 241 is obtained.

[0077] For example, an input image containing a person's face can refer to Figure 5 As shown, Figure 5 A plurality of human faces 501 are displayed.

[0078] Sub-step 242, when the number of faces matches the image quality score, performs a step of enhancing the image quality of the input image through the image quality enhancement model to obtain an output image.

[0079] In some embodiments, the first score interval includes multiple score subintervals, each score subinterval corresponding to a quantity subinterval. In the case where the number of faces belongs to the first quantity subinterval, if the image quality score belongs to the first score subinterval, it is determined that the number of faces matches the image quality score. If the image quality score does not belong to the first score subinterval, it is determined that the number of faces does not match the image quality score. The first score subinterval is a score subinterval among the multiple score subintervals, and the first quantity subinterval is a quantity subinterval corresponding to the first score subinterval.

[0080] The sub-interval division of the first score interval and the division of each quantity sub-interval are set by the technical personnel according to the score matching requirements, and this application does not limit them.

[0081] The size of the score sub-interval and the size of the quantity sub-interval are positively correlated, that is, as the score sub-interval gradually increases, the quantity sub-interval also gradually increases.

[0082] Exemplarily, if the first score interval is 1.5 to 4.5, multiple score sub-intervals can be divided into 1.5 to 2, 2 to 2.5, 2.5 to 3, 3 to 3.5, 3.5 to 4, and 4 to 4.5, and the quantity sub-intervals corresponding to each score sub-interval are 0 to 2, 2 to 4, 4 to 6, 6 to 8, 8 to 10, and greater than 10 respectively.

[0083] For example, when the number of faces belongs to the number sub-interval of 2 to 4, if the image quality score belongs to 2 to 2.5, it is determined that the number of faces matches the image quality score, and if the image quality score does not belong to 2 to 2.5, it is determined that the number of faces does not match the image quality score. That is, when the number of faces belongs to the number sub-interval of 2 to 4, if the image quality score belongs to 1.5 to 2, or 2.5 to 4.5, it is determined that the number of faces does not match the image quality score.

[0084] By enhancing the quality of the input image when the number of faces matches the image quality score, the input image is further filtered, so that the image quality requirement is higher when the number of faces is greater, and the possibility of defects is reduced. The image quality of the input image that does not match the number of faces and the image quality score is not enhanced, and the image quality of the input image is not enhanced when the number of faces is small. The input image can be directly displayed on the target display device, and the image quality of the input image can be avoided when the number of faces is large. The image quality enhancement is avoided to avoid the image after the image quality enhancement still not achieving a good display effect, thereby saving computing resources.

[0085] Figure 6 A schematic diagram of the image quality enhancement process is shown, where the input image is an image containing a person. Based on the input image, the basic features of the input image are obtained, and the input image, the basic features of the input image, and the reference resolution are input into the image quality evaluation model to obtain the image quality score of the input image. The input image is input into the face detection model to obtain the number of faces contained in the input image, and when the number of faces matches the image quality score, it is determined whether the input image contains a virtual character. When the input image contains a virtual character, the input image is enhanced by the virtual image quality enhancement model to obtain an output image; when the input image does not contain a virtual character, the input image is enhanced by the realistic image quality enhancement model to obtain an output image.

[0086] In some embodiments, when the image quality score satisfies the first image quality condition, the input image is displayed on the target display device at the first exposure rate. The first image quality condition includes one of the following: the image quality score is less than the lower limit of the first score range; the image quality score belongs to the first score range, and when the number of faces belongs to the first number sub-range, the image quality score is less than the lower limit of the first score sub-range.

[0087] The first image quality condition is used to indicate that the display quality of the input image on the reference display device is poor, and the input image whose quality score satisfies the first image quality condition is used to indicate a low-quality image.

[0088] Optionally, the first image quality condition may be that the image quality score is less than a lower limit of the first score range. For example, if the first score range is 1.5 to 4.5, the first image quality condition is that the image quality score is less than 1.5.

[0089] Optionally, the first image quality condition may be that the image quality score belongs to a first score interval, and when the number of faces belongs to a first number sub-interval, the image quality score is less than a lower limit value of the first score sub-interval. For example, if the first score interval is 1.5 to 4.5, multiple score sub-intervals can be divided into 1.5 to 2, 2 to 2.5, 2.5 to 3, 3 to 3.5, 3.5 to 4, and 4 to 4.5, and the quantity sub-intervals corresponding to each score sub-interval are 0 to 2, 2 to 4, 4 to 6, 6 to 8, 8 to 10, and greater than 10, respectively. Then, the first image quality condition is that the image quality score belongs to 1.5 to 4.5, and when the number of faces belongs to 2 to 4, the image quality score is less than 2, when the number of faces belongs to 4 to 6, the image quality score is less than 2.5, when the number of faces belongs to 6 to 8, the image quality score is less than 3, when the number of faces belongs to 8 to 10, the image quality score is less than 3.5, and when the number of faces belongs to a range greater than 10, the image quality score is less than 4.

[0090] The specific value of the first exposure rate is set by the technician according to the exposure requirements of the image, and this application does not limit it. For example, the first exposure rate can be 0. It should be noted that the first exposure rate is less than the exposure rate of the output image on the target display device.

[0091] By setting the first image quality condition, an input image whose image quality score meets the first image quality condition is displayed on a target display device at a first exposure rate, thereby reducing the exposure rate of the input image and preventing images with low display quality from being seen by users. This ensures the overall display quality of the image while saving computing resources.

[0092] In some embodiments, when the image quality score satisfies the second image quality condition, the input image is displayed on the target display device at a second exposure rate, and the second exposure rate is greater than the first exposure rate. The second image quality condition includes one of the following: the image quality score is greater than an upper limit of the first score range; the image quality score belongs to the first score range, and when the number of faces belongs to the first number sub-range, the image quality score is greater than an upper limit of the first score sub-range.

[0093] The second image quality condition is used to indicate that the display quality of the input image on the reference display device is relatively high, and the input image whose quality score satisfies the second image quality condition is used as an indication draft quality image.

[0094] Optionally, the second image quality condition may be that the image quality score is greater than an upper limit of the first score range. For example, if the first score range is 1.5 to 4.5, the second image quality condition is that the image quality score is greater than 4.5.

[0095] Optionally, the second image quality condition may be that the image quality score belongs to the first score interval, and when the number of faces belongs to the first quantity sub-interval, the image quality score is greater than the upper limit of the first score sub-interval. For example, if the first score interval is 1.5 to 4.5, multiple score sub-intervals can be divided into 1.5 to 2, 2 to 2.5, 2.5 to 3, 3 to 3.5, 3.5 to 4, 4 to 4.5, and the quantity sub-intervals corresponding to each score sub-interval are 0 to 2, 2 to 4, 4 to 6, 6 to 8, 8 to 10, and greater than 10, respectively, then the first image quality condition is that the image quality score belongs to 1.5 to 4.5, and when the number of faces belongs to 0 to 2, the image quality score is greater than 2, when the number of faces belongs to 2 to 4, the image quality score is greater than 2.5, when the number of faces belongs to 4 to 6, the image quality score is greater than 3, when the number of faces belongs to 6 to 8, the image quality score is greater than 3.5, and when the number of faces belongs to the interval of 8 to 10, the image quality score is greater than 4.

[0096] The second exposure rate is greater than the first exposure rate. The specific value of the first exposure rate is set by the technician according to the exposure requirements of the image, and this application does not limit it. For example, if the first exposure rate is 0, the second exposure rate is any value greater than 0.

[0097] In some embodiments, the second exposure ratio is equal to an exposure ratio of the output image on the target display device.

[0098] By setting the second image quality condition, an input image whose image quality score meets the second image quality condition is displayed on a target display device at a second exposure rate, thereby ensuring normal exposure of the input image. While saving computing resources, images with higher display quality can be viewed by users at a normal exposure rate, thereby improving the overall display quality of the image.

[0099] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0100] Please refer to Figure 7, which shows a block diagram of an image quality enhancement device provided by an embodiment of the present application. The device has the function of implementing the above-mentioned image quality enhancement method, and the function can be implemented by hardware, or by hardware executing corresponding software. The device can be the computer device described above, or it can be set in a computer device. Figure 7 As shown, the device 700 may include: an image acquisition module 710 , an image quality scoring module 720 and an image quality enhancement module 730 .

[0101] The image acquisition module 710 is used to acquire an input image.

[0102] The image quality scoring module 720 is used to obtain the image quality score of the input image according to the input image and the reference resolution through the image quality evaluation model. The reference resolution is the display resolution of the reference display device, and the image quality score is used to measure the display quality of the input image on the reference display device.

[0103] The image quality enhancement module 730 is used to enhance the image quality of the input image through the image quality enhancement model when the image quality score belongs to the first score range, so as to obtain an output image, and the output image is used to be displayed on the target display device.

[0104] In some embodiments, the image quality scoring module 720 is used to:

[0105] Acquire basic features of the input image;

[0106] Obtaining, by means of a first sub-network of the image quality evaluation model, image features of the input image according to the input image and basic features of the input image;

[0107] Obtaining a basic score of the input image according to image features of the input image through a second sub-network of the image quality evaluation model;

[0108] The image quality score of the input image is obtained according to the basic score of the input image and the reference resolution through the third subnetwork of the image quality evaluation model.

[0109] In some embodiments, the basic features of the input image include at least one of the following: noise, resolution, brightness, sharpness, contrast, exposure, and motion blur.

[0110] In some embodiments, the image quality enhancement module 730 is used to:

[0111] When the image quality score belongs to the first score range, determining the number of faces included in the input image by using a face detection model according to the input image;

[0112] In the case where the number of faces matches the image quality score, the step of enhancing the image quality of the input image by using the image quality enhancement model to obtain an output image is performed.

[0113] In some embodiments, the first score interval includes a plurality of score sub-intervals, each score sub-interval corresponds to a quantity sub-interval, and the image quality enhancement module 730 is used to:

[0114] In the case where the number of faces belongs to the first number sub-interval, if the image quality score belongs to the first score sub-interval, determining that the number of faces matches the image quality score;

[0115] If the image quality score does not belong to the first score sub-interval, determining that the number of faces does not match the image quality score;

[0116] The first score sub-interval is a score sub-interval among the multiple score sub-intervals, and the first quantity sub-interval is a quantity sub-interval corresponding to the first score sub-interval.

[0117] In some embodiments, the apparatus 700 further includes an image display module, wherein the image display module is configured to:

[0118] When the image quality score satisfies a first image quality condition, displaying the input image on the target display device at a first exposure rate;

[0119] The first image quality condition includes one of the following:

[0120] The image quality score is less than the lower limit of the first score range;

[0121] The image quality score belongs to the first score range, and when the number of faces belongs to the first number sub-range, the image quality score is less than a lower limit value of the first score sub-range.

[0122] In some embodiments, the image display module is used to:

[0123] When the image quality score satisfies a second image quality condition, displaying the input image on the target display device at a second exposure rate, wherein the second exposure rate is greater than the first exposure rate;

[0124] The second image quality condition includes one of the following:

[0125] The image quality score is greater than the upper limit of the first score range;

[0126] The image quality score belongs to the first score range, and when the number of faces belongs to the first number sub-range, the image quality score is greater than an upper limit value of the first score sub-range.

[0127] In some embodiments, the image quality enhancement module 730 is used to:

[0128] In the case where the input image includes a virtual character, the input image is enhanced by a virtual image quality enhancement model to obtain the output image, wherein the virtual image quality enhancement model is a image quality enhancement model for enhancing the image including the virtual character;

[0129] In the case that the input image does not contain the virtual character, the input image is enhanced by a realistic image quality enhancement model to obtain the output image. The realistic super-resolution enhancement model is a quality enhancement model for enhancing the image that does not contain the virtual character.

[0130] It should be noted that the device provided in the above embodiment, when implementing its functions, is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the content structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0131] Please refer to Figure 8 , which shows a block diagram of a computer device 800 provided in one embodiment of the present application. The computer device 800 may be any electronic device with data calculation, processing and storage functions. The computer device 800 may be used to implement the image quality enhancement method provided in the above embodiment.

[0132] Typically, the computer device 800 includes a processor 801 and a memory 802 .

[0133] The processor 801 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like. The processor 801 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field Programmable Gate Array), and PLA (Programmable Logic Array). The processor 801 may also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 801 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 801 may also include an AI processor for processing computing operations related to machine learning.

[0134] The memory 802 may include one or more computer-readable storage media, which may be non-transitory. The memory 802 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 802 is used to store a computer program, which is configured to be executed by one or more processors to implement the above-mentioned image quality enhancement method.

[0135] Those skilled in the art will understand that Figure 8 The structure shown in the figure does not constitute a limitation on the computer device 800, and the computer device 800 may include more or less components than those shown in the figure, or combine some components, or adopt a different arrangement of components.

[0136] In an illustrative embodiment, a computer-readable storage medium is also provided, wherein a computer program is stored in the storage medium, and the computer program implements the above-mentioned image quality enhancement method when executed by a processor of a computer device. Optionally, the above-mentioned computer-readable storage medium may be a ROM (Read-Only Memory), a RAM (Random Access Memory), a CD-ROM (Compact Disc Read-Only Memory), a magnetic tape, a floppy disk, an optical data storage device, etc.

[0137] In an exemplary embodiment, a computer program product is also provided, the computer program product comprising a computer program, the computer program being stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device performs the above-mentioned image quality enhancement method.

[0138] It should be understood that the "multiple" mentioned in this article refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. In addition, the step numbers described in this article only illustrate a possible execution sequence between the steps. In some other embodiments, the above steps may not be executed in the order of the numbers, such as two steps with different numbers are executed at the same time, or two steps with different numbers are executed in the opposite order to the diagram. The embodiments of the present application are not limited to this.

[0139] The above description is only an exemplary embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for enhancing image quality, characterized in that: The method comprises: Get the input image; Obtaining, by means of a picture quality evaluation model, a picture quality score of the input image according to the input image and a reference resolution, wherein the reference resolution is a display resolution of a reference display device, and the picture quality score is used to measure a display quality of the input image on the reference display device; When the image quality score belongs to the first score range, the image quality of the input image is enhanced by using an image quality enhancement model to obtain an output image, and the output image is used for display on a target display device.

2. The method according to claim 1, characterized in that The step of obtaining a picture quality score of the input image according to the input image and the reference resolution by using the picture quality evaluation model includes: Acquire basic features of the input image; Obtaining, by means of a first sub-network of the image quality evaluation model, image features of the input image according to the input image and basic features of the input image; Obtaining a basic score of the input image according to image features of the input image through a second sub-network of the image quality evaluation model; The image quality score of the input image is obtained according to the basic score of the input image and the reference resolution through the third subnetwork of the image quality evaluation model.

3. The method according to claim 2, characterized in that The basic features of the input image include at least one of the following: noise, resolution, brightness, sharpness, contrast, exposure, and motion blur.

4. The method according to claim 1, characterized in that: The method further comprises: When the image quality score belongs to the first score range, determining the number of faces included in the input image by using a face detection model according to the input image; In the case where the number of faces matches the image quality score, the step of enhancing the image quality of the input image by using the image quality enhancement model to obtain an output image is performed.

5. The method according to claim 4, characterized in that The first score interval includes a plurality of score sub-intervals, each score sub-interval corresponds to a quantity sub-interval, and the method further includes: In the case where the number of faces belongs to the first number sub-interval, if the image quality score belongs to the first score sub-interval, determining that the number of faces matches the image quality score; If the image quality score does not belong to the first score sub-interval, determining that the number of faces does not match the image quality score; The first score sub-interval is a score sub-interval among the multiple score sub-intervals, and the first quantity sub-interval is a quantity sub-interval corresponding to the first score sub-interval.

6. The method according to claim 5, characterized in that The method further comprises: When the image quality score satisfies a first image quality condition, displaying the input image on the target display device at a first exposure rate; The first image quality condition includes one of the following: The image quality score is less than the lower limit of the first score range; The image quality score belongs to the first score range, and when the number of faces belongs to the first number sub-range, the image quality score is less than a lower limit value of the first score sub-range.

7. The method according to claim 6, characterized in that The method further comprises: When the image quality score satisfies a second image quality condition, displaying the input image on the target display device at a second exposure rate, wherein the second exposure rate is greater than the first exposure rate; The second image quality condition includes one of the following: The image quality score is greater than the upper limit of the first score range; The image quality score belongs to the first score range, and when the number of faces belongs to the first number sub-range, the image quality score is greater than an upper limit value of the first score sub-range.

8. The method according to any one of claims 1 to 7, characterized in that: The step of enhancing the image quality of the input image by using the image quality enhancement model to obtain an output image includes: In the case where the input image includes a virtual character, the input image is enhanced by a virtual image quality enhancement model to obtain the output image, wherein the virtual image quality enhancement model is a image quality enhancement model for enhancing the image including the virtual character; In the case that the input image does not contain the virtual character, the input image is enhanced by a realistic image quality enhancement model to obtain the output image. The realistic super-resolution enhancement model is a quality enhancement model for enhancing the image that does not contain the virtual character.

9. An image quality enhancement device, characterized in that: The device comprises: An image acquisition module, used for acquiring an input image; A picture quality scoring module, used to obtain a picture quality score of the input image according to the input image and a reference resolution through a picture quality evaluation model, wherein the reference resolution is a display resolution of a reference display device, and the picture quality score is used to measure the display quality of the input image on the reference display device; The image quality enhancement module is used to enhance the image quality of the input image through the image quality enhancement model when the image quality score belongs to the first score range, so as to obtain an output image, and the output image is used to be displayed on the target display device.

10. A computer device, characterized in that: The computer device includes a processor and a memory, wherein a computer program is stored in the memory, and the computer program is loaded and executed by the processor to implement the image quality enhancement method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is loaded and executed by a processor to implement the image quality enhancement method according to any one of claims 1 to 8.

12. A computer program product, characterized in that The computer program product comprises a computer program, and the computer program is loaded and executed by a processor to implement the image quality enhancement method according to any one of claims 1 to 8.

Images