Image processing method and apparatus and terminal device

Abstract

The present disclosure provides an image processing method and apparatus and a terminal device. The method comprises: acquiring a first image; performing blurring processing on the first image to obtain a blurred image; performing sharpening processing on the first image to obtain a sharpened image; on the basis of the first image, the blurred image and the sharpened image, determining the perceived quality of the sharpening of the first image; on the basis of the first image, determining the sharpness of the first image; and on the basis of the perceived quality of the sharpening and the sharpness, determining the sharpening quality of the first image.

Description

Image processing method, device and terminal equipment

[0001] The present application claims priority to the Chinese patent application No. 202310908148.4, filed on July 21, 2023, entitled "Image processing method, device and terminal equipment", the whole content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] Embodiments of the present disclosure relate to the technical field of image processing, in particular to an image processing method, device and terminal equipment. BACKGROUND

[0003] Image sharpening technology can enhance the sharpness of an image and improve the quality of human eye perception of the image. However, an image that is sharpened too much can also reduce the quality of human eye perception of the image. Therefore, it is particularly important to determine the quality of the image after sharpening.

[0004] At present, a terminal device can input a sharpened image to an image sharpening evaluation model, the image sharpening evaluation model can output a sharpening score corresponding to the sharpened image, and a user can determine whether the image is sharpened too much or not enough based on the sharpening score. However, the user can only estimate the sharpening quality of the sharpened image based on experience, and cannot accurately determine whether the poor sharpening quality is caused by over-sharpening or insufficient sharpening. How to accurately evaluate the sharpening quality of the sharpened image is a problem to be solved.

[0005] SUMMARY

[0006] The present disclosure provides an image processing method, device and terminal equipment to solve one or more technical problems in the prior art.

[0007] In a first aspect, the present disclosure provides an image processing method, comprising: obtaining a first image; performing blurring processing on the first image to obtain a blurred image; performing sharpening processing on the first image to obtain a sharpened image; determining a perceived quality of sharpening of the first image based on the first image, the blurred image and the sharpened image; determining a sharpness of the first image based on the first image; and determining a sharpening quality of the first image based on the perceived quality of sharpening and the sharpness.

[0008] In a second aspect, the present disclosure provides an image processing apparatus, comprising an obtaining module, a processing module, a first determining module, a second determining module and a third determining module, wherein: the obtaining module is configured to obtain a first image; the processing module is configured to perform a blurring process on the first image to obtain a blurred image; the processing module is further configured to perform a sharpening process on the first image to obtain a sharpened image; the first determining module is configured to determine a perceived quality of sharpening of the first image based on the first image, the blurred image and the sharpened image; the second determining module is configured to determine a sharpness of the first image based on the first image; and the third determining module is configured to determine a sharpening quality of the first image based on the perceived quality of sharpening and the sharpness.

[0009] In a third aspect, the present disclosure provides a terminal device, comprising a processor and a memory; the memory stores computer-executable instructions; and the processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the image processing method according to the first aspect and various possible aspects related to the first aspect.

[0010] In a fourth aspect, the present disclosure provides a computer-readable storage medium, which stores computer-executable instructions; when a processor executes the computer-executable instructions, the image processing method according to the first aspect and various possible aspects related to the first aspect is implemented. BRIEF DESCRIPTION OF DRAWINGS

[0011] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure or the prior art, the drawings needed in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present disclosure, and other drawings can also be obtained by those skilled in the art without any creative labor.

[0012] FIG. 1 is a schematic diagram of an application scenario according to an embodiment of the present disclosure;

[0013] FIG. 2 is a flowchart of an image processing method according to an embodiment of the present disclosure;

[0014] FIG. 3 is a schematic diagram of a process of determining a blurred image according to an embodiment of the present disclosure;

[0015] FIG. 4 is a schematic diagram of a process of determining a sharpened image according to an embodiment of the present disclosure;

[0016] FIG. 5 is a schematic diagram of a relationship between a perceived quality of sharpening and a sharpness according to an embodiment of the present disclosure;

[0017] FIG. 6 is a schematic diagram of determining a sharpness of a first image according to an embodiment of the present disclosure;

[0018] FIG. 7 is a schematic diagram of a method for determining the perceived quality of sharpening according to an embodiment of the present disclosure;

[0019] FIG. 8 is a schematic diagram of a process for determining the first image similarity according to an embodiment of the present disclosure;

[0020] FIG. 9 is a schematic diagram of a process for determining the second image similarity according to an embodiment of the present disclosure;

[0021] FIG. 10 is a schematic diagram of a process for determining the weight according to an embodiment of the present disclosure;

[0022] FIG. 11 is a schematic diagram of a process for training the sharpening quality evaluation model according to an embodiment of the present disclosure;

[0023] FIG. 12 is a schematic diagram of a structure of an image processing apparatus according to an embodiment of the present disclosure; and

[0024] FIG. 13 is a schematic diagram of a structure of a terminal device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0025] The exemplary embodiments will be described in detail herein below with reference to the drawings. The following description is merely exemplary in nature and is in no way intended to limit the scope of the present disclosure, as described throughout this patent document. Conversely, specific embodiments of the present disclosure are presented for the purposes of illustration and not limitation.

[0026] For ease of understanding, the concepts involved in the embodiments of the present disclosure are described below.

[0027] Terminal device: a device with wireless transceiver function. The terminal device can be deployed on land, including indoor or outdoor, handheld, wearable or vehicle-mounted. The terminal device can be a mobile phone, a tablet computer (Pad), a computer with wireless transceiver function, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a wireless terminal in industrial control, a vehicle-mounted terminal device, a wireless terminal in self driving, a wireless terminal device in remote medical, a wireless terminal device in smart grid, a wireless terminal device in transportation safety, a wireless terminal device in smart city, a wireless terminal device in smart home, a wearable terminal device, etc. The terminal device involved in the embodiments of the present disclosure can also be referred to as a terminal, a user equipment (UE), an access terminal device, a vehicle-mounted terminal, an industrial control terminal, a UE unit, a UE station, a mobile station, a mobile station, a remote station, a remote terminal device, a mobile device, a UE terminal device, a wireless communication device, a UE agent or a UE apparatus, etc. The terminal device can also be fixed or mobile.

[0028] In the related art, image sharpening can enhance the sharpness of the image and improve the quality of human eye perception of the image. However, both over-sharpening and under-sharpening of the image will reduce the quality of human eye perception of the image, and therefore, it is particularly important to determine the sharpening quality of the sharpened image. For example, if the sharpened image is over-sharpened, the sharpening parameter of the image can be reduced, and the image is re-sharpened by using the reduced sharpening parameter. Similarly, if the sharpened image is under-sharpened, the sharpening parameter of the image can be increased, and the image is re-sharpened based on the increased sharpening parameter. Currently, the terminal device can input the sharpened image to the image sharpening evaluation model, the image sharpening evaluation model can output the sharpening score corresponding to the sharpened image, and the user can determine whether the image is over-sharpened or under-sharpened based on the sharpening score. However, since the image sharpening is proportional to the sharpening score, the higher the image sharpening degree, the higher the sharpening score output by the image sharpening evaluation model, and therefore, the user can only estimate the sharpening quality of the sharpened image based on experience, and the relationship between the perception quality of the image by the human eye and the image sharpening degree is first increased and then decreased, and therefore, the user cannot accurately determine whether the poor image sharpening quality is caused by over-sharpening or under-sharpening based on the sharpening score, which further leads to low accuracy of the evaluation of the sharpening quality of the sharpened image.

[0029] To solve the technical problems in the related art, the embodiment of the present disclosure provides an image processing method. A terminal device can obtain a first image, perform blur processing on the first image to obtain a blurred image, and perform sharpening processing on the first image to obtain a sharpened image. Based on the first image and the blurred image, a first image similarity between the first image and the blurred image is determined. Based on the first image and the sharpened image, a second image similarity between the first image and the sharpened image is determined. Based on the first image, the first image similarity, and the second image similarity, a perceived quality of sharpening of the first image is determined. Based on the first image, a sharpness of the first image is determined. Based on the perceived quality of sharpening and the sharpness, a sharpening quality of the first image is determined. In the above method, the terminal device can use the first image similarity between the first image and the blurred image and the second image similarity between the first image and the sharpened image as a reference to determine the sharpening quality. Therefore, the terminal device can accurately evaluate the perceived quality of sharpening of the first image, and the terminal device can accurately determine the sharpening quality of the first image after sharpening by combining the sharpness and the perceived quality of sharpening, thereby improving the accuracy of determining the sharpening quality of the image.

[0030] Next, in combination with FIG. 1, the application scenario of the embodiment of the present disclosure is described.

[0031] FIG. 1 is a schematic diagram of an application scenario provided by the embodiment of the present disclosure. Referring to FIG. 1, it includes a terminal device. The display page of the terminal device can include a first image. The terminal device can evaluate the sharpening quality of the first image. The terminal device can display the sharpness of the first image and the perceived quality of sharpening of the first image, wherein the sharpness of the first image can be 0.5, the perceived quality of sharpening of the first image can be 70, and the terminal device can determine the sharpening quality of the first image based on the sharpness of the first image and the perceived quality of sharpening of the first image. In this way, the terminal device can obtain the perceived quality of sharpening of the first image based on the blurred image obtained by blurring the first image and the sharpened image obtained by sharpening the first image. Therefore, the accuracy of the perceived quality of sharpening is high, and the terminal device can determine the sharpening quality of the first image by combining the sharpness of the first image and the perceived quality of sharpening. Therefore, the terminal device can accurately determine the reason why the sharpening quality of the first image is poor, and can accurately determine the sharpening quality of the first image, thereby improving the accuracy of determining the sharpening quality.

[0032] It should be noted that FIG. 1 is only an exemplary schematic diagram of the application scenario of the embodiment of the present disclosure, and is not a limitation on the application scenario of the embodiment of the present disclosure.

[0033] The technical solutions of the present disclosure and how the present disclosure solves the above technical problems will be described in detail below with specific examples. The following specific examples can be combined with each other, and the same or similar concepts or processes can not be described again in some examples. The embodiments of the present disclosure will be described below with reference to the accompanying drawings.

[0034] FIG. 2 is a flowchart of an image processing method according to an embodiment of the present disclosure. Referring to FIG. 2, the method can include:

[0035] S201, obtaining a first image.

[0036] The execution subject of the embodiments of the present disclosure can be a terminal device, or an image processing apparatus arranged in the terminal device. The image processing apparatus can be implemented based on software, or implemented based on the combination of software and hardware, which is not limited in the embodiments of the present disclosure. Optionally, the terminal device can be any device with on-device computing capability.

[0037] Optionally, the first image can be an image after sharpening processing. For example, the terminal device can perform sharpening processing on any image, and then obtain the first image. For example, the first image can be an image after sharpening processing, which is to be evaluated for sharpening quality.

[0038] It should be noted that since any image has sharpness, the first image can also be any image, which is not limited in the embodiments of the present disclosure.

[0039] Optionally, the terminal device can receive the first image sent by another device. For example, the terminal device can receive at least one image to be evaluated for sharpening quality associated with a request for sharpening quality evaluation sent by another device (the terminal device can perform the operation of evaluating the sharpening quality after receiving the request). The terminal device can determine the at least one image to be evaluated for sharpening quality as the first image.

[0040] Optionally, the terminal device can obtain the first image from a database. For example, the database can pre-store a plurality of images after sharpening processing, and the terminal device can obtain a plurality of images (i.e., the first image) from the database. For example, the database can pre-store a plurality of images after sharpening processing, and each image after sharpening processing has a unique identifier. The terminal device can receive a request for sharpening quality evaluation sent by another device, the request for sharpening quality evaluation can include identifiers of a plurality of images after sharpening processing, and the terminal device can obtain the associated first image from the database based on the identifiers.

[0041] It should be noted that the terminal device can obtain the first image based on any feasible implementation manner, which is not limited in the embodiments of the present disclosure.

[0042] S202, blur processing is performed on the first image to obtain a blurred image.

[0043] The blurred image is an image obtained by performing blur processing on the first image. For example, the terminal device can perform Gaussian blur processing on the first image to obtain a blurred image corresponding to the first image. For example, the terminal device can perform image degradation processing on the first image to obtain a blurred image corresponding to the first image. For example, the terminal device can perform blurred image simulation processing on the first image to generate a blurred simulated image corresponding to the first image. For example, the terminal device can perform down-sampling processing on the first image to reduce the number of pixels in the first image to obtain a blurred image corresponding to the first image.

[0044] It should be noted that the terminal device can obtain a blurred image corresponding to the first image based on any feasible implementation manner, which is not limited in the embodiments of the present disclosure.

[0045] It should be noted that the clarity of the blurred image obtained by the terminal device after performing blur processing on the first image is lower than that of the first image.

[0046] Optionally, the blurred image can be a negative sample of the first image, and the higher the blur degree of the blurred image, the better the effect of the blurred image as a negative sample. For example, the terminal device can process the first image based on Gaussian blur to obtain a blurred image. Since the blurred image is obtained based on Gaussian blur processing, the blurred image has a high blur degree, so the blurred image has a good effect as a negative sample of the first image.

[0047] Next, the process of determining the blurred image will be described in conjunction with FIG. 3.

[0048] FIG. 3 is a process diagram for determining a blurred image according to an embodiment of the present disclosure. Please refer to FIG. 3, which includes a first image. The terminal device (not shown in FIG. 3) can perform Gaussian blur processing on the first image to obtain a blurred image corresponding to the first image. The size of the first image is the same as that of the blurred image. In this way, the terminal device can obtain a blurred image with good quality based on Gaussian blur, and the terminal device can improve the accuracy of determining the perceived quality of the sharpening of the first image based on the blurred image as a reference, thereby improving the accuracy of determining the sharpening quality of the first image.

[0049] S203, sharpening processing is performed on the first image to obtain a sharpened image.

[0050] The sharpened image is an image obtained by sharpening the first image. For example, the terminal device can sharpen the first image based on a sharpening algorithm, and then obtain the sharpened image corresponding to the first image. The sharpening algorithm used by the terminal device can be any feasible algorithm, which is not limited in the embodiments of the present disclosure.

[0051] It should be noted that the terminal device can generate the sharpened image corresponding to the first image based on any feasible implementation manner, which is not limited in the embodiments of the present disclosure.

[0052] Optionally, the sharpened image can be a negative sample of the first image. The higher the sharpening degree of the sharpened image, the better the effect of the sharpened image as a negative sample. For example, the terminal device can process the first image based on a higher sharpening parameter to obtain the sharpened image. Since the sharpened image is obtained based on a higher sharpening parameter, the sharpening degree of the sharpened image is higher, so that the sharpened image has a better effect as a negative sample of the first image.

[0053] It should be noted that in actual application, the sharpened image can be an image obtained by sharpening the first image by the terminal device based on the highest sharpening parameter, so that the effect of the sharpened image as a negative sample of the first image can be improved.

[0054] In the following, the process of determining the sharpened image is described in combination with FIG. 4.

[0055] FIG. 4 is a process schematic diagram of determining a sharpened image provided by an embodiment of the present disclosure. Please refer to FIG. 4, which includes a first image. The terminal device (not shown in FIG. 4) can sharpen the first image, and then obtain the sharpened image corresponding to the first image. The size of the first image and the sharpened image is the same. The sharpening parameter of the terminal device when performing sharpening (which can indicate the sharpening degree of the image, and the larger the sharpening parameter, the deeper the sharpening degree) can be the highest sharpening parameter of the sharpening algorithm. In this way, the terminal device can obtain a sharpened image with good quality, and the terminal device can improve the accuracy of determining the perceived quality of the sharpening of the first image based on the sharpened image as a reference, and then improve the accuracy of determining the quality of the sharpening of the first image.

[0056] S204, determining the perceived quality of the sharpening of the first image based on the first image, the blurred image and the sharpened image.

[0057] The perceived quality of the sharpening of the first image can be the perceived quality of the image by the human eye of the user. For example, in actual application, the perceived quality of the sharpening can be a score, and the user can determine the perceived quality of the first image by the human eye based on the score. The higher the score, the better the perceived quality of the first image by the human eye. The lower the score, the lower the perceived quality of the first image by the human eye.

[0058] It should be noted that the perceived quality of the sharpening of the first image has an optimal sharpness, that is, if the sharpness of the first image is the optimal sharpness, the score of the perceived quality of the sharpening of the first image is the highest, and if the sharpness of the first image is less than or greater than the optimal sharpness, the score of the perceived quality of the sharpening of the first image is less than the highest score corresponding to the optimal sharpness.

[0059] Next, the relationship between the perceived quality of the sharpening and the sharpness will be described in conjunction with FIG. 5.

[0060] FIG. 5 is a schematic diagram of the relationship between the perceived quality of the sharpening and the sharpness according to an embodiment of the present disclosure. Referring to FIG. 5, a coordinate system between the perceived quality of the sharpening and the sharpness related to the first image is included. In the coordinate system, the horizontal axis is the sharpness, and the vertical axis is the perceived quality of the sharpening. In the coordinate system shown in FIG. 5, the perceived quality of the sharpening first increases and then decreases with the increase of the sharpness, wherein A is the point at which the perceived quality of the sharpening of the first image is the highest, and B is the optimal sharpness of the first image, that is, if the sharpness of the first image is set to the sharpness corresponding to the point B, the perceived quality of the sharpening of the first image is the best.

[0061] It should be noted that the optimal sharpness of the first image is different when the first image is different. For example, if the texture in the first image is less (for example, the first image includes more sky images), the perceived quality of the sharpening of the first image changes slowly after the sharpness of the first image is increased, and the perceived quality of the sharpening of the first image is the best when the sharpness of the first image is large; if the texture in the first image is more (for example, the first image includes more grassland images), the perceived quality of the sharpening of the first image changes quickly after the sharpness of the first image is increased, and the perceived quality of the sharpening of the first image is the best when the sharpness of the first image is small.

[0062] The terminal device can determine the perceived quality of the sharpening of the first image based on the following possible implementation manners. The first image similarity between the first image and the blurred image is determined based on the first image and the blurred image. The second image similarity between the first image and the sharpened image is determined based on the first image and the sharpened image. The perceived quality of the sharpening of the first image is determined based on the first image, the first image similarity, and the second image similarity.

[0063] The first image similarity can be a similarity between the first image and the blurred image, and the second image similarity can be a similarity between the first image and the sharpened image. Since the blurred image has a high degree of blurring, the perception quality of sharpening corresponding to the blurred image is low. Since the sharpened image has a high degree of sharpening, the perception quality of sharpening corresponding to the sharpened image is also low. The terminal device takes the blurred image and the sharpened image as references. Based on the first image similarity, the second image similarity, and the first image, the perception quality of sharpening of the first image can be accurately determined.

[0064] S205, determining the sharpness of the first image based on the first image.

[0065] The sharpness of the first image can indicate the degree of sharpening of the first image. For example, the greater the sharpness of the first image, the higher the degree of sharpening of the first image. The smaller the sharpness of the first image, the lower the degree of sharpening of the first image.

[0066] Optionally, the terminal device can determine the sharpness of the first image based on the following possible implementation manner: performing convolution processing on the first image to obtain a feature image of the first image. The feature image of the first image is processed based on a sharpness calculation model to obtain the sharpness of the first image. For example, the sharpness calculation model is used to generate the sharpness corresponding to the image. The terminal device can input the sharpness of the first image into the sharpness calculation model, and the sharpness calculation model can output the sharpness of the first image. For example, the sharpness calculation model can include a convolutional neural network (CNN) in advance. The terminal device can input the first image into the convolutional neural network, and the convolutional neural network can output the feature image of the first image and input the feature image of the first image into the sharpness calculation model. The sharpness calculation model can output the sharpness of the first image based on the feature image of the first image.

[0067] Next, the process of determining the sharpness of the first image will be described in conjunction with FIG. 6.

[0068] FIG. 6 is a schematic diagram of determining the sharpness of the first image according to an embodiment of the present disclosure. Referring to FIG. 6, it includes a first image, a convolutional neural network, and a sharpness calculation model. The terminal device (not shown in FIG. 6) can input the first image into the convolutional neural network, and the convolutional neural network can output the feature image associated with the first image and input the feature image into the sharpness calculation model. The sharpness calculation model can process the feature image to obtain the sharpness corresponding to the first image. In this way, the terminal device can accurately determine the sharpness corresponding to the first image, and thus the accuracy of determining the sharpening quality of the first image can be improved.

[0069] Optionally, the terminal device can generate the sharpness calculation model based on the following feasible implementation manners: obtaining the second image, the blurred image of the second image, and the sharpened image of the second image, determining that the sharpness of the second image is M, the sharpness of the blurred image of the second image is L, and the sharpness of the sharpened image of the second image is N. The sharpness calculation model is trained based on the second image, the blurred image of the second image, the sharpened image of the second image, and the sharpness of the second image, the sharpness of the blurred image of the second image, and the sharpness of the sharpened image of the second image.

[0070] The perceived quality of the sharpening of the second image is greater than or equal to a preset threshold. For example, the sharpness of the second image can be the optimal sharpness, and the perceived quality of the sharpening of the second image is the highest, that is, the perceived quality of the sharpening of the second image is greater than or equal to the preset threshold. For example, the terminal device can obtain a plurality of second images in an aesthetic data set (an open-source image database, the sharpness of the images in the aesthetic data set is the optimal sharpness, and the perceived quality of the sharpening of the images is the highest), and the terminal device can also obtain a plurality of second images based on other any feasible implementation manners, which are not limited by the embodiments of the present disclosure.

[0071] It should be noted that the terminal device can refer to the process of obtaining the blurred image of the first image and the sharpened image of the first image to obtain the blurred image of the second image and the sharpened image of the second image, and the embodiments of the present disclosure will not be repeated.

[0072] Optionally, after the terminal device obtains the second image, the blurred image of the second image, and the sharpened image of the second image, the terminal device can set the sharpness of the second image as M, set the sharpness of the blurred image of the second image as L, and set the sharpness of the sharpened image of the second image as N. The L, M, and N can be labels for subsequent sharpness calculation model training. For example, in the actual application process, the terminal device can set the sharpness of the second image to 0.5, set the sharpness of the blurred image of the second image to 0, and set the sharpness of the sharpened image of the second image to 1. In this way, after the terminal device obtains the sharpness of the image, the terminal device can accurately evaluate the sharpening quality of the image based on the value of the sharpness, and improve the accuracy of determining the sharpening quality of the image.

[0073] Optionally, the terminal device trains the sharpness calculation model based on the second image, the blurred image of the second image, the sharpened image of the second image, and the sharpness of the second image, the sharpness of the blurred image of the second image, and the sharpness of the sharpened image of the second image. Specifically, the terminal device can process the second image based on the sharpness calculation model to obtain a predicted sharpness of the second image. The terminal device can process the blurred image of the second image based on the sharpness calculation model to obtain a predicted sharpness of the blurred image of the second image. The terminal device can process the sharpened image of the second image based on the sharpness calculation model to obtain a predicted sharpness of the sharpened image of the second image. The terminal device trains the sharpness calculation model based on a loss between the sharpness of the second image and the predicted sharpness of the second image, a loss between the sharpness of the blurred image of the second image and the predicted sharpness of the blurred image of the second image, and a loss between the sharpness of the sharpened image of the second image and the predicted sharpness of the sharpened image of the second image.

[0074] It should be noted that the process of training the sharpness calculation model based on the above losses can refer to the process of training any model, and the embodiments of the present disclosure will not be repeated.

[0075] S206, determine the sharpening quality of the first image based on the perceived quality of sharpening and the sharpness.

[0076] Optionally, the greater the perceived quality of sharpening of the first image, the higher the sharpening quality of the first image, and the smaller the difference between the sharpness and M, the higher the sharpening quality of the first image. For example, in the process of actual application, if the score of the perceived quality of sharpening is higher, it means that the first image is sharpened better, and the sharpening quality of the first image is higher. Since M is the optimal sharpness in the process of training the sharpness calculation model, if the sharpness of the first image is closer to M, it means that the sharpening quality of the first image is better. Therefore, the terminal device processes the perceived quality of sharpening and the sharpness based on any feasible implementation manner to obtain the sharpening quality of the first image, such as determining the product of the perceived quality of sharpening of the first image and the sharpness as the sharpening quality of the first image, and the like. The embodiments of the present disclosure do not limit this.

[0077] Optionally, after the terminal device determines the sharpening quality of the first image, it can also obtain the sharpening quality corresponding to the original image before the first image is sharpened, and determine whether to perform sharpening processing on the original image based on the sharpening quality of the first image and the sharpening quality corresponding to the original image. For example, if the sharpening quality of the first image is greater than the sharpening quality of the original image before the first image is sharpened, it means that the terminal device can improve the sharpening quality of the image after sharpening the original image. If the sharpening quality of the first image is less than or equal to the sharpening quality of the original image, the terminal device can not perform sharpening processing on the original image.

[0078] Optionally, after the terminal device determines the sharpness of the first image, the terminal device can determine whether the first image is over-sharpened. For example, the sharpness output by the sharpness calculation model is a number between 0 and 1. If the sharpness of the first image is 0.5, the terminal device can determine that the sharpness of the first image is good. If the sharpness of the first image is greater than 0.75, the terminal device can determine that the first image is over-sharpened.

[0079] Optionally, the terminal device can determine the sharpening parameter of the sharpening algorithm based on the sharpening quality of the first image. For example, the terminal device can obtain a first image obtained by the sharpening algorithm based on different sharpening parameters for sharpening the original image. The terminal device can obtain the sharpening quality of each first image, and determine the sharpening parameter corresponding to the first image with the highest sharpening quality as the optimal sharpening parameter.

[0080] The embodiment of the present disclosure provides an image processing method. The terminal device can obtain a first image, blur the first image to obtain a blurred image, and sharpen the first image to obtain a sharpened image. The first image similarity between the first image and the blurred image is determined based on the first image and the blurred image. The second image similarity between the first image and the sharpened image is determined based on the first image and the sharpened image. The perceived quality of the sharpening of the first image is determined based on the first image, the first image similarity, and the second image similarity. The sharpness of the first image is determined based on the first image. The sharpening quality of the first image is determined based on the perceived quality of the sharpening and the sharpness. In the above method, the terminal device can use the first image similarity and the second image similarity as a reference to determine the perceived quality of the sharpening. Therefore, the terminal device can accurately determine the perceived quality of the sharpening of the first image. In addition, the terminal device can evaluate the sharpening quality of the first image by combining the sharpness of the first image and the perceived quality of the sharpening, thereby improving the accuracy of determining the sharpening quality of the image.

[0081] Based on the embodiment shown in FIG. 2, the method for determining the perceived quality of the sharpening of the first image based on the first image, the blurred image, and the sharpened image in the above image processing method will be described below in combination with FIG. 7.

[0082] FIG. 7 is a schematic diagram of a method for determining the perceived quality of the sharpening according to an embodiment of the present disclosure. Please refer to FIG. 7. The method flow includes the following steps:

[0083] S701, determine the first image similarity between the first image and the blurred image based on the first image and the blurred image.

[0084] Optionally, the terminal device can determine the first image similarity between the first image and the blurred image based on the following feasible implementation manner: performing convolution processing on the first image to obtain a feature image of the first image, performing convolution processing on the blurred image to obtain a feature image of the blurred image, and calculating the similarity between the feature image of the first image and the feature image of the blurred image to obtain the first image similarity.

[0085] Optionally, the terminal device can perform convolution processing on the first image based on the convolutional neural network to obtain a feature image of the first image, perform processing on the blurred image of the first image based on the convolutional neural network to obtain a feature image of the blurred image of the first image, and determine the feature image of the first image and the feature image of the blurred image based on any feasible implementation manner, which is not limited in the embodiments of the present disclosure.

[0086] Optionally, the similarity between the feature image of the first image and the feature image of the blurred image can be the first image similarity. For example, the terminal device can calculate the cosine similarity between the feature image of the first image and the feature image of the blurred image, and determine the cosine similarity as the first image similarity. For example, the terminal device can calculate the Euclidean distance between the feature image of the first image and the feature image of the blurred image, and determine the Euclidean distance as the first image similarity.

[0087] It should be noted that the terminal device can process the feature image of the first image and the feature image of the blurred image of the first image based on any feasible implementation manner to obtain the first image similarity, which is not limited in the embodiments of the present disclosure.

[0088] Next, the process of determining the first image similarity will be described in combination with FIG. 8.

[0089] FIG. 8 is a process schematic diagram of determining the first image similarity provided by an embodiment of the present disclosure. Please refer to FIG. 8, which includes a first image, a blurred image of the first image, a convolutional neural network A and a convolutional neural network B. Wherein, the convolutional neural network A and the convolutional neural network B share parameters (i.e., the parameters in the convolutional neural network A and the convolutional neural network B are the same). The terminal device (not shown in FIG. 8) can process the first image based on the convolutional neural network A to obtain a feature image 1 of the first image, and process the blurred image based on the convolutional neural network B to obtain a feature image 2 of the blurred image of the first image.

[0090] Referring to FIG. 8, the terminal device performs dimension reduction processing on the feature image 1 and the feature image 2 to obtain an image dimension reduction feature a corresponding to the feature image 1 and an image dimension reduction feature b corresponding to the feature image 2. The terminal device can calculate a cosine similarity between the image dimension reduction feature a and the image dimension reduction feature b to obtain a first image similarity between the first image and the blurred image of the first image. In this way, the terminal device can take the blurred image as a reference and accurately evaluate the perceived quality of the sharpening of the first image through the first image similarity, thereby improving the accuracy of the evaluation of the sharpening quality of the first image.

[0091] S702, based on the first image and the sharpened image, determining a second image similarity between the first image and the sharpened image.

[0092] Optionally, the terminal device can determine the second image similarity between the first image and the sharpened image based on the following feasible implementation manner: performing convolution processing on the first image to obtain a feature image of the first image, performing convolution processing on the sharpened image to obtain a feature image of the sharpened image, and calculating a similarity between the feature image of the first image and the feature image of the sharpened image to obtain the second image similarity.

[0093] Optionally, the terminal device can perform convolution processing on the first image based on a convolutional neural network to obtain a feature image of the first image, the terminal device can perform processing on the sharpened image of the first image based on the convolutional neural network to obtain a feature image of the sharpened image of the first image, and the terminal device can also determine the feature image of the first image and the feature image of the sharpened image based on any feasible implementation manner, which is not limited in the embodiments of the present disclosure.

[0094] Optionally, the similarity between the feature image of the first image and the feature image of the sharpened image can be the second image similarity. For example, the terminal device can calculate a cosine similarity between the feature image of the first image and the feature image of the sharpened image, and determine the cosine similarity as the second image similarity. For example, the terminal device can calculate an Euclidean distance between the feature image of the first image and the feature image of the sharpened image, and determine the Euclidean distance as the second image similarity.

[0095] It should be noted that the terminal device can process the feature image of the first image and the feature image of the sharpened image of the first image based on any feasible implementation manner to obtain the second image similarity, which is not limited in the embodiments of the present disclosure.

[0096] Next, the process of determining the second image similarity will be described in conjunction with FIG. 9.

[0097] FIG. 9 is a schematic diagram of a process for determining a second image similarity according to an embodiment of the present disclosure. As shown in FIG. 9, the process includes a first image, a sharpened image of the first image, a convolutional neural network A, and a convolutional neural network B. The convolutional neural network A and the convolutional neural network B share parameters. A terminal device (not shown in FIG. 9) can process the first image based on the convolutional neural network A to obtain a feature image 1 of the first image, and can process the sharpened image based on the convolutional neural network B to obtain a feature image 2 of the sharpened image.

[0098] As shown in FIG. 9, the terminal device can perform dimension reduction processing on the feature image 1 and the feature image 2 to obtain an image dimension reduction feature a corresponding to the feature image 1 and an image dimension reduction feature b corresponding to the feature image 2. The terminal device can calculate a cosine similarity between the image dimension reduction feature a and the image dimension reduction feature b to obtain a second image similarity between the first image and the sharpened image of the first image. In this way, the terminal device can take the sharpened image as a reference and accurately evaluate the perceived quality of the sharpening of the first image through the second image similarity, thereby improving the accuracy of the evaluation of the sharpening quality of the first image.

[0099] It should be noted that the convolutional neural network in the embodiments of the present disclosure can be a VGG convolutional neural network. After the features at different levels in the VGG network are subjected to pooling and concatenation operations, the multi-level fusion features generated can improve the expression capability of the features.

[0100] It should be noted that in the embodiments of the present disclosure, the terminal device can train the convolutional neural network for extracting the feature image based on any feasible implementation manner, which is not limited in the embodiments of the present disclosure.

[0101] S703, determining the perceived quality of the sharpening of the first image based on the first image, the first image similarity, and the second image similarity.

[0102] The terminal device can determine the perceived quality of the sharpening of the first image based on the following feasible implementation manner: performing convolution processing on the first image to obtain a feature image of the first image. Based on the feature image of the first image, determining a weight of the first image similarity or the second image similarity in the perceived quality of the sharpening. Based on the weight, the first similarity, and the second similarity, calculating the perceived quality of the sharpening of the first image.

[0103] The process of performing convolution processing on the first image to obtain the feature image of the first image will not be described herein. It should be noted that in the embodiments of the present disclosure, the convolution neural network used by the terminal device in the process of performing convolution processing on the first image to obtain the feature image of the first image is the same, and the parameters in the convolution neural network are also the same. In actual application, the terminal device can obtain the feature image of the first image when calculating the first image similarity or the second image similarity.

[0104] The weight can be a weight of the first image similarity in the sharpened perceptual quality, or a weight of the second image similarity in the sharpened perceptual quality. For example, if the weight is the proportion of the first image similarity in the sharpened perceptual quality, the proportion of the second image similarity in the sharpened perceptual quality is 1-the weight; if the weight is the proportion of the second image similarity in the sharpened perceptual quality, the proportion of the first image similarity in the sharpened perceptual quality is 1-the weight.

[0105] Optionally, the terminal device can determine the weight based on the following possible implementation manner: performing multiple linear convolution processing on the feature image of the first image to obtain the weight of the first image similarity or the second image similarity in the sharpened perceptual quality. For example, the terminal device can process the feature image of the first image based on multiple linear layers (Linear), and then obtain a fusion coefficient, that is, the fusion coefficient can be the proportion of the first image similarity or the second image similarity in the sharpened perceptual quality, the fusion coefficient is greater than or equal to 0 and less than or equal to 1.

[0106] Next, the process of determining the weight of the first image similarity or the second image similarity in the sharpened perceptual quality will be described in conjunction with FIG. 10.

[0107] FIG. 10 is a schematic diagram of determining a weight provided by an embodiment of the present disclosure. Please refer to FIG. 10, which includes a first image, a convolutional neural network, and multiple linear convolution layers. The terminal device (not shown in FIG. 10) can input the first image to the convolutional neural network, the convolutional neural network can output the feature image corresponding to the first image, and the terminal device can input the feature image to the multiple linear convolution layers, and the multiple linear convolution layers can output the weight, which can be a number between 0 and 1. In this way, the terminal device can accurately determine the position of the maximum perceptual quality of the sharpness of the first image, the distance from the over-fuzzy distance and the over-sharpened distance based on the weight, and then accurately obtain the perceptual quality of the sharpness of the first image.

[0108] It should be noted that, in the embodiments of the present disclosure, since the optimal sharpness corresponding to different first images is different (for example, the optimal sharpness of an image with less texture is larger, and the optimal sharpness of an image with more texture is smaller), the weight determined by the terminal device based on the feature image of the first image can accurately adjust the proportion of the blurred image and the sharp image in the perceived quality of the sharpness of the first image, thereby improving the accuracy of the perceived quality of the sharpness of the first image.

[0109] Optionally, the terminal device can calculate the perceived quality of the sharpening of the first image based on the following formula: q = a x Dist1 + (1-a) x Dist2

[0110] wherein q is the perceived quality of the sharpening of the first image; a is the proportion of the first image similarity in the perceived quality of the sharpening; Dist1 is the first image similarity; and Dist2 is the second image similarity. The terminal device can obtain the perceived quality of the sharpening of the first image based on the above formula.

[0111] Optionally, the above image processing method can be obtained based on a sharpening quality evaluation model, for example, the terminal device can input the first image into the sharpening quality evaluation model, and the sharpening quality evaluation model can output the perceived quality of the sharpening of the first image and the sharpness. In the following, the training process of the sharpening quality evaluation model will be described in detail in combination with FIG. 11.

[0112] FIG. 11 is a training diagram of a sharpening quality evaluation model provided by an embodiment of the present disclosure. Please refer to FIG. 11, which includes a sharpening quality evaluation model 1, a sharpening quality evaluation model 2, an image A and an image B. Wherein the image A is an image with optimal sharpness, the image B is an image obtained by blurring the image A, the sharpening quality evaluation model 1 and the sharpening quality evaluation model 2 share model parameters, the sharpening quality evaluation model 1 includes a similarity calculation module A, a similarity calculation module B, a sharpness calculation module, a weight calculation module and a weighting module, and the sharpening quality evaluation model 2 includes a similarity calculation module C, a similarity calculation module D, a weighting module, a weight calculation module and a sharpness calculation module.

[0113] Please refer to FIG. 11, the sharpening quality evaluation model 1 can process the image A. Wherein the image A is blurred to obtain a blurred image, and the image A is sharpened to obtain a sharpened image. The similarity calculation module A processes the image A and the blurred image to obtain the image similarity 1 between the image A and the blurred image. The similarity calculation module B processes the image A and the sharpened image to obtain the image similarity 2 between the image A and the sharpened image.

[0114] Referring to FIG. 11, the sharpening quality evaluation model 1 inputs the feature image of the image A output by the similarity calculation module A to the sharpness calculation module to obtain the predicted sharpness of the image A, and inputs the feature image of the image A to the weight calculation module to obtain the weight m. The sharpening quality evaluation model 1 inputs the image similarity 1 between the image A and the blurred image, the image similarity 2 between the image A and the sharpened image, and the weight m to the weighting module to obtain the perceived quality of the sharpening of the image A. The sharpening quality evaluation model 1 can obtain the first loss based on the predicted sharpness of the image A and the true sharpness of the image A.

[0115] Referring to FIG. 11, the sharpening quality evaluation model 2 can process the image B obtained by blurring the image A. The image B is blurred to obtain a blurred image, and the image B is sharpened to obtain a sharpened image. The similarity calculation module C processes the image B and the blurred image to obtain the image similarity 3 between the image A and the blurred image. The similarity calculation module D processes the image B and the sharpened image to obtain the image similarity 4 between the image B and the sharpened image.

[0116] Referring to FIG. 11, the sharpening quality evaluation model 2 inputs the feature image of the image B output by the similarity calculation module D to the sharpness calculation module to obtain the predicted sharpness of the image B, and inputs the feature image of the image B to the weight calculation module to obtain the weight n. The sharpening quality evaluation model 2 inputs the image similarity 3 between the image B and the blurred image, the image similarity 4 between the image B and the sharpened image, and the weight n to the weighting module to obtain the perceived quality of the sharpening of the image B. The sharpening quality evaluation model 2 can obtain the second loss based on the predicted sharpness of the image B and the true sharpness of the image B.

[0117] Referring to FIG. 11, since the image B is obtained by blurring the image A, and the image A is an optimal sharpness image, the perceived quality a of the sharpening of the image A output by the sharpening quality evaluation model 1 should be greater than the perceived quality b of the sharpening of the image B output by the sharpening quality evaluation model 2. Therefore, the sizes of the perceived quality a of the sharpening and the perceived quality b of the sharpening are compared, and the third loss is determined based on the result of the comparison and the perceived quality a of the sharpening being greater than the perceived quality b of the sharpening. The sharpening quality evaluation model 1 can be trained based on the first loss and the third loss, and the sharpening quality evaluation model 2 can be trained based on the second loss and the third loss.

[0118] In this way, the training method based on the twin model can improve the accuracy of the generated perceived quality of the sharpening and the accuracy of the sharpness, and further improve the accuracy of determining the sharpening quality of the image.

[0119] It should be noted that the calculation process of the similarity calculation module in FIG. 11 can refer to the embodiments shown in FIG. 8 or FIG. 9, and the present disclosure will not repeat them.

[0120] Optionally, the training data set of the sharpening quality evaluation model can be constructed based on the following method: obtaining a full-reference quality evaluation data set with subjective score annotation to constrain the parameters in the similarity calculation module. As shown in FIG. 11, image A and image B, images in a large-scale aesthetic data set can be selected, and the images in the aesthetic data set are determined as the images with optimal sharpness, so that after sharpening or blurring of the images, the perceived quality of the sharpened images will decrease. For example, if the image with optimal sharpness is selected as the input of the first sharpening quality evaluation model, the image with optimal sharpness can be sharpened or blurred to obtain a sharpened image or a blurred image, and the sharpened image or the blurred image is used as the input of the second sharpening quality evaluation model. In this way, the perceived quality of sharpening output by the first sharpening quality evaluation model can be greater than the perceived quality of sharpening output by the second sharpening quality evaluation model (i.e., if the perceived quality of sharpening output by the first sharpening quality evaluation model is greater than the perceived quality of sharpening output by the second sharpening quality evaluation model, 1 is output, if the perceived quality of sharpening output by the first sharpening quality evaluation model is less than or equal to the perceived quality of sharpening output by the second sharpening quality evaluation model, 0 is output, and the real label is also a 0-1 label).

[0121] Optionally, after blurring or sharpening the image with optimal sharpness, the blurred or sharpened image can be scored, and then the twin model can be trained. For example, the score of the image with optimal sharpness is 5, and the score of the distorted image after blurring or sharpening can be less than 5, so that a ranking of multiple images in a training sample set can be obtained, and a real label can be obtained.

[0122] The embodiments of the present disclosure provide a method for determining the perceived quality of sharpening, determining a first image similarity between a first image and a blurred image based on the first image and the blurred image. A second image similarity between the first image and a sharpened image is determined based on the first image and the sharpened image. The first image is convoluted to obtain a feature image of the first image. The weight of the first image similarity or the second image similarity in the perceived quality of sharpening is determined based on the feature image of the first image. The perceived quality of sharpening of the first image is calculated based on the weight, the first similarity and the second similarity. In this way, the terminal device can determine the perceived quality of sharpening of the first image by combining the similarity between the first image and the highly blurred image, and the similarity between the first image and the over-sharpened image, thereby improving the accuracy of determining the perceived quality of sharpening.

[0123] FIG. 12 is a structural schematic diagram of an image processing apparatus provided by an embodiment of the present disclosure. Referring to FIG. 12, the image processing apparatus 120 comprises an acquisition module 121, a processing module 122, a first determination module 123, a second determination module 124, and a third determination module 125, wherein:

[0124] The acquisition module 121 is configured to acquire a first image.

[0125] The processing module 122 is configured to perform blur processing on the first image to obtain a blurred image.

[0126] The processing module 122 is further configured to perform sharpening processing on the first image to obtain a sharpened image.

[0127] The first determination module 123 is configured to determine a perceived quality of sharpening of the first image based on the first image, the blurred image, and the sharpened image.

[0128] The second determination module 124 is configured to determine a sharpness of the first image based on the first image.

[0129] The third determination module 125 is configured to determine a sharpening quality of the first image based on the perceived quality of sharpening and the sharpness.

[0130] According to one or more embodiments of the present disclosure, the first determination module 123 is specifically configured to:

[0131] determine a first image similarity between the first image and the blurred image based on the first image and the blurred image;

[0132] determine a second image similarity between the first image and the sharpened image based on the first image and the sharpened image;

[0133] determine the perceived quality of sharpening of the first image based on the first image, the first image similarity, and the second image similarity.

[0134] According to one or more embodiments of the present disclosure, the first determination module 123 is specifically configured to:

[0135] perform convolution processing on the first image to obtain a feature image of the first image;

[0136] determine a weight of the first image similarity or the second image similarity in the perceived quality of sharpening based on the feature image of the first image;

[0137] calculate the perceived quality of sharpening of the first image based on the weight, the first image similarity, and the second image similarity.

[0138] According to one or more embodiments of the present disclosure, the first determining module 123 is specifically configured to:

[0139] performing multiple linear convolution processing on the feature image of the first image to obtain a weight of the first image similarity or the second image similarity in the sharpened perceptual quality.

[0140] According to one or more embodiments of the present disclosure, the first determining module 123 is specifically configured to:

[0141] performing convolution processing on the first image to obtain a feature image of the first image;

[0142] performing convolution processing on the blurred image to obtain a feature image of the blurred image;

[0143] calculating a similarity between the feature image of the first image and the feature image of the blurred image to obtain the first image similarity.

[0144] According to one or more embodiments of the present disclosure, the first determining module 123 is specifically configured to:

[0145] performing convolution processing on the first image to obtain a feature image of the first image;

[0146] performing convolution processing on the sharpened image to obtain a feature image of the sharpened image;

[0147] calculating a similarity between the feature image of the first image and the feature image of the sharpened image to obtain the second image similarity.

[0148] According to one or more embodiments of the present disclosure, the second determining module 124 is specifically configured to:

[0149] performing convolution processing on the first image to obtain a feature image of the first image;

[0150] processing the feature image of the first image based on a sharpness calculation model to obtain a sharpness of the first image.

[0151] According to one or more embodiments of the present disclosure, the second determining module 124 is specifically configured to:

[0152] obtaining a second image, a blurred image of the second image, and a sharpened image of the second image, a sharpened perceptual quality of the second image being greater than or equal to a preset threshold;

[0153] determine a sharpness of the second image as M, a sharpness of a blurred image of the second image as L, a sharpness of a sharpened image of the second image as N, the L being less than the M, the M being less than the N;

[0154] train the sharpness calculation model based on the second image, the blurred image of the second image, the sharpened image of the second image, and the sharpness of the second image, the sharpness of the blurred image of the second image, and the sharpness of the sharpened image of the second image.

[0155] The image processing apparatus provided by the embodiments of the present disclosure can be used to execute the technical solutions of the method embodiments, and has similar implementation principles and technical effects, which will not be described here again.

[0156] FIG. 13 is a structural schematic diagram of a terminal device provided by an embodiment of the present disclosure. Please refer to FIG. 13, which shows a structural schematic diagram of a terminal device 1300 suitable for implementing the embodiments of the present disclosure. The terminal device can include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, personal digital assistants (PDA), tablet computers (PAD), portable media players (PMP), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. The terminal device shown in FIG. 13 is only an example, and should not bring any limitation to the functions and use range of the embodiments of the present disclosure.

[0157] As shown in FIG. 13, the terminal device 1300 can include a processing apparatus (such as a central processor, a graphics processor, etc.) 1301, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 1302 or loaded from a storage apparatus 1308 to a random access memory (RAM) 1303. In the RAM 1303, various programs and data required for the operation of the terminal device 1300 are also stored. The processing apparatus 1301, the ROM 1302, and the RAM 1303 are connected to each other through a bus 1304. An input / output (I / O) interface 1305 is also connected to the bus 1304.

[0158] In general, the following devices can be connected to the I / O interface 1305: input devices 1306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, and the like; output devices 1307 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage devices 1308 including, for example, a magnetic tape, a hard disk, and the like; and communication devices 1309. The communication devices 1309 can allow the terminal device 1300 to communicate wirelessly or wired with other devices to exchange data. While FIG. 13 shows the terminal device 1300 with various devices, it is understood that not all of the shown devices are required to be implemented or present. More or less devices can alternatively be implemented or present.

[0159] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for executing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network through the communication devices 1309, or installed from the storage devices 1308, or installed from the ROM 1302. When the computer program is executed by the processing devices 1301, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are performed.

[0160] It should be noted that the computer readable medium in the above disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples of computer readable storage media can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or apparatus. In the present disclosure, the computer readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer readable program code. Such a propagated data signal can take various forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination of the above. The computer readable signal medium can also be any computer readable medium other than the computer readable storage medium, which can send, propagate or transmit a program for use by or in conjunction with an instruction execution system, device or apparatus. The program code contained in the computer readable medium can be transmitted by any suitable medium, including but not limited to a wire, a cable, an RF (radio frequency) or the like, or any suitable combination of the above.

[0161] The computer readable medium described above can be contained in the terminal device described above; or can exist separately and not be assembled into the terminal device.

[0162] The computer readable medium described above carries one or more programs, which, when executed by the terminal device, cause the terminal device to perform the method shown in the above embodiments.

[0163] The embodiment of the present disclosure provides a computer readable storage medium, which stores computer execution instructions, and when a processor executes the computer execution instructions, the image processing method described above is realized.

[0164] The embodiment of the present disclosure provides a computer program product, which includes a computer program, and when a processor executes the computer program, the image processing method described above is realized.

[0165] Computer program code for carrying out operations of the present disclosure can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0166] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flow diagrams or block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may

[0167] The units described in the embodiments of the present disclosure can be implemented by software, or by hardware. In some cases, the name of the unit does not constitute a limitation on the unit itself. For example, the first obtaining unit can also be described as a unit for obtaining at least two Internet protocol addresses.

[0168] The functions described above in the specification of the present disclosure can be performed by one or more hardware logic components. For example, and without limitation, illustrative types of hardware logic components that can be used include Field-programmable Gate Arrays (FPGAs), Program-specific Integrated Circuits (ASICs), Program-specific Standard Products (ASSPs), System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), etc.

[0169] In the context of this disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more of: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0170] It should be noted that the modification of "one" or "multiple" mentioned in this disclosure is illustrative rather than limiting, and those skilled in the art should understand that "one" or "multiple" should be understood unless the context clearly indicates otherwise.

[0171] The names of the messages or information exchanged between the plurality of devices in the embodiments of the present disclosure are only for illustrative purposes, and are not intended to limit the scope of the messages or information.

[0172] It can be understood that, before using the technical solutions disclosed in the embodiments of the present disclosure, the type, use range, use scenario, etc. of the personal information involved in the present disclosure should be informed to the user and the authorization of the user should be obtained through appropriate means according to relevant laws and regulations.

[0173] For example, in response to receiving the active request of the user, a prompt information is sent to the user to explicitly prompt the user that the operation requested to be performed will require obtaining and using the personal information of the user. Thus, the user can voluntarily choose whether to provide the personal information to the terminal device, application program, server or storage medium, etc. software or hardware performing the operation of the technical solutions of the present disclosure according to the prompt information. As an optional but non-limiting implementation manner, in response to receiving the active request of the user, the manner of sending the prompt information to the user may, for example, be a pop-up window manner, and the prompt information may, for example, be presented in the form of text in the pop-up window. In addition, the pop-up window may, for example, carry a selection control for the user to select "agree" or "disagree" to provide the personal information to the terminal device.

[0174] It can be understood that the above notification and user authorization process is only illustrative, and does not limit the implementation manner of the present disclosure, and other manners meeting the relevant laws and regulations can also be applied to the implementation manner of the present disclosure.

[0175] It can be understood that the data (including but not limited to the data itself, acquisition or use of the data) involved in the technical solution should comply with the requirements of the corresponding laws and regulations and relevant provisions. The data can include information, parameters and messages, etc., such as the flow splitting indication information.

[0176] The above description is merely preferred embodiments of the present disclosure and a description of the principles of the technology used. Those skilled in the art should understand that the disclosed scope of the present disclosure is not limited to the technical solutions formed by the specific combinations of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features disclosed in the present disclosure (but not limited to) having similar functions to form technical solutions. It should be noted that steps S201, S202, etc. in the embodiments of the present disclosure are not limited to the execution order.

[0177] In addition, although each operation is described in a specific order, this should not be understood as requiring the operations to be performed in the specific order shown or in a sequential order. In certain circumstances, multitasking and parallel processing can be advantageous. Similarly, although specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Certain features described in the context of separate embodiments can also be combined in a single embodiment. Conversely, various features described in the context of a single embodiment can also be implemented separately or in any suitable subcombination in multiple embodiments.

[0178] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. An image processing method comprising: obtaining a first image; obtaining a blurred image by blurring the first image; obtaining a sharpened image by sharpening the first image; determining a perceived quality of sharpening of the first image based on the first image, the blurred image and the sharpened image; determining a sharpness of the first image based on the first image; determining a quality of sharpening of the first image based on the perceived quality of sharpening and the sharpness.

2. The method of claim 1, wherein the determining the perceived quality of sharpening of the first image based on the first image, the blurred image and the sharpened image comprises: determining a first image similarity between the first image and the blurred image based on the first image and the blurred image; determining a second image similarity between the first image and the sharpened image based on the first image and the sharpened image; determining the perceived quality of sharpening of the first image based on the first image, the first image similarity and the second image similarity.

3. The method of claim 2, wherein the determining the perceived quality of sharpening of the first image based on the first image, the first image similarity and the second image similarity comprises: obtaining a feature image of the first image by convolving the first image; determining a weight of the first image similarity or the second image similarity in the perceived quality of sharpening based on the feature image of the first image; calculating the perceived quality of sharpening of the first image based on the weight, the first image similarity and the second image similarity.

4. The method of claim 3, wherein the determining the weight of the first image similarity or the second image similarity in the perceived quality of sharpening based on the feature image of the first image comprises: obtaining the weight of the first image similarity or the second image similarity in the perceived quality of sharpening by performing multiple linear convolutions on the feature image of the first image.

5. The method of claim 2, wherein the determining the first image similarity between the first image and the blurred image based on the first image and the blurred image comprises: obtaining a feature image of the first image by convolving the first image; obtaining a feature image of the blurred image by convolving the blurred image; calculating the first image similarity between the feature image of the first image and the feature image of the blurred image.

6. The method of claim 2, wherein the determining the second image similarity between the first image and the sharpened image based on the first image and the sharpened image comprises: obtaining a feature image of the first image by convolving the first image; obtaining a feature image of the sharpened image by convolving the sharpened image; calculate a similarity between the feature image of the first image and the feature image of the sharpened image, to obtain the second image similarity.

7. The method of any one of claims 1-6, wherein the determining the sharpness of the first image based on the first image comprises: performing convolution processing on the first image to obtain a feature image of the first image; processing the feature image of the first image based on a sharpness calculation model to obtain the sharpness of the first image.

8. The method of claim 6, wherein the sharpness calculation model is obtained based on the following steps: obtaining a second image, a blurred image of the second image, and a sharpened image of the second image, the sharpened image of the second image having a perceptual quality of sharpening greater than or equal to a preset threshold; determining that a sharpness of the second image is M, a sharpness of the blurred image of the second image is L, and a sharpness of the sharpened image of the second image is N, the L being less than the M, and the M being less than the N; training the sharpness calculation model based on the second image, the blurred image of the second image, the sharpened image of the second image, and the sharpness of the second image, the sharpness of the blurred image of the second image, and the sharpness of the sharpened image of the second image.

9. An image processing apparatus, comprising an obtaining module, a processing module, a first determining module, a second determining module, and a third determining module, wherein: the obtaining module is configured to obtain a first image; the processing module is configured to perform blurring processing on the first image to obtain a blurred image; the processing module is further configured to perform sharpening processing on the first image to obtain a sharpened image; the first determining module is configured to determine a perceptual quality of sharpening of the first image based on the first image, the blurred image, and the sharpened image; the second determining module is configured to determine a sharpness of the first image based on the first image; the third determining module is configured to determine a sharpening quality of the first image based on the perceptual quality of sharpening and the sharpness.

10. A terminal device comprising: a processor and a memory; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that the processor performs the image processing method of any one of claims 1-8.

11. A computer-readable storage medium, the computer-readable storage medium storing computer-executable instructions, when a processor executes the computer-executable instructions, implementing the image processing method of any one of claims 1-8.

Images