Image processing method and device and terminal equipment

Abstract

The invention provides an image processing method and apparatus, and a terminal device. The method comprises the steps of obtaining a first image; performing fuzzy processing on the first image to obtain a fuzzy image; performing sharpening processing on the first image to obtain a sharpened image; based on the first image, the blurred image and the sharpened image, determining the perceived quality of sharpening of the first image; determining the sharpness of the first image based on the first image; and based on the sharpened perceived quality and the sharpness, determining the sharpening quality of the first image.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of image processing technology, and in particular, to an image processing method, apparatus, and terminal device. Background Art

[0002] Image sharpening technology can enhance the sharpness of an image and improve the quality of the image perceived by the human eye. However, over-sharpening an image can also reduce the quality of the image perceived by the human eye. Therefore, it is particularly important to determine the image quality after image sharpening.

[0003] At present, the terminal device can input the sharpened image into the image sharpening evaluation model, and the image sharpening evaluation model can output the sharpening score corresponding to the sharpened image. The user can determine whether the image is over-sharpened or under-sharpened based on the sharpening score. However, the user can only estimate the sharpening quality of the sharpened image based on experience, and cannot accurately determine whether the poor sharpening quality is caused by over-sharpening or under-sharpening. How to accurately evaluate the sharpening quality of the sharpened image is an urgent problem to be solved. Summary of the invention

[0004] The present disclosure provides an image processing method, apparatus and terminal device, which are used to solve one or more technical problems in the prior art.

[0005] In a first aspect, the present disclosure provides an image processing method, the method comprising:

[0006] acquiring a first image;

[0007] Performing blur processing on the first image to obtain a blurred image;

[0008] Performing sharpening processing on the first image to obtain a sharpened image;

[0009] determining a perceptual quality of sharpening of the first image based on the first image, the blurred image, and the sharpened image;

[0010] Based on the first image, determining a sharpness of the first image;

[0011] A sharpening quality of the first image is determined based on the perceived quality of the sharpening and the sharpness.

[0012] In a second aspect, the present disclosure provides an image processing device, the image processing device comprising an acquisition module, a processing module, a first determination module, a second determination module and a third determination module, wherein:

[0013] The acquisition module is used to acquire a first image;

[0014] The processing module is used to perform blur processing on the first image to obtain a blurred image;

[0015] The processing module is further used to perform sharpening processing on the first image to obtain a sharpened image;

[0016] The first determination module is used to determine the sharpened perceptual quality of the first image based on the first image, the blurred image and the sharpened image;

[0017] The second determination module is used to determine the sharpness of the first image based on the first image;

[0018] A sharpening quality of the first image is determined based on the perceived quality of the sharpening and the sharpness.

[0019] In a third aspect, an embodiment of the present disclosure provides a terminal device including: a processor and a memory;

[0020] The memory stores computer-executable instructions;

[0021] The processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the image processing method as described in the first aspect and various possible aspects of the first aspect.

[0022] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, in which computer execution instructions are stored. When a processor executes the computer execution instructions, the image processing method as described in the first aspect and various possible aspects of the first aspect are implemented.

[0023] The present disclosure provides an image processing method, apparatus and terminal device. The terminal device can obtain a first image, blur the first image to obtain a blurred image, and sharpen the first image to obtain a sharpened image. The terminal device can determine the perceptual quality of sharpening of the first image based on the first image, the blurred image and the sharpened image, determine the sharpening of the first image based on the first image, and determine the sharpening quality of the first image based on the perceptual quality and sharpness of the sharpening. In the above method, since the terminal device can evaluate the perceptual quality of sharpening of the first image based on the blurred image and the sharpened image as references, thereby improving the accuracy of determining the perceptual quality of sharpening, and the terminal device can determine the sharpening quality of the first image after sharpening in combination with the perceptual quality of sharpening of the first image and the sharpness of the first image, thereby improving the accuracy of determining the sharpening quality of the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0025] Figure 1 A schematic diagram of an application scenario provided by an embodiment of the present disclosure;

[0026] Figure 2 A flowchart of an image processing method provided by an embodiment of the present disclosure;

[0027] Figure 3 A schematic diagram of a process for determining a blurred image provided by an embodiment of the present disclosure;

[0028] Figure 4 A schematic diagram of a process for determining a sharpened image provided by an embodiment of the present disclosure;

[0029] Figure 5 A schematic diagram of the relationship between the perceived quality and sharpness of sharpening provided in an embodiment of the present disclosure;

[0030] Figure 6 A schematic diagram of determining the sharpness of a first image provided by an embodiment of the present disclosure;

[0031] Figure 7 A schematic diagram of a method for determining the perceptual quality of sharpening provided by an embodiment of the present disclosure;

[0032] Figure 8 A schematic diagram of a process for determining a first image similarity provided by an embodiment of the present disclosure;

[0033] Fig. 9 A schematic diagram of a process for determining a second image similarity provided by an embodiment of the present disclosure;

[0034] Fig.10 A schematic diagram of determining weights provided in an embodiment of the present disclosure;

[0035] Fig.11 A training diagram of a sharpening quality assessment model provided in an embodiment of the present disclosure;

[0036] Fig.12 A schematic diagram of the structure of an image processing device provided by an embodiment of the present disclosure; and

[0037] Fig.13 A schematic diagram of the structure of a terminal device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0038] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0039] To facilitate understanding, the concepts involved in the embodiments of the present disclosure are explained below.

[0040] Terminal device: a device with wireless transceiver function. The terminal device can be deployed on land, including indoors or outdoors, handheld, wearable or vehicle-mounted. The terminal device can be a mobile phone, a tablet computer, a computer with wireless transceiver function, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a wireless terminal in industrial control, a vehicle-mounted terminal device, a wireless terminal in self-driving, a wireless terminal device in remote medical, a wireless terminal device in smart grid, a wireless terminal device in transportation safety, a wireless terminal device in smart city, a wireless terminal device in smart home, a wearable terminal device, etc. The terminal device involved in the embodiments of the present disclosure can also be called a terminal, a user equipment (UE), an access terminal device, a vehicle-mounted terminal, an industrial control terminal, a UE unit, a UE station, a mobile station, a mobile station, a remote station, a remote terminal device, a mobile device, a UE terminal device, a wireless communication device, a UE agent or a UE device, etc. Terminal devices can also be fixed or mobile.

[0041] In the related art, image sharpening can enhance the sharpness of an image and improve the quality of the image perceived by the human eye. However, over-sharpening or under-sharpening will reduce the quality of the image perceived by the human eye. Therefore, it is particularly important to determine the sharpening quality of the sharpened image. For example, if the sharpened image is over-sharpened, the sharpening parameters of the image can be reduced, and the image can be re-sharpened using the reduced sharpening parameters. Similarly, if the sharpened image is under-sharpened, the sharpening parameters of the image can be increased, and the image can be re-sharpened based on the increased sharpening parameters. At present, the terminal device can input the sharpened image into the image sharpening evaluation model, and the image sharpening evaluation model can output the sharpening score corresponding to the sharpened image. The user can determine whether the image is over-sharpened or under-sharpened based on the sharpening score. However, since the sharpness of an image is proportional to the sharpening score, the higher the image sharpening degree, the higher the sharpening score output by the image sharpening evaluation model. Therefore, users can only estimate the sharpening quality of the sharpened image based on experience. In addition, the relationship between the perceived quality of an image by the human eye and the degree of image sharpness is that it first increases and then decreases. Therefore, based on the sharpening score, users cannot accurately determine whether the poor sharpening quality of the image is caused by over-sharpening or under-sharpening, which leads to low accuracy in the evaluation of the sharpening quality of the sharpened image.

[0042] In order to solve the technical problems in the related art, the embodiment of the present disclosure provides an image processing method, wherein a terminal device can obtain a first image, blur the first image to obtain a blurred image, and sharpen the first image to obtain a sharpened image, determine the first image similarity between the first image and the blurred image based on the first image and the blurred image, determine the second image similarity between the first image and the sharpened image based on the first image and the sharpened image, determine the perceptual quality of sharpening of the first image based on the first image, the first image similarity, and the second image similarity, determine the sharpness of the first image based on the first image, and determine the sharpening quality of the first image based on the perceptual quality of sharpening and the sharpness. In the above method, since the terminal device can use the first image similarity between the first image and the blurred image and the second image similarity between the first image and the sharpened image as references for determining the sharpening quality, the terminal device can accurately evaluate the perceptual quality of sharpening of the first image, and the terminal device can accurately determine the sharpening quality of the first image after sharpening in combination with the sharpness and the perceptual quality of sharpening, thereby improving the accuracy of determining the sharpening quality of the image.

[0043] Next, combine Figure 1 , the application scenarios of the embodiments of the present disclosure are described.

[0044] Figure 1 A schematic diagram of an application scenario provided by an embodiment of the present disclosure. Figure 1, including: a terminal device. The display page of the terminal device may include a first image. The terminal device may evaluate the sharpening quality of the first image. The terminal device may display the sharpness of the first image and the perceptual quality of the sharpening of the first image, wherein the sharpness of the first image may be 0.5, the perceptual quality of the sharpening of the first image may be 70, and the terminal device may determine the sharpening quality of the first image based on the sharpness of the first image and the perceptual quality of the sharpening of the first image. In this way, since the terminal device may obtain the perceptual quality of the sharpening of the first image based on the blurred image after the blurring process of the first image and the sharpened image after the sharpening process of the first image as a reference, the accuracy of the perceptual quality of the sharpening is high, and the terminal device may determine the sharpening quality of the first image in combination with the sharpness of the first image and the perceptual quality of the sharpening, so the terminal device may accurately determine the reason when the sharpening quality of the first image is poor, and the sharpening quality of the first image may be accurately determined, thereby improving the accuracy of determining the sharpening quality.

[0045] It should be noted that Figure 1 The application scenarios of the embodiments of the present disclosure are merely illustrative and are not intended to limit the application scenarios of the embodiments of the present disclosure.

[0046] The technical solution of the present invention and how the technical solution of the present invention solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will be described below in conjunction with the accompanying drawings.

[0047] Figure 2 A flowchart of an image processing method provided by an embodiment of the present disclosure. Figure 2 , the method may include:

[0048] S201: Acquire a first image.

[0049] The execution subject of the embodiments of the present disclosure may be a terminal device, or an image processing device provided in the terminal device. The image processing device may be implemented based on software, or may be implemented based on a combination of software and hardware, which is not limited in the embodiments of the present disclosure. Optionally, the terminal device may be any device with terminal computing capabilities.

[0050] Optionally, the first image may be an image after sharpening processing. For example, the terminal device may perform sharpening processing on any image to obtain the first image. For example, the first image may be an image after sharpening processing and the sharpening quality is to be evaluated.

[0051] It should be noted that, since any image has sharpness, the first image can also be any image, and the embodiments of the present disclosure are not limited to this.

[0052] Optionally, the terminal device may receive a first image sent by another device. For example, the terminal device may associate at least one image to be evaluated for sharpness quality with a request for sharpness quality evaluation sent by another device (the terminal device may perform a sharpness quality evaluation operation after receiving the request), and the terminal device may determine the at least one image to be evaluated for sharpness quality as the first image.

[0053] Optionally, the terminal device may obtain the first image from a database. For example, a plurality of sharpened images may be pre-stored in the database, and the terminal device may obtain the plurality of images (i.e., first images) from the database. For example, a plurality of sharpened images may be pre-stored in the database, and each sharpened image may have a unique identifier. The terminal device may receive a sharpening quality assessment request sent by other devices, and the sharpening quality assessment request may include identifiers of the plurality of sharpened images. The terminal device may obtain the associated first image from the database based on the identifier.

[0054] It should be noted that the terminal device can obtain the first image based on any feasible implementation method, and the embodiments of the present disclosure are not limited to this.

[0055] S202: Perform blur processing on the first image to obtain a blurred image.

[0056] The blurred image is an image after blurring the first image. For example, the terminal device may perform Gaussian blurring on the first image to obtain a blurred image corresponding to the first image. For example, the terminal device may perform image degradation processing on the first image to obtain a blurred image corresponding to the first image. For example, the terminal device may perform blurred image simulation processing on the first image to generate a blurred simulated image corresponding to the first image. For example, the terminal device may perform downsampling processing on the first image to reduce the number of pixels in the first image to obtain a blurred image corresponding to the first image.

[0057] It should be noted that the terminal device can obtain the blurred image corresponding to the first image based on any feasible implementation method, and the embodiments of the present disclosure are not limited to this.

[0058] It should be noted that after the terminal device blurs the first image, the clarity of the obtained blurred image is lower than the clarity of the first image.

[0059] Optionally, the blurred image may be a negative sample of the first image, and the higher the blurriness of the blurred image, the better the blurred image is as a negative sample. For example, the terminal device may process the first image based on Gaussian blur to obtain a blurred image. Since the blurred image is obtained based on Gaussian blur processing, the blurriness of the blurred image is high, so the blurred image is better as a negative sample of the first image.

[0060] Next, combine Figure 3 , the process of determining the blurred image is explained.

[0061] Figure 3 A schematic diagram of a process for determining a blurred image provided by an embodiment of the present disclosure. Figure 3 , including: a first image. Terminal device ( Figure 3 The first image may be subjected to Gaussian blur processing (not shown) to obtain a blurred image corresponding to the first image. The first image and the blurred image have the same size. In this way, the terminal device may obtain a blurred image with better quality based on Gaussian blur, and the terminal device may improve the accuracy of determining the perceived quality of the sharpening of the first image based on the blurred image as a reference, thereby improving the accuracy of determining the sharpening quality of the first image.

[0062] S203: Perform sharpening processing on the first image to obtain a sharpened image.

[0063] The sharpened image is an image after the first image is sharpened. For example, the terminal device may perform sharpening processing on the first image based on a sharpening algorithm to obtain a sharpened image corresponding to the first image. The sharpening algorithm used by the terminal device may be any feasible algorithm, which is not limited in the embodiments of the present disclosure.

[0064] It should be noted that the terminal device can generate a sharpened image corresponding to the first image based on any feasible implementation method, and the embodiments of the present disclosure are not limited to this.

[0065] Optionally, the sharpened image may be a negative sample of the first image. The higher the sharpening degree of the sharpened image, the better the effect of the sharpened image as a negative sample. For example, the terminal device may process the first image based on a higher sharpening parameter to obtain a sharpened image. Since the sharpened image is obtained based on a higher sharpening parameter, the sharpening degree of the sharpened image is higher. In this way, the sharpened image has a better effect as a negative sample of the first image.

[0066] It should be noted that, in actual application, the sharpened image may be an image obtained after the terminal device sharpens the first image based on the highest sharpening parameter, which can improve the effect of the sharpened image as a negative sample of the first image.

[0067] Next, combine Figure 4 , the process of determining a sharpened image is described.

[0068] Figure 4 A schematic diagram of a process for determining a sharpened image provided by an embodiment of the present disclosure. Figure 4 , including: a first image. Terminal device ( Figure 4 The first image may be sharpened by a terminal device (not shown) to obtain a sharpened image corresponding to the first image. The first image and the sharpened image have the same size. The sharpening parameter (which may indicate the degree of sharpening of the image, the larger the sharpening parameter, the deeper the degree of sharpening) when the terminal device performs the sharpening process may be the highest sharpening parameter of the sharpening algorithm. In this way, the terminal device may obtain a sharpened image of better quality. Based on the sharpened image as a reference, the terminal device may improve the accuracy of determining the perceived quality of the sharpening of the first image, thereby improving the accuracy of determining the sharpening quality of the first image.

[0069] S204: Determine a sharpening perceptual quality of the first image based on the first image, the blurred image, and the sharpened image.

[0070] The perceived quality of the sharpening of the first image may be the perceived quality of the image by the user's human eyes. For example, in actual applications, the perceived quality of the sharpening may be a score, and the user may determine the perceived quality of the first image by the human eyes based on the score. The higher the score, the better the perceived quality of the first image by the human eyes, and the lower the score, the lower the perceived quality of the first image by the human eyes.

[0071] It should be noted that there is an optimal sharpness for the perceptual quality of sharpening of the first image, that is, if the sharpness of the first image is the optimal sharpness, the score of the perceptual quality of sharpening of the first image is the highest; if the sharpness of the first image is less than or greater than the optimal sharpness, the score of the perceptual quality of sharpening of the first image is less than the highest score corresponding to the optimal sharpness.

[0072] Next, combine Figure 5 , the relationship between the perceptual quality of sharpening and sharpness is explained.

[0073] Figure 5 A schematic diagram of the relationship between the perceived quality and sharpness of a sharpening provided by an embodiment of the present disclosure. Figure 5 , including: a coordinate system between the perceived quality of sharpening and sharpness related to the first image. The horizontal axis of the coordinate system is sharpness, and the vertical axis of the coordinate system is the perceived quality of sharpening. Figure 5In the coordinate system shown, the perceptual quality of sharpening increases first and then decreases as the sharpness increases, wherein point A is the point where the perceptual quality of sharpening corresponding to the first image is the largest, and point B is the optimal sharpness of the first image, that is, if the sharpness of the first image is set to the sharpness corresponding to point B, the perceptual quality of sharpening corresponding to the first image is the best.

[0074] It should be noted that when the first image is different, the optimal sharpness corresponding to the first image is also different. For example, if there are fewer textures in the first image (for example, the first image includes more sky images), after the sharpness of the first image is increased, the perceived quality of the sharpening of the first image changes slowly, and when the sharpness of the first image is large, the perceived quality of the sharpening of the first image is the best; if there are more textures in the first image (for example, the first image includes more grass images), after the sharpness of the first image is increased, the perceived quality of the sharpening of the first image changes quickly, and when the sharpness of the first image is small, the perceived quality of the sharpening of the first image is the best.

[0075] Among them, the terminal device can determine the perceptual quality of sharpening of the first image based on the following feasible implementation methods: based on the first image and the blurred image, determine the first image similarity between the first image and the blurred image; based on the first image and the sharpened image, determine the second image similarity between the first image and the sharpened image; based on the first image, the first image similarity and the second image similarity, determine the perceptual quality of sharpening of the first image.

[0076] Among them, the first image similarity can be the similarity between the first image and the blurred image, and the second image similarity can be the similarity between the first image and the sharpened image. Since the blurred image has a higher degree of blur, the perceived quality of sharpening corresponding to the blurred image is lower, and since the sharpening degree of the sharpened image is higher, the perceived quality of sharpening corresponding to the sharpened image is also lower. The terminal device uses the blurred image and the sharpened image as a reference, and based on the first image similarity, the second image similarity and the first image, can accurately determine the perceived quality of sharpening of the first image.

[0077] S205: Determine the sharpness of the first image based on the first image.

[0078] The sharpness of the first image may indicate the degree of sharpening of the first image. For example, if the sharpness of the first image is greater, it means that the degree of sharpening of the first image is higher, and if the sharpness of the first image is smaller, it means that the degree of sharpening of the first image is lower.

[0079] Optionally, the terminal device may determine the sharpness of the first image based on the following feasible implementation method: performing convolution processing on the first image to obtain a feature image of the first image, and processing the feature image of the first image based on a sharpness calculation model to obtain the sharpness of the first image. For example, the sharpness calculation model is used to generate the sharpness corresponding to the image, and the terminal device may input the sharpness of the first image into the sharpness calculation model, and the sharpness calculation model may output the sharpness of the first image. For example, the sharpness calculation model may include a convolutional neural network CNN, and the terminal device may input the first image into the convolutional neural network, and the convolutional neural network may output the feature image of the first image, and input the feature image of the first image into the sharpness calculation model, and the sharpness calculation model may output the sharpness of the first image based on the feature image of the first image.

[0080] Next, combine Figure 6 , a process of determining the sharpness of the first image is described.

[0081] Figure 6 A schematic diagram of determining the sharpness of a first image provided by an embodiment of the present disclosure. Figure 6 , including: a first image, a convolutional neural network and a sharpness calculation model. Among them, the terminal device ( Figure 6 The first image may be input into a convolutional neural network (not shown), and the convolutional neural network may output a feature image associated with the first image, and the feature image may be input into a sharpness calculation model, and the sharpness calculation model may process the feature image to obtain the sharpness corresponding to the first image. In this way, the terminal device may accurately determine the sharpness corresponding to the first image, thereby improving the accuracy of determining the sharpening quality of the first image.

[0082] Optionally, the terminal device can generate a sharpness calculation model based on the following feasible implementation method: obtain the second image, the blurred image of the second image and the sharpened image of the second image, determine the sharpness of the second image to be M, the sharpness of the blurred image of the second image to be L, and the sharpness of the sharpened image of the second image to be N; and train the sharpness calculation model based on the second image, the blurred image of the second image, the sharpened image of the second image, and the sharpness of the second image, the sharpness of the blurred image of the second image and the sharpness of the sharpened image of the second image.

[0083] The perceived quality of the sharpening of the second image is greater than or equal to a preset threshold. For example, the sharpness of the second image may be an optimal sharpness, and the perceived quality of the sharpening of the second image by the human eye is the highest, that is, the perceived quality of the sharpening of the second image is greater than or equal to the preset threshold. For example, the terminal device may obtain multiple second images from an aesthetic data set (an open source image database, in which the sharpness of the images in the aesthetic data set is the optimal sharpness, and the perceived quality of the sharpening of the images is the greatest), and the terminal device may also obtain multiple second images based on any other feasible implementation method, which is not limited in the embodiments of the present disclosure.

[0084] It should be noted that the process of the terminal device obtaining the blurred image of the second image and the sharpened image of the second image can refer to the process of the terminal device obtaining the blurred image of the first image and the sharpened image of the first image, and the present disclosed embodiment will not elaborate on this.

[0085] Optionally, after the terminal device obtains the second image, the blurred image of the second image, and the sharpened image of the second image, the sharpness of the second image can be set to M, the sharpness of the blurred image of the second image can be set to L, and the sharpness of the sharpened image of the second image can be set to N. L, M, and N can be labels for subsequent sharpness calculation model training. For example, in actual application, the terminal device can set the sharpness of the second image to 0.5, the sharpness of the blurred image of the second image to 0, and the sharpness of the sharpened image of the second image to 1. In this way, after the terminal device obtains the sharpness of the image, it can accurately evaluate the sharpening quality of the image based on the sharpness value, thereby improving the accuracy of determining the sharpening quality of the image.

[0086] Optionally, the terminal device trains a sharpness calculation model based on the second image, the blurred image of the second image, the sharpened image of the second image, the sharpness of the second image, the sharpness of the blurred image of the second image and the sharpness of the sharpened image of the second image. Specifically, the second image may be processed based on the sharpness calculation model to obtain the predicted sharpness of the second image, the blurred image of the second image may be processed based on the sharpness calculation model to obtain the predicted sharpness of the blurred image of the second image, the sharpened image of the second image may be processed based on the sharpness calculation model to obtain the predicted sharpness of the sharpened image of the second image, and the sharpness calculation model may be trained based on the loss between the sharpness of the second image and the predicted sharpness of the second image, the loss between the sharpness of the blurred image of the second image and the predicted sharpness of the blurred image of the second image, and the loss between the sharpness of the sharpened image of the second image and the predicted sharpness of the sharpened image of the second image.

[0087] It should be noted that the process of training the sharpness calculation model based on the above-mentioned loss can refer to the process of training any model, and the embodiments of the present disclosure will not be repeated here.

[0088] S206: Determine the sharpening quality of the first image based on the perceived quality and sharpness of the sharpening.

[0089] Optionally, the greater the perceived quality of sharpening of the first image, the higher the sharpening quality of the first image, and the smaller the difference between the sharpness and M, the higher the sharpening quality of the first image. For example, in actual application, if the score of the perceived quality of sharpening is higher, it means that the first image is better sharpened and the sharpening quality of the first image is higher. Since M is the optimal sharpness during the training of the sharpness calculation model, the closer the sharpness of the first image is to M, the better the sharpening quality of the first image. Therefore, the terminal device processes the perceived quality of sharpening and the sharpness based on any feasible implementation method to obtain the sharpening quality of the first image, such as determining the product of the perceived quality of sharpening and the sharpness of the first image as the sharpening quality of the first image, etc. The embodiments of the present disclosure are not limited to this.

[0090] Optionally, after determining the sharpening quality of the first image, the terminal device may also obtain the sharpening quality corresponding to the original image before the first image is sharpened, and determine whether to perform sharpening processing on the original image based on the sharpening quality of the first image and the sharpening quality corresponding to the original image. For example, if the sharpening quality of the first image is greater than the sharpening quality of the original image before the first image is sharpened, it means that the sharpening quality of the image can be improved after the terminal device sharpens the original image; if the sharpening quality of the first image is less than or equal to the sharpening quality of the original image, the terminal device may not perform sharpening processing on the original image.

[0091] Optionally, after determining the sharpness of the first image, the terminal device may determine whether the first image is over-sharpened. For example, the sharpness output by the sharpness calculation model is a number between 0 and 1. If the sharpness of the first image is 0.5, the terminal device may determine that the sharpness of the first image is good. If the sharpness of the first image is greater than 0.75, the terminal device may determine that the first image is over-sharpened.

[0092] Optionally, the terminal device may determine the sharpening parameters of the sharpening algorithm based on the sharpening quality of the first image. For example, the terminal device may obtain the first image obtained after the sharpening algorithm sharpens the original image based on different sharpening parameters. The terminal device may obtain the sharpening quality of each first image, and determine the sharpening parameters corresponding to the first image with the highest sharpening quality as the optimal sharpening parameters.

[0093] The disclosed embodiment provides an image processing method, wherein a terminal device can obtain a first image, blur the first image to obtain a blurred image, and sharpen the first image to obtain a sharpened image, determine the first image similarity between the first image and the blurred image based on the first image and the blurred image, determine the second image similarity between the first image and the sharpened image based on the first image and the sharpened image, determine the perceptual quality of sharpening of the first image based on the first image, the first image similarity, and the second image similarity, determine the sharpness of the first image based on the first image, and determine the sharpening quality of the first image based on the perceptual quality of sharpening and the sharpness. In the above method, since the terminal device can use the first image similarity and the second image similarity as a reference for determining the perceptual quality of sharpening, the terminal device can accurately determine the perceptual quality of sharpening of the first image, and since the terminal device can evaluate the sharpening quality of the first image in combination with the sharpness of the first image and the perceptual quality of sharpening, the accuracy of determining the sharpening quality of the image can be improved.

[0094] exist Figure 2 Based on the embodiment shown below, combined with Figure 7 , a method for determining the perceptual quality of sharpening of the first image based on the first image, the blurred image and the sharpened image in the above-mentioned image processing method is described.

[0095] Figure 7 A schematic diagram of a method for determining the perceptual quality of sharpening provided by an embodiment of the present disclosure. Figure 7 The method flow includes:

[0096] S701: Determine a first image similarity between the first image and the blurred image based on the first image and the blurred image.

[0097] Optionally, the terminal device can determine the first image similarity between the first image and the blurred image based on the following feasible implementation method: perform convolution processing on the first image to obtain a feature image of the first image, perform convolution processing on the blurred image to obtain a feature image of the blurred image, and calculate the similarity between the feature image of the first image and the feature image of the blurred image to obtain the first image similarity.

[0098] Optionally, the terminal device may perform convolution processing on the first image based on a convolutional neural network to obtain a feature image of the first image. The terminal device may process the blurred image of the first image based on the convolutional neural network to obtain a feature image of the blurred image of the first image. The terminal device may also determine the feature image of the first image and the feature image of the blurred image based on any feasible implementation method, which is not limited to the embodiments of the present disclosure.

[0099] Optionally, the similarity between the feature image of the first image and the feature image of the blurred image may be the first image similarity. For example, the terminal device may calculate the cosine similarity between the feature image of the first image and the feature image of the blurred image, and determine the cosine similarity as the first image similarity. For example, the terminal device may calculate the Euclidean distance between the feature image of the first image and the feature image of the blurred image, and determine the Euclidean distance as the first image similarity.

[0100] It should be noted that the terminal device can process the feature image of the first image and the feature image of the blurred image of the first image based on any feasible implementation method to obtain the first image similarity, which is not limited in this embodiment of the present disclosure.

[0101] Next, combine Figure 8 , a process of determining the similarity of the first image is described.

[0102] Figure 8 A schematic diagram of a process for determining the similarity of a first image provided by an embodiment of the present disclosure. Figure 8 , including: a first image, a blurred image of the first image, a convolutional neural network A, and a convolutional neural network B. The convolutional neural network A and the convolutional neural network B share parameters (that is, the parameters in the convolutional neural network A and the convolutional neural network B are the same). Terminal device ( Figure 8 (not shown) The first image can be processed based on convolutional neural network A to obtain a feature image 1 of the first image, and the terminal device can process the blurred image based on convolutional neural network B to obtain a feature image 2 of the blurred image of the first image.

[0103] See also Figure 8 , the terminal device performs dimensionality reduction processing on feature image 1 and feature image 2 to obtain image dimensionality reduction feature a corresponding to feature image 1 and image dimensionality reduction feature b corresponding to feature image 2. The terminal device can calculate the cosine similarity between the image dimensionality reduction feature a and the image dimensionality reduction feature b to obtain the first image similarity between the first image and the blurred image of the first image. In this way, the terminal device can use the blurred image as a reference and accurately evaluate the perceptual quality of the sharpening of the first image through the first image similarity, thereby improving the accuracy of the evaluation of the sharpening quality of the first image.

[0104] S702: Determine a second image similarity between the first image and the sharpened image based on the first image and the sharpened image.

[0105] Optionally, the terminal device can determine the second image similarity between the first image and the sharpened image based on the following feasible implementation method: perform convolution processing on the first image to obtain a feature image of the first image, perform convolution processing on the sharpened image to obtain a feature image of the sharpened image, and calculate the similarity between the feature image of the first image and the feature image of the sharpened image to obtain the second image similarity.

[0106] Optionally, the terminal device may perform convolution processing on the first image based on a convolutional neural network to obtain a feature image of the first image. The terminal device may process the sharpened image of the first image based on the convolutional neural network to obtain a feature image of the sharpened image of the first image. The terminal device may also determine the feature image of the first image and the feature image of the sharpened image based on any feasible implementation method, which is not limited to the embodiments of the present disclosure.

[0107] Optionally, the similarity between the feature image of the first image and the feature image of the sharpened image may be the second image similarity. For example, the terminal device may calculate the cosine similarity between the feature image of the first image and the feature image of the sharpened image, and determine the cosine similarity as the second image similarity. For example, the terminal device may calculate the Euclidean distance between the feature image of the first image and the feature image of the sharpened image, and determine the Euclidean distance as the second image similarity.

[0108] It should be noted that the terminal device can process the feature image of the first image and the feature image of the sharpened image of the first image based on any feasible implementation method to obtain the second image similarity, which is not limited in the embodiments of the present disclosure.

[0109] Next, combine Fig. 9 , the process of determining the similarity of the second image is described.

[0110] Fig. 9 A schematic diagram of a process for determining the similarity of a second image provided by an embodiment of the present disclosure. Fig. 9 , including: a first image, a sharpened image of the first image, a convolutional neural network A, and a convolutional neural network B. The convolutional neural network A and the convolutional neural network B share parameters. The terminal device ( Fig. 9 (not shown) The first image can be processed based on convolutional neural network A to obtain a feature image 1 of the first image, and the terminal device can process the sharpened image based on convolutional neural network B to obtain a feature image 2 of the sharpened image of the first image.

[0111] See also Fig. 9, the terminal device performs dimensionality reduction processing on feature image 1 and feature image 2 to obtain image dimensionality reduction feature a corresponding to feature image 1 and image dimensionality reduction feature b corresponding to feature image 2. The terminal device can calculate the cosine similarity between the image dimensionality reduction feature a and the image dimensionality reduction feature b to obtain the second image similarity between the first image and the sharpened image of the first image. In this way, the terminal device can use the sharpened image as a reference and accurately evaluate the perceptual quality of the sharpening of the first image through the second image similarity, thereby improving the accuracy of the evaluation of the sharpening quality of the first image.

[0112] It should be noted that the convolutional neural network in the embodiment of the present disclosure may be a VGG convolutional neural network. After pooling and concatenation operations on features at different levels in the VGG network, the generated multi-level fusion features can improve the expressiveness of the features.

[0113] It should be noted that in the embodiments of the present disclosure, the terminal device can train the convolutional neural network for extracting feature images based on any feasible implementation method, and the embodiments of the present disclosure are not limited to this.

[0114] S703: Determine a sharpened perceptual quality of the first image based on the first image, the first image similarity, and the second image similarity.

[0115] Among them, the terminal device can determine the perceptual quality of sharpening of the first image based on the following feasible implementation method: perform convolution processing on the first image to obtain a feature image of the first image, determine the weight of the first image similarity or the second image similarity in the perceptual quality of sharpening based on the feature image of the first image, and calculate the perceptual quality of sharpening of the first image based on the weight, the first similarity and the second similarity.

[0116] Among them, the process of performing convolution processing on the first image to obtain the feature image of the first image is not described in detail in the embodiments of the present disclosure. It should be noted that, in the embodiments of the present disclosure, in all processes of performing convolution processing on the first image to obtain the feature image of the first image, the convolutional neural network structure used by the terminal device is the same, the parameters in the convolutional neural network are also the same, and, in actual application, the terminal device can obtain the feature image of the first image when calculating the similarity of the first image or the similarity of the second image.

[0117] The weight may be the weight of the first image similarity in the perceived quality of sharpening, or the weight of the second image similarity in the perceived quality of sharpening. For example, if the weight is the proportion of the first image similarity in the perceived quality of sharpening, then the proportion of the second image similarity in the perceived quality of sharpening is 1-the weight; if the weight is the proportion of the second image similarity in the perceived quality of sharpening, then the proportion of the first image similarity in the perceived quality of sharpening is 1-the weight.

[0118] Optionally, the terminal device may determine the weight based on the following feasible implementation: perform multiple linear convolution processes on the feature image of the first image to obtain the weight of the similarity of the first image or the similarity of the second image in the perceived quality of sharpening. For example, the terminal device may process the feature image of the first image based on multiple linear layers (Linear), and then obtain a fusion coefficient, that is, the fusion coefficient may be the proportion of the similarity of the first image or the similarity of the second image in the perceived quality of sharpening, and the fusion coefficient is a number greater than or equal to 0 and less than or equal to 1.

[0119] Next, combine Fig.10 , a process of determining a weight of the first image similarity or the second image similarity in the perceived quality of sharpening is described.

[0120] Fig.10 A schematic diagram of determining weights provided in an embodiment of the present disclosure. Fig.10 , including: a first image, a convolutional neural network and a plurality of linear convolutional layers. The terminal device ( Fig.10 (not shown) can input the first image into the convolutional neural network, the convolutional neural network can output the feature image corresponding to the first image, the terminal device can input the feature image into multiple linear convolutional layers, and the multiple linear convolutional layers can output weights, which can be numbers between 0 and 1. In this way, the terminal device can accurately determine the position where the perceived quality of the sharpness of the first image is the largest and the distance of excessive blur and excessive sharpness based on the weight, and thus can accurately obtain the perceived quality of the sharpness of the first image.

[0121] It should be noted that in the embodiments of the present disclosure, since different first images correspond to different optimal sharpnesses (for example, an image with less texture has a larger optimal sharpness, and an image with more texture has a smaller optimal sharpness), the terminal device can accurately adjust the proportion of the blurred image and the sharp image in the perceived quality of the sharpness of the first image based on the weights determined by the feature image of the first image, thereby improving the accuracy of the perceived quality of the sharpness of the first image.

[0122] Optionally, the terminal device may calculate the sharpened perceptual quality of the first image based on the following formula:

[0123] q = α × Dist1 + (1-α) × Dist2

[0124] Wherein, q is the sharpening perceived quality of the first image; α is the proportion of the first image similarity in the sharpening perceived quality; Dist1 is the first image similarity; Dist2 is the second image similarity. Based on the above formula, the terminal device can obtain the sharpening perceived quality of the first image.

[0125] Optionally, the above-mentioned image processing method can be obtained based on the sharpening quality assessment model of the image. For example, the terminal device can input the first image into the sharpening quality assessment model, and the sharpening quality assessment model can output the sharpening perceptual quality and sharpness of the first image. Fig.11 , the training process of the sharpening quality assessment model is explained in detail.

[0126] Fig.11 A training diagram of a sharpening quality assessment model provided by an embodiment of the present disclosure. Fig.11 , including: sharpening quality assessment model 1, sharpening quality assessment model 2, image A and image B. Among them, image A is an image with optimal sharpness, image B is an image after image A is blurred, model parameters are shared between sharpening quality assessment model 1 and sharpening quality assessment model 2, sharpening quality assessment model 1 includes similarity calculation module A, similarity calculation module B, sharpness calculation module, weight calculation module and weighting module, and sharpening quality assessment model 2 includes similarity calculation module C, similarity calculation module D, weighting module, weight calculation module and sharpness calculation module.

[0127] See also Fig.11 , the sharpening quality assessment model 1 can process image A. Among them, image A is blurred to obtain a blurred image, image A is sharpened to obtain a sharpened image, image A and the blurred image are processed based on the similarity calculation module A to obtain the image similarity 1 between image A and the blurred image. Image A and the sharpened image are processed based on the similarity calculation module B to obtain the image similarity 2 between image A and the sharpened image.

[0128] See also Fig.11, the sharpening quality assessment model 1 inputs the feature image of image A output by the similarity calculation module A to the sharpness calculation module to obtain the predicted sharpness of image A, and inputs the feature image of image A to the weight calculation module to obtain the weight m. The sharpening quality assessment model 1 inputs the image similarity 1 between image A and the blurred image, the image similarity 2 between image A and the sharpened image, and the weight m to the weighting module to obtain the perceptual quality of the sharpening of image A. Among them, the sharpening quality assessment model 1 can obtain the first loss based on the predicted sharpness of image A and the actual sharpness of image A.

[0129] See also Fig.11 , the sharpening quality assessment model 2 can process the image B after the image A is blurred. Among them, the image B is blurred to obtain a blurred image, the image B is sharpened to obtain a sharpened image, and the image B and the blurred image are processed based on the similarity calculation module C to obtain the image similarity 3 between the image A and the blurred image. The image B and the sharpened image are processed based on the similarity calculation module D to obtain the image similarity 4 between the image B and the sharpened image.

[0130] See also Fig.11 , the sharpening quality assessment model 2 inputs the feature image of image B output by the similarity calculation module D to the sharpness calculation module to obtain the predicted sharpness of image B, and inputs the feature image of image B to the weight calculation module to obtain the weight n. The sharpening quality assessment model 2 inputs the image similarity 3 between image B and the blurred image, the image similarity 4 between image B and the sharpened image, and the weight n to the weighting module to obtain the sharpening perceptual quality of image B. Among them, the sharpening quality assessment model 2 can obtain the second loss based on the predicted sharpness of image B and the actual sharpness of image B.

[0131] See also Fig.11 , since image B is the image after blurring of image A, and image A is the image with the best sharpness, the sharpening perceptual quality a of image A output by sharpening quality assessment model 1 should be greater than the sharpening perceptual quality b of image B output by sharpening quality assessment model 2. Therefore, the sharpening perceptual quality a is compared with the sharpening perceptual quality b, and the third loss is determined based on the result of the comparison and the sharpening perceptual quality a being greater than the sharpening perceptual quality b. Among them, the sharpening quality assessment model 1 can be trained based on the first loss and the third loss, and the sharpening quality assessment model 2 can be trained based on the second loss and the third loss.

[0132] In this way, the twin model-based training method can improve the accuracy of the generated sharpening perceptual quality and the accuracy of the sharpness, thereby improving the accuracy of determining the sharpening quality of the image.

[0133] It should be noted that Fig.11The calculation process of the similarity calculation module in can refer to Figure 8 or Fig. 9 The embodiments shown are not described in detail in the embodiments of the present disclosure.

[0134] Optionally, the training data set of the sharpening quality assessment model can be constructed based on the following method: obtain a full reference quality assessment data set with subjective score annotations to constrain the parameters in the similarity calculation module. Fig.11 For the images A and B in , images in a large-scale aesthetic dataset can be selected, and the images in the aesthetic dataset are determined as the images with the best sharpness. Therefore, after the image is sharpened or blurred, the corresponding sharpening perceptual quality of the image will be reduced. For example, if the image with the best sharpness is selected as the input of the first sharpening quality assessment model, the image with the best sharpness can be sharpened or blurred to obtain a sharpened image or a blurred image, and the sharpened image or the blurred image can be used as the input of the second sharpening quality assessment model. In this way, the perceptual quality of sharpening output by the first sharpening quality assessment model can be greater than the perceptual quality of sharpening output by the second sharpening quality assessment model (that is, if the perceptual quality of sharpening output by the first sharpening quality assessment model is greater than the perceptual quality of sharpening output by the second sharpening quality assessment model, the output is 1, and if the perceptual quality of sharpening output by the first sharpening quality assessment model is less than or equal to the perceptual quality of sharpening output by the second sharpening quality assessment model, the output is 0, and the true label is also a 0-1 label).

[0135] Optionally, after blurring or sharpening the optimally sharpened image, the blurred or sharpened image can be scored, and then the twin model can be trained. For example, the score of the optimally sharpened image is 5 points, and the score of the distorted image after blurring or sharpening can be less than 5 points. In this way, the ranking of multiple images in a set of training samples can be obtained, and the true label can be obtained.

[0136] The disclosed embodiment provides a method for determining the perceptual quality of sharpening, which comprises determining the first image similarity between the first image and the blurred image based on a first image and a blurred image, determining the second image similarity between the first image and the sharpened image based on the first image and the sharpened image, performing convolution processing on the first image to obtain a feature image of the first image, determining the weight of the first image similarity or the second image similarity in the perceptual quality of sharpening based on the feature image of the first image, and calculating the perceptual quality of sharpening of the first image based on the weight, the first similarity and the second similarity. In this way, the terminal device can determine the perceptual quality of sharpening of the first image in combination with the similarity between the first image and the highly blurred image and the similarity between the first image and the over-sharpened image, thereby improving the accuracy of determining the perceptual quality of sharpening.

[0137] Fig.12 This is a schematic diagram of the structure of an image processing device provided by an embodiment of the present disclosure. Fig.12 The image processing device 120 includes an acquisition module 121, a processing module 122, a first determination module 123, a second determination module 124 and a third determination module 125, wherein:

[0138] The acquisition module 121 is used to acquire a first image;

[0139] The processing module 122 is used to perform blur processing on the first image to obtain a blurred image;

[0140] The processing module 122 is further used to perform sharpening processing on the first image to obtain a sharpened image;

[0141] The first determination module 123 is used to determine the sharpened perceptual quality of the first image based on the first image, the blurred image and the sharpened image;

[0142] The second determination module 124 is used to determine the sharpness of the first image based on the first image;

[0143] The third determination module 125 is configured to determine the sharpening quality of the first image based on the perceived quality of the sharpening and the sharpness.

[0144] According to one or more embodiments of the present disclosure, the first determining module 123 is specifically configured to:

[0145] determining, based on the first image and the blurred image, a first image similarity between the first image and the blurred image;

[0146] determining a second image similarity between the first image and the sharpened image based on the first image and the sharpened image;

[0147] A perceptual quality of sharpening of the first image is determined based on the first image, the first image similarity, and the second image similarity.

[0148] According to one or more embodiments of the present disclosure, the first determining module 123 is specifically configured to:

[0149] Performing convolution processing on the first image to obtain a feature image of the first image;

[0150] Determining, based on a feature image of the first image, a weight of the first image similarity or the second image similarity in the perceived quality of the sharpening;

[0151] The sharpened perceptual quality of the first image is calculated based on the weights, the first image similarity, and the second image similarity.

[0152] According to one or more embodiments of the present disclosure, the first determining module 123 is specifically configured to:

[0153] Perform multiple linear convolution processes on the feature image of the first image to obtain the weight of the first image similarity or the second image similarity in the sharpened perceptual quality.

[0154] According to one or more embodiments of the present disclosure, the first determining module 123 is specifically configured to:

[0155] Performing convolution processing on the first image to obtain a feature image of the first image;

[0156] Performing convolution processing on the blurred image to obtain a feature image of the blurred image;

[0157] The similarity between the feature image of the first image and the feature image of the blurred image is calculated to obtain the first image similarity.

[0158] According to one or more embodiments of the present disclosure, the first determining module 123 is specifically configured to:

[0159] Performing convolution processing on the first image to obtain a feature image of the first image;

[0160] Performing convolution processing on the sharpened image to obtain a feature image of the sharpened image;

[0161] The similarity between the feature image of the first image and the feature image of the sharpened image is calculated to obtain the second image similarity.

[0162] According to one or more embodiments of the present disclosure, the second determining module 124 is specifically configured to:

[0163] Performing convolution processing on the first image to obtain a feature image of the first image;

[0164] The feature image of the first image is processed based on a sharpness calculation model to obtain the sharpness of the first image.

[0165] According to one or more embodiments of the present disclosure, the second determining module 124 is specifically configured to:

[0166] Acquire a second image, a blurred image of the second image, and a sharpened image of the second image, wherein a sharpened perceptual quality of the second image is greater than or equal to a preset threshold;

[0167] Determine that the sharpness of the second image is M, the sharpness of the blurred image of the second image is L, the sharpness of the sharpened image of the second image is N, the L is smaller than the M, and the M is smaller than the N;

[0168] The sharpness calculation model is trained based on the second image, the blurred image of the second image, the sharpened image of the second image, and the sharpness of the second image, the sharpness of the blurred image of the second image, and the sharpness of the sharpened image of the second image.

[0169] The image processing device provided in the embodiment of the present disclosure may be used to execute the technical solution of the above method embodiment, and its implementation principle and technical effect are similar, which will not be described in detail in this embodiment.

[0170] Fig.13 This is a schematic diagram of the structure of a terminal device provided by an embodiment of the present disclosure. Fig.13 , which shows a schematic diagram of the structure of a terminal device 1300 suitable for implementing the embodiments of the present disclosure. The terminal device may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (Portable Android Devices, PADs), portable multimedia players (PMPs), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Fig.13 The terminal device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0171] like Fig.13 As shown, the terminal device 1300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 1301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1302 or a program loaded from a storage device 1308 to a random access memory (RAM) 1303. Various programs and data required for the operation of the terminal device 1300 are also stored in the RAM 1303. The processing device 1301, the ROM 1302, and the RAM 1303 are connected to each other via a bus 1304. An input / output (I / O) interface 1305 is also connected to the bus 1304.

[0172] Typically, the following devices may be connected to the I / O interface 1305: input devices 1306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 1307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1308 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1309. The communication devices 1309 may allow the terminal device 1300 to communicate with other devices wirelessly or by wire to exchange data. Although Fig.13 The terminal device 1300 is shown as having various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0173] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 1309, or installed from a storage device 1308, or installed from a ROM 1302. When the computer program is executed by the processing device 1301, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0174] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0175] The computer-readable medium may be included in the terminal device, or may exist independently without being installed in the terminal device.

[0176] The computer-readable medium carries one or more programs. When the one or more programs are executed by the terminal device, the terminal device executes the method shown in the above embodiment.

[0177] An embodiment of the present disclosure provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the image processing method that may be involved in various embodiments as described above is implemented.

[0178] An embodiment of the present disclosure provides a computer program product, including a computer program, and when the computer program is executed by a processor, the image processing method that may be involved in various embodiments as described above is implemented.

[0179] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0180] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0181] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of a unit does not limit the unit itself in some cases. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses".

[0182] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0183] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0184] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0185] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0186] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0187] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can independently choose whether to provide personal information to software or hardware such as a terminal device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure based on the prompt message. As an optional but non-limiting implementation method, in response to receiving an active request from a user, the method of sending a prompt message to the user can be, for example, a pop-up window, and the prompt message can be presented in text form in the pop-up window. In addition, the pop-up window can also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the terminal device.

[0188] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0189] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of the relevant laws and regulations. The data may include information, parameters and messages, such as flow switching indication information.

[0190] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solution formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with the technical features with similar functions disclosed in the present disclosure (but not limited to). It should be noted that step S201, step S202, etc. in the embodiments of the present disclosure are not limitations on the execution order.

[0191] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0192] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. An image processing method, characterized in that: include: acquiring a first image; Performing blur processing on the first image to obtain a blurred image; Performing sharpening processing on the first image to obtain a sharpened image; determining a perceptual quality of sharpening of the first image based on the first image, the blurred image, and the sharpened image; Based on the first image, determining a sharpness of the first image; Based on the perceived quality of the sharpening and the sharpness, a sharpening quality of the first image is determined.

2. The method according to claim 1, characterized in that: The determining, based on the first image, the blurred image and the sharpened image, a sharpened perceptual quality of the first image comprises: determining, based on the first image and the blurred image, a first image similarity between the first image and the blurred image; determining a second image similarity between the first image and the sharpened image based on the first image and the sharpened image; A perceptual quality of sharpening of the first image is determined based on the first image, the first image similarity, and the second image similarity.

3. The method according to claim 2, characterized in that The determining, based on the first image, the first image similarity and the second image similarity, of a sharpened perceptual quality of the first image comprises: Performing convolution processing on the first image to obtain a feature image of the first image; Determining, based on a feature image of the first image, a weight of the first image similarity or the second image similarity in the perceived quality of the sharpening; The sharpened perceptual quality of the first image is calculated based on the weights, the first image similarity, and the second image similarity.

4. The method according to claim 3, characterized in that: Determining, based on the feature image of the first image, a weight of the first image similarity or the second image similarity in the perceived quality of the sharpening, includes: Perform multiple linear convolution processes on the feature image of the first image to obtain the weight of the first image similarity or the second image similarity in the sharpened perceptual quality.

5. The method according to claim 2, characterized in that: The determining, based on the first image and the blurred image, a first image similarity between the first image and the blurred image comprises: Performing convolution processing on the first image to obtain a feature image of the first image; Performing convolution processing on the blurred image to obtain a feature image of the blurred image; The similarity between the feature image of the first image and the feature image of the blurred image is calculated to obtain the first image similarity.

6. The method according to claim 2, characterized in that The determining, based on the first image and the sharpened image, a second image similarity between the first image and the sharpened image comprises: Performing convolution processing on the first image to obtain a feature image of the first image; Performing convolution processing on the sharpened image to obtain a feature image of the sharpened image; The similarity between the feature image of the first image and the feature image of the sharpened image is calculated to obtain the second image similarity.

7. The method according to any one of claims 1 to 6, characterized in that: The determining, based on the first image, the sharpness of the first image comprises: Performing convolution processing on the first image to obtain a feature image of the first image; The feature image of the first image is processed based on a sharpness calculation model to obtain the sharpness of the first image.

8. The method according to claim 6, characterized in that The sharpness calculation model is obtained based on the following steps: Acquire a second image, a blurred image of the second image, and a sharpened image of the second image, wherein a sharpened perceptual quality of the second image is greater than or equal to a preset threshold; Determine that the sharpness of the second image is M, the sharpness of the blurred image of the second image is L, the sharpness of the sharpened image of the second image is N, the L is smaller than the M, and the M is smaller than the N; The sharpness calculation model is trained based on the second image, the blurred image of the second image, the sharpened image of the second image, and the sharpness of the second image, the sharpness of the blurred image of the second image, and the sharpness of the sharpened image of the second image.

9. An image processing device, characterized in that: It includes an acquisition module, a processing module, a first determination module, a second determination module and a third determination module, wherein: The acquisition module is used to acquire a first image; The processing module is used to perform blur processing on the first image to obtain a blurred image; The processing module is further used to perform sharpening processing on the first image to obtain a sharpened image; The first determination module is used to determine the sharpened perceptual quality of the first image based on the first image, the blurred image and the sharpened image; The second determination module is used to determine the sharpness of the first image based on the first image; The third determination module is used to determine the sharpening quality of the first image based on the perceptual quality of the sharpening and the sharpness.

10. A terminal device, characterized in that: include: Processor and memory; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the image processing method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions. When a processor executes the computer-executable instructions, the image processing method according to any one of claims 1 to 8 is implemented.

Images