Image processing method and apparatus and terminal device

Abstract

The present disclosure provides an image processing method and apparatus and a terminal device. The method comprises: acquiring a first image; performing blurring processing on the first image to obtain a blurred image; performing sharpening processing on the first image to obtain a sharpened image; on the basis of the first image, the blurred image and the sharpened image, determining the perceived quality of the sharpening of the first image; on the basis of the first image, determining the sharpness of the first image; and on the basis of the perceived quality of the sharpening and the sharpness, determining the sharpening quality of the first image.

Description

Image processing method, device and terminal equipment

[0001] This application claims priority to the Chinese invention patent application entitled “Image processing method, device and terminal device” and application number 202310908148.4 filed on July 21, 2023. The entire contents of that application are incorporated by reference into this application. Technical Field

[0002] The embodiments of the present disclosure relate to the field of image processing technology, and in particular to an image processing method, apparatus, and terminal device. Background Art

[0003] Image sharpening techniques can enhance image sharpness and improve the quality of the image perceived by the human eye. However, over-sharpening can also reduce the perceived quality of the image. Therefore, it is particularly important to determine the image quality after image sharpening.

[0004] Currently, a terminal device can input a sharpened image into an image sharpening assessment model, which then outputs a sharpness score corresponding to the sharpened image. Based on this sharpness score, the user can determine whether the image is over-sharpened or under-sharpened. However, users can only estimate the sharpness quality of the sharpened image based on experience, and cannot accurately determine whether the poor sharpening quality is due to over-sharpening or under-sharpening. Therefore, accurately assessing the sharpness quality of a sharpened image is an urgent problem to be solved.

[0005] Summary of the Invention

[0006] The present disclosure provides an image processing method, apparatus, and terminal device for solving one or more technical problems in the prior art.

[0007] In a first aspect, the present disclosure provides an image processing method, which includes: acquiring a first image; blurring the first image to obtain a blurred image; sharpening the first image to obtain a sharpened image; determining the perceptual quality of sharpening of the first image based on the first image, the blurred image and the sharpened image; determining the sharpness of the first image based on the first image; and determining the sharpening quality of the first image based on the perceptual quality of sharpening and the sharpness.

[0008] In a second aspect, the present disclosure provides an image processing device, which includes an acquisition module, a processing module, a first determination module, a second determination module and a third determination module, wherein: the acquisition module is used to acquire a first image; the processing module is used to blur the first image to obtain a blurred image; the processing module is also used to sharpen the first image to obtain a sharpened image; the first determination module is used to determine the perceptual quality of sharpening of the first image based on the first image, the blurred image and the sharpened image; the second determination module is used to determine the sharpness of the first image based on the first image; and the sharpening quality of the first image is determined based on the perceptual quality of sharpening and the sharpness.

[0009] In a third aspect, an embodiment of the present disclosure provides a terminal device comprising: a processor and a memory; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the image processing method as described in the first aspect above and various possible aspects involved in the first aspect.

[0010] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the image processing method as described in the first aspect and various possible aspects of the first aspect are implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0012] FIG1 is a schematic diagram of an application scenario provided by an embodiment of the present disclosure;

[0013] FIG2 is a schematic flow chart of an image processing method provided by an embodiment of the present disclosure;

[0014] FIG3 is a schematic diagram of a process for determining a blurred image provided by an embodiment of the present disclosure;

[0015] FIG4 is a schematic diagram of a process for determining a sharpened image according to an embodiment of the present disclosure;

[0016] FIG5 is a schematic diagram showing the relationship between the perceived quality and sharpness of a sharpening process provided by an embodiment of the present disclosure;

[0017] FIG6 is a schematic diagram of determining the sharpness of a first image according to an embodiment of the present disclosure;

[0018] FIG7 is a schematic diagram of a method for determining the perceptual quality of sharpening provided by an embodiment of the present disclosure;

[0019] FIG8 is a schematic diagram of a process for determining the similarity of a first image according to an embodiment of the present disclosure;

[0020] FIG9 is a schematic diagram of a process for determining the similarity of a second image according to an embodiment of the present disclosure;

[0021] FIG10 is a schematic diagram of determining a weight according to an embodiment of the present disclosure;

[0022] FIG11 is a schematic diagram of training a sharpening quality assessment model provided by an embodiment of the present disclosure;

[0023] FIG12 is a schematic structural diagram of an image processing device provided by an embodiment of the present disclosure; and

[0024] FIG13 is a schematic structural diagram of a terminal device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0025] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

[0026] To facilitate understanding, the concepts involved in the embodiments of the present disclosure are explained below.

[0027] Terminal device: is a device with wireless transceiver function. The terminal device can be deployed on land, including indoors or outdoors, handheld, wearable or vehicle-mounted. The terminal device can be a mobile phone, a tablet computer, a computer with wireless transceiver function, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a wireless terminal in industrial control, a vehicle-mounted terminal device, a wireless terminal in self-driving, a wireless terminal device in remote medical, a wireless terminal device in smart grid, a wireless terminal device in transportation safety, a wireless terminal device in smart city, a wireless terminal device in smart home, a wearable terminal device, etc. The terminal device involved in the embodiments of the present disclosure can also be called a terminal, user equipment (UE), an access terminal device, a vehicle-mounted terminal, an industrial control terminal, a UE unit, a UE station, a mobile station, a mobile station, a remote station, a remote terminal device, a mobile device, a UE terminal device, a wireless communication device, a UE agent or a UE device, etc. Terminal devices can also be fixed or mobile.

[0028] In related technologies, image sharpening can enhance the sharpness of an image and improve the quality of the image perceived by the human eye. However, over-sharpening or under-sharpening will reduce the quality of the image perceived by the human eye. Therefore, it is particularly important to determine the sharpening quality of the sharpened image. For example, if the sharpened image is over-sharpened, the sharpening parameters of the image can be reduced, and the image can be re-sharpened using the reduced sharpening parameters. Similarly, if the sharpened image is not sharp enough, the sharpening parameters of the image can be increased, and the image can be re-sharpened based on the increased sharpening parameters. Currently, the terminal device can input the sharpened image into the image sharpening evaluation model, and the image sharpening evaluation model can output the sharpening score corresponding to the sharpened image. The user can determine whether the image is over-sharpened or under-sharpened based on the sharpening score. However, since the sharpness of an image is proportional to the sharpness score, the higher the image sharpness, the higher the sharpness score output by the image sharpening evaluation model. Therefore, users can only estimate the sharpness quality of the sharpened image based on experience. In addition, the relationship between the human eye's perceived image quality and the image sharpness is that it first increases and then decreases. Therefore, based on the sharpness score, users cannot accurately determine whether the poor sharpening quality of the image is caused by over-sharpening or under-sharpening, which leads to low accuracy in the evaluation of the sharpening quality of the sharpened image.

[0029] To address technical issues in related technologies, embodiments of the present disclosure provide an image processing method. A terminal device may acquire a first image, blur the first image to obtain a blurred image, and sharpen the first image to obtain a sharpened image. A first image similarity between the first image and the blurred image is determined based on the first image and the blurred image. A second image similarity between the first image and the sharpened image is determined based on the first image and the sharpened image. A perceptual quality of sharpening of the first image is determined based on the first image, the first image similarity, and the second image similarity. The sharpness of the first image is determined based on the first image. The sharpening quality of the first image is determined based on the perceptual quality of sharpening and the sharpness. In the above method, since the terminal device may use the first image similarity between the first image and the blurred image and the second image similarity between the first image and the sharpened image as references for determining the sharpening quality, the terminal device can accurately evaluate the perceptual quality of sharpening of the first image. Furthermore, the terminal device can accurately determine the sharpening quality of the sharpened first image based on the sharpness and the perceptual quality of sharpening, thereby improving the accuracy of determining the sharpening quality of the image.

[0030] The application scenario of the embodiment of the present disclosure is described below with reference to FIG1 .

[0031] Figure 1 is a schematic diagram of an application scenario provided by an embodiment of the present disclosure. Referring to Figure 1 , the embodiment includes a terminal device. The terminal device may include a first image on its display page. The terminal device may evaluate the sharpening quality of the first image. The terminal device may display the sharpness of the first image and the perceived quality of the sharpening of the first image. The sharpness of the first image may be 0.5, and the perceived quality of the sharpening of the first image may be 70. The terminal device may determine the sharpening quality of the first image based on the sharpness of the first image and the perceived quality of the sharpening of the first image. In this way, since the terminal device can use the blurred image after blurring and the sharpened image after sharpening as references to determine the perceived quality of the sharpening of the first image, the accuracy of the perceived quality of the sharpening is relatively high. Furthermore, the terminal device may determine the sharpening quality of the first image based on the sharpness and perceived quality of the sharpening of the first image. Therefore, the terminal device can accurately determine the cause of the poor sharpening quality of the first image and accurately determine the sharpening quality of the first image, thereby improving the accuracy of the determination of the sharpening quality.

[0032] It should be noted that FIG1 is only an illustrative illustration of the application scenario of the embodiment of the present disclosure, and does not limit the application scenario of the embodiment of the present disclosure.

[0033] The following detailed description of the technical solution of the present disclosure and how the technical solution of the present disclosure solves the above-mentioned technical problems is provided with specific embodiments. The following specific embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments. The embodiments of the present disclosure will be described below in conjunction with the accompanying drawings.

[0034] FIG2 is a flow chart of an image processing method provided by an embodiment of the present disclosure. Referring to FIG2 , the method may include:

[0035] S201: Acquire a first image.

[0036] The execution subject of the embodiments of the present disclosure may be a terminal device or an image processing device provided in the terminal device. The image processing device may be implemented based on software or a combination of software and hardware, which is not limited in the embodiments of the present disclosure. Alternatively, the terminal device may be any device with on-device computing capabilities.

[0037] Optionally, the first image may be a sharpened image. For example, the terminal device may perform sharpening on any image to obtain the first image. For example, the first image may be a sharpened image whose sharpening quality is to be evaluated.

[0038] It should be noted that, since any image has sharpness, the first image can also be any image, and the embodiment of the present disclosure does not limit this.

[0039] Optionally, the terminal device may receive a first image sent by another device. For example, the terminal device may associate at least one image whose sharpness quality is to be evaluated with a request for sharpness quality evaluation sent by the other device (the terminal device may perform a sharpness quality evaluation operation after receiving the request), and the terminal device may determine the at least one image whose sharpness quality is to be evaluated as the first image.

[0040] Optionally, the terminal device may obtain the first image from a database. For example, a plurality of sharpened images may be pre-stored in the database, and the terminal device may obtain the plurality of images (i.e., first images) from the database. For example, a plurality of sharpened images may be pre-stored in the database, and each sharpened image may have a unique identifier. The terminal device may receive a sharpening quality assessment request sent by another device, and the sharpening quality assessment request may include identifiers of the plurality of sharpened images. The terminal device may obtain the associated first image from the database based on the identifiers.

[0041] It should be noted that the terminal device can obtain the first image based on any feasible implementation method, and the embodiments of the present disclosure are not limited to this.

[0042] S202: Perform blur processing on the first image to obtain a blurred image.

[0043] The blurred image is an image obtained by blurring the first image. For example, the terminal device may perform Gaussian blur processing on the first image to obtain a blurred image corresponding to the first image. For example, the terminal device may perform image degradation processing on the first image to obtain a blurred image corresponding to the first image. For example, the terminal device may perform blurred image simulation processing on the first image to generate a blurred simulated image corresponding to the first image. For example, the terminal device may perform downsampling processing on the first image to reduce the number of pixels in the first image to obtain a blurred image corresponding to the first image.

[0044] It should be noted that the terminal device can obtain the blurred image corresponding to the first image based on any feasible implementation method, and the embodiment of the present disclosure is not limited to this.

[0045] It should be noted that after the terminal device performs blurring processing on the first image, the clarity of the obtained blurred image is lower than the clarity of the first image.

[0046] Optionally, the blurred image may be a negative sample of the first image. The higher the blurriness of the blurred image, the better the blurred image's effect as a negative sample. For example, the terminal device may process the first image based on Gaussian blur to obtain a blurred image. Since the blurred image is obtained based on Gaussian blur, the blurriness of the blurred image is relatively high, and thus the blurred image is more effective as a negative sample of the first image.

[0047] The process of determining a blurred image is described below with reference to FIG3 .

[0048] FIG3 is a schematic diagram of a process for determining a blurred image provided by an embodiment of the present disclosure. Referring to FIG3 , it includes: a first image. A terminal device (not shown in FIG3 ) can perform Gaussian blur processing on the first image to obtain a blurred image corresponding to the first image. The first image and the blurred image have the same size. In this way, the terminal device can obtain a blurred image of better quality based on Gaussian blur. The terminal device can improve the accuracy of determining the perceived quality of sharpening of the first image based on the blurred image as a reference, thereby improving the accuracy of determining the sharpening quality of the first image.

[0049] S203: Perform sharpening processing on the first image to obtain a sharpened image.

[0050] The sharpened image is an image obtained by sharpening the first image. For example, the terminal device may sharpen the first image based on a sharpening algorithm to obtain a sharpened image corresponding to the first image. The sharpening algorithm used by the terminal device may be any feasible algorithm, which is not limited in the present embodiment.

[0051] It should be noted that the terminal device can generate a sharpened image corresponding to the first image based on any feasible implementation method, and the embodiments of the present disclosure are not limited to this.

[0052] Optionally, the sharpened image may be a negative sample of the first image. The higher the sharpness of the sharpened image, the better the effect of the sharpened image as a negative sample. For example, the terminal device may process the first image based on a higher sharpening parameter to obtain a sharpened image. Since the sharpened image is obtained based on the higher sharpening parameter, the sharpened image has a higher degree of sharpening. Thus, the sharpened image has a better effect as a negative sample of the first image.

[0053] It should be noted that, in actual application, the sharpened image may be an image obtained after the terminal device sharpens the first image based on the highest sharpening parameter, which can improve the effect of the sharpened image as a negative sample of the first image.

[0054] The process of determining the sharpened image will be described below with reference to FIG. 4 .

[0055] FIG4 is a schematic diagram of a process for determining a sharpened image provided by an embodiment of the present disclosure. Please refer to FIG4 , which includes: a first image. The terminal device (not shown in FIG4 ) can perform sharpening processing on the first image, thereby obtaining a sharpened image corresponding to the first image. The first image and the sharpened image have the same size. The sharpening parameter (which can indicate the degree of sharpening of the image, and the larger the sharpening parameter, the deeper the degree of sharpening) when the terminal device performs sharpening processing can be the highest sharpening parameter of the sharpening algorithm. In this way, the terminal device can obtain a sharpened image of better quality. Based on the sharpened image as a reference, the terminal device can improve the accuracy of determining the perceived quality of the sharpening of the first image, thereby improving the accuracy of determining the sharpening quality of the first image.

[0056] S204: Determine a perceptual quality of sharpening of the first image based on the first image, the blurred image, and the sharpened image.

[0057] The perceived quality of the sharpening of the first image may be the perceived quality of the image by the user's eyes. For example, in actual applications, the perceived quality of the sharpening may be a score, and the user may determine the perceived quality of the first image by the user's eyes based on the score. A higher score indicates a better perceived quality of the first image by the user's eyes, and a lower score indicates a lower perceived quality of the first image by the user's eyes.

[0058] It should be noted that there is an optimal sharpness for the perceptual quality of sharpening of the first image, that is, if the sharpness of the first image is the optimal sharpness, the score of the perceptual quality of sharpening of the first image is the highest; if the sharpness of the first image is less than or greater than the optimal sharpness, the score of the perceptual quality of sharpening of the first image is less than the highest score corresponding to the optimal sharpness.

[0059] The relationship between the perceptual quality of sharpening and sharpness is described below with reference to FIG. 5 .

[0060] FIG5 is a schematic diagram of the relationship between the perceived quality of sharpening and sharpness provided by an embodiment of the present disclosure. Please refer to FIG5 , which includes: a coordinate system between the perceived quality of sharpening and sharpness related to the first image. The horizontal axis of the coordinate system is sharpness, and the vertical axis of the coordinate system is the perceived quality of sharpening. In the coordinate system shown in FIG5 , the perceived quality of sharpening first increases and then decreases as the sharpness increases, wherein point A is the point where the perceived quality of sharpening corresponding to the first image is the largest, and point B is the optimal sharpness of the first image, that is, if the sharpness of the first image is set to the sharpness corresponding to point B, the perceived quality of sharpening corresponding to the first image is the best.

[0061] It should be noted that the optimal sharpness corresponding to different first images is also different. For example, if the first image contains less texture (e.g., the first image includes more sky images), then increasing the sharpness of the first image will cause the perceptual quality of the sharpening of the first image to change more slowly. When the sharpness of the first image is greater, the perceptual quality of the sharpening of the first image is the best. If the first image contains more texture (e.g., the first image includes more grass images), then increasing the sharpness of the first image will cause the perceptual quality of the sharpening of the first image to change more rapidly. When the sharpness of the first image is less, the perceptual quality of the sharpening of the first image is the best.

[0062] The terminal device may determine the perceptual quality of the sharpening of the first image based on the following feasible implementation: determining a first image similarity between the first image and the blurred image based on the first image and the blurred image; determining a second image similarity between the first image and the sharpened image based on the first image and the sharpened image; and determining the perceptual quality of the sharpening of the first image based on the first image, the first image similarity, and the second image similarity.

[0063] The first image similarity may be the similarity between the first image and the blurred image, and the second image similarity may be the similarity between the first image and the sharpened image. Since the blurred image has a higher degree of blur, the perceived quality of the sharpening corresponding to the blurred image is lower. Furthermore, since the sharpened image has a higher degree of sharpening, the perceived quality of the sharpening corresponding to the sharpened image is also lower. The terminal device uses the blurred image and the sharpened image as references. Based on the first image similarity, the second image similarity, and the first image, the perceived quality of the sharpening of the first image can be accurately determined.

[0064] S205: Determine the sharpness of the first image based on the first image.

[0065] The sharpness of the first image may indicate the degree of sharpening of the first image. For example, a greater sharpness of the first image indicates a higher degree of sharpening of the first image, and a smaller sharpness of the first image indicates a lower degree of sharpening of the first image.

[0066] Optionally, the terminal device may determine the sharpness of the first image based on the following feasible implementation method: performing convolution processing on the first image to obtain a feature image of the first image. Processing the feature image of the first image based on a sharpness calculation model to obtain the sharpness of the first image. For example, the sharpness calculation model is used to generate the sharpness corresponding to the image, and the terminal device may input the sharpness of the first image into the sharpness calculation model, and the sharpness calculation model may output the sharpness of the first image. For example, the sharpness calculation model may include a convolutional neural network CNN before, and the terminal device may input the first image into the convolutional neural network, and the convolutional neural network may output the feature image of the first image, and input the feature image of the first image into the sharpness calculation model, and the sharpness calculation model may output the sharpness of the first image based on the feature image of the first image.

[0067] The process of determining the sharpness of the first image is described below with reference to FIG6 .

[0068] FIG6 is a schematic diagram of a method for determining the sharpness of a first image provided by an embodiment of the present disclosure. Referring to FIG6 , it includes: a first image, a convolutional neural network, and a sharpness calculation model. The terminal device (not shown in FIG6 ) can input the first image into the convolutional neural network, and the convolutional neural network can output a feature image associated with the first image, and input the feature image into the sharpness calculation model. The sharpness calculation model can process the feature image to obtain the sharpness corresponding to the first image. In this way, the terminal device can accurately determine the sharpness corresponding to the first image, thereby improving the accuracy of determining the sharpening quality of the first image.

[0069] Optionally, the terminal device may generate a sharpness calculation model based on the following feasible implementation: obtaining a second image, a blurred image of the second image, and a sharpened image of the second image, determining the sharpness of the second image to be M, the sharpness of the blurred image of the second image to be L, and the sharpness of the sharpened image of the second image to be N. The sharpness calculation model is trained based on the second image, the blurred image of the second image, the sharpened image of the second image, the sharpness of the second image, the sharpness of the blurred image of the second image, and the sharpness of the sharpened image of the second image.

[0070] The perceived quality of the sharpening of the second image is greater than or equal to a preset threshold. For example, the sharpness of the second image may be optimal sharpness, and the human eye has the highest perceived quality of the sharpening of the second image, that is, the perceived quality of the sharpening of the second image is greater than or equal to the preset threshold. For example, the terminal device may obtain multiple second images from an aesthetic dataset (an open source image database, where the sharpness of the images in the aesthetic dataset is optimal sharpness and the perceived quality of the sharpening of the images is the highest). The terminal device may also obtain multiple second images based on any other feasible implementation method, and the embodiments of the present disclosure are not limited to this.

[0071] It should be noted that the process of the terminal device obtaining the blurred image of the second image and the sharpened image of the second image can refer to the process of the terminal device obtaining the blurred image of the first image and the sharpened image of the first image, and the present embodiment will not go into details about this.

[0072] Optionally, after obtaining the second image, the blurred image of the second image, and the sharpened image of the second image, the terminal device may set the sharpness of the second image to M, the sharpness of the blurred image of the second image to L, and the sharpness of the sharpened image of the second image to N. L, M, and N may be labels for subsequent sharpness calculation model training. For example, in actual application, the terminal device may set the sharpness of the second image to 0.5, the sharpness of the blurred image of the second image to 0, and the sharpness of the sharpened image of the second image to 1. In this way, after obtaining the sharpness of the image, the terminal device may accurately evaluate the sharpening quality of the image based on the sharpness value, thereby improving the accuracy of determining the sharpening quality of the image.

[0073] Optionally, the terminal device trains a sharpness calculation model based on the second image, the blurred image of the second image, the sharpened image of the second image, the sharpness of the second image, the sharpness of the blurred image of the second image, and the sharpness of the sharpened image of the second image. Specifically, the second image is processed based on the sharpness calculation model to obtain the predicted sharpness of the second image. The blurred image of the second image is processed based on the sharpness calculation model to obtain the predicted sharpness of the blurred image of the second image. The sharpness calculation model is trained based on the loss between the sharpness of the second image and the predicted sharpness of the second image, the loss between the sharpness of the blurred image of the second image and the predicted sharpness of the blurred image of the second image, and the loss between the sharpness of the sharpened image of the second image and the predicted sharpness of the sharpened image of the second image.

[0074] It should be noted that the process of training the sharpness calculation model based on the above-mentioned loss can refer to the process of training any model, and the embodiments of the present disclosure will not be repeated here.

[0075] S206: Determine the sharpening quality of the first image based on the sharpening perception quality and sharpness.

[0076] Optionally, the greater the perceived quality of sharpening of the first image, the higher the sharpening quality of the first image, and the smaller the difference between the sharpness and M, the higher the sharpening quality of the first image. For example, in actual application, if the score of the perceived quality of sharpening is higher, it means that the first image is better sharpened and the sharpening quality of the first image is higher. Since M is the optimal sharpness during the training of the sharpness calculation model, the closer the sharpness of the first image is to M, the better the sharpening quality of the first image. Therefore, the terminal device processes the perceived quality of sharpening and the sharpness based on any feasible implementation method to obtain the sharpening quality of the first image, such as determining the product of the perceived quality of sharpening and the sharpness of the first image as the sharpening quality of the first image, etc. The embodiments of the present disclosure are not limited to this.

[0077] Optionally, after determining the sharpening quality of the first image, the terminal device may also obtain the sharpening quality corresponding to the original image before the first image is sharpened, and determine whether to perform sharpening processing on the original image based on the sharpening quality of the first image and the sharpening quality corresponding to the original image. For example, if the sharpening quality of the first image is greater than the sharpening quality of the original image before the first image is sharpened, it means that the sharpening quality of the original image can be improved after the terminal device sharpens the original image; if the sharpening quality of the first image is less than or equal to the sharpening quality of the original image, the terminal device may not perform sharpening processing on the original image.

[0078] Optionally, after determining the sharpness of the first image, the terminal device may determine whether the first image is over-sharpened. For example, the sharpness output by the sharpness calculation model is a number between 0 and 1. If the sharpness of the first image is 0.5, the terminal device may determine that the sharpness of the first image is good. If the sharpness of the first image is greater than 0.75, the terminal device may determine that the first image is over-sharpened.

[0079] Optionally, the terminal device may determine the sharpening parameters of the sharpening algorithm based on the sharpening quality of the first image. For example, the terminal device may obtain first images obtained by sharpening the original image using the sharpening algorithm based on different sharpening parameters. The terminal device may obtain the sharpening quality of each first image and determine the sharpening parameters corresponding to the first image with the highest sharpening quality as the optimal sharpening parameters.

[0080] The embodiment of the present disclosure provides an image processing method, in which a terminal device can obtain a first image, blur the first image to obtain a blurred image, and sharpen the first image to obtain a sharpened image. Based on the first image and the blurred image, a first image similarity between the first image and the blurred image is determined. Based on the first image and the sharpened image, a second image similarity between the first image and the sharpened image is determined. Based on the first image, the first image similarity, and the second image similarity, the perceptual quality of sharpening of the first image is determined. Based on the first image, the sharpness of the first image is determined. Based on the perceptual quality of sharpening and the sharpness, the sharpening quality of the first image is determined. In the above method, since the terminal device can use the first image similarity and the second image similarity as a reference for determining the perceptual quality of sharpening, the terminal device can accurately determine the perceptual quality of sharpening of the first image, and since the terminal device can evaluate the sharpening quality of the first image in combination with the sharpness of the first image and the perceptual quality of sharpening, the accuracy of determining the sharpening quality of the image can be improved.

[0081] Based on the embodiment shown in FIG. 2 , the method for determining the perceptual quality of sharpening of the first image based on the first image, the blurred image, and the sharpened image in the above image processing method will be described below in conjunction with FIG. 7 .

[0082] FIG7 is a schematic diagram of a method for determining the perceptual quality of sharpening provided by an embodiment of the present disclosure. Referring to FIG7 , the method process includes:

[0083] S701: Determine a first image similarity between the first image and the blurred image based on the first image and the blurred image.

[0084] Optionally, the terminal device can determine the first image similarity between the first image and the blurred image based on the following feasible implementation method: perform convolution processing on the first image to obtain a feature image of the first image, perform convolution processing on the blurred image to obtain a feature image of the blurred image, calculate the similarity between the feature image of the first image and the feature image of the blurred image, and obtain the first image similarity.

[0085] Optionally, the terminal device can perform convolution processing on the first image based on a convolutional neural network to obtain a feature image of the first image. The terminal device can process the blurred image of the first image based on a convolutional neural network to obtain a feature image of the blurred image of the first image. The terminal device can also determine the feature image of the first image and the feature image of the blurred image based on any feasible implementation method. The embodiments of the present disclosure are not limited to this.

[0086] Optionally, the similarity between the feature image of the first image and the feature image of the blurred image may be the first image similarity. For example, the terminal device may calculate the cosine similarity between the feature image of the first image and the feature image of the blurred image, and determine the cosine similarity as the first image similarity. For example, the terminal device may calculate the Euclidean distance between the feature image of the first image and the feature image of the blurred image, and determine the Euclidean distance as the first image similarity.

[0087] It should be noted that the terminal device can process the feature image of the first image and the feature image of the blurred image of the first image based on any feasible implementation method to obtain the first image similarity, which is not limited in this embodiment of the present disclosure.

[0088] The process of determining the first image similarity is described below with reference to FIG8 .

[0089] FIG8 is a schematic diagram of a process for determining the similarity of a first image provided by an embodiment of the present disclosure. Please refer to FIG8 , which includes: a first image, a blurred image of the first image, a convolutional neural network A, and a convolutional neural network B. The convolutional neural network A and the convolutional neural network B share parameters (i.e., the parameters in the convolutional neural network A and the convolutional neural network B are the same). The terminal device (not shown in FIG8 ) can process the first image based on the convolutional neural network A to obtain a feature image 1 of the first image, and the terminal device can process the blurred image based on the convolutional neural network B to obtain a feature image 2 of the blurred image of the first image.

[0090] Referring to Figure 8 , the terminal device performs dimensionality reduction processing on feature image 1 and feature image 2 to obtain image dimensionality reduction feature a corresponding to feature image 1 and image dimensionality reduction feature b corresponding to feature image 2. The terminal device can calculate the cosine similarity between image dimensionality reduction feature a and image dimensionality reduction feature b to obtain a first image similarity between the first image and the blurred version of the first image. In this way, the terminal device can use the blurred image as a reference and accurately assess the perceived sharpening quality of the first image based on the first image similarity, thereby improving the accuracy of the assessment of the sharpening quality of the first image.

[0091] S702: Determine a second image similarity between the first image and the sharpened image based on the first image and the sharpened image.

[0092] Optionally, the terminal device can determine the second image similarity between the first image and the sharpened image based on the following feasible implementation method: performing convolution processing on the first image to obtain a feature image of the first image, performing convolution processing on the sharpened image to obtain a feature image of the sharpened image, and calculating the similarity between the feature image of the first image and the feature image of the sharpened image to obtain the second image similarity.

[0093] Optionally, the terminal device can perform convolution processing on the first image based on a convolutional neural network to obtain a feature image of the first image. The terminal device can process the sharpened image of the first image based on a convolutional neural network to obtain a feature image of the sharpened image of the first image. The terminal device can also determine the feature image of the first image and the feature image of the sharpened image based on any feasible implementation method. The embodiments of the present disclosure are not limited to this.

[0094] Optionally, the similarity between the feature image of the first image and the feature image of the sharpened image may be the second image similarity. For example, the terminal device may calculate the cosine similarity between the feature image of the first image and the feature image of the sharpened image, and determine the cosine similarity as the second image similarity. For example, the terminal device may calculate the Euclidean distance between the feature image of the first image and the feature image of the sharpened image, and determine the Euclidean distance as the second image similarity.

[0095] It should be noted that the terminal device can process the feature image of the first image and the feature image of the sharpened image of the first image based on any feasible implementation method to obtain the second image similarity, which is not limited in this embodiment of the present disclosure.

[0096] The process of determining the second image similarity is described below with reference to FIG. 9 .

[0097] FIG9 is a schematic diagram of a process for determining the similarity of a second image provided by an embodiment of the present disclosure. Referring to FIG9 , the process includes: a first image, a sharpened image of the first image, a convolutional neural network A, and a convolutional neural network B. Convolutional neural network A and convolutional neural network B share parameters. A terminal device (not shown in FIG9 ) can process the first image based on convolutional neural network A to obtain a feature image 1 of the first image, and the terminal device can process the sharpened image based on convolutional neural network B to obtain a feature image 2 of the sharpened image of the first image.

[0098] As shown in Figure 9, the terminal device performs dimensionality reduction processing on Feature Image 1 and Feature Image 2 to obtain image dimensionality reduction feature a corresponding to Feature Image 1 and image dimensionality reduction feature b corresponding to Feature Image 2. The terminal device can calculate the cosine similarity between image dimensionality reduction feature a and image dimensionality reduction feature b to obtain a second image similarity between the first image and the sharpened version of the first image. In this way, the terminal device can use the sharpened image as a reference and accurately assess the perceived quality of the sharpening of the first image based on the second image similarity, thereby improving the accuracy of the assessment of the sharpening quality of the first image.

[0099] It should be noted that the convolutional neural network in the embodiment of the present disclosure can be a VGG convolutional neural network. After the features of different layers in the VGG network are pooled and concatenated, the multi-level fusion features generated can improve the expressive ability of the features.

[0100] It should be noted that in the embodiment of the present disclosure, the terminal device can train the convolutional neural network for extracting feature images based on any feasible implementation method, and the embodiment of the present disclosure is not limited to this.

[0101] S703: Determine a sharpened perceptual quality of the first image based on the first image, the first image similarity, and the second image similarity.

[0102] The terminal device may determine the perceptual quality of sharpening of the first image based on the following feasible implementation: performing convolution processing on the first image to obtain a feature image of the first image; determining, based on the feature image of the first image, a weight of the first image similarity or the second image similarity in the perceptual quality of sharpening; and calculating the perceptual quality of sharpening of the first image based on the weight, the first similarity, and the second similarity.

[0103] Among them, the process of performing convolution processing on the first image to obtain the characteristic image of the first image will not be repeated in the embodiments of the present disclosure. It should be noted that, in the embodiments of the present disclosure, in all processes of performing convolution processing on the first image to obtain the characteristic image of the first image, the convolutional neural network structure used by the terminal device is the same, and the parameters in the convolutional neural network are also the same. Moreover, in actual application, the terminal device can obtain the characteristic image of the first image when calculating the similarity of the first image or the similarity of the second image.

[0104] The weight may be the weight of the first image similarity in the perceived quality of sharpening, or the weight of the second image similarity in the perceived quality of sharpening. For example, if the weight is the proportion of the first image similarity in the perceived quality of sharpening, then the proportion of the second image similarity in the perceived quality of sharpening is 1-the weight; if the weight is the proportion of the second image similarity in the perceived quality of sharpening, then the proportion of the first image similarity in the perceived quality of sharpening is 1-the weight.

[0105] Optionally, the terminal device may determine the weight based on the following feasible implementation: performing multiple linear convolution processes on the feature image of the first image to obtain the weight of the similarity of the first image or the similarity of the second image in the perceptual quality of sharpening. For example, the terminal device may process the feature image of the first image based on multiple linear layers (Linear), and then obtain a fusion coefficient, that is, the fusion coefficient may be the proportion of the similarity of the first image or the similarity of the second image in the perceptual quality of sharpening, and the fusion coefficient is a number greater than or equal to 0 and less than or equal to 1.

[0106] The following describes a process of determining the weight of the first image similarity or the second image similarity in the perceptual quality of sharpening with reference to FIG. 10 .

[0107] FIG10 is a schematic diagram of a method for determining weights provided by an embodiment of the present disclosure. Please refer to FIG10 , which includes: a first image, a convolutional neural network, and a plurality of linear convolutional layers. The terminal device (not shown in FIG10 ) can input the first image into the convolutional neural network, and the convolutional neural network can output a feature image corresponding to the first image. The terminal device can input the feature image into a plurality of linear convolutional layers, and the plurality of linear convolutional layers can output weights, which can be numbers between 0 and 1. In this way, the terminal device can accurately determine the position where the perceived quality of the sharpness of the first image is the largest, the distance from excessive blurring, and the distance from excessive sharpening based on the weight, and thus accurately obtain the perceived quality of the sharpness of the first image.

[0108] It should be noted that in the embodiment of the present disclosure, since different first images correspond to different optimal sharpness (for example, the optimal sharpness of an image with less texture is larger, and the optimal sharpness of an image with more texture is smaller), the terminal device determines the weight based on the characteristic image of the first image, and can accurately adjust the proportion of the blurred image and the sharp image in the perceived quality of the sharpness of the first image, thereby improving the accuracy of the perceived quality of the sharpness of the first image.

[0109] Optionally, the terminal device may calculate the sharpened perceptual quality of the first image based on the following formula: q = α × Dist1 + (1-α) × Dist2

[0110] Where q is the perceived quality of sharpening of the first image; α is the proportion of the first image similarity in the perceived quality of sharpening; Dist1 is the similarity of the first image; and Dist2 is the similarity of the second image. Based on the above formula, the terminal device can obtain the perceived quality of sharpening of the first image.

[0111] Optionally, the above-mentioned image processing method can be obtained based on a sharpening quality assessment model of the image. For example, the terminal device can input a first image into the sharpening quality assessment model, and the sharpening quality assessment model can output the sharpening perceptual quality and sharpness of the first image. Below, in combination with Figure 11, the training process of the sharpening quality assessment model is described in detail.

[0112] FIG11 is a schematic diagram of a training process for a sharpening quality assessment model provided by an embodiment of the present disclosure. Referring to FIG11 , the training process includes: a sharpening quality assessment model 1, a sharpening quality assessment model 2, image A, and image B. Image A is an image with optimal sharpness, and image B is an image after image A is blurred. Model parameters are shared between sharpening quality assessment model 1 and sharpening quality assessment model 2. Sharpening quality assessment model 1 includes similarity calculation module A, similarity calculation module B, sharpness calculation module, weight calculation module, and weighting module. Sharpening quality assessment model 2 includes similarity calculation module C, similarity calculation module D, weighting module, weight calculation module, and sharpness calculation module.

[0113] Referring to Figure 11, sharpening quality assessment model 1 can process image A. Image A is blurred to obtain a blurred image, and then sharpened to obtain a sharpened image. Image A and the blurred image are processed using similarity calculation module A to obtain image similarity 1 between them. Image A and the sharpened image are processed using similarity calculation module B to obtain image similarity 2 between them.

[0114] Referring to Figure 11, Sharpening Quality Assessment Model 1 inputs the feature image of Image A, output by Similarity Calculation Module A, into the Sharpness Calculation Module to obtain the predicted sharpness of Image A. It then inputs the feature image of Image A into the Weight Calculation Module to obtain the weight m. Sharpening Quality Assessment Model 1 then inputs the Image Similarity 1 between Image A and the blurred image, the Image Similarity 2 between Image A and the sharpened image, and the weight m into the Weighting Module to obtain the perceived sharpness of Image A. Sharpening Quality Assessment Model 1 can derive a first loss based on the predicted sharpness of Image A and the actual sharpness of Image A.

[0115] Referring to Figure 11, sharpening quality assessment model 2 can process image B after blurring image A. Image B is blurred to obtain a blurred image, and sharpened to obtain a sharpened image. Similarity calculation module C processes image B and the blurred image to obtain image similarity 3 between image A and the blurred image. Similarity calculation module D processes image B and the sharpened image to obtain image similarity 4 between image B and the sharpened image.

[0116] Referring to Figure 11 , Sharpening Quality Assessment Model 2 inputs the feature image of Image B, output by Similarity Calculation Module D, into the Sharpness Calculation Module to obtain the predicted sharpness of Image B. It then inputs the feature image of Image B into the Weight Calculation Module to obtain the weight n. Sharpening Quality Assessment Model 2 then inputs the image similarity 3 between Image B and the blurred image, the image similarity 4 between Image B and the sharpened image, and the weight n into the Weighting Module to obtain the perceived sharpness of Image B. Sharpening Quality Assessment Model 2 can derive a second loss based on the predicted sharpness of Image B and the actual sharpness of Image B.

[0117] Please refer to Figure 11. Since Image B is the blurred image of Image A, and Image A is the image with optimal sharpness, the sharpened perceptual quality a of Image A output by Sharpening Quality Assessment Model 1 should be greater than the sharpened perceptual quality b of Image B output by Sharpening Quality Assessment Model 2. Therefore, the sharpened perceptual quality a is compared with the sharpened perceptual quality b, and the third loss is determined based on the result of the comparison and the fact that the sharpened perceptual quality a is greater than the sharpened perceptual quality b. Sharpening Quality Assessment Model 1 can be trained based on the first loss and the third loss, and Sharpening Quality Assessment Model 2 can be trained based on the second loss and the third loss.

[0118] In this way, the training method based on the twin model can improve the accuracy of the perceptual quality of the generated sharpening and the accuracy of the sharpness, thereby improving the accuracy of determining the sharpening quality of the image.

[0119] It should be noted that the calculation process of the similarity calculation module in FIG11 may refer to the embodiment shown in FIG8 or FIG9 , and will not be described in detail in the embodiment of the present disclosure.

[0120] Optionally, a training dataset for the sharpening quality assessment model can be constructed using the following method: A full-reference quality assessment dataset with subjective score annotations is obtained to constrain the parameters in the similarity calculation module. For example, for Images A and B in Figure 11, images from a large-scale aesthetic dataset can be selected and identified as the optimally sharp images. Therefore, sharpening or blurring this image will degrade the perceived quality of the corresponding sharpening. For example, if an image with optimal sharpness is selected as the input of the first sharpening quality assessment model, the image with optimal sharpness can be sharpened or blurred to obtain a sharpened image or a blurred image, and the sharpened image or the blurred image can be used as the input of the second sharpening quality assessment model. In this way, the perceptual quality of sharpening output by the first sharpening quality assessment model can be greater than the perceptual quality of sharpening output by the second sharpening quality assessment model (that is, if the perceptual quality of sharpening output by the first sharpening quality assessment model is greater than the perceptual quality of sharpening output by the second sharpening quality assessment model, the output is 1; if the perceptual quality of sharpening output by the first sharpening quality assessment model is less than or equal to the perceptual quality of sharpening output by the second sharpening quality assessment model, the output is 0, and the true label is also a 0-1 label).

[0121] Optionally, after blurring or sharpening the optimally sharpened image, the blurred or sharpened image can be scored to train the twin model. For example, the optimally sharpened image can have a score of 5, while the blurred or sharpened distorted image can have a score less than 5. This way, the ranking of multiple images in a set of training samples can be obtained, resulting in the true label.

[0122] The embodiment of the present disclosure provides a method for determining the perceptual quality of sharpening, which comprises determining the first image similarity between the first image and the blurred image based on a first image and a blurred image. Determining the second image similarity between the first image and the sharpened image based on the first image and the sharpened image, performing convolution processing on the first image to obtain a feature image of the first image. Based on the feature image of the first image, determining the weight of the first image similarity or the second image similarity in the perceptual quality of sharpening. Based on the weight, the first similarity, and the second similarity, calculating the perceptual quality of sharpening of the first image. In this way, the terminal device can determine the perceptual quality of sharpening of the first image based on the similarity between the first image and the highly blurred image, and the similarity between the first image and the over-sharpened image, thereby improving the accuracy of determining the perceptual quality of sharpening.

[0123] FIG12 is a schematic diagram of the structure of an image processing device provided by an embodiment of the present disclosure. Referring to FIG12 , the image processing device 120 includes an acquisition module 121, a processing module 122, a first determination module 123, a second determination module 124, and a third determination module 125, wherein:

[0124] The acquisition module 121 is used to acquire a first image;

[0125] The processing module 122 is configured to perform blur processing on the first image to obtain a blurred image;

[0126] The processing module 122 is further configured to perform sharpening processing on the first image to obtain a sharpened image;

[0127] The first determining module 123 is configured to determine a sharpening perceptual quality of the first image based on the first image, the blurred image, and the sharpened image;

[0128] The second determining module 124 is configured to determine the sharpness of the first image based on the first image;

[0129] The third determination module 125 is configured to determine a sharpening quality of the first image based on the sharpening perceptual quality and the sharpness.

[0130] According to one or more embodiments of the present disclosure, the first determining module 123 is specifically configured to:

[0131] determining a first image similarity between the first image and the blurred image based on the first image and the blurred image;

[0132] determining a second image similarity between the first image and the sharpened image based on the first image and the sharpened image;

[0133] A perceptual quality of sharpening of the first image is determined based on the first image, the first image similarity, and the second image similarity.

[0134] According to one or more embodiments of the present disclosure, the first determining module 123 is specifically configured to:

[0135] performing convolution processing on the first image to obtain a feature image of the first image;

[0136] determining, based on a feature image of the first image, a weight of the first image similarity or the second image similarity in the perceived quality of the sharpening;

[0137] The sharpened perceptual quality of the first image is calculated based on the weights, the first image similarity, and the second image similarity.

[0138] According to one or more embodiments of the present disclosure, the first determining module 123 is specifically configured to:

[0139] Performing multiple linear convolution processes on the feature image of the first image to obtain a weight of the first image similarity or the second image similarity in the sharpening perception quality.

[0140] According to one or more embodiments of the present disclosure, the first determining module 123 is specifically configured to:

[0141] performing convolution processing on the first image to obtain a feature image of the first image;

[0142] Performing convolution processing on the blurred image to obtain a feature image of the blurred image;

[0143] The similarity between the feature image of the first image and the feature image of the blurred image is calculated to obtain the first image similarity.

[0144] According to one or more embodiments of the present disclosure, the first determining module 123 is specifically configured to:

[0145] performing convolution processing on the first image to obtain a feature image of the first image;

[0146] Performing convolution processing on the sharpened image to obtain a feature image of the sharpened image;

[0147] The similarity between the feature image of the first image and the feature image of the sharpened image is calculated to obtain the second image similarity.

[0148] According to one or more embodiments of the present disclosure, the second determining module 124 is specifically configured to:

[0149] performing convolution processing on the first image to obtain a feature image of the first image;

[0150] The feature image of the first image is processed based on a sharpness calculation model to obtain the sharpness of the first image.

[0151] According to one or more embodiments of the present disclosure, the second determining module 124 is specifically configured to:

[0152] Acquire a second image, a blurred image of the second image, and a sharpened image of the second image, wherein a perceptual quality of the sharpening of the second image is greater than or equal to a preset threshold;

[0153] Determine that the sharpness of the second image is M, the sharpness of the blurred image of the second image is L, and the sharpness of the sharpened image of the second image is N, where L is smaller than M, and M is smaller than N;

[0154] The sharpness calculation model is trained based on the second image, the blurred image of the second image, the sharpened image of the second image, and the sharpness of the second image, the sharpness of the blurred image of the second image, and the sharpness of the sharpened image of the second image.

[0155] The image processing device provided in the embodiment of the present disclosure can be used to execute the technical solution of the above-mentioned method embodiment. Its implementation principle and technical effects are similar and will not be described in detail in this embodiment.

[0156] FIG13 is a schematic diagram of the structure of a terminal device provided by an embodiment of the present disclosure. Please refer to FIG13 , which shows a schematic diagram of the structure of a terminal device 1300 suitable for implementing an embodiment of the present disclosure. The terminal device may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (Portable Android Devices, PADs), portable multimedia players (PMPs), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and fixed terminals such as digital TVs and desktop computers. The terminal device shown in FIG13 is merely an example and should not impose any limitations on the functions and scope of use of the embodiments of the present disclosure.

[0157] As shown in FIG13 , terminal device 1300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 1301, which can perform various appropriate actions and processes based on programs stored in a read-only memory (ROM) 1302 or programs loaded from a storage device 1308 into a random access memory (RAM) 1303. RAM 1303 also stores various programs and data required for the operation of terminal device 1300. Processing device 1301, ROM 1302, and RAM 1303 are connected to each other via a bus 1304. An input / output (I / O) interface 1305 is also connected to bus 1304.

[0158] Typically, the following devices may be connected to the I / O interface 1305: an input device 1306 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 1307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1308 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1309. The communication device 1309 may allow the terminal device 1300 to communicate with other devices wirelessly or by wire to exchange data. Although FIG13 illustrates a terminal device 1300 having various devices, it should be understood that not all of the devices shown are required to be implemented or present. More or fewer devices may alternatively be implemented or present.

[0159] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network via the communication device 1309, or installed from the storage device 1308, or installed from the ROM 1302. When the computer program is executed by the processing device 1301, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0160] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0161] The computer-readable medium may be included in the terminal device, or may exist independently without being incorporated into the terminal device.

[0162] The computer-readable medium carries one or more programs. When the one or more programs are executed by the terminal device, the terminal device executes the method shown in the above embodiment.

[0163] An embodiment of the present disclosure provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions. When a processor executes the computer-executable instructions, the image processing method that may be involved in various embodiments above is implemented.

[0164] An embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the image processing methods that may be involved in various embodiments as described above.

[0165] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0166] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0167] The units involved in the embodiments described in this disclosure may be implemented in software or hardware. In some cases, the name of a unit does not limit the unit itself. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses."

[0168] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0169] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0170] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0171] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0172] It is understandable that before using the technical solutions disclosed in the various embodiments of this disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved in this disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0173] For example, in response to receiving an active request from the user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thereby, the user can independently choose whether to provide personal information to the software or hardware such as the terminal device, application, server or storage medium that performs the operation of the technical solution of the present disclosure based on the prompt message. As an optional but non-limiting implementation method, in response to receiving an active request from the user, the method of sending the prompt message to the user can be, for example, a pop-up window, and the prompt information can be presented in the form of text in the pop-up window. In addition, the pop-up window can also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the terminal device.

[0174] It is understandable that the above notification and user authorization process are merely illustrative and do not limit the implementation of the present disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0175] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws and regulations. Data may include information, parameters and messages, such as flow switching indication information.

[0176] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with the technical features with similar functions disclosed in this disclosure (but not limited to). It should be noted that step S201, step S202, etc. in the embodiment of the present disclosure are not limitations on the execution order.

[0177] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details have been included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0178] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims

1. An image processing method, comprising: acquiring a first image; Performing blur processing on the first image to obtain a blurred image; Performing sharpening processing on the first image to obtain a sharpened image; determining a perceptual quality of sharpening of the first image based on the first image, the blurred image, and the sharpened image; Based on the first image, determining a sharpness of the first image; Based on the perceived quality of the sharpening and the sharpness, a sharpening quality of the first image is determined.

2. The method of claim 1 , wherein determining a perceptual quality of sharpening of the first image based on the first image, the blurred image, and the sharpened image comprises: determining, based on the first image and the blurred image, a first image similarity between the first image and the blurred image; determining a second image similarity between the first image and the sharpened image based on the first image and the sharpened image; A perceptual quality of sharpening of the first image is determined based on the first image, the first image similarity, and the second image similarity.

3. The method of claim 2, wherein determining the perceptual quality of sharpening of the first image based on the first image, the first image similarity, and the second image similarity comprises: Performing convolution processing on the first image to obtain a feature image of the first image; Determining, based on a feature image of the first image, a weight of the first image similarity or the second image similarity in the perceived quality of the sharpening; The sharpened perceptual quality of the first image is calculated based on the weights, the first image similarity, and the second image similarity.

4. The method according to claim 3, wherein determining the weight of the first image similarity or the second image similarity in the perceived quality of the sharpening based on the feature image of the first image comprises: Perform multiple linear convolution processes on the feature image of the first image to obtain the weight of the first image similarity or the second image similarity in the sharpened perceptual quality.

5. The method according to claim 2, wherein the determining, based on the first image and the blurred image, a first image similarity between the first image and the blurred image comprises: Performing convolution processing on the first image to obtain a feature image of the first image; Performing convolution processing on the blurred image to obtain a feature image of the blurred image; The similarity between the feature image of the first image and the feature image of the blurred image is calculated to obtain the first image similarity.

6. The method according to claim 2, wherein the determining, based on the first image and the sharpened image, a second image similarity between the first image and the sharpened image comprises: Performing convolution processing on the first image to obtain a feature image of the first image; Performing convolution processing on the sharpened image to obtain a feature image of the sharpened image; The similarity between the feature image of the first image and the feature image of the sharpened image is calculated to obtain the second image similarity.

7. The method according to any one of claims 1 to 6, wherein determining the sharpness of the first image based on the first image comprises: Performing convolution processing on the first image to obtain a feature image of the first image; The feature image of the first image is processed based on a sharpness calculation model to obtain the sharpness of the first image.

8. The method according to claim 6, wherein the sharpness calculation model is obtained based on the following steps: Acquire a second image, a blurred image of the second image, and a sharpened image of the second image, wherein a sharpened perceptual quality of the second image is greater than or equal to a preset threshold; Determine that the sharpness of the second image is M, the sharpness of the blurred image of the second image is L, the sharpness of the sharpened image of the second image is N, the L is smaller than the M, and the M is smaller than the N; The sharpness calculation model is trained based on the second image, the blurred image of the second image, the sharpened image of the second image, and the sharpness of the second image, the sharpness of the blurred image of the second image, and the sharpness of the sharpened image of the second image.

9. An image processing device, comprising an acquisition module, a processing module, a first determination module, a second determination module and a third determination module, wherein: The acquisition module is used to acquire a first image; The processing module is used to perform blur processing on the first image to obtain a blurred image; The processing module is further used to perform sharpening processing on the first image to obtain a sharpened image; The first determination module is used to determine the sharpened perceptual quality of the first image based on the first image, the blurred image and the sharpened image; The second determination module is used to determine the sharpness of the first image based on the first image; The third determination module is used to determine the sharpening quality of the first image based on the perceptual quality of the sharpening and the sharpness.

10. A terminal device, comprising: Processor and memory; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the image processing method according to any one of claims 1 to 8.

11. A computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when a processor executes the computer-executable instructions, the image processing method according to any one of claims 1 to 8 is implemented.

Images