Image sharpness evaluation model obtaining method and image sharpness evaluation method

Abstract

The application discloses an image sharpness evaluation model obtaining method and an image sharpness evaluation method. The image sharpness evaluation model obtaining method comprises the following steps: obtaining a sharpness evaluation result of an original image; performing image degradation processing or image reconstruction processing on the original image according to the sharpness evaluation result of the original image, to obtain a processed image; obtaining a sharpness evaluation result of the processed image; and obtaining a first image sharpness evaluation model for evaluating image sharpness according to the original image, the sharpness evaluation result of the original image, the processed image and the sharpness evaluation result of the processed image. The image sharpness evaluation model obtained by the method can simulate the visual perception of the human eye to sharpness, and can be used in a no-reference image sharpness evaluation scene.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to a method for obtaining an image sharpness evaluation model. This application also relates to an apparatus, electronic device, and computer-readable storage medium for obtaining an image sharpness evaluation model. This application further relates to an image sharpness evaluation method, apparatus, electronic device, and computer-readable storage medium. This application also relates to a training sample acquisition method, apparatus, electronic device, and computer-readable storage medium. This application further relates to another method, apparatus, electronic device, and computer-readable storage medium for obtaining an image sharpness evaluation model. Background Technology

[0002] In online-to-offline (O2O) service platforms, image data is one of their core assets, playing a crucial role in enabling users to obtain product information and complete purchases. When a user visits a service page on an O2O platform, the clarity of the product images displayed directly impacts the user's visual experience and interest in the product. Therefore, selecting high-quality, clear images from a vast pool of data for display is essential.

[0003] Traditional image sharpness assessment methods require reference to standard images and evaluation of image sharpness based on objective evaluation indicators. This method is not suitable for image sharpness assessment scenarios without reference and cannot simulate the human eye's visual perception of sharpness. Summary of the Invention

[0004] This application provides a method for obtaining an image sharpness evaluation model to address the problems that existing image sharpness evaluation methods are not applicable to image sharpness evaluation scenarios without references and cannot simulate the human eye's visual perception of sharpness. This application also provides an apparatus, electronic device, and computer-readable storage medium for obtaining an image sharpness evaluation model. This application further provides an image sharpness evaluation method, apparatus, electronic device, and computer-readable storage medium. This application also provides a method, apparatus, electronic device, and computer-readable storage medium for obtaining training samples. This application also provides another method, apparatus, electronic device, and computer-readable storage medium for obtaining an image sharpness evaluation model.

[0005] This application provides a method for obtaining an image sharpness evaluation model, comprising: obtaining a sharpness evaluation result for an original image; performing image degradation processing or image reconstruction processing on the original image based on the sharpness evaluation result for the original image to obtain a processed image; obtaining a sharpness evaluation result for the processed image; and obtaining a first image sharpness evaluation model for evaluating image sharpness based on the original image and the sharpness evaluation result for the original image, as well as the processed image and the sharpness evaluation result for the processed image.

[0006] Optionally, obtaining the sharpness assessment result for the original image includes: establishing a correspondence between the sharpness category to which the image belongs and the sharpness score range; obtaining the sharpness category to which the original image belongs according to the image sharpness classification model; and obtaining the sharpness score for the original image according to the sharpness score range corresponding to the sharpness category to which the original image belongs, as the sharpness assessment result for the original image.

[0007] Optionally, obtaining the sharpness assessment result for the original image includes: establishing a correspondence between the probability value of the sharpness category to which the image belongs and the sharpness score; obtaining the probability value of the sharpness category to which the original image belongs based on the image sharpness classification model; and obtaining the sharpness score corresponding to the probability value of the sharpness category to which the original image belongs, as the sharpness assessment result for the original image.

[0008] Optionally, the step of performing image degradation processing or image reconstruction processing on the original image based on the sharpness evaluation result of the original image to obtain a processed image includes: if the sharpness evaluation result of the original image indicates that the sharpness of the original image is higher than a first preset threshold, then performing image degradation processing on the background portion of the original image to obtain a background degradation image as the processed image; the step of obtaining the sharpness evaluation result of the processed image includes: obtaining the sharpness evaluation result of the processed image based on the sharpness evaluation result of the original image, wherein the sharpness evaluation result of the processed image indicates that the sharpness of the processed image is lower than the sharpness of the original image, and the difference between the sharpness of the processed image and the sharpness of the original image meets a preset range.

[0009] Optionally, the step of performing image degradation processing or image reconstruction processing on the original image based on the sharpness evaluation result of the original image to obtain a processed image includes: if the sharpness evaluation result of the original image indicates that the sharpness of the original image is lower than the first preset threshold and higher than the second preset threshold, then performing image degradation processing on the original image to obtain a full-image degradation image as the processed image; the step of obtaining the sharpness evaluation result of the processed image includes: obtaining the sharpness evaluation result of the processed image based on the sharpness evaluation result of the original image, wherein the sharpness evaluation result of the processed image indicates that the sharpness of the processed image is lower than the sharpness of the original image.

[0010] Optionally, the step of performing image degradation processing or image reconstruction processing on the original image based on the sharpness evaluation result of the original image to obtain the processed image includes: if the sharpness evaluation result of the original image indicates that the sharpness of the original image is lower than the second preset threshold, then performing image reconstruction processing on the original image to obtain a full-image reconstructed image as the processed image;

[0011] The step of obtaining a sharpness evaluation result for the processed image includes: obtaining a sharpness evaluation result for the processed image based on the sharpness evaluation result for the original image, wherein the sharpness evaluation result for the processed image indicates that the sharpness of the processed image is higher than that of the original image.

[0012] Optionally, the step of performing image reconstruction processing on the original image to obtain a full-image reconstructed image as the processed image includes: inputting the original image into an image reconstruction model to obtain a full-image reconstructed image; using an image sharpness classification discriminator to determine whether the sharpness category to which the full-image reconstructed image belongs is a sharp category, and using an image relative sharpness discriminator to determine whether the sharpness of the full-image reconstructed image is higher than the sharpness of the original image; if the sharpness category to which the full-image reconstructed image belongs is a sharp category, and the sharpness of the full-image reconstructed image is higher than the sharpness of the original image, then the full-image reconstructed image is used as the processed image.

[0013] Optionally, obtaining a first image sharpness evaluation model for evaluating image sharpness based on the original image and the sharpness evaluation result for the original image, and the processed image and the sharpness evaluation result for the processed image, includes: obtaining a first image sharpness evaluation initial model, wherein the first image sharpness evaluation initial model is a regression model; inputting the original image and the processed image as input samples into the first image sharpness evaluation initial model to obtain a sharpness prediction result for the input samples; and adjusting the parameters of the first image sharpness evaluation initial model based on the sharpness prediction result for the input samples and the sharpness evaluation result for the input samples to obtain the first image sharpness evaluation model.

[0014] Optionally, adjusting the parameters of the first image sharpness assessment initial model based on the sharpness prediction result and the sharpness assessment result for the input sample to obtain the first image sharpness assessment model includes: obtaining a sharpness assessment difference loss function for evaluating the degree of difference between the sharpness prediction result and the sharpness assessment result for the input sample; and adjusting the parameters of the first image sharpness assessment initial model based on the sharpness assessment difference loss function to obtain the first image sharpness assessment model.

[0015] Optionally, adjusting the parameters of the first image sharpness assessment initial model according to the sharpness assessment difference loss function to obtain the first image sharpness assessment model includes: calculating the value of the sharpness assessment difference loss function based on the sharpness prediction result and the sharpness assessment result for the input sample; adjusting the parameters of the first image sharpness assessment initial model according to the value of the sharpness assessment difference loss function; stopping the adjustment of the parameters of the first image sharpness assessment initial model until the value of the sharpness assessment difference loss function reaches a first preset condition, thereby obtaining the first image sharpness assessment model.

[0016] Optionally, the sharpness assessment difference loss function is as follows:

[0017]

[0018] Where L represents the sharpness assessment difference loss between the sharpness prediction result and the sharpness assessment result for the input sample; N represents the total number of input samples; i represents the i-th input sample; j represents the j-th input sample; y i This represents the sharpness evaluation result for the i-th input sample; This represents the sharpness prediction result for the i-th input sample; y j This represents the sharpness evaluation result for the j-th input sample; Let l1 and l2 represent the sharpness prediction result for the j-th input sample; l1 and l2 both represent distances. τ represents the preset hyperparameter; ∑ represents summation; max(.) represents taking the maximum value; |.| represents taking the absolute value.

[0019] Optionally, the method further includes: using the first image sharpness evaluation model as a teacher model, obtaining a student model through knowledge distillation, and using this student model as a second image sharpness evaluation model for evaluating image sharpness.

[0020] Optionally, the step of using the first image sharpness evaluation model as a teacher model and obtaining a student model through knowledge distillation includes: obtaining an initial student model; obtaining a teacher model sharpness prediction result and a teacher model feature map based on the teacher model; obtaining an initial student model sharpness prediction result and an initial student model feature map based on the initial student model; obtaining a knowledge distillation loss function for evaluating the degree of difference between the teacher model sharpness prediction result and the initial student model sharpness prediction result; obtaining a feature map difference loss function for evaluating the degree of difference between the teacher model feature map and the initial student model feature map; and adjusting the parameters of the initial student model based on the knowledge distillation loss function and the feature map difference loss function to obtain the student model.

[0021] Optionally, adjusting the parameters of the initial student model based on the knowledge distillation loss function and the feature map difference loss function to obtain the student model includes: calculating the value of the knowledge distillation loss function based on the clarity prediction results of the teacher model and the initial student model; calculating the value of the feature map difference loss function based on the feature map of the teacher model and the feature map of the initial student model; adjusting the parameters of the initial student model based on the sum of the values ​​of the knowledge distillation loss function and the feature map difference loss function; and stopping the adjustment of the parameters of the initial student model until the sum of the values ​​of the knowledge distillation loss function and the feature map difference loss function reaches a second preset condition, thereby obtaining the student model.

[0022] Optionally, the knowledge distillation loss function is as follows:

[0023]

[0024] Among them, L kd This represents the knowledge distillation loss between the clarity prediction results of the teacher model and the clarity prediction results of the initial student model. This represents the initial student model clarity prediction results; This represents the clarity prediction result of the teacher model; |.| represents taking the absolute value.

[0025] Optionally, the feature map difference loss function is as follows:

[0026]

[0027] in, Let represent the difference loss for the i-th feature map; i represents the i-th feature map; n represents the total number of feature maps; This represents the feature map feature of the i-th initial student model; Let represent the feature map feature of the i-th teacher model; vec(.) denotes vectorization; ‖.‖2 denotes L2 normalization.

[0028] This application also provides an image sharpness evaluation model acquisition device, comprising: a first acquisition unit for acquiring a sharpness evaluation result for an original image; a second acquisition unit for performing image degradation processing or image reconstruction processing on the original image based on the sharpness evaluation result for the original image to acquire a processed image; a third acquisition unit for acquiring a sharpness evaluation result for the processed image; and a fourth acquisition unit for acquiring a first image sharpness evaluation model for evaluating image sharpness based on the original image and the sharpness evaluation result for the original image, as well as the processed image and the sharpness evaluation result for the processed image.

[0029] This application also provides an image sharpness assessment method, comprising: obtaining an image to be assessed; obtaining a first image sharpness assessment model for assessing image sharpness, wherein the first image sharpness assessment model is a regression model; and obtaining a sharpness prediction result for the image to be assessed based on the first image sharpness assessment model.

[0030] Optionally, obtaining the first image sharpness evaluation model for evaluating image sharpness includes: obtaining a sharpness evaluation result for the original image; performing image degradation processing or image reconstruction processing on the original image based on the sharpness evaluation result for the original image to obtain a processed image; obtaining a sharpness evaluation result for the processed image; and obtaining the first image sharpness evaluation model based on the original image and the sharpness evaluation result for the original image, as well as the processed image and the sharpness evaluation result for the processed image.

[0031] Optionally, obtaining a sharpness prediction result for the image to be evaluated based on the first image sharpness evaluation model includes: inputting the image to be evaluated into the first image sharpness evaluation model to obtain a sharpness prediction result for the image to be evaluated.

[0032] Optionally, obtaining the sharpness prediction result for the image to be evaluated based on the first image sharpness evaluation model includes: using the first image sharpness evaluation model as a teacher model, obtaining a student model through knowledge distillation, and using it as a second image sharpness evaluation model for evaluating image sharpness; inputting the image to be evaluated into the second image sharpness evaluation model to obtain the sharpness prediction result for the image to be evaluated.

[0033] This application also provides an image sharpness evaluation device, comprising: a fifth obtaining unit for obtaining an image to be evaluated; a sixth obtaining unit for obtaining a first image sharpness evaluation model for evaluating image sharpness, wherein the first image sharpness evaluation model is a regression model; and a seventh obtaining unit for obtaining a sharpness prediction result for the image to be evaluated based on the first image sharpness evaluation model.

[0034] This application also provides a method for obtaining training samples, comprising: obtaining an original image; obtaining a sharpness evaluation result for the original image; performing image degradation processing or image reconstruction processing on the original image based on the sharpness evaluation result for the original image to obtain a processed image; obtaining a sharpness evaluation result for the processed image; and obtaining training samples for training an image sharpness evaluation model based on the original image and the sharpness evaluation result for the original image, as well as the processed image and the sharpness evaluation result for the processed image, wherein the image sharpness evaluation model is a model for evaluating image sharpness.

[0035] Optionally, obtaining the sharpness assessment result for the original image includes: establishing a correspondence between the sharpness category to which the image belongs and the sharpness score range; obtaining the sharpness category to which the original image belongs according to the image sharpness classification model; and obtaining the sharpness score for the original image according to the sharpness score range corresponding to the sharpness category to which the original image belongs, as the sharpness assessment result for the original image.

[0036] Optionally, obtaining the sharpness evaluation result for the original image includes: establishing a correspondence between the probability value of the sharpness category to which the image belongs and the sharpness score; obtaining the probability value of the sharpness category to which the original image belongs based on the image sharpness classification model; and obtaining the sharpness score corresponding to the probability value of the sharpness category to which the original image belongs, as the sharpness evaluation result for the original image.

[0037] Optionally, the step of performing image degradation processing or image reconstruction processing on the original image based on the sharpness evaluation result of the original image to obtain a processed image includes: if the sharpness evaluation result of the original image indicates that the sharpness of the original image is higher than a first preset threshold, then performing image degradation processing on the background portion of the original image to obtain a background degradation image as the processed image; the step of obtaining the sharpness evaluation result of the processed image includes: obtaining the sharpness evaluation result of the processed image based on the sharpness evaluation result of the original image, wherein the sharpness evaluation result of the processed image indicates that the sharpness of the processed image is lower than the sharpness of the original image, and the difference between the sharpness of the processed image and the sharpness of the original image meets a preset range.

[0038] Optionally, the step of performing image degradation processing or image reconstruction processing on the original image based on the sharpness evaluation result of the original image to obtain a processed image includes: if the sharpness evaluation result of the original image indicates that the sharpness of the original image is lower than the first preset threshold and higher than the second preset threshold, then performing image degradation processing on the original image to obtain a full-image degradation image as the processed image; the step of obtaining the sharpness evaluation result of the processed image includes: obtaining the sharpness evaluation result of the processed image based on the sharpness evaluation result of the original image, wherein the sharpness evaluation result of the processed image indicates that the sharpness of the processed image is lower than the sharpness of the original image.

[0039] Optionally, the step of performing image degradation processing or image reconstruction processing on the original image based on the sharpness evaluation result of the original image to obtain a processed image includes: if the sharpness evaluation result of the original image indicates that the sharpness of the original image is lower than the second preset threshold, then performing image reconstruction processing on the original image to obtain a full-image reconstructed image as the processed image; the step of obtaining the sharpness evaluation result of the processed image includes: obtaining the sharpness evaluation result of the processed image based on the sharpness evaluation result of the original image, wherein the sharpness evaluation result of the processed image indicates that the sharpness of the processed image is higher than the sharpness of the original image.

[0040] Optionally, the step of performing image reconstruction processing on the original image to obtain a full-image reconstructed image as the processed image includes: inputting the original image into an image reconstruction model to obtain a full-image reconstructed image; using an image sharpness classification discriminator to determine whether the sharpness category to which the full-image reconstructed image belongs is a sharp category, and using an image relative sharpness discriminator to determine whether the sharpness of the full-image reconstructed image is higher than the sharpness of the original image; if the sharpness category to which the full-image reconstructed image belongs is a sharp category, and the sharpness of the full-image reconstructed image is higher than the sharpness of the original image, then the full-image reconstructed image is used as the processed image.

[0041] Optionally, obtaining training samples for training the image sharpness evaluation model based on the original image and the sharpness evaluation result for the original image, as well as the processed image and the sharpness evaluation result for the processed image, includes: using the original image and the processed image as input samples respectively, using the sharpness evaluation result for the original image and the sharpness evaluation result for the processed image as sample labels respectively, and obtaining training samples for training the image sharpness evaluation model based on the input samples and the sample labels.

[0042] This application also provides a training sample acquisition device, comprising: an eighth acquisition unit for acquiring an original image; a ninth acquisition unit for acquiring a sharpness evaluation result for the original image; a tenth acquisition unit for performing image degradation processing or image reconstruction processing on the original image based on the sharpness evaluation result for the original image to obtain a processed image; an eleventh acquisition unit for acquiring a sharpness evaluation result for the processed image; and a twelfth acquisition unit for acquiring training samples for training an image sharpness evaluation model based on the original image and the sharpness evaluation result for the original image, as well as the processed image and the sharpness evaluation result for the processed image, wherein the image sharpness evaluation model is a model for evaluating image sharpness.

[0043] This application also provides a method for obtaining an image sharpness evaluation model, comprising: obtaining a first image sharpness evaluation model for evaluating image sharpness; using the first image sharpness evaluation model as a teacher model, obtaining a student model through knowledge distillation, which serves as a second image sharpness evaluation model for evaluating image sharpness, wherein the student model is trained based on the degree of difference between the sharpness prediction results of the teacher model and the sharpness prediction results of the initial student model, and the degree of difference between the feature maps of the teacher model and the feature maps of the initial student model.

[0044] Optionally, the step of using the first image sharpness evaluation model as a teacher model and obtaining a student model through knowledge distillation includes: obtaining an initial student model; obtaining a teacher model sharpness prediction result and a teacher model feature map based on the teacher model; obtaining an initial student model sharpness prediction result and an initial student model feature map based on the initial student model; obtaining a knowledge distillation loss function for evaluating the degree of difference between the teacher model sharpness prediction result and the initial student model sharpness prediction result; obtaining a feature map difference loss function for evaluating the degree of difference between the teacher model feature map and the initial student model feature map; and adjusting the parameters of the initial student model based on the knowledge distillation loss function and the feature map difference loss function to obtain the student model.

[0045] Optionally, adjusting the parameters of the initial student model based on the knowledge distillation loss function and the feature map difference loss function to obtain the student model includes: calculating the value of the knowledge distillation loss function based on the clarity prediction results of the teacher model and the initial student model; calculating the value of the feature map difference loss function based on the feature map of the teacher model and the feature map of the initial student model; adjusting the parameters of the initial student model based on the sum of the values ​​of the knowledge distillation loss function and the feature map difference loss function; and stopping the adjustment of the parameters of the initial student model until the sum of the values ​​of the knowledge distillation loss function and the feature map difference loss function reaches a second preset condition, thereby obtaining the student model.

[0046] This application also provides an image sharpness evaluation model acquisition device, comprising: a thirteenth acquisition unit for acquiring a first image sharpness evaluation model for evaluating image sharpness; and a fourteenth acquisition unit for using the first image sharpness evaluation model as a teacher model and obtaining a student model through knowledge distillation as a second image sharpness evaluation model for evaluating image sharpness, wherein the student model is trained based on the degree of difference between the sharpness prediction result of the teacher model and the sharpness prediction result of the initial student model, and the degree of difference between the feature map of the teacher model and the feature map of the initial student model.

[0047] This application also provides an electronic device, including a processor and a memory; the memory is used to store programs and data, and the processor calls the programs stored in the memory to execute the above-described image sharpness evaluation model acquisition method, or the above-described image sharpness evaluation method, or the above-described training sample acquisition method, or the above-described image sharpness evaluation model acquisition method.

[0048] This application also provides a storage medium storing a program and data, the program being executed by a processor to implement the above-described image sharpness evaluation model acquisition method, or the above-described image sharpness evaluation method, or the above-described training sample acquisition method, or the above-described image sharpness evaluation model acquisition method.

[0049] Compared with the prior art, this application has the following advantages:

[0050] The image sharpness assessment model obtained in this application involves performing image degradation or image reconstruction processing on the original image to obtain images with different sharpness levels, thereby constructing a sample set that meets different sharpness requirements. Based on the original image and the sharpness assessment result for the original image, as well as the processed image and the sharpness assessment result for the processed image, a first image sharpness assessment model for evaluating image sharpness is obtained. This model can simulate the human eye's visual perception of sharpness and output image sharpness prediction results, and can be used in image sharpness assessment scenarios without reference. Attached Figure Description

[0051] Figure 1 This is a flowchart of the image sharpness evaluation model training and inference method provided in this application.

[0052] Figure 2 This is a schematic diagram of the classification model auxiliary data labeling process in the training and inference method of the image sharpness evaluation model provided in this application.

[0053] Figure 3 This is a schematic diagram of the image degradation processing process in the training and inference method of the image sharpness evaluation model provided in this application.

[0054] Figure 4 This is a schematic diagram of the image reconstruction process in the training and inference method of the image sharpness evaluation model provided in this application.

[0055] Figure 5 This is a schematic diagram of the structure of the first image sharpness evaluation model in the image sharpness evaluation model training and inference method provided in this application.

[0056] Figure 6 This is a schematic diagram of the knowledge distillation process in the training and inference method of the image sharpness evaluation model provided in this application.

[0057] Figure 7 This is a flowchart of the method for obtaining the image sharpness evaluation model provided in the first embodiment of this application.

[0058] Figure 8 This is a schematic diagram of the image sharpness evaluation model acquisition device provided in the second embodiment of this application.

[0059] Figure 9 This is a flowchart of the image sharpness evaluation method provided in the third embodiment of this application.

[0060] Figure 10 This is a schematic diagram of the image sharpness evaluation device provided in the fourth embodiment of this application.

[0061] Figure 11 This is a flowchart of the training sample acquisition method provided in the fifth embodiment of this application.

[0062] Figure 12 This is a schematic diagram of the training sample acquisition device provided in the sixth embodiment of this application.

[0063] Figure 13 This is a flowchart of the method for obtaining the image sharpness evaluation model provided in the seventh embodiment of this application.

[0064] Figure 14 This is a schematic diagram of the image sharpness evaluation model acquisition device provided in the eighth embodiment of this application.

[0065] Figure 15 This is a schematic diagram of the electronic device provided in the ninth embodiment of this application. Detailed Implementation

[0066] To make the objectives, advantages, and features of this application clearer, the following detailed description, in conjunction with the accompanying drawings and specific embodiments, further illustrates this application. In the following description, numerous specific details are set forth to provide a thorough understanding of this application. However, this application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this application. Therefore, this application is not limited to the specific embodiments disclosed below.

[0067] It should be noted that in the description of this application, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance, or a specific order or sequence. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances. Furthermore, in the description of this application, unless otherwise stated, the term "multiple" refers to two or more. The term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. The terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or devices.

[0068] In online-to-offline (O2O) service platforms, image data is one of their core assets, playing a crucial role in enabling users to obtain product information and complete purchases. When a user visits a service page on an O2O platform, the clarity of the product images displayed directly impacts the user's visual experience and interest in the product. Therefore, selecting high-quality, clear images from a vast pool of data for display is essential.

[0069] For online-to-offline service platforms, in addition to the need for high-quality, clear images for daily product displays, many services also rely on high-quality, clear images. For example, using high-quality, clear image materials paired with predefined templates can generate more creative and attractive images, thereby drawing users' attention and interest. However, with the information explosion and increasing diversity of content, the number of images in massive image libraries has reached billions. How to select high-quality, clear images from this vast pool has become a very urgent and challenging problem.

[0070] Sharpness is a crucial concept in image processing, typically referring to the clarity of details and contours of objects in an image. In online-to-offline (OTO) service platform scenarios, sharpness is a core metric for evaluating image quality, primarily reflecting the contrast and richness of detail at local edges. In OTO service platform scenarios, many images exhibit high resolution, low detail, and blurred contours. Traditional error-based evaluation metrics, such as MSE (Mean Square Error) and PSNR (Peak Signal-to-Noise Ratio), and metrics based on perceptual models and image structure information, such as SSIM (Structural Similarity), are no longer suitable for evaluating sharpness in OTO service platform scenarios where there are no references.

[0071] In summary, traditional image sharpness assessment methods typically require reference to standard images and evaluate image sharpness based on objective evaluation metrics. However, in online-to-offline service platform scenarios, sharpness assessment needs to be performed on individual, unreferenced images, using human subjective visual perception as the standard. Therefore, traditional methods are not suitable for unreferenced image sharpness assessment scenarios and cannot simulate the human eye's visual perception of sharpness.

[0072] To address the aforementioned issues, this application provides an image sharpness assessment method. This method utilizes a regression model within a machine learning model to output a sharpness prediction result for a single image that aligns with the visual perception of most people, thereby selecting high-quality, sharp images for product display or image synthesis. Furthermore, it employs knowledge distillation to improve model efficiency while maintaining accuracy, enabling end-to-end real-time inference.

[0073] The following describes the image sharpness evaluation method provided in this application, as well as the method for obtaining the image sharpness evaluation model and the method for obtaining training samples for training the image sharpness evaluation model, using an application scenario as an example.

[0074] Figure 1 The diagram shows a flowchart of the end-to-end image sharpness assessment model training and inference method in an online-to-offline service platform scenario. It mainly includes two parts: data construction (including...) Figure 2 , 3 (as shown in Figure 4) and model training inference (including...) Figure 5 , 6(See the content shown). Data construction refers to the process of annotating, degrading, and reconstructing original images from online-to-offline service platform scenarios to create image sample sets that meet different sharpness requirements, and generating corresponding subjective sharpness scores (MOS). Model training and inference refers to training a neural network model adapted to different sharpness requirements using deep learning methods based on the constructed image sample set, and simplifying the neural network model structure through knowledge distillation, thereby applying it end-to-end to image sharpness evaluation tasks in online-to-offline service platform scenarios. Specifically, it includes the following steps:

[0075] S101, Auxiliary data annotation for classification models.

[0076] First, the original images from online-to-offline service platform scenarios are labeled to obtain a subjective score for the clarity of the original images.

[0077] Generally, for original images in online-to-offline service platform scenarios, the images are labeled through the subjective judgment of a large number of evaluators. Each evaluator assigns a sharpness score based on their perception of image quality and edge detail, ranging from 1 (representing blurred to indistinguishable subject) to 10 (representing extremely sharpness). These scores are then aggregated and averaged to form the subjective sharpness rating for the original image. For images in online-to-offline service platform scenarios, this subjective evaluation method aligns more closely with user perception than objective evaluation methods.

[0078] To reduce the cost of manual annotation, an image sharpness classification model can be used to assist in data annotation. This model can quickly and effectively classify the original image into three categories: sharp, intermediate (sharp and blurry), and blurry. Then, the image is labeled according to its sharpness category, providing evaluators with a relevant assessment reference and avoiding excessive subjective evaluation differences.

[0079] Image sharpness classification models are machine learning models that determine the sharpness level of an input image and categorize it into different classes, such as sharp, intermediate between sharp and blurry, and blurry. These models are typically built using deep learning algorithms and trained on labeled images. During training, the model learns to extract image features and establish a correlation between these features and sharpness. Once trained, the model can determine the sharpness of an input image based on its features and output a probability value indicating whether the image belongs to a different sharpness category. The image's sharpness category is then determined based on these probability values.

[0080] The process of classification model-assisted data annotation is as follows Figure 2 As shown, a three-category image sharpness classification model is trained using a small number of original images from an online-to-offline service platform scenario. The three categories are sharp, between sharp and blurry, and blurry. The image sharpness classification model structure includes Conv1×1 (1×1 convolutional layer), D-wise3×3 (3×3 depthwise separable convolution), and Conv1×1 (1×1 convolutional layer), connected by residuals. A large number of original images are input into the image sharpness classification model to obtain the sharpness category to which the original images belong, and they are sorted based on the probability values ​​of their respective sharpness categories. The subjective sharpness rating range corresponding to each sharpness category is determined; for example, the subjective sharpness rating range for the sharp category is 7-9.9, for the between sharp and blurry category it is 3.1-6.9, and for the blurry category it is 0.1-3. Evaluators label each original image according to the ranking, within the subjective sharpness rating range corresponding to its respective sharpness category, thereby obtaining a sharpness evaluation score for the original image. For the same original image, the sharpness evaluation scores of all evaluators are summed and averaged to obtain the subjective sharpness score for that original image.

[0081] In order to construct an image sample set that meets different sharpness requirements, and to make the trained image sharpness evaluation model focus more on sharpness rather than other specific dimensions, the original image is subjected to the following image degradation or image reconstruction processing based on the subjective sharpness score of the original image to obtain the processed image.

[0082] Of course, the processed images can also be classified using an image sharpness classification model, followed by manual annotation to obtain a subjective sharpness score for each image. Finally, the subjective sharpness scores of all images can be integrated and filtered to ensure that the subjective sharpness scores of all images in the image sample set are within the range of 0-10.

[0083] S102, Image degradation processing.

[0084] To ensure the trained image sharpness assessment model focuses more on the main content of the image, background images are extracted from particularly sharp or slightly sharp original images using a cutout technique. These particularly sharp or slightly sharp original images can be selected based on subjective sharpness scores for the original images; for example, images with subjective sharpness scores higher than a first preset threshold are selected. Then, the background images undergo random degradation processing to obtain degraded background images, which serve as the processed images. The processed images are then manually annotated to obtain a sharpness evaluation score, which in turn yields a subjective sharpness score for the processed image. Here, the subjective sharpness score for the processed image should still fall within the sharpness evaluation score range corresponding to the sharpness category of the original image, but should be slightly lower than the subjective sharpness score for the original image.

[0085] To avoid the trained image sharpness evaluation model focusing solely on the image content, random degradation processing is applied to the entire original image (sharp, slightly sharp, and slightly blurry) to obtain a fully degraded image, which serves as the processed image. The sharp, slightly sharp, and slightly blurry original images are selected based on subjective sharpness ratings, with scores below a first preset threshold and above a second preset threshold. The processed images are then manually annotated to obtain a sharpness evaluation score, which in turn yields a subjective sharpness rating for the processed image. The subjective sharpness rating for the processed image should be significantly lower than that for the original image; the more random degradation steps involved, the greater the reduction in the subjective sharpness rating for the processed image.

[0086] The process of image degradation processing is as follows Figure 3 As shown, it includes at least one random degradation stage, and each random degradation stage includes multiple degradation modules, that is, multiple degradation steps. Examples include anisotropic degradation modules, isotropic degradation modules, Gaussian blur degradation modules, JPEG compression degradation modules, bicubic interpolation degradation modules, bilinear interpolation degradation modules, Gaussian noise degradation modules, and gray noise degradation modules. Each degradation module's activation is controlled by a corresponding random factor, so the activation of each degradation module is random, resulting in random interleaving and connection between the various degradation modules. Here, three random degradation stages are used (the third stage in the figure is not shown), and each random degradation stage includes 12 degradation modules, each of which is randomly activated.

[0087] The processed image is manually annotated to obtain a subjective sharpness score. Alternatively, the processed image can be first input into an image sharpness classification model to determine its sharpness category. Then, an evaluator annotates the processed image according to the subjective sharpness score range corresponding to its category, obtaining a sharpness evaluation score and thus a subjective sharpness score. The more random degradation steps are applied, the lower the subjective sharpness score of the processed image, but it must still meet the scoring requirements for background degradation and full-image degradation.

[0088] By performing image degradation processing on the original image, images with various degradation effects can be obtained. To avoid imbalance problems during the training process of the image sharpness evaluation model, the original image also needs to undergo the following image reconstruction processing.

[0089] S103, Image reconstruction processing.

[0090] For a blurred original image, an image reconstruction model is used to reconstruct the details and sharpness of the original image, resulting in a fully reconstructed image, which serves as the processed image. The blurred original image can be selected based on a subjective sharpness rating lower than a second preset threshold. The processed image is then manually annotated to obtain a sharpness evaluation score, which in turn yields a subjective sharpness rating for the processed image. The subjective sharpness rating for the processed image should be higher than that for the original image.

[0091] The process of image reconstruction is as follows Figure 4 As shown, the original image is compressed by a shallow neural network (conv), and then further processed through three stacked RRDP (Residual in Residual Dense Pooling) modules, each consisting of several residual dense blocks. The image is then reconstructed after upsampling. Two discriminators (conv) are introduced simultaneously. The first discriminator is a lightweight network used in the auxiliary data annotation process, namely an image sharpness classification discriminator, used to determine whether the sharpness category of the reconstructed image is sharp. The second discriminator is a relative discriminator, namely an image relative sharpness discriminator, used to determine whether the sharpness of the reconstructed image is higher than that of the original image, expressed by the formula:

[0092] D(x ori ,x new )=σ(C(xori )-E[C(x new )])

[0093] Where D(.) represents the discriminator; x ori Represents the original image; x new σ represents the reconstructed full-image; C(.) represents the discriminator; E represents the feature output after the image reconstruction model; and E represents the expected output of the full-reconstructed full-image from a small batch of original images in the discriminator.

[0094] If the sharpness category of the reconstructed image is sharp, and the sharpness of the reconstructed image is higher than that of the original image, then the reconstructed image is used as the processed image.

[0095] The processed image is manually annotated to obtain a subjective sharpness score. Alternatively, the processed image can be first input into an image sharpness classification model to determine its sharpness category. Then, an evaluator annotates the processed image according to the subjective sharpness score range corresponding to its category, obtaining a sharpness evaluation score and thus a subjective sharpness score. The subjective sharpness score for the processed image should be higher than that for the original image.

[0096] At this point, images that meet different sharpness requirements and subjective sharpness scores for these images have been obtained. These images are used as input samples, and the subjective sharpness scores for these images are used as sample labels, thus forming training samples for training the image sharpness evaluation model.

[0097] Because training samples are constructed from existing raw images, the labels are very sparse and imbalanced. Training a model to evaluate image sharpness using this data can easily lead the model to veer towards irrelevant categories during training. The image degradation and reconstruction processes described above, which expand the training sample data, can resolve the class imbalance problem.

[0098] S104, Training and inference of the first image sharpness evaluation model.

[0099] The first image sharpness assessment model is a regression model, which can be the EfficientNetB6 convolutional neural network model.

[0100] A regression model is a model used to predict continuous numerical output variables, such as predicting house prices or sales revenue. Unlike classification models, which output discrete categories, regression models can output arbitrary numerical values.

[0101] EfficientNetB6 is one of the EfficientNet series, which is a series of efficient neural network architectures proposed by the Google team in 2019.

[0102] The architecture of the first image sharpness assessment model is as follows: Figure 5 As shown, the architecture includes a Backbone, a Global Average Pooling (GAP) layer, and a Fully Connected (FC) head. The Backbone consists of multiple stacked convolutional layers. After passing through these stacked convolutional layers, the image features are integrated using a Global Average Pooling (GAP) layer and several fully connected layers. Finally, a sharpness prediction score for the image is output.

[0103] During the training and inference process, the input samples are fed into the first image sharpness evaluation initial model, which then outputs a sharpness prediction score for the input samples. The term "first image sharpness evaluation initial model" simply refers to the first image sharpness evaluation model that has not yet met the required accuracy; that is, it refers to the first image sharpness evaluation model before the end of training, and its architecture is as described above.

[0104] Based on the sharpness prediction score and the subjective sharpness score of the input sample, the value of the sharpness assessment difference loss function is calculated. Then, based on the value of the sharpness assessment difference loss function, the parameters of the initial model for first image sharpness assessment are adjusted until the value of the sharpness assessment difference loss function reaches a first preset condition. At this point, the adjustment of the parameters of the initial model for first image sharpness assessment stops, resulting in a first image sharpness assessment model that can be used in the application scenario. The first preset condition may be that the value of the sharpness assessment difference loss function no longer decreases, indicating that the model accuracy will no longer improve, or that the value of the sharpness assessment difference loss function is lower than the first preset loss value, indicating that the model accuracy has met the requirements.

[0105] The sharpness assessment difference loss function is as follows:

[0106]

[0107] L represents the sharpness assessment difference loss between the sharpness prediction result and the sharpness assessment result for the input sample; N represents the total number of input samples; i represents the i-th input sample; j represents the j-th input sample; y i This represents the sharpness evaluation result for the i-th input sample; This represents the sharpness prediction result for the i-th input sample; y j This represents the sharpness evaluation result for the j-th input sample; Let l1 and l2 represent the sharpness prediction result for the j-th input sample; l1 and l2 both represent distances. τ represents the preset hyperparameter; ∑ represents summation; max(.) represents taking the maximum value; |.| represents taking the absolute value.

[0108] Thus, the first image sharpness assessment model, after training, can evaluate the sharpness of a single image without reference, simulate the human eye's visual perception of sharpness, and output a sharpness prediction score for the image.

[0109] S105, knowledge distillation acceleration.

[0110] The first image sharpness evaluation model uses the EfficientNetB6 convolutional neural network model, which has a complex structure and a large number of parameters, resulting in significant time consumption when processing high-resolution images. However, directly using a lightweight model as the sharpness evaluation model would lead to underfitting. Therefore, for the first image sharpness evaluation model, knowledge distillation is employed to transfer knowledge from the complex model to the lightweight model.

[0111] The process of knowledge distillation is as follows Figure 6 As shown, the first image sharpness evaluation model is used as the teacher model, and its parameters are frozen. Furthermore, convolutional blocks in each stage of EfficientNetB6 are removed, retaining only at least one block per stage, while the rest of the structure remains consistent with the original EfficientNetB6. This is used as the initial student model. Similarly, the term "initial student model" refers to the student model that has not yet met the accuracy requirements, i.e., the student model before the knowledge distillation process is complete. Input samples are fed into the teacher model and the initial student model respectively to obtain the sharpness prediction results and feature maps of the teacher model, as well as the sharpness prediction results and feature maps of the initial student model.

[0112] Based on the sharpness prediction results of the teacher model and the initial student model, the value of the knowledge distillation loss function is calculated, and the value of the feature map difference loss function is calculated based on the feature maps of the teacher model and the initial student model. The parameters of the initial student model are adjusted according to the sum of the knowledge distillation loss function and the feature map difference loss function until the sum reaches a second preset condition. Adjustment of the initial student model's parameters is then stopped, resulting in a student model, which serves as a second image sharpness evaluation model that can be used in application scenarios. The smaller the sum of the knowledge distillation loss function and the feature map difference loss function, the higher the accuracy of the student model. The second preset condition can be either that the sum of the knowledge distillation loss function and the feature map difference loss function no longer decreases, or that the sum of the knowledge distillation loss function and the feature map difference loss function is lower than a second preset loss value.

[0113] The knowledge distillation loss function is as follows:

[0114]

[0115] L kd This represents the knowledge distillation loss between the clarity prediction results of the teacher model and the clarity prediction results of the initial student model. This represents the initial student model clarity prediction results; This represents the clarity prediction result of the teacher model; |.| represents taking the absolute value.

[0116] The feature map difference loss function is as follows:

[0117]

[0118] Let represent the difference loss for the i-th feature map; i represents the i-th feature map; n represents the total number of feature maps; This represents the feature map feature of the i-th initial student model; Let represent the feature map feature of the i-th teacher model; vec(.) denotes vectorization; ‖.‖2 denotes L2 normalization.

[0119] Thus, a lightweight second image sharpness evaluation model was obtained for evaluating image sharpness. This model can also evaluate the sharpness of a single image without reference, simulate the human eye's visual perception of sharpness, and output a sharpness prediction score for the image.

[0120] The aforementioned model distillation achieves a smooth transition from complex to lightweight models, resolving the issue of time-consuming inference for complex models in online-to-offline service platform scenarios. Furthermore, unlike the distillation loss used for classification tasks, a specific loss function is designed for regression tasks. This function not only calculates the difference between the predicted sharpness score and the subjective sharpness score but also considers the difference loss of feature maps during the model distillation process, thus making the entire distillation process more stable.

[0121] In the context of an online-to-offline service platform, for images uploaded by users, the aforementioned second image sharpness assessment model can be used to quickly evaluate the image's sharpness and return a predicted sharpness score to the user. Alternatively, for images whose predicted sharpness score does not meet the requirements, a message indicating that the uploaded image does not meet the requirements can be returned to the user. Furthermore, for messages sent by users requesting images of a certain sharpness, the aforementioned second image sharpness assessment model can be used to quickly filter images that meet the sharpness requirements from the image library and send them to the user. Here, the user can be a user using the online-to-offline service platform to obtain services, a user using the platform to provide services, or a user used by the platform's operators. Of course, the above application scenarios can also be implemented using the first image sharpness assessment model.

[0122] The above is an introduction to the application scenario of the image sharpness evaluation method provided in this application. The following is a detailed description of the image sharpness evaluation model acquisition method, image sharpness evaluation method, training sample acquisition method and other related methods involved in this application.

[0123] First Embodiment

[0124] The first embodiment of this application provides a method for obtaining an image sharpness evaluation model, such as... Figure 7 As shown, the method includes the following steps:

[0125] S701, obtain the sharpness assessment results for the original image.

[0126] The original image can be an original image from an online-to-offline service platform scenario, such as an image from an image library.

[0127] The sharpness assessment result for the original image is used to represent the sharpness status of the original image, such as a sharpness score or sharpness level. In this embodiment, the sharpness score is used as the sharpness assessment result. This sharpness score, also known as the Mean Opinion Score (MOS) mentioned above, is obtained by labeling an original image with the subjective judgments of a large number of evaluators, obtaining a sharpness evaluation score for each evaluator. For example, the sharpness evaluation score ranges from 1 (representing blurriness to the point of being indistinguishable) to 10 (representing extreme sharpness). Then, all the sharpness evaluation scores for the original image are aggregated, and the average score is calculated, which is used as the subjective sharpness score for the original image.

[0128] For massive amounts of raw images, relying entirely on manual annotation to obtain sharpness scores, i.e., sharpness assessment results, is extremely time-consuming and labor-intensive, and prone to significant subjective differences. Therefore, image sharpness classification models can be used to assist in data annotation; two methods are provided below.

[0129] The first approach is to establish a correspondence between the image's sharpness category and its sharpness score range. For example, the sharpness score range for the sharpness category is 7-9.9, the sharpness score range for the category between sharp and blurry is 3.1-6.9, and the sharpness score range for the blurry category is 0.1-3.

[0130] Secondly, based on the image sharpness classification model, the sharpness category of the original image is obtained. A three-category image sharpness classification model can be trained using a small number of original images from online-to-offline service platform scenarios. The three categories can be sharp, intermediate between sharp and blurry, and blurry. The image sharpness classification model structure includes Conv1×1 (1×1 convolutional layer), D-wise3×3 (3×3 depthwise separable convolution), and Conv1×1 (1×1 convolutional layer), connected by residuals. A large number of original images are input into the image sharpness classification model to obtain the sharpness category of the original images, and then sorted based on the probability values ​​of the categories to which the original images belong.

[0131] Finally, based on the sharpness score range corresponding to the sharpness category to which the original image belongs, a sharpness score is obtained for the original image, which serves as the sharpness evaluation result for the original image. Evaluators can label each original image within the sharpness score range corresponding to its respective sharpness category according to the ranking, obtaining a sharpness evaluation score for each original image. For the same original image, the sharpness evaluation scores from all evaluators are summed and averaged to obtain a subjective sharpness score for that original image, i.e., the sharpness score for the original image. This sharpness score is used as the sharpness evaluation result for the original image.

[0132] The second approach involves first establishing a correspondence between the probability value of an image's sharpness category and its sharpness score. Since the image sharpness classification model outputs the probability value of an image belonging to different sharpness categories, the sharpness category with the highest probability value is usually taken as the image's sharpness category. Therefore, the probability value of an image's sharpness category can be directly mapped one-to-one with its sharpness score.

[0133] Secondly, based on the image sharpness classification model, the probability value of the sharpness category to which the original image belongs is obtained. The image sharpness classification model can be the same as described above.

[0134] Finally, the sharpness score corresponding to the probability value of the sharpness category to which the original image belongs is obtained, which serves as the sharpness evaluation result for the original image.

[0135] S702, based on the sharpness assessment result of the original image, perform image degradation processing or image reconstruction processing on the original image to obtain the processed image.

[0136] In order to construct an image sample set that meets different sharpness requirements, and to make the trained image sharpness evaluation model focus more on sharpness rather than other specific dimensions, the original image is subjected to image degradation processing or image reconstruction processing based on the sharpness evaluation results of the original image to obtain the processed image.

[0137] If the clarity assessment result of the original image indicates that the clarity of the original image is higher than a first preset threshold, then the background portion of the original image is subjected to image degradation processing to obtain a background-degraded image, which is used as the processed image.

[0138] Specifically, original images with a sharpness score higher than a first preset threshold can be selected, for example, original images with a sharpness score higher than 7.5. To make the trained image sharpness evaluation model focus more on the main content of the image, the selected original images are processed by image matting to obtain the background image. Then, the background image is randomly degraded to obtain a degraded background image, which is used as the processed image.

[0139] If the sharpness evaluation result of the original image indicates that the sharpness of the original image is lower than the first preset threshold and higher than the second preset threshold, then the original image is subjected to image degradation processing to obtain a full-image degradation image, which is used as the processed image.

[0140] Specifically, original images with a sharpness score lower than a first preset threshold and higher than a second preset threshold can be selected, for example, original images with a sharpness score of 3-7.5. To avoid the trained image sharpness evaluation model focusing on the image content itself, the selected original images are subjected to random degradation processing to obtain a fully degraded image, which is used as the processed image.

[0141] Image degradation processing includes at least one random degradation stage, which comprises multiple degradation modules, or multiple degradation steps. Examples include anisotropic, isotropic, Gaussian blur, JPEG compression, bicubic, bilinear, Gaussian noise, and grayscale noise modules. Each degradation module's activation is controlled by a corresponding random factor, thus its activation is random, allowing for random interleaving and connection between the various degradation modules.

[0142] If the sharpness evaluation result of the original image indicates that the sharpness of the original image is lower than the second preset threshold, then the original image is subjected to image reconstruction processing to obtain a full-image reconstructed image, which is used as the processed image.

[0143] Specifically, original images with a sharpness score lower than a second preset threshold can be selected, for example, original images with a sharpness score lower than 3. The original images are input into an image reconstruction model, and the image reconstruction model reconstructs the details and sharpness of the selected original images to obtain a full-image reconstructed image, which is used as the processed image.

[0144] The image reconstruction process includes compressing the original image using a shallow neural network (conv), followed by passing it through three stacked RRDP (Residual in Residual Dense Pooling) modules, each consisting of several residual dense blocks. The image is then reconstructed after upsampling. Two discriminators (conv) are introduced: the first is a lightweight network used in the auxiliary annotation process, i.e., an image sharpness classification discriminator, used to determine whether the sharpness category of the reconstructed image is sharp; the second is a relative discriminator, i.e., an image relative sharpness discriminator, used to determine whether the sharpness of the reconstructed image is higher than that of the original image, expressed by the formula:

[0145] D(x ori ,x new )=σ(C(x ori )-E[C(x new )])

[0146] Where D(.) represents the discriminator; x ori Represents the original image; x new σ represents the reconstructed full image; C(.) represents the discriminator; E represents the expected output of the full reconstructed full image of a small batch of original images in the discriminator.

[0147] If the sharpness category of the reconstructed image is sharp, and the sharpness of the reconstructed image is higher than that of the original image, then the reconstructed image is used as the processed image.

[0148] S703, obtain the sharpness evaluation result for the processed image.

[0149] Based on the sharpness assessment results for the original image, a sharpness assessment result for the processed image is obtained.

[0150] Specifically, for a processed image with a degraded background, the sharpness evaluation result of the processed image indicates that the sharpness of the processed image is lower than that of the original image, and the difference between the sharpness of the processed image and the original image meets a preset range. In other words, the processed image is manually annotated to obtain a sharpness evaluation score, which in turn yields a subjective sharpness score, or sharpness rating. This sharpness rating for the processed image should still fall within the sharpness rating range corresponding to the sharpness category of the original image, and should be slightly lower than the sharpness rating for the original image.

[0151] For a fully degraded image as the processed image, the sharpness evaluation result for the processed image indicates that the sharpness of the processed image is lower than that of the original image. In other words, the processed image is manually annotated to obtain a sharpness evaluation score, which is then used to obtain a subjective sharpness score for the processed image. This sharpness score for the processed image should be significantly lower than the sharpness score for the original image; the more random degradation steps, the greater the reduction in the sharpness score for the processed image.

[0152] For a fully reconstructed image as the processed image, the sharpness evaluation result for the processed image indicates that the sharpness of the processed image is higher than that of the original image. In other words, the processed image is manually annotated to obtain a sharpness evaluation score, which in turn yields a subjective sharpness rating for the processed image, i.e., a sharpness score. This sharpness score for the processed image should be higher than the sharpness score for the original image.

[0153] At this point, images that meet different sharpness requirements and their sharpness scores have been obtained. These images are used as input samples, and their sharpness scores are used as sample labels, thus forming training samples for training the image sharpness evaluation model.

[0154] S704, based on the original image and the sharpness evaluation result for the original image, and the processed image and the sharpness evaluation result for the processed image, a first image sharpness evaluation model for evaluating image sharpness is obtained.

[0155] Specifically, the first initial model for image sharpness assessment is obtained, which is a regression model. The term "first initial model for image sharpness assessment" refers to the first image sharpness assessment model that has not yet met the required accuracy; that is, the first image sharpness assessment model before training is complete.

[0156] A regression model is a model used to predict continuous numerical output variables, such as predicting house prices or sales revenue. Unlike classification models, which output discrete categories, regression models can output arbitrary numerical values.

[0157] The initial model for the first image sharpness evaluation can be the EfficientNetB6 convolutional neural network model. EfficientNetB6 is one of the EfficientNet series, a series of efficient neural network architectures proposed by the Google team in 2019.

[0158] The initial model architecture for image sharpness assessment consists of a Backbone network, a Global Average Pooling (GAP) layer, and a Fully Connected (FC) head. The Backbone network comprises multiple stacked convolutional layers. After passing through these stacked convolutional layers, the image features are integrated via a Global Average Pooling (GAP) layer and several fully connected layers. Finally, the sharpness prediction result is output, which can be a sharpness prediction score.

[0159] The original image and the processed image are used as input samples and input into the first image sharpness evaluation initial model to obtain sharpness prediction results for the input samples. Based on the sharpness prediction results and the sharpness evaluation results for the input samples, the parameters of the first image sharpness evaluation initial model are adjusted to obtain the first image sharpness evaluation model.

[0160] Specifically, a sharpness assessment difference loss function is obtained to evaluate the degree of difference between the sharpness prediction result and the sharpness assessment result for the input sample. Based on the sharpness assessment difference loss function, the parameters of the first image sharpness assessment initial model are adjusted to obtain the first image sharpness assessment model.

[0161] The sharpness assessment difference loss function is as follows:

[0162]

[0163] Where L represents the sharpness assessment difference loss between the sharpness prediction result and the sharpness assessment result for the input sample; N represents the total number of input samples; i represents the i-th input sample; j represents the j-th input sample; y i This represents the sharpness evaluation result for the i-th input sample; This represents the sharpness prediction result for the i-th input sample; y j This represents the sharpness evaluation result for the j-th input sample; Let l1 and l2 represent the sharpness prediction result for the j-th input sample; l1 and l2 both represent distances. τ represents the preset hyperparameter; ∑ represents summation; max(.) represents taking the maximum value; |.| represents taking the absolute value.

[0164] Based on the sharpness prediction result and the sharpness evaluation result of the input sample, the value of the sharpness evaluation difference loss function is calculated. Based on the value of the sharpness evaluation difference loss function, the parameters of the first image sharpness evaluation initial model are adjusted until the value of the sharpness evaluation difference loss function reaches a first preset condition, for example, the value of the sharpness evaluation difference loss function no longer decreases, or the value of the sharpness evaluation difference loss function is lower than the first preset loss value. Then, the adjustment of the parameters of the first image sharpness evaluation initial model is stopped, and the first image sharpness evaluation model is obtained.

[0165] Thus, the first image sharpness assessment model, after training, can evaluate the sharpness of a single image without reference, simulate the human eye's visual perception of sharpness, and output a sharpness prediction score for the image.

[0166] Since the first image sharpness evaluation model uses the EfficientNetB6 convolutional neural network model, which has a complex structure and a large number of parameters, it will cause huge time consumption when processing high-resolution images. On the other hand, directly using a lightweight model as the sharpness evaluation model will result in underfitting. Therefore, the method for obtaining the image sharpness evaluation model may further include:

[0167] The first image sharpness evaluation model is used as the teacher model, and the student model is obtained through knowledge distillation, which is used as the second image sharpness evaluation model for evaluating image sharpness.

[0168] Specifically, the trained first image sharpness evaluation model is used as the teacher model, and the parameters of the first image sharpness evaluation model are frozen.

[0169] Obtain the initial student model. Delete the convolutional blocks of each stage of EfficientNetB6, retaining only at least one block from each stage, while keeping the rest of the structure consistent with the original EfficientNetB6. This is used as the initial student model. Similarly, the term "initial student model" here refers to the student model that has not yet reached the required accuracy, that is, the student model before the knowledge distillation process is complete.

[0170] The input samples are input into the teacher model and the initial student model respectively. Based on the teacher model, the sharpness prediction result and the feature map of the teacher model are obtained. Based on the initial student model, the sharpness prediction result and the feature map of the initial student model are obtained.

[0171] A knowledge distillation loss function is obtained to evaluate the degree of difference between the sharpness prediction results of the teacher model and the sharpness prediction results of the initial student model, and a feature map difference loss function is obtained to evaluate the degree of difference between the feature maps of the teacher model and the feature maps of the initial student model. Based on the knowledge distillation loss function and the feature map difference loss function, the parameters of the initial student model are adjusted to obtain the student model.

[0172] The knowledge distillation loss function is as follows:

[0173]

[0174] Among them, L kd This represents the knowledge distillation loss between the clarity prediction results of the teacher model and the clarity prediction results of the initial student model. This represents the initial student model clarity prediction results; This represents the clarity prediction result of the teacher model; |.| represents taking the absolute value.

[0175] The feature map difference loss function is as follows:

[0176]

[0177] in, Let represent the difference loss for the i-th feature map; i represents the i-th feature map; n represents the total number of feature maps; This represents the feature map feature of the i-th initial student model; Let represent the feature map feature of the i-th teacher model; vec(.) denotes vectorization; ‖.‖2 denotes L2 normalization.

[0178] Based on the clarity prediction results of the teacher model and the clarity prediction results of the initial student model, the value of the knowledge distillation loss function is calculated, and based on the feature map of the teacher model and the feature map of the initial student model, the value of the feature map difference loss function is calculated.

[0179] The parameters of the initial student model are adjusted based on the sum of the values ​​of the knowledge distillation loss function and the feature map difference loss function until this sum reaches a second preset condition. For example, if the sum no longer decreases, or if it falls below the second preset loss value, the adjustment of the initial student model's parameters stops, and the student model is obtained. Clearly, the smaller the sum of the knowledge distillation loss function and the feature map difference loss function, the higher the accuracy of the student model and the better the knowledge distillation effect.

[0180] Thus, a lightweight second image sharpness evaluation model was obtained for evaluating image sharpness. This model can also evaluate the sharpness of a single image without reference, simulate the human eye's visual perception of sharpness, and output a sharpness prediction score for the image.

[0181] Second Embodiment

[0182] The second embodiment of this application provides an apparatus for obtaining an image sharpness evaluation model, such as... Figure 8 As shown. This device corresponds to the image sharpness evaluation model acquisition method provided in the first embodiment. Since the device embodiment is similar to the method embodiment, it is described simply here. For relevant details, please refer to the content of the first embodiment.

[0183] The image sharpness evaluation model acquisition device 800 provided in this embodiment includes:

[0184] The first obtaining unit 801 is used to obtain the sharpness evaluation result of the original image;

[0185] The second obtaining unit 802 is used to perform image degradation processing or image reconstruction processing on the original image based on the sharpness evaluation result of the original image to obtain the processed image;

[0186] The third obtaining unit 803 is used to obtain the sharpness evaluation result of the processed image;

[0187] The fourth obtaining unit 804 is used to obtain a first image sharpness evaluation model for evaluating image sharpness based on the original image and the sharpness evaluation result for the original image, as well as the processed image and the sharpness evaluation result for the processed image.

[0188] Third Embodiment

[0189] The third embodiment of this application provides an image sharpness evaluation method, such as... Figure 9 As shown, the method includes the following steps:

[0190] S901, obtain the image to be evaluated.

[0191] The image to be evaluated can be an image uploaded by the user, or an image obtained from an image library to meet the user's needs.

[0192] S902, obtain a first image sharpness evaluation model for evaluating image sharpness, wherein the first image sharpness evaluation model is a regression model.

[0193] The step of obtaining a first image sharpness evaluation model for evaluating image sharpness includes: obtaining a sharpness evaluation result for an original image; performing image degradation processing or image reconstruction processing on the original image based on the sharpness evaluation result for the original image to obtain a processed image; obtaining a sharpness evaluation result for the processed image; and obtaining the first image sharpness evaluation model based on the original image and the sharpness evaluation result for the original image, as well as the processed image and the sharpness evaluation result for the processed image.

[0194] For the method of obtaining the first image sharpness evaluation model, please refer to the first embodiment.

[0195] S903 obtains the sharpness prediction result for the image to be evaluated based on the first image sharpness evaluation model.

[0196] Specifically, the image to be evaluated can be input into the first image sharpness evaluation model to obtain a sharpness prediction result for the image to be evaluated.

[0197] Alternatively, the first image sharpness assessment model can be used as the teacher model, and a student model can be obtained through knowledge distillation, which serves as the second image sharpness assessment model for evaluating image sharpness. Then, the image to be evaluated is input into the second image sharpness assessment model to obtain a sharpness prediction result for the image to be evaluated.

[0198] For the method of obtaining the second image sharpness evaluation model, please refer to the first embodiment.

[0199] In the context of an online-to-offline service platform, for images uploaded by users, the aforementioned first or second image sharpness evaluation model can be used to assess image sharpness and return a sharpness prediction score to the user. Alternatively, for images whose sharpness prediction scores do not meet the requirements, a message indicating that the uploaded image does not meet the requirements can be returned to the user. Furthermore, for messages sent by users requesting images of a certain sharpness, the aforementioned first or second image sharpness evaluation model can be used to quickly filter images that meet the sharpness requirements from the image library and send them to the user. Here, "user" can refer to the user's terminal used to obtain services through the online-to-offline service platform, the user's terminal used to provide services through the online-to-offline service platform, or the user's terminal used by the platform's operators.

[0200] Fourth embodiment

[0201] The fourth embodiment of this application provides an image sharpness evaluation device, such as... Figure 10 As shown. This device corresponds to the image sharpness evaluation method provided in the third embodiment. Since the device embodiment is similar to the method embodiment, it is described simply here. For relevant details, please refer to the content of the third embodiment.

[0202] The image sharpness evaluation device 1000 provided in this embodiment includes:

[0203] The fifth acquisition unit 1001 is used to acquire the image to be evaluated;

[0204] The sixth obtaining unit 1002 is used to obtain a first image sharpness evaluation model for evaluating image sharpness, wherein the first image sharpness evaluation model is a regression model;

[0205] The seventh obtaining unit 1003 is used to obtain a sharpness prediction result for the image to be evaluated based on the first image sharpness evaluation model.

[0206] Fifth Embodiment

[0207] The fifth embodiment of this application provides a method for obtaining training samples, such as... Figure 11 As shown, the method includes the following steps:

[0208] S1101, Obtain the original image.

[0209] The original image can be an original image from an online-to-offline service platform scenario, such as an image from an image library.

[0210] S1102, Obtain the sharpness evaluation result for the original image.

[0211] The sharpness assessment result for the original image can be obtained entirely by manually annotating the sharpness score as the sharpness assessment result, or an image sharpness classification model can be used to assist in data annotation.

[0212] The first approach is to establish a correspondence between the image's sharpness category and the sharpness score range. Based on the image sharpness classification model, the sharpness category of the original image is obtained. Based on the sharpness score range corresponding to the sharpness category of the original image, a sharpness score for the original image is obtained, which serves as the sharpness evaluation result for the original image.

[0213] The second approach is to establish a correspondence between the probability value of the image's sharpness category and the sharpness score. Based on the image sharpness classification model, the probability value of the original image's sharpness category is obtained, and the sharpness score corresponding to the probability value of the original image's sharpness category is obtained, which serves as the sharpness evaluation result for the original image.

[0214] S1103, Based on the sharpness evaluation result of the original image, perform image degradation processing or image reconstruction processing on the original image to obtain the processed image.

[0215] Specifically, if the sharpness evaluation result of the original image indicates that the sharpness of the original image is higher than a first preset threshold, then the background part of the original image is subjected to image degradation processing to obtain a background degradation image, which is used as the processed image.

[0216] If the sharpness evaluation result of the original image indicates that the sharpness of the original image is lower than the first preset threshold and higher than the second preset threshold, then the original image is subjected to image degradation processing to obtain a full-image degradation image, which is used as the processed image.

[0217] If the sharpness evaluation result of the original image indicates that the sharpness of the original image is lower than the second preset threshold, then the original image is subjected to image reconstruction processing to obtain a full-image reconstructed image, which is used as the processed image.

[0218] The step of performing image reconstruction processing on the original image to obtain a full-image reconstructed image as the processed image includes: inputting the original image into an image reconstruction model to obtain a full-image reconstructed image; using an image sharpness classification discriminator to determine whether the sharpness category to which the full-image reconstructed image belongs is a sharp category, and using an image relative sharpness discriminator to determine whether the sharpness of the full-image reconstructed image is higher than the sharpness of the original image; if the sharpness category to which the full-image reconstructed image belongs is a sharp category, and the sharpness of the full-image reconstructed image is higher than the sharpness of the original image, then the full-image reconstructed image is used as the processed image.

[0219] S1104, Obtain the sharpness evaluation result for the processed image.

[0220] Specifically, a sharpness assessment result for the processed image is obtained based on the sharpness assessment result for the original image.

[0221] For an image processed using a degraded background image, the sharpness evaluation result indicates that the sharpness of the processed image is lower than that of the original image, and the difference between the sharpness of the processed image and the sharpness of the original image meets a preset range.

[0222] For images processed from a fully degraded image, the sharpness evaluation results indicate that the sharpness of the processed image is lower than that of the original image.

[0223] For the image reconstructed from the whole image as the processed image, the sharpness evaluation result indicates that the sharpness of the processed image is higher than that of the original image.

[0224] S1105, Based on the original image and the sharpness evaluation result for the original image, and the processed image and the sharpness evaluation result for the processed image, training samples for training the image sharpness evaluation model are obtained, wherein the image sharpness evaluation model is a model for evaluating image sharpness.

[0225] Specifically, the original image and the processed image are used as input samples, and the sharpness evaluation results for the original image and the processed image are used as sample labels. Based on the input samples and the sample labels, training samples for training the image sharpness evaluation model are obtained.

[0226] Because training samples are constructed from existing raw images, the labels are very sparse and imbalanced. Training a model to evaluate image sharpness using this data can easily lead the model to veer towards irrelevant categories during training. The image degradation and reconstruction processes described above, which expand the training sample data, can resolve the class imbalance problem.

[0227] For details on the method for obtaining training samples, please refer to the first embodiment.

[0228] Sixth Embodiment

[0229] The sixth embodiment of this application provides a training sample acquisition device, such as... Figure 12 As shown. This device corresponds to the training sample acquisition method provided in the fifth embodiment. Since the device embodiment is similar to the method embodiment, it is described simply here. For relevant details, please refer to the content of the fifth embodiment.

[0230] The training sample acquisition device 1200 provided in this embodiment includes:

[0231] The eighth acquisition unit 1201 is used to acquire the original image;

[0232] The ninth obtaining unit 1202 is used to obtain the sharpness evaluation result for the original image;

[0233] The tenth obtaining unit 1203 is used to perform image degradation processing or image reconstruction processing on the original image based on the sharpness evaluation result of the original image to obtain the processed image;

[0234] The eleventh obtaining unit 1204 is used to obtain a sharpness evaluation result for the processed image;

[0235] The twelfth obtaining unit 1205 is used to obtain training samples for training an image sharpness evaluation model based on the original image and the sharpness evaluation result for the original image, as well as the processed image and the sharpness evaluation result for the processed image. The image sharpness evaluation model is a model used to evaluate image sharpness.

[0236] Seventh Embodiment

[0237] The seventh embodiment of this application provides a method for obtaining an image sharpness evaluation model, such as... Figure 13 As shown, the method includes the following steps:

[0238] S1301, A first image sharpness evaluation model is obtained for evaluating image sharpness.

[0239] The first image sharpness assessment model can be a regression model, employing the EfficientNetB6 convolutional neural network model. For the specific structure and training method of the first image sharpness assessment model, please refer to the first embodiment.

[0240] S1302, the first image sharpness evaluation model is used as the teacher model, and a student model is obtained through knowledge distillation as the second image sharpness evaluation model for evaluating image sharpness. The student model is trained based on the degree of difference between the sharpness prediction results of the teacher model and the sharpness prediction results of the initial student model, as well as the degree of difference between the feature maps of the teacher model and the feature maps of the initial student model.

[0241] Specifically, we first obtain an initial student model. We remove the convolutional blocks from each stage of EfficientNetB6, keeping only at least one block from each stage, while maintaining the rest of the structure consistent with the original EfficientNetB6. This is used as the initial student model.

[0242] Based on the teacher model, obtain the teacher model sharpness prediction result and the teacher model feature map; based on the initial student model, obtain the initial student model sharpness prediction result and the initial student model feature map. Obtain a knowledge distillation loss function to evaluate the degree of difference between the teacher model sharpness prediction result and the initial student model sharpness prediction result; obtain a feature map difference loss function to evaluate the degree of difference between the teacher model feature map and the initial student model feature map.

[0243] The parameters of the initial student model are adjusted based on the knowledge distillation loss function and the feature map difference loss function to obtain the student model. Specifically, the value of the knowledge distillation loss function is calculated based on the clarity prediction results of the teacher model and the initial student model, and the value of the feature map difference loss function is calculated based on the feature maps of the teacher model and the initial student model. The parameters of the initial student model are adjusted based on the sum of the values ​​of the knowledge distillation loss function and the feature map difference loss function until the sum of the values ​​of the knowledge distillation loss function and the feature map difference loss function reaches a second preset condition, at which point the adjustment of the parameters of the initial student model stops, and the student model is obtained.

[0244] For details on how to obtain the second image sharpness evaluation model, please refer to the first embodiment.

[0245] Eighth embodiment

[0246] The eighth embodiment of this application provides an apparatus for obtaining an image sharpness evaluation model, such as... Figure 14As shown. This device corresponds to the resource allocation method provided in the seventh embodiment. Since the device embodiment is similar to the method embodiment, it is described simply here. For relevant details, please refer to the content of the seventh embodiment.

[0247] The image sharpness evaluation model acquisition device 1400 provided in this embodiment includes:

[0248] The thirteenth obtaining unit 1401 is used to obtain a first image sharpness evaluation model for evaluating image sharpness;

[0249] The fourteenth obtaining unit 1402 is used to obtain a student model by knowledge distillation using the first image sharpness evaluation model as a teacher model, and to use it as a second image sharpness evaluation model for evaluating image sharpness. The student model is trained based on the degree of difference between the sharpness prediction results of the teacher model and the sharpness prediction results of the initial student model, as well as the degree of difference between the feature maps of the teacher model and the feature maps of the initial student model.

[0250] Ninth Embodiment

[0251] The ninth embodiment of this application provides an electronic device, such as... Figure 15 As shown, the electronic device includes at least one processor 1501, at least one memory 1502, at least one communication interface 1503, and at least one communication bus 1504. Optionally, the processor 1501 may be a CPU, an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application. The memory 1502 may include high-speed RAM and may also include non-volatile memory, such as at least one disk storage device. The communication interface 1503 may be an interface for a communication module, such as an interface for a GSM module. The memory 1502 stores programs and data, and the processor 1501 calls the programs stored in the memory 1502 to execute the above-described image sharpness evaluation model acquisition method, or the above-described image sharpness evaluation method, or the above-described training sample acquisition method, or the above-described image sharpness evaluation model acquisition method.

[0252] Tenth Embodiment

[0253] The tenth embodiment of this application provides a storage medium storing a program and data. The program is executed by a processor to implement the above-described image sharpness evaluation model acquisition method, or the above-described image sharpness evaluation method, or the above-described training sample acquisition method, or the above-described image sharpness evaluation model acquisition method.

[0254] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0255] Although several modules or units for action execution have been mentioned in the detailed description above, this division is not mandatory. In fact, according to specific embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0256] Furthermore, although the steps of the method in this application are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0257] It should be noted that the embodiments of this application can be implemented in hardware, software, or a combination of both. The hardware portion can be implemented using dedicated logic; the software portion can be stored in memory and executed by a suitable instruction execution system, such as a microprocessor or dedicated-design hardware. Those skilled in the art will understand that the above-described devices and methods can be implemented using computer-executable instructions and / or included in processor control code, for example, such code provided on a carrier medium such as a disk, CD, or DVD-ROM, a programmable memory such as read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and modules of this application can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field-programmable gate arrays, programmable logic devices, etc., or by software executed by various types of processors, or by a combination of the above-described hardware circuits and software, such as firmware.

[0258] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the scope of the technology disclosed in this application, and within the spirit and principles of this application, should be included within the scope of protection of this application.

Claims

1. A method for obtaining an image sharpness assessment model, characterized in that, include: Obtain the sharpness assessment results for the original image; Based on the sharpness assessment result of the original image, image degradation processing or image reconstruction processing is performed on the original image to obtain a processed image, including: if the sharpness assessment result of the original image indicates that the sharpness of the original image is higher than a first preset threshold, then image degradation processing is performed on the background portion of the original image to obtain a background-degraded image, which is used as the processed image; if the sharpness assessment result of the original image indicates that the sharpness of the original image is lower than the first preset threshold but higher than a second preset threshold, then image degradation processing is performed on the original image to obtain a full-image-degraded image, which is used as the processed image; if the sharpness assessment result of the original image indicates that the sharpness of the original image is lower than the second preset threshold, then image reconstruction processing is performed on the original image to obtain a full-image-reconstructed image, which is used as the processed image. Obtaining a sharpness evaluation result for the processed image includes: when the processed image is a background degradation image, obtaining a sharpness evaluation result for the processed image based on the sharpness evaluation result for the original image, wherein the sharpness evaluation result for the processed image indicates that the sharpness of the processed image is lower than that of the original image, and the difference between the sharpness of the processed image and the sharpness of the original image meets a preset range; when the processed image is a full-image degradation image, obtaining a sharpness evaluation result for the processed image based on the sharpness evaluation result for the original image, wherein the sharpness evaluation result for the processed image indicates that the sharpness of the processed image is lower than that of the original image; when the processed image is a full-image reconstruction image, obtaining a sharpness evaluation result for the processed image based on the sharpness evaluation result for the original image, wherein the sharpness evaluation result for the processed image indicates that the sharpness of the processed image is higher than that of the original image; Based on the original image and the sharpness evaluation result for the original image, as well as the processed image and the sharpness evaluation result for the processed image, a first image sharpness evaluation model for evaluating image sharpness is obtained.

2. The method for obtaining the image sharpness evaluation model according to claim 1, characterized in that, Obtaining the sharpness assessment result for the original image includes: Establish the correspondence between the image's sharpness category and the sharpness score range; Based on the image sharpness classification model, the sharpness category to which the original image belongs is obtained; Based on the sharpness score range corresponding to the sharpness category to which the original image belongs, a sharpness score is obtained for the original image, which serves as the sharpness evaluation result for the original image.

3. The method for obtaining the image sharpness evaluation model according to claim 1, characterized in that, Obtaining the sharpness assessment result for the original image includes: Establish the correspondence between the probability value of the image's sharpness category and the sharpness score; Based on the image sharpness classification model, obtain the probability value of the sharpness category to which the original image belongs; Obtain the sharpness score corresponding to the probability value of the sharpness category to which the original image belongs, and use it as the sharpness evaluation result for the original image.

4. The method for obtaining the image sharpness evaluation model according to claim 1, characterized in that, The step of performing image reconstruction processing on the original image to obtain a fully reconstructed image, as the processed image, includes: The original image is input into the image reconstruction model to obtain the full-image reconstruction image; The image sharpness classification discriminator determines whether the sharpness category of the reconstructed full-image is sharp, and the image relative sharpness discriminator determines whether the sharpness of the reconstructed full-image is higher than that of the original image. If the sharpness category of the reconstructed full-image is sharp, and the sharpness of the reconstructed full-image is higher than that of the original image, then the reconstructed full-image is used as the processed image.

5. The method for obtaining the image sharpness evaluation model according to claim 1, characterized in that, The step of obtaining a first image sharpness assessment model for evaluating image sharpness based on the original image and the sharpness assessment result for the original image, and the processed image and the sharpness assessment result for the processed image, includes: A first image sharpness assessment initial model is obtained, wherein the first image sharpness assessment initial model is a regression model; The original image and the processed image are used as input samples and input into the first image sharpness evaluation initial model to obtain the sharpness prediction result for the input samples. Based on the sharpness prediction results and sharpness evaluation results for the input sample, the parameters of the first image sharpness evaluation initial model are adjusted to obtain the first image sharpness evaluation model.

6. The method for obtaining the image sharpness evaluation model according to claim 5, characterized in that, The step of adjusting the parameters of the first image sharpness assessment initial model based on the sharpness prediction result and the sharpness assessment result of the input sample to obtain the first image sharpness assessment model includes: Obtain a sharpness assessment difference loss function to evaluate the degree of difference between the sharpness prediction result and the sharpness assessment result for the input sample; Based on the sharpness assessment difference loss function, the parameters of the first image sharpness assessment initial model are adjusted to obtain the first image sharpness assessment model.

7. The method for obtaining the image sharpness evaluation model according to claim 6, characterized in that, The step of adjusting the parameters of the first image sharpness assessment initial model according to the sharpness assessment difference loss function to obtain the first image sharpness assessment model includes: The value of the sharpness assessment difference loss function is calculated based on the sharpness prediction result and the sharpness assessment result of the input sample. Based on the value of the sharpness assessment difference loss function, adjust the parameters of the first image sharpness assessment initial model; The parameters of the first image sharpness assessment initial model are adjusted until the value of the sharpness assessment difference loss function reaches the first preset condition, thus obtaining the first image sharpness assessment model.

8. The method for obtaining the image sharpness evaluation model according to claim 6, characterized in that, The sharpness assessment difference loss function is as follows: in, This represents the loss due to the difference in sharpness assessment between the sharpness prediction result and the sharpness assessment result for the input sample. Indicates the total number of input samples; Indicates the first One input sample; Indicates the first One input sample; Indicates that for the first Clarity assessment results for each input sample; Indicates that for the first Clarity prediction results for each input sample; Indicates that for the first Clarity assessment results for each input sample; Indicates that for the first Clarity prediction results for each input sample; , Both represent distance. , , Indicates the preset hyperparameters; To express summation; This indicates taking the maximum value; This indicates taking the absolute value.

9. The method for obtaining the image sharpness evaluation model according to claim 1, characterized in that, Also includes: The first image sharpness evaluation model is used as the teacher model, and the student model is obtained through knowledge distillation, which is used as the second image sharpness evaluation model for evaluating image sharpness.

10. The method for obtaining the image sharpness evaluation model according to claim 9, characterized in that, The step of using the first image sharpness assessment model as the teacher model and obtaining the student model through knowledge distillation includes: Obtain the initial student model; Based on the teacher model, the clarity prediction result and the feature map of the teacher model are obtained; Based on the initial student model, the sharpness prediction result and feature map of the initial student model are obtained; Obtain a knowledge distillation loss function to evaluate the degree of difference between the clarity prediction results of the teacher model and the clarity prediction results of the initial student model; Obtain a feature map difference loss function to evaluate the degree of difference between the teacher model feature map and the initial student model feature map; Based on the knowledge distillation loss function and the feature map difference loss function, the parameters of the initial student model are adjusted to obtain the student model.

11. The method for obtaining the image sharpness evaluation model according to claim 10, characterized in that, The step of adjusting the parameters of the initial student model based on the knowledge distillation loss function and the feature map difference loss function to obtain the student model includes: The value of the knowledge distillation loss function is calculated based on the clarity prediction results of the teacher model and the clarity prediction results of the initial student model. Calculate the value of the feature map difference loss function based on the teacher model feature map and the initial student model feature map; The parameters of the initial student model are adjusted based on the sum of the values ​​of the knowledge distillation loss function and the feature map difference loss function. The adjustment of the parameters of the initial student model is stopped until the sum of the values ​​of the knowledge distillation loss function and the feature map difference loss function reaches a second preset condition, thus obtaining the student model.

12. The method for obtaining the image sharpness evaluation model according to claim 10, characterized in that, The knowledge distillation loss function is as follows: in, This represents the knowledge distillation loss between the clarity prediction results of the teacher model and the clarity prediction results of the initial student model. This represents the initial student model clarity prediction results; This indicates the clarity prediction results of the teacher model; This indicates taking the absolute value.

13. The method for obtaining the image sharpness evaluation model according to claim 10, characterized in that, The feature map difference loss function is as follows: in, Indicates the first Each feature map difference loss; Indicates the first Each feature map; Indicates the total number of feature maps; Indicates the first Features of the initial student model feature map; Indicates the first Features of a teacher model feature map; Vectorization; This indicates L2 standardization.

14. An apparatus for obtaining an image sharpness evaluation model, characterized in that, include: The first obtaining unit is used to obtain the sharpness assessment result for the original image; The second obtaining unit is configured to perform image degradation processing or image reconstruction processing on the original image based on the sharpness evaluation result of the original image to obtain a processed image, including: if the sharpness evaluation result of the original image indicates that the sharpness of the original image is higher than a first preset threshold, then perform image degradation processing on the background portion of the original image to obtain a background degradation image, which is used as the processed image; if the sharpness evaluation result of the original image indicates that the sharpness of the original image is lower than the first preset threshold but higher than a second preset threshold, then perform image degradation processing on the original image to obtain a full-image degradation image, which is used as the processed image; if the sharpness evaluation result of the original image indicates that the sharpness of the original image is lower than the second preset threshold, then perform image reconstruction processing on the original image to obtain a full-image reconstructed image, which is used as the processed image. The third obtaining unit is configured to obtain a sharpness evaluation result for the processed image, including: when the processed image is the background degradation image, obtaining a sharpness evaluation result for the processed image based on the sharpness evaluation result for the original image, wherein the sharpness evaluation result for the processed image indicates that the sharpness of the processed image is lower than that of the original image, and the difference between the sharpness of the processed image and the sharpness of the original image meets a preset range; when the processed image is the full-image degradation image, obtaining a sharpness evaluation result for the processed image based on the sharpness evaluation result for the original image, wherein the sharpness evaluation result for the processed image indicates that the sharpness of the processed image is lower than that of the original image; when the processed image is the full-image reconstruction image, obtaining a sharpness evaluation result for the processed image based on the sharpness evaluation result for the original image, wherein the sharpness evaluation result for the processed image indicates that the sharpness of the processed image is higher than that of the original image; The fourth obtaining unit is used to obtain a first image sharpness evaluation model for evaluating image sharpness based on the original image and the sharpness evaluation result for the original image, as well as the processed image and the sharpness evaluation result for the processed image.

15. A method for evaluating image sharpness, characterized in that, include: Obtain the image to be evaluated; A first image sharpness evaluation model is obtained for evaluating image sharpness, wherein the first image sharpness evaluation model is a model obtained by the method of any one of claims 1 to 13, and the first image sharpness evaluation model is a regression model; Based on the first image sharpness assessment model, a sharpness prediction result is obtained for the image to be evaluated.

16. The image sharpness evaluation method according to claim 15, characterized in that, The step of obtaining a sharpness prediction result for the image to be evaluated based on the first image sharpness evaluation model includes: The image to be evaluated is input into the first image sharpness evaluation model to obtain a sharpness prediction result for the image to be evaluated.

17. The image sharpness evaluation method according to claim 15, characterized in that, The step of obtaining a sharpness prediction result for the image to be evaluated based on the first image sharpness evaluation model includes: The first image sharpness evaluation model is used as the teacher model, and the student model is obtained through knowledge distillation, which is used as the second image sharpness evaluation model for evaluating image sharpness. The image to be evaluated is input into the second image sharpness evaluation model to obtain a sharpness prediction result for the image to be evaluated.

18. An image sharpness evaluation device, characterized in that, include: The fifth acquisition unit is used to acquire the image to be evaluated; The sixth obtaining unit is used to obtain a first image sharpness evaluation model for evaluating image sharpness, wherein the first image sharpness evaluation model is a model obtained by the method described in any one of claims 1 to 13, and the first image sharpness evaluation model is a regression model; The seventh obtaining unit is used to obtain a sharpness prediction result for the image to be evaluated based on the first image sharpness evaluation model.

19. A method for obtaining training samples, characterized in that, include: Obtain the original image; Obtain the sharpness assessment result for the original image; Based on the sharpness assessment result of the original image, image degradation processing or image reconstruction processing is performed on the original image to obtain a processed image, including: if the sharpness assessment result of the original image indicates that the sharpness of the original image is higher than a first preset threshold, then image degradation processing is performed on the background portion of the original image to obtain a background-degraded image, which is used as the processed image; if the sharpness assessment result of the original image indicates that the sharpness of the original image is lower than the first preset threshold but higher than a second preset threshold, then image degradation processing is performed on the original image to obtain a full-image-degraded image, which is used as the processed image; if the sharpness assessment result of the original image indicates that the sharpness of the original image is lower than the second preset threshold, then image reconstruction processing is performed on the original image to obtain a full-image-reconstructed image, which is used as the processed image. Obtaining a sharpness evaluation result for the processed image includes: when the processed image is a background degradation image, obtaining a sharpness evaluation result for the processed image based on the sharpness evaluation result for the original image, wherein the sharpness evaluation result for the processed image indicates that the sharpness of the processed image is lower than that of the original image, and the difference between the sharpness of the processed image and the sharpness of the original image meets a preset range; when the processed image is a full-image degradation image, obtaining a sharpness evaluation result for the processed image based on the sharpness evaluation result for the original image, wherein the sharpness evaluation result for the processed image indicates that the sharpness of the processed image is lower than that of the original image; when the processed image is a full-image reconstruction image, obtaining a sharpness evaluation result for the processed image based on the sharpness evaluation result for the original image, wherein the sharpness evaluation result for the processed image indicates that the sharpness of the processed image is higher than that of the original image; Based on the original image and the sharpness evaluation result for the original image, as well as the processed image and the sharpness evaluation result for the processed image, training samples are obtained for training the image sharpness evaluation model, which is a model for evaluating image sharpness.

20. The training sample acquisition method according to claim 19, characterized in that, Obtaining the sharpness assessment result for the original image includes: Establish the correspondence between the image's sharpness category and the sharpness score range; Based on the image sharpness classification model, the sharpness category to which the original image belongs is obtained; Based on the sharpness score range corresponding to the sharpness category to which the original image belongs, a sharpness score is obtained for the original image, which serves as the sharpness evaluation result for the original image.

21. The training sample acquisition method according to claim 19, characterized in that, Obtaining the sharpness assessment result for the original image includes: Establish the correspondence between the probability value of the image's sharpness category and the sharpness score; Based on the image sharpness classification model, obtain the probability value of the sharpness category to which the original image belongs; Obtain the sharpness score corresponding to the probability value of the sharpness category to which the original image belongs, and use it as the sharpness evaluation result for the original image.

22. The training sample acquisition method according to claim 19, characterized in that, The step of performing image reconstruction processing on the original image to obtain a fully reconstructed image, as the processed image, includes: The original image is input into the image reconstruction model to obtain the full-image reconstruction image; The image sharpness classification discriminator determines whether the sharpness category of the reconstructed full-image is sharp, and the image relative sharpness discriminator determines whether the sharpness of the reconstructed full-image is higher than that of the original image. If the sharpness category of the reconstructed full-image is sharp, and the sharpness of the reconstructed full-image is higher than that of the original image, then the reconstructed full-image is used as the processed image.

23. The training sample acquisition method according to claim 19, characterized in that, The step of obtaining training samples for training the image sharpness assessment model based on the original image and the sharpness assessment result for the original image, and the processed image and the sharpness assessment result for the processed image, includes: The original image and the processed image are used as input samples, and the sharpness evaluation results for the original image and the processed image are used as sample labels. Based on the input samples and the sample labels, training samples for training the image sharpness evaluation model are obtained.

24. A training sample acquisition device, characterized in that, include: The eighth acquisition unit is used to acquire the original image; The ninth obtaining unit is used to obtain the sharpness evaluation result for the original image; The tenth obtaining unit is configured to perform image degradation processing or image reconstruction processing on the original image based on the sharpness evaluation result of the original image to obtain a processed image, including: if the sharpness evaluation result of the original image indicates that the sharpness of the original image is higher than a first preset threshold, then perform image degradation processing on the background portion of the original image to obtain a background degradation image, which is used as the processed image; if the sharpness evaluation result of the original image indicates that the sharpness of the original image is lower than the first preset threshold but higher than a second preset threshold, then perform image degradation processing on the original image to obtain a full-image degradation image, which is used as the processed image; if the sharpness evaluation result of the original image indicates that the sharpness of the original image is lower than the second preset threshold, then perform image reconstruction processing on the original image to obtain a full-image reconstructed image, which is used as the processed image. The eleventh obtaining unit is used to obtain a sharpness evaluation result for the processed image, including: when the processed image is the background degradation image, obtaining a sharpness evaluation result for the processed image based on the sharpness evaluation result for the original image, wherein the sharpness evaluation result for the processed image indicates that the sharpness of the processed image is lower than that of the original image, and the difference between the sharpness of the processed image and the sharpness of the original image meets a preset range; when the processed image is the full-image degradation image, obtaining a sharpness evaluation result for the processed image based on the sharpness evaluation result for the original image, wherein the sharpness evaluation result for the processed image indicates that the sharpness of the processed image is lower than that of the original image; when the processed image is the full-image reconstruction image, obtaining a sharpness evaluation result for the processed image based on the sharpness evaluation result for the original image, wherein the sharpness evaluation result for the processed image indicates that the sharpness of the processed image is higher than that of the original image; The twelfth obtaining unit is used to obtain training samples for training an image sharpness evaluation model based on the original image and the sharpness evaluation result for the original image, as well as the processed image and the sharpness evaluation result for the processed image. The image sharpness evaluation model is a model used to evaluate image sharpness.

25. An electronic device, characterized in that, Including processor and memory; The memory is used to store programs and data. The processor calls the programs stored in the memory to execute the image sharpness evaluation model acquisition method according to any one of claims 1-13, or the image sharpness evaluation method according to any one of claims 15-17, or the training sample acquisition method according to any one of claims 19-23.

26. A storage medium, characterized in that, The storage medium stores a program and data, the program being executed by a processor to implement the image sharpness evaluation model acquisition method according to any one of claims 1-13, or the image sharpness evaluation method according to any one of claims 15-17, or the training sample acquisition method according to any one of claims 19-23.

Images