Image quality assessment method, device, terminal and storage medium
Through the comparison-based method, the reference image quality evaluation standard changes are solved, and the problem of low accuracy and flexibility in image quality evaluation in the prior art is solved, and higher evaluation accuracy and flexibility are achieved .
Patent Information
- Application Number
- CN202010902856.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-08-31
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2040-08-31
AI Technical Summary
In the prior art, the accuracy and flexibility of image quality evaluation are low, mainly because there is subjective deviation in manual annotation and the need to retrain the model when the evaluation criteria are changed.
Using a comparison-based method, by obtaining the target image and reference image set, using the training-completed comparison model to perform image quality comparison, determine the image quality level of the target image based on the comparison results, and provide reference image adaptation evaluation criteria. Change, avoid retraining the model.
It improves the accuracy and flexibility of image quality evaluation, reduces the impact of subjective deviations of labeling personnel and changes in evaluation criteria, and enhances the model's ability to distinguish image quality.
Smart Images

Figure CN114118191B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to an image quality assessment method, device, terminal and storage medium. Background Art
[0002] Image quality assessment is one of the basic technologies in image processing. It mainly analyzes the characteristics of images and then evaluates the quality of images. By evaluating the image quality, we can distinguish the quality of images and then perform corresponding operations based on the quality of the images. For example, in the process of pushing images, we can evaluate the quality of each image and then push images with better quality and filter out images with poorer quality.
[0003] Currently, using deep learning models to evaluate image quality is one of the mainstream methods, which specifically includes using a large number of sample images with labeled image quality levels to train the model so that the model can learn the image quality characteristics of images at different levels. However, due to the subjectivity of image quality comparison, different people may have deviations in labeling image quality levels, resulting in incorrect labeling results. When the level evaluation standard changes, the model needs to be retrained to evaluate the image quality of the changed images. Therefore, the accuracy and flexibility of image quality evaluation are low. Summary of the invention
[0004] The embodiments of the present invention provide an image quality assessment method, device, terminal and storage medium, which can assess the image quality based on comparison, thereby improving the accuracy and flexibility of image quality assessment.
[0005] In one aspect, an embodiment of the present invention provides an image quality assessment method, the method comprising:
[0006] Acquire a target image and a reference image set, wherein the reference image set includes at least one reference image;
[0007] Calling the trained comparison model to perform image quality comparison between the target image and each reference image in the reference image set, and obtaining at least one image quality comparison result, wherein the image quality comparison result includes a comparison result between the image quality of the target image and the image quality of the reference images in the reference image set;
[0008] Evaluate the image quality of the target image according to the at least one image quality comparison result to obtain at least one classification result, each of the classification results including the image quality level to which the target image belongs;
[0009] A target image quality level with the largest frequency is determined from at least one image quality level to which the target image belongs as a target classification result corresponding to the target image.
[0010] In one aspect, an embodiment of the present invention provides an image quality assessment device, the device comprising:
[0011] An acquisition module, used to acquire a target image and a reference image set, wherein the reference image set includes at least one reference image;
[0012] A calling module, used for calling the trained comparison model to compare the image quality of the target image with each reference image in the reference image set, and obtaining at least one image quality comparison result, wherein the image quality comparison result includes a comparison result between the image quality of the target image and the image quality of the reference images in the reference image set;
[0013] A classification module, configured to evaluate the image quality of the target image according to the at least one image quality comparison result to obtain at least one classification result, each of which includes the image quality level to which the target image belongs;
[0014] The determination module is used to determine the target image quality level with the largest frequency from at least one image quality level to which the target image belongs, as the target classification result corresponding to the target image.
[0015] On the one hand, an embodiment of the present invention provides a terminal, including a processor, an input interface, an output interface and a memory, wherein the processor, input interface, output interface and memory are interconnected, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to execute the image quality assessment method.
[0016] On the one hand, an embodiment of the present invention provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program, the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes the image quality assessment method.
[0017] In an embodiment of the present invention, a terminal obtains a target image and a reference image set, and calls a trained comparison model to perform image quality comparison between the target image and each reference image in the reference image set, and obtains at least one image quality comparison result. The terminal evaluates the image quality of the target image according to the at least one image quality comparison result, and obtains at least one classification result. Each classification result includes the image quality level to which the target image belongs. The terminal determines the target image quality level with the largest frequency from the at least one image quality level to which the target image belongs, as the target classification result corresponding to the target image, and evaluates the image quality in a comparison-based manner to improve the accuracy and flexibility of the image quality evaluation. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.
[0019] Figure 1 It is a schematic diagram of a process for evaluating image quality based on a deep neural network provided by an embodiment of the present invention;
[0020] Figure 2 is a flow chart of an image quality assessment method provided by an embodiment of the present invention;
[0021] Figure 3 is a flow chart of another image quality assessment method provided by an embodiment of the present invention;
[0022] Figure 4 It is a flowchart of a training method for a comparison model provided by an embodiment of the present invention;
[0023] Figure 5 is a flow chart of another image quality assessment method provided by an embodiment of the present invention;
[0024] Figure 6 is a schematic diagram of a display interface provided by an embodiment of the present invention;
[0025] Figure 7 is a structural schematic diagram of an image quality assessment device provided by an embodiment of the present invention;
[0026] Figure 8 It is a schematic diagram of the structure of a terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0027] Image quality assessment based on deep neural networks is one of the common ways of image quality assessment. By evaluating the image quality through a deep neural network model, the image quality level to which the image belongs can be determined, and the image can be classified based on the image quality. The image quality of images at different quality levels is different. For example, the image quality level is divided into the first level, the second level and the third level. The image quality standard is that the highest quality is 100 and the lowest quality is 0. The first level can correspond to a quality of 60-100, the second level corresponds to a quality of 40-60, and the third level corresponds to a quality less than 0-40. The image quality can be specifically judged by indicators such as clarity and resolution. At present, the specific steps for image quality assessment based on deep neural networks are as follows: ① Manually label the quality level of sample images; ② Train the deep learning model based on the labeled sample images to obtain a trained model, which has the ability to distinguish the quality level of the image; ③ Evaluate the image quality based on the trained model to obtain the quality level of the image. Figure 1 As shown, currently, images of different gears can be input to train the initial classification model to obtain a trained classification model, and then images of unknown gears can be input into the trained classification model to obtain the image quality gear to which the image belongs.
[0028] However, the above method has the following defects: ① In the process of manually labeling the image quality levels, since the image quality judgment is relatively subjective, it is easy to produce erroneous labeling results due to subjective factors, which affects the model learning and makes the model evaluation accuracy low; ② The model directly outputs the image quality level to which the image belongs. When the evaluation standard of the image quality level changes, the model needs to be retrained to make the output image quality evaluation result meet the changed evaluation standard. For example, the initial first level corresponds to 60-100 image quality. After the evaluation standard is changed, the first level corresponds to 50-100 image quality. After the standard is changed, the model needs to be retrained for all images so that the model can correctly input the image quality level to which the image belongs. The evaluation flexibility of the model is low.
[0029] Based on this, the present invention proposes an image quality assessment method to solve the problems of low assessment accuracy and flexibility in the image quality assessment process. The image quality assessment method specifically applies an artificial intelligence solution. Artificial intelligence is a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that the machines have the functions of perception, reasoning and decision-making. In the image quality assessment scheme proposed in the embodiment of the present invention, unlike the current image quality assessment scheme, in the sample image annotation stage, this scheme does not directly annotate the image quality level, but annotates the image quality comparison result between the sample image and the sample reference image, such as the image quality of the sample image is higher than the sample reference image, lower than the sample reference image, or the same as the sample reference image, wherein the sample reference image can be an image of a certain quality selected in advance by humans. By providing the sample reference image, the annotator can annotate the image based on the image quality comparison between the sample reference image and the sample image, which can greatly reduce the annotation result error caused by the subjective fluctuation of the annotator, and stabilize the annotation results of different annotators as much as possible. In addition, in the model training process, this scheme also trains the model based on the combination of the sample image and the sample reference image, so that the model can correctly distinguish the image quality of the input image and the sample reference image. In the process of using the model, this scheme determines the image quality level of the target image based on the image quality difference between the target image and the reference image. By adopting the above method, when the evaluation standard of the image quality level changes, it is only necessary to change the image quality of the reference image compared with the target image, so that the model can correctly output the image quality level of the target image. For example, the initial first level corresponds to the image quality of 60-100, and the image quality of the initial reference image is 60. When the model determines that the image quality of the target image is higher than that of the initial reference image, it can be determined that the target image belongs to the first level. When the evaluation standard is changed to the first level corresponding to the image quality of 50-100, a reference image with an image quality of 50 can be selected. When the model determines that the image quality of the target image is higher than that of the changed reference image, it can be determined that the target image belongs to the first level without retraining the model, thereby improving the flexibility of image quality evaluation.
[0030] Specifically, the specific process of the image quality evaluation method provided by this solution is as follows: Figure 2As shown, it specifically includes: ① model pre-training, specifically, performing noise processing on multiple first sample images, obtaining multiple second sample images, using the first sample images and the corresponding second sample images to train the initial comparison model, adjusting some parameters in the initial comparison model, so that the initial comparison model outputs the result that the image quality of the first sample image is higher than that of the second sample image, and obtaining a basic comparison model, which has a basic ability to distinguish high and low image quality. ② model training and tuning, specifically, obtaining a training image set, and iteratively training the basic comparison model based on the image combination in the training image set, optimizing the parameters in the basic comparison model, so that the basic comparison model's ability to distinguish high and low image quality is enhanced, and a comparison model that has been trained is obtained. Among them, the training image set includes multiple image combinations, each image combination includes a sample image and a sample reference image, and each sample image in the sample image contains annotation information, and the annotation information indicates a preset comparison result between the image quality of the sample image and the image quality of the sample reference image. ③ Based on the trained comparison model, the image quality comparison results of the target image and multiple reference images are obtained, and the image quality level of the target image is determined based on each image quality comparison result, and the final target image quality level is screened out from the image quality levels of the target image based on preset rules.
[0031] In the above scheme, by providing a sample reference image, the annotator can annotate the image based on the image quality comparison between the sample reference image and the sample image, thereby improving the accuracy of the annotator's image annotation. By pre-training and training optimization of the model, the model can distinguish the high and low relationship between image quality. When the model judges the image quality level of the target image, it determines the image quality level to which the target image ultimately belongs based on multiple comparison results, thereby improving the accuracy of the image quality level evaluation.
[0032] Based on the above description, an embodiment of the present invention provides an image quality assessment method. Figure 3 , the image quality evaluation process may include the following steps S301-S304:
[0033] S301: The terminal obtains a target image and a reference image set.
[0034] In an embodiment of the present invention, the reference image set includes at least one reference image, each reference image can be pre-defined by a user and used for comparison with a target image to determine a quality classification of the target image. The terminal can obtain the user-defined reference image and the target image, wherein the type of content included in each reference image can be different, such as there are 6 reference images, two of which include human bodies, two include scenes, and two include objects. The quality of the reference images is known, and the quality of each reference image can be the same, such as the quality of each reference image is 60, and the corresponding quality level is the second level, or the quality of the reference images can be different, such as the quality of three reference images is 40, and the corresponding quality level is the third level, and the quality of three reference images is 40, and the corresponding quality level is the second level, wherein the quality of the image with the highest quality among all images is pre-defined as 100, and the quality of the image with the lowest quality is 0.
[0035] S302: The terminal calls the trained comparison model to compare the target image with each reference image in the reference image set to obtain at least one image quality comparison result.
[0036] In an embodiment of the present invention, after the terminal obtains the target image and the reference image set, it can call the trained comparison model to perform image quality comparison between the target image and each reference image in the reference image set to obtain at least one image quality comparison result, and the image quality comparison result includes the comparison result between the image quality of the target image and the image quality of the reference images in the reference image set, such as the image quality of the target image is higher than that of the reference image, the image quality of the target image is the same as that of the reference image, or the image quality of the target image is lower than that of the reference image.
[0037] In one implementation, the trained comparison model includes two input interfaces. The terminal can input the target image and a reference image in the reference image set into the trained comparison model through the two input interfaces in one comparison. The trained comparison model outputs the image quality comparison result for the target image and the reference image. After obtaining one image quality comparison result, the terminal again inputs the target image and another reference image in the reference image set into the trained comparison model through the two input interfaces to obtain another image quality comparison result. Based on the above method, the terminal can obtain the image quality comparison result of the target image and each reference image in the reference image set, that is, at least one image quality comparison result.
[0038] In one implementation, the trained comparison model includes multiple input interfaces. The terminal can input the target image and each reference image in the reference image set into the trained comparison model through multiple input interfaces at one time in one comparison. The comparison model inputs the image quality comparison results between the target image and each reference image to obtain at least one image quality comparison result.
[0039] It should be noted that the trained comparison model has the function of comparing the image quality between images. For the specific training process, see Figure 4 The embodiment shown.
[0040] S303: The terminal evaluates the image quality of the target image according to at least one image quality comparison result to obtain at least one classification result.
[0041] In the embodiment of the present invention, after obtaining at least one image quality comparison result, the terminal can evaluate the image quality of the target image according to the at least one image quality comparison result to obtain at least one classification result, each classification result including the image quality level to which the target image belongs.
[0042] In one implementation, one image quality comparison result corresponds to one classification result, the image quality level of each reference image in the reference image set is the second level, and the image quality comparison result includes that the image quality of the target image is higher than the reference image, the image quality of the target image is the same as the reference image, or the image quality of the target image is lower than the reference image. Then, the specific process of the terminal evaluating the image quality of the target image according to at least one image quality comparison result is as follows: if the image quality of the target image is higher than the reference image, the image quality level of the target image is determined to be the first level; if the image quality of the target image is the same as the reference image, the image quality level of the target image is determined to be the second level; if the image quality of the target image is lower than the reference image, the image quality level of the target image is determined to be the first level. There are three gears, wherein the image quality of the first gear is higher than that of the second gear, and the image quality of the second gear is higher than that of the first gear. Through the above method, a classification result can be obtained based on each image quality comparison result. For example, the reference image set includes reference image 1, reference image 2 and reference image 3, and the comparison result 1 indicates that the image quality of the target image is higher than that of reference image 1, then the classification result 1 is determined to be the first gear, and the comparison result 2 indicates that the image quality of the target image is higher than that of reference image 2, then the classification result 2 is determined to be the first gear, and the comparison result 3 indicates that the image quality of the target image is the same as that of reference image 3, then the classification result 3 is the second gear. Then, at least one classification result finally obtained is "first gear, first gear and second gear".
[0043] In one implementation, multiple image quality comparison results correspond to one classification result, and the image quality comparison result includes that the image quality of the target image is higher than that of the reference image or that the image quality of the target image is lower than that of the reference image. For example, the reference image set includes reference image 4 and reference image 5, wherein the image quality of reference image 4 is lower than that of reference image 5. When the comparison result indicates that the image quality of the target image is higher than that of reference image 4 and reference image 5, the classification result of the target image is determined to be the first gear; when the comparison result indicates that the image quality of the target image is higher than that of reference image 4 but lower than that of reference image 5, the classification result of the target image is determined to be the second gear; when the comparison result indicates that the image quality of the target image is lower than that of reference image 4 and reference image 5, the classification result of the target image is determined to be the third gear. The above method realizes determining a classification result based on two comparison results.
[0044] S304: The terminal determines a target image quality level with the largest frequency from at least one image quality level to which the target image belongs, as a target classification result corresponding to the target image.
[0045] In the embodiment of the present invention, after obtaining at least one classification result, the terminal may determine a target image quality level with the largest frequency from at least one image quality level to which the target image belongs as a target classification result corresponding to the target image.
[0046] In one implementation, the frequency is the number of occurrences of each image quality level in at least one classification result, and the terminal obtains the number of occurrences of each image quality level from at least one classification result, and determines the image quality level with the highest number of occurrences as the target classification result corresponding to the target image. For example, if the number of at least one classification result is 3, namely "first level, first level, and second level", the terminal determines that the frequency of the first level is 2, the frequency of the second level is 1, and the image quality level with the largest frequency is the first level, and the terminal determines the first level as the target classification result corresponding to the target image.
[0047] In one implementation, the frequency is a weighted value of the number of occurrences of each picture quality level in at least one classification result. Specifically, the terminal obtains the weight of each classification result, and performs weighted processing on the picture quality level in the classification result based on the weight to obtain the weighted value of the picture quality level. Furthermore, the terminal counts the weighted values of the picture quality levels in each classification result as the weighted value of the number of occurrences of each picture quality level, that is, the frequency. Among them, the weight of each classification result is determined by the similarity between the target image and the content in the reference image corresponding to the classification result. The higher the similarity, the higher the corresponding weight. For example, if the content included in the target image is a scene and the content included in the reference image 1 is also a scene, the similarity is determined to be 100%, and the weight of the classification result 1 obtained based on the comparison between the target image and the reference image 1 is 2. If the content included in the target image is a scene and the content included in the reference image 2 is a human body, the similarity is determined to be 50%, and the weight of the classification result 2 obtained based on the comparison between the target image and the reference image 2 is 1. In the above manner, the weight corresponding to each classification result can be determined. Further, the terminal extracts the corresponding image quality level from each classification result, and uses the weight to perform weighted processing on the image quality level to obtain the frequency of each image quality level. For example, the number of at least one classification result is 3, namely "first level, first level and second level", and the corresponding weights are "2, 1 and 0.5" respectively. Then the terminal determines that the frequency corresponding to the first level is: 2*1+1*1=3, and the frequency corresponding to the second level is: 0.5*1=0.5.
[0048] In an embodiment of the present invention, a terminal obtains a target image and a reference image set, and calls a trained comparison model to perform image quality comparison of the target image with each reference image in the reference image set to obtain at least one image quality comparison result. The terminal evaluates the image quality of the target image according to the at least one image quality comparison result to obtain at least one classification result, each classification result including the image quality level to which the target image belongs. The terminal determines the target image quality level with the largest frequency from at least one image quality level to which the target image belongs as the target classification result corresponding to the target image. The image quality is evaluated in a comparison-based manner to improve the accuracy and flexibility of the image quality evaluation. Furthermore, the final classification result is determined based on multiple classification results of the image, which further improves the accuracy of the image classification.
[0049] Based on the above description, the present invention provides a training method for a comparison model. Figure 4 , the image quality evaluation process may include the following steps S401-S406:
[0050] S401: The terminal obtains N first training images, and performs noise addition processing on the N first training images to obtain N second training images.
[0051] In an embodiment of the present invention, the terminal needs to obtain training images and train an initial comparison model based on the training images, so that the trained comparison model has the ability to distinguish between high and low image quality. Specifically, the terminal obtains N first training images, and performs noise processing on the above N first training images to obtain N second training images, where N is a positive integer, wherein the specific method of noise processing may include a compression method, that is, compressing each first training image, or the specific method of noise processing may also be a method of adding noise points, or the specific method of noise processing may also be a blurring method, and the noise processing is specifically used to reduce the quality of the first training image, so that the quality of the second training image obtained by noise processing is lower than that of the first training image.
[0052] S402: The terminal iteratively trains the initial comparison model based on the N first training images and the N second training images to update the second parameter in the initial comparison model.
[0053] In an embodiment of the present invention, after the terminal obtains N first training images and N second training images, the initial comparison model can be trained based on the above training images. The specific training process can be that the terminal constructs N training image combinations based on the N first training images and the N second training images, and each of the N training image combinations includes a first training image and a second training image obtained by compressing the first training image. The terminal calls the initial comparison model to perform iterative operations on each training image combination, and continuously updates the second parameter in the initial comparison model based on the operation result until the initial comparison model after the parameter update meets the second preset condition, wherein the initial comparison model can include multiple layers, such as an input layer, a fully connected layer, a convolutional layer, a pooling layer, and an output layer, and the second parameter can specifically be a parameter of a part of the layers of the initial comparison model, such as a parameter of a fully connected layer, or the second parameter can be a part of the parameters in each layer, and the second preset condition can be that the comparison accuracy of the initial comparison model for the N first training images and the N second training images is higher than the second preset accuracy.
[0054] S403: When it is detected that the initial comparison model after parameter update meets the second preset condition, the terminal determines the initial comparison model after parameter update as the basic comparison model.
[0055] In an embodiment of the present invention, the second preset condition includes that the initial comparison model has a comparison accuracy rate higher than the second preset accuracy rate for N first training images and N second training images. Specifically, after the terminal calls the initial comparison model to perform iterative operations on each training image combination in the N training image combinations, the image quality relationship between the first training image and the second training image in each image combination can be obtained, and the comparison accuracy rate of the N first training images and the N second training images is determined based on the image quality comparison results of the N image combinations, wherein if the comparison result output by the model for a training image combination indicates that the image quality of the first training image is higher than that of the second training image, then the comparison is determined to be correct, and the terminal determines the ratio between the number of correctly compared training image combinations in the N training image combinations and the total number of training image combinations as the comparison accuracy rate. Further, the terminal verifies whether the comparison accuracy rate is greater than the second preset accuracy rate, and if so, it is determined that the pre-training of the initial comparison model is completed, and the initial comparison model at this time is determined as the basic comparison model.
[0056] S404: The terminal obtains a training image set.
[0057] In an embodiment of the present invention, after the terminal obtains a basic comparison model based on the training of the initial comparison model, the terminal can further train and tune the basic comparison model to obtain a comparison model that has been trained. In the process of training and tuning, the terminal first obtains a training image set, wherein the training image set includes multiple image combinations, each image combination includes a sample image and a sample reference image, and each sample image in the sample image contains annotation information, and the annotation information indicates a preset comparison result between the image quality of the sample image and the image quality of the sample reference image, and the preset comparison result can be that the image quality of the sample image is higher than that of the sample reference image, the image quality of the sample image is the same as that of the sample reference image, or the image quality of the sample image is lower than that of the sample reference image.
[0058] It should be noted that, in the process of the user annotating the sample image, an operation can be input to the terminal to generate a corresponding image annotation instruction, and the terminal can receive the image annotation instruction. Specifically, when the terminal receives the annotation instruction for the sample image in the sample image set, the target category corresponding to the content in the sample image can be obtained, wherein the category of the content in the sample image can be pre-divided into human body, scene, and object, and the target category is human body, scene or object. After the terminal obtains the target category corresponding to the content in the sample image, it can find the corresponding sample reference image based on the target category and output the sample reference image for display, wherein the category of the content in the sample reference image can be the target category, such as when the category of the content in the sample image is scene, the category of the content in the sample reference image is also scene. Further, the terminal receives the annotation information input based on the image quality difference between the sample reference image and the sample image, and adds the annotation information to the sample image. By providing the sample reference image, the user can annotate the image based on the image quality comparison between the sample reference image and the sample image, which can greatly reduce the annotation result error caused by the fluctuation of the user's subjective feeling and improve the accuracy of the image quality annotation.
[0059] S405: The terminal iteratively trains the basic comparison model based on the image combination in the sample image set to update the second parameter in the basic comparison model.
[0060] In an embodiment of the present invention, after the terminal obtains the training image set, it can iteratively train the basic comparison model based on the image combination in the training image set to update the second parameter in the basic comparison model. Specifically, the terminal calls the basic comparison model to perform iterative operations on each training image combination, and continuously updates the first parameter in the initial comparison model based on the operation result until the initial comparison model after the parameter update meets the first preset condition, wherein the basic comparison model may include multiple layers, such as an input layer, a fully connected layer, a convolutional layer, a pooling layer, and an output layer, and the first parameter can specifically be a parameter of a part of the layers of the initial comparison model, such as a parameter of a convolutional layer and a parameter of a pooling layer, or the second parameter can be a part of the parameters in each layer, and the first preset condition can be that the comparison accuracy of the basic comparison model for the image combination in the training image set is higher than the first preset accuracy.
[0061] S406: When it is detected that the basic comparison model after parameter update meets the first preset condition, the terminal determines the basic comparison model after parameter update as the comparison model after training.
[0062] In an embodiment of the present invention, the first preset condition includes that the comparison accuracy of the basic comparison model for the image combination in the training image set is higher than the first preset accuracy. Specifically, after the terminal calls the basic comparison model to perform iterative operations on each image combination in the training image set, the image quality relationship between the training image and the sample reference image in each image combination can be obtained, and the comparison accuracy is determined based on the matching of the image quality comparison results of the images in each image combination with the preset comparison results, wherein, if the image quality comparison results output by the model for the sample image and the sample reference image in the image combination match the preset comparison results marked in the sample image, then it is determined that the comparison is correct, and the comparison accuracy is determined by the ratio of the number of image combinations that are correctly compared in the training image set to the total number of image combinations, and further, the terminal verifies whether the comparison accuracy is greater than the first preset accuracy, and if so, it is determined that the pre-training of the basic comparison model is completed, and the basic comparison model is determined as the comparison model that has been trained.
[0063] In an embodiment of the present invention, the terminal pre-trains an initial comparison model based on N first training images and the N second training images, adjusts some parameters in the model to obtain a basic comparison model, and then trains and tunes the basic comparison model based on a combination of training images, and adjusts some parameters in the model to obtain a trained comparison model. Through the above-mentioned training and training and tuning processes, the trained model can have the ability to distinguish image quality, thereby improving the accuracy of image quality classification.
[0064] Based on the above description, an embodiment of the present invention provides an image quality assessment method. Figure 5 , the image quality evaluation process may include the following steps S501-S510:
[0065] S501. The terminal uses N first training images and N second training images to pre-train an initial comparison model to obtain a basic comparison model.
[0066] In an embodiment of the present invention, pre-training is mainly used to adjust the second parameter in the initial comparison model so that the basic comparison model has the ability to distinguish image quality. The process of pre-training the initial comparison model by the terminal refers to steps S401-S403, which will not be repeated here.
[0067] S502: The terminal uses the training image set to train and optimize the basic comparison model to obtain a trained comparison model.
[0068] In the implementation example of the present invention, after the terminal obtains the basic comparison model, the training image set can be used to train and tune the basic comparison model. The training and tuning is mainly used to adjust the first parameter in the basic comparison model so that the basic comparison model's ability to distinguish image quality is enhanced. When the enhanced basic comparison model meets the conditions, the basic comparison model at this time is determined as the comparison model that has been trained, wherein the first parameter and the second parameter can be parameters of the same type, such as the first parameter and the second parameter are both parameters of the convolution layer in the model, etc., or the first parameter and the second parameter can also be parameters of different types, such as the first parameter is a parameter in the fully connected layer of the model, and the second parameter is a parameter in the convolution layer and the pooling layer in the model, etc. The process of the terminal pre-training the basic comparison model is referred to steps S404-S406, which will not be described in detail here.
[0069] S503: The terminal obtains a screening rule for the target image.
[0070] In an embodiment of the present invention, a user can predefine a screening rule for a target image, where the screening rule indicates a quality classification standard for the target image, such as screening out target images with a quality higher than or equal to 50 as push images, which are directly pushed to other users on the Internet, and screening out target images with a quality lower than 50 as interception images, which are intercepted so that other users on the Internet cannot obtain the target image, wherein the predefined image quality standard is 100 for the highest quality and 0 for the lowest quality.
[0071] S504: The terminal selects at least one reference image from the database according to the screening rule.
[0072] In an embodiment of the present invention, after the terminal determines the filtering rules for the target image, it can select the corresponding reference image from the database based on the above filtering rules. For example, if the filtering rule is that the target image with a quality higher than or equal to 50 is used as the pushed image, and the target image with a quality lower than 50 is used as the intercepted image, that is, the target image with a quality higher than or equal to 50 satisfies the filtering rule, and the target image with a quality lower than 50 does not satisfy the filtering rule, then the terminal can select at least one image with a quality of 50 from the database as the reference image, that is, the target image with a quality higher than the reference image or the same as the reference image satisfies the filtering rule, and the target image with a quality lower than the reference image does not satisfy the filtering rule. Alternatively, the filtering rule is to recommend target images with a quality higher than 70, store images with a quality between 40-70, and intercept images with a quality lower than 40, then the terminal can select images with a quality of 40 and 70 respectively from the database as reference images.
[0073] S505: The terminal obtains a target image and a reference image set.
[0074] S506: The terminal calls the trained comparison model to perform image quality comparison between the target image and each reference image in the reference image set, and obtains at least one image quality comparison result.
[0075] S507: The terminal evaluates the image quality of the target image according to at least one image quality comparison result to obtain at least one classification result.
[0076] S508: The terminal determines a target image quality level with the largest frequency from at least one image quality level to which the target image belongs, as a target classification result corresponding to the target image.
[0077] S509: The terminal verifies whether the target classification result meets the screening rules.
[0078] In an embodiment of the present invention, after the terminal determines the target classification result corresponding to the target image, it will verify whether the target classification result meets the filtering rules. Specifically, when the target classification result is the first gear or the second gear, it is determined that the image meets the filtering rules. When the target classification result is the third gear, it is determined that the image does not meet the filtering rules.
[0079] S510: If yes, the terminal determines an image pushing method corresponding to the target classification result, and pushes the target image according to the image pushing method.
[0080] In an embodiment of the present invention, when the terminal determines that the target classification result corresponding to the target image meets the screening rules, it will determine the image push method corresponding to the target classification result, and push the target image according to the image push method. Among them, the push methods corresponding to images with different image quality levels can be different, such as the push method corresponding to the first-level image is direct push, the push method corresponding to the second-level image is storage, and push when the push conditions are met, and the push method corresponding to the third-level image is interception and no push. Through screening, images that meet the requirements can be pushed in a corresponding manner, and images with poor image quality are avoided from being pushed to users, thereby improving the intelligence of the image push process.
[0081] In an embodiment of the present invention, a terminal obtains a target image and a set of sample reference images, and calls a trained comparison model to perform image quality comparison between the target image and each sample reference image in the set of sample reference images, and obtains at least one image quality comparison result. The terminal evaluates the image quality of the target image according to the at least one image quality comparison result, and obtains at least one classification result. Each classification result includes the image quality level to which the target image belongs. The terminal determines the target image quality level with the largest frequency from at least one image quality level to which the target image belongs, as the target classification result corresponding to the target image, and determines the final classification result based on multiple classification results for the image, thereby further improving the accuracy of image classification.
[0082] In one implementation scenario, the terminal may provide Figure 6 The display interface shown, in which the user can input multiple images and input the image quality screening rules, the screening rules define the image quality requirements for the image, that is, screen out the images that meet the requirements, after the terminal receives the above image, it will obtain the reference image from the database based on the screening rules, and call the trained comparison model to compare the image input by the user and the reference image, and determine the image quality level of the reference image based on the comparison result, and output and display the image quality level of each reference image, and the user can modify the image quality level output by the terminal. In the implementation of the embodiment of the present application, the terminal can automatically judge the image quality level of the image input by the user, and can also accept the user's modification operation on the image quality level at any time. When the image quality level classification is wrong, it can be modified based on the user's operation, which improves the accuracy of the image quality level classification, and the user does not need to mark the image quality level of each image, which improves the convenience of user operation.
[0083] Based on the description of the above-mentioned image quality assessment method embodiment, the embodiment of the present invention further discloses an image quality assessment device, which can be a computer program (including program code) running in a terminal, or a physical device included in the terminal. The image quality assessment device can execute Figure 3-Figure 5 See the method shown in Figure 7 The image quality assessment device 70 includes: an acquisition module 701, a calling module 702, a classification module 703, a determination module 704 and a training module 705.
[0084] An acquisition module 701 is used to acquire a target image and a reference image set, wherein the reference image set includes at least one reference image;
[0085] A calling module 702 is used to call the trained comparison model to compare the image quality of the target image with each reference image in the reference image set, and obtain at least one image quality comparison result, wherein the image quality comparison result includes a comparison result between the image quality of the target image and the image quality of the reference images in the reference image set;
[0086] A classification module 703, configured to evaluate the image quality of the target image according to the at least one image quality comparison result to obtain at least one classification result, each of which includes the image quality level to which the target image belongs;
[0087] The determination module 704 is used to determine the target image quality level with the largest frequency from at least one image quality level to which the target image belongs, as the target classification result corresponding to the target image.
[0088] In one implementation, the training module 705 is specifically configured to:
[0089] Acquire a training image set, the training image set comprising a plurality of image combinations, each of the image combinations comprising a sample image and a sample reference image, each of the sample images comprising annotation information, the annotation information indicating a preset comparison result between the image quality of the sample image and the image quality of the sample reference image;
[0090] Iteratively training a basic comparison model based on each image combination in the training image set to update a first parameter in the basic comparison model;
[0091] When it is detected that the basic comparison model after parameter update meets the first preset condition, the basic comparison model after parameter update is determined as the trained comparison model, and the first preset condition includes that the comparison accuracy of the basic comparison model for the image combination in the training image set is higher than the first preset accuracy.
[0092] In one implementation, the training module 705 is further configured to:
[0093] Acquire N first training images, and perform noise addition processing on the N first training images to obtain N second training images, wherein the noise addition processing is used to make the image quality of the second training images lower than that of the first training images, where N is a positive integer;
[0094] Iteratively training the initial comparison model based on the N first training images and the N second training images to update the second parameter in the initial comparison model;
[0095] When it is detected that the initial comparison model after parameter update satisfies a second preset condition, the initial comparison model after parameter update is determined as a basic comparison model, and the second preset condition includes that the comparison accuracy of the initial comparison model for the N first training images and the N second training images is higher than a second preset accuracy.
[0096] In one implementation, the acquisition module 701 is further configured to:
[0097] When receiving a labeling instruction for a sample image in the sample image set, obtaining a target category corresponding to the content in the sample image;
[0098] Finding a corresponding sample reference image based on the target category, and outputting and displaying the sample reference image;
[0099] Receiving annotation information input based on the image quality difference between the sample reference image and the sample image, and adding the annotation information to the sample image.
[0100] In one implementation, the image quality level of the reference image in the reference image set is the second level, and the image quality comparison result includes that the image quality of the target image is higher than that of the reference image, the image quality of the target image is the same as that of the reference image, or the image quality of the target image is lower than that of the reference image. The determination module 704 is specifically configured to:
[0101] If the image quality of the target image is higher than that of the reference image, the image quality level of the target image is determined to be the first level;
[0102] If the image quality of the target image is the same as that of the reference image, the image quality level of the target image is determined to be the second level;
[0103] If the image quality of the target image is lower than that of the reference image, the image quality level of the target image is determined to be a third level, the image quality of the first level is higher than that of the second level, and the image quality of the second level is higher than that of the first level.
[0104] In one implementation, the acquisition module 701 is further configured to:
[0105] Acquire a screening rule for a target image, wherein the screening rule indicates an image quality requirement for the target image;
[0106] Selecting at least one reference image from a database according to the screening rule;
[0107] A reference image set is constructed based on the at least one reference image.
[0108] In one implementation, the obtaining module 701 is further used to determine the target image quality level with the largest frequency from the at least one image quality level to which the target image belongs:
[0109] Verifying whether the target classification result satisfies the screening rule;
[0110] If so, determine the image pushing method corresponding to the target classification result, and push the target image according to the image pushing method.
[0111] In an embodiment of the present invention, an acquisition module 701 acquires a target image and a reference image set, a calling module 702 calls a trained comparison model to perform image quality comparison between the target image and each reference image in the reference image set, and obtains at least one image quality comparison result, a classification module 703 evaluates the image quality of the target image according to at least one image quality comparison result, and obtains at least one classification result, each classification result including an image quality level to which the target image belongs, a determination module 704 determines a target image quality level with the largest frequency from at least one image quality level to which the target image belongs, as a target classification result corresponding to the target image, and evaluates the image quality by a comparison-based method, thereby improving the accuracy and flexibility of the image quality evaluation.
[0112] See also Figure 8 , is a schematic diagram of the structure of a terminal provided by an embodiment of the present invention. Figure 8 As shown, the terminal includes: at least one processor 801, an input device 803, an output device 804, a memory 805, and at least one communication bus 802. The communication bus 802 is used to realize the connection and communication between these components. The memory 805 can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. The memory 805 can also be at least one storage device located away from the aforementioned processor 801. The processor 801 can be combined with Figure 7 In the described apparatus, a set of program codes is stored in the memory 805, and the processor 801, the input device 803, and the output device 804 call the program codes stored in the memory 805 to perform the following operations:
[0113] Processor 801, configured to acquire a target image and a reference image set, wherein the reference image set includes at least one reference image;
[0114] The processor 801 is configured to call the trained comparison model to perform image quality comparison between the target image and each reference image in the reference image set, and obtain at least one image quality comparison result, wherein the image quality comparison result includes a comparison result between the image quality of the target image and the image quality of the reference images in the reference image set;
[0115] The processor 801 is used to evaluate the image quality of the target image according to the at least one image quality comparison result to obtain at least one classification result, each of which includes the image quality level to which the target image belongs;
[0116] The processor 801 is used to determine a target image quality level with the largest frequency from at least one image quality level to which the target image belongs, as a target classification result corresponding to the target image.
[0117] In one implementation, the processor 801 is specifically configured to:
[0118] Acquire a training image set, the training image set comprising a plurality of image combinations, each of the image combinations comprising a sample image and a sample reference image, each of the sample images comprising annotation information, the annotation information indicating a preset comparison result between the image quality of the sample image and the image quality of the sample reference image;
[0119] Iteratively training a basic comparison model based on each image combination in the training image set to update a first parameter in the basic comparison model;
[0120] When it is detected that the basic comparison model after parameter update meets the first preset condition, the basic comparison model after parameter update is determined as the trained comparison model, and the first preset condition includes that the comparison accuracy of the basic comparison model for the image combination in the training image set is higher than the first preset accuracy.
[0121] In one implementation, the processor 801 is specifically configured to:
[0122] Acquire N first training images, and perform noise addition processing on the N first training images to obtain N second training images, wherein the noise addition processing is used to make the image quality of the second training images lower than that of the first training images, where N is a positive integer;
[0123] Iteratively training the initial comparison model based on the N first training images and the N second training images to update the second parameter in the initial comparison model;
[0124] When it is detected that the initial comparison model after parameter update satisfies a second preset condition, the initial comparison model after parameter update is determined as a basic comparison model, and the second preset condition includes that the comparison accuracy of the initial comparison model for the N first training images and the N second training images is higher than a second preset accuracy.
[0125] In one implementation, the processor 801 is specifically configured to:
[0126] When receiving a labeling instruction for a sample image in the sample image set, obtaining a target category corresponding to the content in the sample image;
[0127] Finding a corresponding sample reference image based on the target category, and outputting and displaying the sample reference image;
[0128] Receiving annotation information input based on the image quality difference between the sample reference image and the sample image, and adding the annotation information to the sample image.
[0129] In one implementation, the image quality level of the reference image in the reference image set is the second level, and the image quality comparison result includes that the image quality of the target image is higher than that of the reference image, the image quality of the target image is the same as that of the reference image, or the image quality of the target image is lower than that of the reference image. The processor 801 is specifically configured to:
[0130] If the image quality of the target image is higher than that of the reference image, the image quality level of the target image is determined to be the first level;
[0131] If the image quality of the target image is the same as that of the reference image, the image quality level of the target image is determined to be the second level;
[0132] If the image quality of the target image is lower than that of the reference image, the image quality level of the target image is determined to be a third level, the image quality of the first level is higher than that of the second level, and the image quality of the second level is higher than that of the first level.
[0133] In one implementation, the processor 801 is specifically configured to:
[0134] Acquire a screening rule for a target image, wherein the screening rule indicates an image quality requirement for the target image;
[0135] Selecting at least one reference image from a database according to the screening rule;
[0136] A reference image set is constructed based on the at least one reference image.
[0137] In one implementation, the processor 801 is specifically configured to:
[0138] Verifying whether the target classification result satisfies the screening rule;
[0139] If so, determine the image pushing method corresponding to the target classification result, and push the target image according to the image pushing method.
[0140] In an embodiment of the present invention, the processor 801 obtains a target image and a reference image set, and calls a trained comparison model to perform image quality comparison between the target image and each reference image in the reference image set to obtain at least one image quality comparison result. The processor 801 evaluates the image quality of the target image according to the at least one image quality comparison result to obtain at least one classification result, each classification result including the image quality level to which the target image belongs. The processor 801 determines the target image quality level with the largest frequency from the at least one image quality level to which the target image belongs as the target classification result corresponding to the target image, and evaluates the image quality by a comparison-based method to improve the accuracy and flexibility of the image quality evaluation.
[0141] The modules described in the embodiments of the present invention may be implemented by a general integrated circuit, such as a CPU (Central Processing Unit), or by an ASIC (Application Specific Integrated Circuit).
[0142] It should be understood that in the embodiment of the present invention, the processor 801 may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.
[0143] The bus 802 may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. The bus 802 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 It is represented by only one thick line, but it does not mean that there is only one bus or one type of bus.
[0144] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The computer-readable storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM).
[0145] The above disclosure is only the preferred embodiment of the present invention, which certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.
Claims
1. A method for evaluating image quality, characterized in that: The method comprises: Acquire a training image set, the training image set comprising a plurality of image combinations, each of the image combinations comprising a sample image and a sample reference image, each of the sample images comprising annotation information, the annotation information indicating a preset comparison result between the image quality of the sample image and the image quality of the sample reference image; Iteratively training a basic comparison model based on each image combination in the training image set to update a first parameter in the basic comparison model; When it is detected that the basic comparison model after parameter update satisfies a first preset condition, the basic comparison model after parameter update is determined as the comparison model after training, wherein the first preset condition includes that the comparison accuracy of the basic comparison model for the image combination in the training image set is higher than a first preset accuracy rate; Acquire a target image and a reference image set, wherein the reference image set includes at least one reference image; Calling the trained comparison model to perform image quality comparison between the target image and each reference image in the reference image set, respectively, to obtain at least one image quality comparison result, wherein the image quality comparison result includes a comparison result between the image quality of the target image and the image quality of the reference images in the reference image set; Evaluate the image quality of the target image according to the at least one image quality comparison result to obtain at least one classification result, each of the classification results including the image quality level to which the target image belongs; A target image quality level with the largest frequency is determined from at least one image quality level to which the target image belongs as a target classification result corresponding to the target image.
2. The method according to claim 1, characterized in that The method further comprises: Acquire N first training images, and perform noise addition processing on the N first training images to obtain N second training images, wherein the noise addition processing is used to make the image quality of the second training images lower than that of the first training images, where N is a positive integer; Iteratively training the initial comparison model based on the N first training images and the N second training images to update the second parameter in the initial comparison model; When it is detected that the initial comparison model after parameter update satisfies a second preset condition, the initial comparison model after parameter update is determined as a basic comparison model, and the second preset condition includes that the comparison accuracy of the initial comparison model for the N first training images and the N second training images is higher than a second preset accuracy.
3. The method according to claim 1, characterized in that Before acquiring the training image set, the method further includes: When receiving a labeling instruction for a sample image in the sample image set, obtaining a target category corresponding to the content in the sample image; Finding a corresponding sample reference image based on the target category, and outputting and displaying the sample reference image; Receiving annotation information input based on the image quality difference between the sample reference image and the sample image, and adding the annotation information to the sample image.
4. The method according to claim 1, characterized in that: The image quality level of the reference image in the reference image set is the second level, the image quality comparison result includes that the image quality of the target image is higher than that of the reference image, the image quality of the target image is the same as that of the reference image, or the image quality of the target image is lower than that of the reference image, and the image quality of the target image is evaluated according to the at least one image quality comparison result to obtain at least one classification result, including: If the image quality of the target image is higher than that of the reference image, the image quality level of the target image is determined to be the first level; If the image quality of the target image is the same as that of the reference image, the image quality level of the target image is determined to be the second level; If the image quality of the target image is lower than that of the reference image, the image quality level of the target image is determined to be a third level, the image quality of the first level is higher than that of the second level, and the image quality of the second level is higher than that of the first level.
5. The method according to claim 1, characterized in that Before acquiring the target image and the reference image set, the method further includes: Acquire a screening rule for a target image, wherein the screening rule indicates an image quality requirement for the target image; Selecting at least one reference image from a database according to the screening rule; A reference image set is constructed based on the at least one reference image.
6. The method according to claim 5, characterized in that After determining the target image quality level with the largest frequency from the at least one image quality level to which the target image belongs as the target classification result corresponding to the target image, the method further includes: Verifying whether the target classification result satisfies the screening rule; If so, determine the image pushing method corresponding to the target classification result, and push the target image according to the image pushing method.
7. An image quality assessment device, characterized in that: The device comprises: A training module is used to obtain a training image set, wherein the training image set includes multiple image combinations, each of which includes a sample image and a sample reference image, and each of the sample images contains annotation information, wherein the annotation information indicates a preset comparison result between the image quality of the sample image and the image quality of the sample reference image; iteratively train a basic comparison model based on each image combination in the training image set to update a first parameter in the basic comparison model; when it is detected that the basic comparison model after the parameter update meets a first preset condition, determine the basic comparison model after the parameter update as the comparison model after the training is completed, wherein the first preset condition includes that the comparison accuracy of the basic comparison model for the image combination in the training image set is higher than a first preset accuracy rate; An acquisition module, used to acquire a target image and a reference image set, wherein the reference image set includes at least one reference image; A calling module, used for calling the trained comparison model to compare the image quality of the target image with each reference image in the reference image set, and obtaining at least one image quality comparison result, wherein the image quality comparison result includes a comparison result between the image quality of the target image and the image quality of the reference images in the reference image set; A classification module, configured to evaluate the image quality of the target image according to the at least one image quality comparison result to obtain at least one classification result, each of which includes the image quality level to which the target image belongs; The determination module is used to determine the target image quality level with the largest frequency from at least one image quality level to which the target image belongs, as the target classification result corresponding to the target image.
8. A terminal, characterized in that: The method comprises a processor, an input interface, an output interface and a memory, wherein the processor, the input interface, the output interface and the memory are interconnected, wherein the memory is used to store a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions to execute the method according to any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor is caused to perform the method according to any one of claims 1 to 6.
