Method and system for evaluating quality of image with perceptual opinions

Abstract

The invention relates to the field of image quality evaluation, and discloses an image quality evaluation method and device with perceptual opinions, and the method comprises the steps: carrying out the scoring of a to-be-trained image through a non-deep learning algorithm, and obtaining a first quality evaluation score and an artificial classification label; inputting the to-be-trained image, the first quality evaluation score and an artificial label into a to-be-trained deep learning model; outputting a second quality evaluation score of a deep learning network model according to the to-be-trained image, and completing the training of the deep learning network model; and inputting a to-be-identified image into the trained deep learning network model to complete image quality evaluation. The novel image quality evaluation method based on the combination of deep learning and traditional image processing, provided by the invention, has perceptual opinions, belongs to a non-reference image quality evaluation method, mainly introduces less manual supervision, and realizes a better image quality evaluation effect at the same time.

Description

Technical Field

[0001] The present invention relates to the technical field of image quality assessment, and in particular to a method and system for image quality assessment with perceptual opinions. Background Art

[0002] In image analysis, a high-quality input image is extremely important. After continuous research, image quality assessment has become an important branch in the image field and plays a vital role in various image analysis tasks.

[0003] Image quality assessment methods can be divided into subjective assessment and objective assessment. Subjective assessment refers to assessing the quality of an image from a person's subjective perception. In practice, it is achieved by asking subjects to rate the original image and the distorted image, and using the mean subjective score (MOS) or the difference in mean subjective score (DMOS) to represent the final result. Objective assessment refers to using a mathematical model to quantify the quality of an image. A common method is to generate a batch of distorted images based on the original image through image processing technology, and then compare the original image with the distorted image. The purpose of the image quality assessment algorithm is to automatically assess the image quality while keeping it consistent with the subjective quality judgment of the person as much as possible. Obviously, subjective assessment is time-consuming and labor-intensive, and it is difficult to apply it to the judgment of the quality of a large number of images.

[0004] Therefore, it is necessary to explore automatic mathematical models to achieve higher consistency with human subjective quality evaluation. Summary of the invention

[0005] The main purpose of the present invention is to solve the technical problem that subjective evaluation in the prior art is time-consuming and labor-intensive and difficult to apply to the judgment of the quality of a large number of images. A method for image quality evaluation with perceptual opinions comprises the following steps: Use a non-deep learning algorithm to score the training images to obtain a first quality assessment score and a manual classification label; Inputting the image to be trained, the first quality assessment score and the manual label into the deep learning model to be trained; Outputting a second quality assessment score of the deep learning network model according to the image to be trained, and obtaining a loss according to the first quality assessment score and the second quality assessment score; adjusting parameters of the deep learning network model according to the loss to complete the training of the deep learning network model; The image to be recognized is input into the trained deep learning network model to complete the image quality assessment.

[0006] The calculation method of the loss loss is: Calculating the degree of deviation MSE between the first quality assessment score and the second quality assessment score; A constraint loss function is defined based on the artificial label, the first quality assessment score corresponding to the artificial label, and the second quality assessment score corresponding to the artificial label. ; According to the degree of deviation MSE and constraint loss function , and the loss loss is calculated. The constraint loss function The calculation method is as follows: Among them, the and The loss of good and bad quality images is generated by the labeling of classification labels. and is defined as follows: Among them, dist() is the Euclidean distance weight calculation, , is the first quality assessment score of good and bad quality images, , is the second quality assessment score of good and bad quality images, , It is the average of the first quality assessment scores of good and bad quality images.

[0007] A second aspect of the present invention also relates to a perceptually informed image quality assessment device, the device comprising: A first evaluation unit, configured to score the training image using a non-deep learning algorithm to obtain a first quality evaluation score and a manual classification label; An input unit, used to input the image to be trained, the first quality assessment score and the manual label into the deep learning model to be trained; A training unit, outputting a second quality assessment score of the deep learning network model according to the image to be trained, obtaining a loss according to the first quality assessment score and the second quality assessment score; adjusting parameters of the deep learning network model according to the loss to complete the training of the deep learning network model; The second evaluation unit is used to input the image to be identified into the trained deep learning network model to complete the image quality evaluation.

[0008] A third aspect of the present invention provides an electronic device, comprising: a memory and at least one processor, wherein instructions are stored in the memory, and the memory and the at least one processor are interconnected via lines; the at least one processor calls the instructions in the memory so that the electronic device executes the above-mentioned image quality assessment method with perceptual opinions.

[0009] A fourth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the above-mentioned image quality assessment method with perceptual opinions.

[0010] The present invention has the following beneficial effects: This patent proposes a novel, perceptual image quality assessment method based on the combination of deep learning and traditional image processing. It is a reference-free image quality assessment method that introduces less manual supervision and achieves better image quality assessment results. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 It is a topological architecture diagram of the existing computer monitoring system; Figure 2 A topological architecture diagram of the computer bypass monitoring system of the present invention; Figure 3 A first flow chart of a method for image quality assessment with perceptual opinions provided by an embodiment of the present invention; Figure 4 A second flow chart of the image quality assessment method with perceptual opinions provided by an embodiment of the present invention; Figure 5 A third flow chart of the image quality assessment method with perceptual opinions provided by an embodiment of the present invention; Figure 6 A fourth flow chart of the image quality assessment method with perceptual opinions provided by an embodiment of the present invention; Figure 7 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0012] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0013] Deep learning network model like Figure 4 and5 Deep learning network models are learning algorithms based on artificial neural networks that process data and recognize patterns by simulating the way the human brain works. These models consist of multiple layers (or "depth") of neural networks, each of which is able to extract features from the input data and pass these features to the next layer.

[0014] Deep learning network models usually contain multiple hidden layers, allowing multi-level feature extraction and abstraction of data. These layers are connected by weight matrices and use nonlinear activation functions to increase the expressive power of the network.

[0015] Non-deep learning network model Non-deep learning network models usually refer to traditional shallow neural networks, which have only one or a few hidden layers.

[0016] Due to the small number of hidden layers, non-deep learning network models have relatively limited capabilities in feature extraction and abstraction. They rely more on manually designed features to represent data.

[0017] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 The first embodiment of the image quality assessment method with perceptual opinions in the embodiment of the present invention includes: Use a non-deep learning algorithm to score the image to be trained to obtain a first quality assessment score and a manual classification label; input the image to be trained, the first quality assessment score and the manual label into the deep learning model to be trained; output a second quality assessment score of the deep learning network model based on the image to be trained, and obtain the loss according to the first quality assessment score and the second quality assessment score; adjust the parameters of the deep learning network model according to the loss to complete the training of the deep learning network model; input the image to be identified into the trained deep learning network model to complete the image quality assessment.

[0018] Specifically: Non-deep learning algorithm scoring: Use non-deep learning algorithms (such as traditional image processing technology, feature extraction methods, etc.) to score the images to be trained and obtain a preliminary quality assessment score, namely the first quality assessment score. At the same time, assign manual classification labels to these images, which represent the quality categories or characteristics of the images.

[0019] Input the deep learning model to be trained: The images to be trained, their first quality assessment scores and manual classification labels are input into the deep learning model to be trained.

[0020] Deep learning model output and loss calculation: The deep learning model outputs a second quality assessment score based on the input image.

[0021] The difference between the first quality assessment score and the second quality assessment score is used to calculate the loss, which represents the accuracy of the deep learning model's prediction.

[0022] Model training: Based on the calculated loss value, the parameters of the deep learning model are adjusted to reduce the loss in the next iteration and improve the accuracy of the model's predictions.

[0023] This process is repeated until the deep learning model achieves satisfactory performance.

[0024] Image quality assessment: The image to be recognized is input into the trained deep learning model. The model outputs a quality assessment score to indicate the quality of the image.

[0025] The method of the present invention combines non-deep learning and deep learning methods, using non-deep learning algorithms to provide preliminary quality assessments, and then using deep learning models to perform more refined quality assessments. This method can take advantage of the advantages of non-deep learning algorithms in specific tasks, while taking advantage of the powerful feature extraction and learning capabilities of deep learning models, thereby improving the accuracy of image quality assessments.

[0026] More specifically, if Figure 2 The step of using a non-deep learning algorithm to score the training image to obtain a first quality assessment score and a manual label includes: Collect a set of images to be trained with different quality levels, which should cover a wide range of quality; The brightness, contrast, clarity, noise level, edge information, color saturation, and frequency domain characteristics of the image are used as evaluation features to extract the image features of the image to be trained; Brightness and contrast evaluation features include the following steps: Grayscale conversion: If the image is in color, first convert it to grayscale to allow for uniform brightness analysis.

[0027] Global brightness average calculation: Calculate the average value of all pixel values ​​in a grayscale image. This average value reflects the overall brightness level of the image.

[0028] Global brightness standard deviation calculation: Calculate the standard deviation of all pixel values ​​in the grayscale image. The size of the standard deviation reflects the contrast level of the image. The larger the standard deviation, the higher the contrast of the image.

[0029] Grayscale histogram analysis: Count the number of pixels at each gray level in the grayscale image and generate a grayscale histogram.

[0030] By analyzing the distribution of the histogram, we can further understand the brightness and contrast characteristics of the image.

[0031] The clarity assessment features include the following steps: Edge Detection: Use algorithms such as Sobel operator, Prewitt operator or Canny edge detector to extract edges in the image.

[0032] Edge count: Count the number of edges detected. The more edges there are, the clearer the image is.

[0033] Edge strength or gradient change statistics: Calculates the strength or gradient of an edge. These values ​​reflect how distinct the edge is.

[0034] A greater change in intensity or gradient indicates a more distinct edge and greater image clarity.

[0035] Comprehensive Assessment: Combining the information of edge quantity and strength or gradient change, a comprehensive clarity evaluation feature can be obtained. The extracted multiple evaluation features are fused to form a comprehensive feature vector, and a scoring model based on non-deep learning is designed; Applying a non-deep learning-based scoring model to the image data to be trained to obtain a first quality assessment score for each image; Based on the manual classification of good and bad quality, the manual classification labels of the images to be trained are obtained.

[0036] In addition, if Figure 3 , the calculation method of the loss loss of the present invention is: Calculating the degree of deviation MSE between the first quality assessment score and the second quality assessment score; A constraint loss function is defined based on the artificial label, the first quality assessment score corresponding to the artificial label, and the second quality assessment score corresponding to the artificial label. ; According to the degree of deviation MSE and constraint loss function , calculate the loss loss.

[0037] The constraint loss function The calculation method is as follows: Among them, the and The loss of good and bad quality images is generated by the labeling of classification labels. and is defined as follows: Among them, dist() is the Euclidean distance weight calculation, , is the first quality assessment score of good and bad quality images, , is the second quality assessment score of good and bad quality images, , It is the average of the first quality assessment scores of good and bad quality images.

[0038] The above describes the image quality assessment method with perceptual opinions in the embodiment of the present invention. The following describes the image quality assessment device with perceptual opinions in the embodiment of the present invention. Figure 2 , a first embodiment of the image quality assessment device with perceptual opinions in the embodiment of the present invention comprises: A first evaluation unit, configured to score the training image using a non-deep learning algorithm to obtain a first quality evaluation score and a manual classification label; An input unit, used to input the image to be trained, the first quality assessment score and the manual label into the deep learning model to be trained; A training unit, outputting a second quality assessment score of the deep learning network model according to the image to be trained, obtaining a loss according to the first quality assessment score and the second quality assessment score; adjusting parameters of the deep learning network model according to the loss to complete the training of the deep learning network model; The second evaluation unit is used to input the image to be identified into the trained deep learning network model to complete the image quality evaluation.

[0039] Figure 77 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. The electronic device 700 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 710 (for example, one or more processors) and a memory 720, and one or more storage media 730 (for example, one or more mass storage devices) storing application programs 733 or data 732. Among them, the memory 720 and the storage medium 730 can be temporary storage or permanent storage. The program stored in the storage medium 730 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the electronic device 700. Furthermore, the processor 710 may be configured to communicate with the storage medium 730 to execute a series of instruction operations in the storage medium 730 on the electronic device 700.

[0040] The electronic device 700 may also include one or more power supplies 740, one or more wired or wireless network interfaces 750, one or more input and output interfaces 750, and / or one or more operating systems 731, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc. It will be appreciated by those skilled in the art that Figure 7 The structure of the electronic device shown does not constitute a limitation on the electronic device, and may include more or less components than shown in the figure, or combine some components, or arrange the components differently.

[0041] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. The computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the steps of a method for assessing image quality with perceptual opinions.

[0042] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, device, or unit can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.

[0043] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0044] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for image quality assessment with perceptual opinion, characterized in that: The following steps are involved: Use a non-deep learning algorithm to score the training images to obtain a first quality assessment score and a manual classification label; Inputting the image to be trained, the first quality assessment score and the manual label into the deep learning model to be trained; Outputting a second quality assessment score of the deep learning network model according to the image to be trained, and obtaining a loss according to the first quality assessment score and the second quality assessment score; adjusting parameters of the deep learning network model according to the loss to complete the training of the deep learning network model; The image to be recognized is input into the trained deep learning network model to complete the image quality assessment.

2. The method for perceptually evaluating image quality according to claim 1, characterized in that: The step of using a non-deep learning algorithm to score the training image to obtain a first quality assessment score and a manual label includes: Collect a set of images to be trained with different quality levels, which should cover a wide range of quality; The brightness, contrast, clarity, noise level, edge information, color saturation, and frequency domain characteristics of the image are used as evaluation features to extract the image features of the image to be trained; The extracted multiple evaluation features are integrated to form a comprehensive feature vector, and a non-deep learning-based scoring model is designed; A non-deep learning-based scoring model is applied to the image data to be trained to obtain a first quality assessment score for each image; based on manual classification of good and bad quality, manual classification labels of the images to be trained are obtained.

3. The method for perceptually evaluating image quality according to claim 2, characterized in that: The calculation steps of the brightness and contrast evaluation features are: The global brightness mean and standard deviation of the image are calculated to evaluate the brightness and contrast levels of the image, and the grayscale histogram statistical method is used for analysis to obtain the brightness and contrast evaluation features.

4. The method for perceptually evaluating image quality according to claim 2, characterized in that: The calculation steps of the clarity evaluation feature are: Use the Sobel operator, Prewitt operator or Canny edge detector to extract edges, count the number, intensity or gradient changes of edges, and obtain clarity evaluation features.

5. The method for perceptually evaluating image quality according to claim 1, characterized in that: The calculation method of the loss loss is: Calculating the degree of deviation MSE between the first quality assessment score and the second quality assessment score; A constraint loss function is defined according to the artificial label, the first quality assessment score corresponding to the artificial label, and the second quality assessment score corresponding to the artificial label. ; According to the degree of deviation MSE and constraint loss function , calculate the loss loss.

6. The method for perceptually evaluating image quality according to claim 5, characterized in that: The constraint loss function The calculation method is as follows: Among them, the and The loss of good and bad quality images is generated by the labeling of classification labels. and is defined as follows: Among them, dist() is the Euclidean distance weight calculation, , is the first quality assessment score of good and bad quality images, , is the second quality assessment score of good and bad quality images, , It is the average of the first quality assessment scores of good and bad quality images.

7. An image quality assessment device with perceptual opinions, characterized in that: The device comprises: A first evaluation unit, configured to score the training image using a non-deep learning algorithm to obtain a first quality evaluation score and a manual classification label; An input unit, used to input the image to be trained, the first quality assessment score and the manual label into the deep learning model to be trained; A training unit, outputting a second quality assessment score of the deep learning network model according to the image to be trained, obtaining a loss according to the first quality assessment score and the second quality assessment score; adjusting parameters of the deep learning network model according to the loss to complete the training of the deep learning network model; The second evaluation unit is used to input the image to be identified into the trained deep learning network model to complete the image quality evaluation.

8. The image quality assessment device with perceptual opinions according to claim 7, characterized in that: The calculation method of the loss loss is: Calculating the degree of deviation MSE between the first quality assessment score and the second quality assessment score; A constraint loss function is defined according to the artificial label, the first quality assessment score corresponding to the artificial label, and the second quality assessment score corresponding to the artificial label. ; According to the degree of deviation MSE and constraint loss function , calculate the loss loss.

9. An electronic device, comprising a memory and at least one processor, wherein instructions are stored in the memory; The at least one processor calls the instructions in the memory to enable the electronic device to perform the various steps of the image quality assessment method with perceptual opinions as described in any one of claims 1-6.

10. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by a processor, the steps of the method for perceptually informed image quality assessment as described in any one of claims 1 to 6 are implemented.

Images