A picture quality evaluation method, a picture quality evaluation model training method, and related products

Abstract

Embodiments of the present application disclose a picture quality evaluation method, a picture quality evaluation model training method and related products, which can be applied to the picture quality detection technology field in the artificial intelligence (AI) field. The picture quality evaluation method comprises obtaining a rendering result, the rendering result comprising N images rendered using the same rendering effect, N being a positive integer; extracting semantic features of each of the N images, the semantic features being K-dimensional features, K being a positive integer; selecting M images from the N images according to the semantic features of each of the N images, M being a positive integer and M being less than N; extracting statistical features of each of the M images, the statistical features being L-dimensional features, L being a positive integer and L being less than K; and obtaining an evaluation value of each of the M images according to the semantic features and the statistical features of each of the M images. The present application can improve the accuracy of picture quality evaluation.

Claims

1. A method for evaluating image quality, characterized in that, include: Obtain the rendering result, which includes N images rendered using the same rendering effect, where N is a positive integer; Semantic features are extracted from each of the N images. The semantic features are K-dimensional features, where K is a positive integer. The semantic features of each of the N images are input into the Gaussian Mixture Model (GMM) iterative model to calculate the first quality score of each of the N images; The N images are divided into multiple scenes based on their first quality scores. Based on the first quality score of each image in each scene, M images are selected from the N images, and the M images include at least one image with the highest distortion score in each scene, where M is a positive integer and M is less than N; Statistical features are extracted from each of the M images. These statistical features are L-dimensional features based on brightness information, where L is a positive integer and L is less than K. Based on the semantic features and statistical features of each of the M images, an evaluation value is obtained for each of the M images.

2. The method according to claim 1, characterized in that, The step of dividing the N images into multiple scenes based on the first quality score of the N images includes: taking the first quality score of each of the N images as an integer, and grouping images with the same first quality score into the same scene; The step of selecting M images from the N images based on the first quality score of each image in each scene includes: normalizing the first quality score of each image in each scene to obtain a second quality score of each image in each scene; selecting the M images from the N images based on the second quality score of each image in each scene; wherein each of the M images is the image with the highest second quality score among the images in each scene, representing the top preset percentage.

3. The method according to claim 1 or 2, characterized in that, The step of extracting statistical features of the brightness of each of the M images includes: Each of the M images is input into the BRISQUE model for image quality assessment in the spatial domain without reference. After processing by the BRISQUE model, the generalized Gaussian statistical features and asymmetric generalized Gaussian statistical features of each of the M images are obtained. The statistical features include the generalized Gaussian statistical features and the asymmetric generalized Gaussian statistical features.

4. The method according to claim 3, characterized in that, The generalized Gaussian statistical features and asymmetric generalized Gaussian statistical features of each of the M images obtained through processing by the BRISQUE model include: The average contrast normalization coefficient MSCN is obtained by locally normalizing the luminance component of each of the M images. The first generalized Gaussian statistical feature is obtained by performing a generalized Gaussian fitting on the MSCN coefficients. Based on the results of the MSCN coefficients, the first asymmetric generalized Gaussian statistical feature in each direction is obtained by performing pairwise multiplication in multiple directions. Each of the M images is downsampled by a preset factor to obtain M downsampled images; The luminance component of each of the M downsampled images is locally normalized to obtain the MSCN; The second generalized Gaussian statistical feature is obtained by performing a generalized Gaussian fitting on the MSCN coefficients. Based on the results of the MSCN coefficients, a pairwise product is performed in multiple directions to obtain the second asymmetric generalized Gaussian statistical feature in each direction; the generalized Gaussian statistical feature includes the first generalized Gaussian statistical feature and the second generalized Gaussian statistical feature; the asymmetric generalized Gaussian statistical feature includes the first asymmetric generalized Gaussian statistical feature and the second asymmetric generalized Gaussian statistical feature.

5. The method according to any one of claims 1-4, characterized in that, The step of obtaining an evaluation value for each of the M images based on the semantic features and statistical features of each image in the M images includes: The semantic features and statistical features of each of the M images are input into the XGBoost model to obtain the evaluation value of each of the M images.

6. A method for training an image quality evaluation model, characterized in that, The image quality evaluation model includes a perceptual network model, a GMM iterative model, a BRISQUE model, and an XGBoost model, and the method includes: Obtain a training sample set, which includes X image samples rendered using the same rendering effect, where X is a positive integer; The X image samples are input into the perceptual network model respectively to obtain the semantic features of each of the X image samples. The semantic features are K-dimensional features, where K is a positive integer. The semantic features of each image in the X image samples are input into the GMM iterative model to calculate the first quality score of each image in the X image samples; The X images are divided into multiple scenes based on their first quality scores; Y images are selected from the X images based on the first quality score of each image in each scene. The Y images include at least one image with the highest distortion score in each scene. Y is a positive integer and Y is less than X. The Y image samples are input into the BRISQUE model to obtain the statistical features of each image in the Y image samples. The statistical features are L-dimensional features based on brightness information, where L is a positive integer and L is less than K. The semantic features and statistical features of each image in the Y image samples are input into the XGBoost model to obtain the evaluation value of each image sample in the Y image samples. The higher the evaluation value, the lower the quality of the image sample. The evaluation model is trained based on the evaluation values ​​and the labeled dataset of the training sample set to obtain the target image quality evaluation model.

7. The method according to claim 6, characterized in that, The method further includes: Obtain the original sample set, which includes Z original image samples rendered using the same rendering effect, where Z is a positive integer; The Z original image samples are preprocessed to obtain an enhanced sample set. The preprocessing includes one or more of blurring, distortion and occlusion processing. The original sample set and the enhanced sample set are integrated into the training sample set.

8. The method according to claim 6 or 7, characterized in that, The method further includes: Obtain a first user-labeled dataset, which includes X image samples rendered using the same rendering effect and label data for each of the X image samples; the label data is obtained by a group of evaluation users labeling each sample in the training sample set; The target image samples and their corresponding label data are removed from the first user-labeled dataset to obtain the second user-labeled dataset; wherein, the target image samples are image samples whose corresponding label data has a statistical error of less than a preset percentage. The label dataset is obtained by proportionally sampling image samples of each degree of distortion in the second user-labeled dataset.

9. The method according to any one of claims 6-8, characterized in that, The step of dividing the X images into multiple scenes based on the first quality score of the X images includes: taking the first quality score of each of the X images as an integer, and grouping images with the same first quality score into the same scene; The process of selecting Y images from X images based on the first quality score of each image in each scene includes: normalizing the first quality score of each image in each scene to obtain a second quality score of each image in each scene; and selecting Y image samples from the X image samples based on the second quality score of each image in each scene; wherein each image in the Y image samples is the image with the highest second quality score in each scene, representing the top preset percentage of images.

10. The method according to any one of claims 6-9, characterized in that, The step of inputting the Y image samples into the BRISQUE model to obtain the statistical features of each image in the Y image samples includes: Each of the Y image samples is input into the BRISQUE model, and the BRISQUE model processes the data to obtain the generalized Gaussian statistical features and asymmetric generalized Gaussian statistical features of each of the Y image samples; the statistical features include the generalized Gaussian statistical features and asymmetric generalized Gaussian statistical features.

11. The method according to claim 10, characterized in that, The generalized Gaussian statistical features and asymmetric generalized Gaussian statistical features of each of the Y image samples obtained through the BRISQUE model processing include: The average contrast normalization coefficient MSCN is obtained by locally normalizing the luminance component of each of the Y image samples. The first generalized Gaussian statistical feature is obtained by performing a generalized Gaussian fitting on the MSCN coefficients. Based on the results of the MSCN coefficients, the first asymmetric generalized Gaussian statistical feature in each direction is obtained by performing pairwise multiplication in multiple directions. Each of the Y image samples is downsampled by a preset factor to obtain Y downsampled images; The luminance component of each of the Y downsampled images is locally normalized to obtain the MSCN; The second generalized Gaussian statistical feature is obtained by performing a generalized Gaussian fitting on the MSCN coefficients. Based on the results of the MSCN coefficients, a pairwise product is performed in multiple directions to obtain the second asymmetric generalized Gaussian statistical feature in each direction; the generalized Gaussian statistical feature includes the first generalized Gaussian statistical feature and the second generalized Gaussian statistical feature; the asymmetric generalized Gaussian statistical feature includes the first asymmetric generalized Gaussian statistical feature and the second asymmetric generalized Gaussian statistical feature.

12. The method according to claim 11, characterized in that, The multiple directions include four directions: horizontal, vertical, -45 degrees and +45 degrees.

13. A method for evaluating image quality, characterized in that, Applied to a terminal device, the method includes: Receive video recording instructions; In response to the video recording instruction, video recording is performed, and a rendering result is generated. The rendering result includes N images rendered using the same rendering effect, where N is a positive integer. The rendering results are sent to the server. These rendering results are used for: extracting semantic features from each of the N images, where each semantic feature is a K-dimensional feature and K is a positive integer; inputting the semantic features of each of the N images into a Gaussian Mixture Model (GMM) iterative model to calculate a first quality score for each of the N images; dividing the N images into multiple scenes based on the first quality scores; selecting M images from the N images based on the first quality score of each image in each scene, where the M images include at least one image with the highest distortion score in each scene, M is a positive integer and M is less than N; extracting statistical features of the brightness of each of the M images, where the statistical features are L-dimensional features based on brightness information, L is a positive integer and L is less than K; and obtaining an evaluation value for each of the M images based on the semantic features and the statistical features of each image. Display the M images and the evaluation value of each of the M images.

14. An image quality evaluation device, characterized in that, include: The acquisition unit is used to acquire the rendering result, which includes N images rendered using the same rendering effect, where N is a positive integer; The first feature extraction unit is used to extract the semantic features of each of the N images, wherein the semantic features are K-dimensional features and K is a positive integer; The filtering unit is used to input the semantic features of each of the N images into the Gaussian Mixture Model (GMM) iterative model and calculate the first quality score of each of the N images. The N images are divided into multiple scenes based on their first quality scores. Based on the first quality score of each image in each scene, M images are selected from the N images, and the M images include at least one image with the highest distortion score in each scene, where M is a positive integer and M is less than N; The second feature extraction unit is used to extract the statistical features of each of the M images respectively. The statistical features are L-dimensional features based on brightness information, where L is a positive integer and L is less than K. An evaluation unit is configured to obtain an evaluation value for each of the M images based on the semantic features and statistical features of each image in the M images; The output unit is used to output the evaluation value of each of the M images.

15. The apparatus according to claim 14, characterized in that, The step of dividing the N images into multiple scenes based on the first quality score of the N images includes: taking the first quality score of each of the N images as an integer, and grouping images with the same first quality score into the same scene; The step of selecting M images from the N images based on the first quality score of each image in each scene includes: normalizing the first quality score of each image in each scene to obtain a second quality score of each image in each scene; selecting the M images from the N images based on the second quality score of each image in each scene; wherein each of the M images is the image with the highest second quality score among the images in each scene, representing the top preset percentage.

16. The apparatus according to claim 14 or 15, characterized in that, The second feature extraction unit is specifically used for: Each of the M images is input into the BRISQUE model, and the BRISQUE model processes the data to obtain the generalized Gaussian statistical features and asymmetric generalized Gaussian statistical features of each of the M images; the statistical features include the generalized Gaussian statistical features and asymmetric generalized Gaussian statistical features.

17. The apparatus according to claim 16, characterized in that, The second feature extraction unit is specifically used for: The average contrast normalization coefficient MSCN is obtained by locally normalizing the luminance component of each of the M images. The first generalized Gaussian statistical feature is obtained by performing a generalized Gaussian fitting on the MSCN coefficients. Based on the results of the MSCN coefficients, the first asymmetric generalized Gaussian statistical feature in each direction is obtained by performing pairwise multiplication in multiple directions. Each of the M images is downsampled by a preset factor to obtain M downsampled images; The luminance component of each of the M downsampled images is locally normalized to obtain the MSCN; The second generalized Gaussian statistical feature is obtained by performing a generalized Gaussian fitting on the MSCN coefficients. Based on the results of the MSCN coefficients, a pairwise product is performed in multiple directions to obtain the second asymmetric generalized Gaussian statistical feature in each direction; the generalized Gaussian statistical feature includes the first generalized Gaussian statistical feature and the second generalized Gaussian statistical feature; the asymmetric generalized Gaussian statistical feature includes the first asymmetric generalized Gaussian statistical feature and the second asymmetric generalized Gaussian statistical feature.

18. The apparatus according to any one of claims 14-17, characterized in that, The evaluation unit is specifically used for: The semantic features and statistical features of each of the M images are input into the XGBoost model to obtain the evaluation value of each of the M images.

19. A training device for an image quality evaluation model, characterized in that, The image quality evaluation model includes a perceptual network model, a GMM iterative model, a BRISQUE algorithm, and an XGBoost model; the device includes: The first acquisition unit is used to acquire a training sample set, which includes X image samples rendered using the same rendering effect, where X is a positive integer. The first feature extraction unit is used to input the X image samples into the perceptual network model respectively to obtain the semantic features of each image sample in the X image samples, wherein the semantic features are K-dimensional features and K is a positive integer; A filtering unit is used to input the semantic features of each image in the X image samples into the GMM iterative model to calculate the first quality score of each image in the X image samples; divide the X images into multiple scenes based on the first quality scores of the X images; and filter Y images from the X images based on the first quality score of each image in each scene, wherein the Y images include at least one image with the highest distortion score in each scene, Y is a positive integer and Y is less than X; The second feature extraction unit is used to input the Y image samples into the BRISQUE model respectively to obtain the statistical features of each image in the Y image samples. The statistical features are L-dimensional features based on brightness information, where L is a positive integer and L is less than K. An evaluation unit is used to input the semantic features and statistical features of each image in the Y image samples into the XGBoost model to obtain an evaluation value for each image sample in the Y image samples. The higher the evaluation value, the lower the quality of the image sample. The training unit is used to train the evaluation model based on the evaluation value and the label dataset of the training sample set to obtain the target image quality evaluation model.

20. The apparatus according to claim 19, characterized in that, The device further includes: The second acquisition unit is used to acquire the original sample set, which includes Z original image samples rendered using the same rendering effect, where Z is a positive integer. The data augmentation unit is used to preprocess the Z original image samples to obtain an augmented sample set, wherein the preprocessing includes one or more of blurring, distortion and occlusion processing. An integration unit is used to integrate the original sample set and the enhanced sample set into the training sample set.

21. The apparatus according to claim 19 or 20, characterized in that, The device further includes: The third acquisition unit is used to acquire a first user-labeled dataset, which includes X image samples rendered using the same rendering effect and label data for each of the X image samples; the label data is obtained by a group of evaluation users labeling each sample in the training sample set. The elimination unit is used to eliminate target image samples and their corresponding label data from the first user-labeled dataset to obtain a second user-labeled dataset; wherein, the target image samples are image samples whose corresponding label data has a statistical error of less than a preset percentage. The balancing unit is used to proportionally sample image samples of various degrees of distortion in the second user-labeled dataset to obtain the label dataset.

22. The apparatus according to any one of claims 19-21, characterized in that, The step of dividing the X images into multiple scenes based on the first quality score of the X images includes: taking the first quality score of each of the X images as an integer, and grouping images with the same first quality score into the same scene; The step of selecting Y images from the X images based on the first quality score of each image in each scene includes: normalizing the first quality score of each image in each scene to obtain a second quality score of each image in each scene; selecting the Y image samples from the X image samples based on the second quality score of each image in each scene; wherein each image in the Y image samples is the image with the highest second quality score in each scene, representing the top preset percentage.

23. The apparatus according to any one of claims 19-22, characterized in that, The second feature extraction unit is specifically used for: Each of the Y image samples is input into the BRISQUE model, and the BRISQUE model processes the data to obtain the generalized Gaussian statistical features and asymmetric generalized Gaussian statistical features of each of the Y image samples; the statistical features include the generalized Gaussian statistical features and asymmetric generalized Gaussian statistical features.

24. The apparatus according to claim 23, characterized in that, The second feature extraction unit is specifically used for: The average contrast normalization coefficient MSCN is obtained by locally normalizing the luminance component of each of the Y image samples. The first generalized Gaussian statistical feature is obtained by performing a generalized Gaussian fitting on the MSCN coefficients. Based on the results of the MSCN coefficients, the first asymmetric generalized Gaussian statistical feature in each direction is obtained by performing pairwise multiplication in multiple directions. Each of the Y image samples is downsampled by a preset factor to obtain Y downsampled images; The luminance component of each of the Y downsampled images is locally normalized to obtain the MSCN; The second generalized Gaussian statistical feature is obtained by performing a generalized Gaussian fitting on the MSCN coefficients. Based on the results of the MSCN coefficients, a pairwise product is performed in multiple directions to obtain the second asymmetric generalized Gaussian statistical feature in each direction; the generalized Gaussian statistical feature includes the first generalized Gaussian statistical feature and the second generalized Gaussian statistical feature; the asymmetric generalized Gaussian statistical feature includes the first asymmetric generalized Gaussian statistical feature and the second asymmetric generalized Gaussian statistical feature.

25. A server, characterized in that, Includes a processor; the processor is used to perform the method of any one of claims 1-5 or 6-12.

26. A computer program product, characterized in that, The method includes a computer program comprising instructions that, when executed by a processor, cause the method of any one of claims 1-5 or 6-12 to be implemented.

27. An electronic device, characterized in that, include: Memory, used to store programs; A processor for executing a program stored in the memory; When the program stored in the memory is executed by the processor, it is used to implement the method of any one of claims 1-5 or 6-12.

28. A terminal device, characterized in that, include: Memory, used to store programs; A processor for executing a program stored in the memory; When the program stored in the memory is executed by the processor, it is used to implement the method of claim 13.

29. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program code for execution by the device, which, when executed by the device, is used to implement the method of any one of claims 1-5 or 6-12.

Images