Image Quality Assessment Method, Apparatus, Device, and Medium

Abstract

The present application relates to the field of computer technology, and provides an image quality evaluation method, apparatus, device, and medium. The method includes obtaining a plurality of images to be evaluated, inputting each image to be evaluated into a quality characterization extraction model trained with image samples of multiple first-level image qualities to obtain first quality features, then determining a quality evaluation value for each image to be evaluated according to the first quality features, and then determining the images to be evaluated with quality evaluation values less than a preset threshold as first-level image quality, or performing clustering processing on the images to be evaluated with quality evaluation values greater than the preset threshold based on the first quality features, and dividing the images to be evaluated into second-level image quality and third-level image quality according to the clustering results. The present application uses the quality characterization extraction model to extract the quality features of the images to be evaluated to determine the quality evaluation values, and uses the quality evaluation values and the clustering method to perform image quality classification, which can achieve objective image quality evaluation and image quality classification.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular, to an image quality assessment method, apparatus, device, and medium. Background Art

[0002] With the continuous development of medical imaging devices and the emergence of new imaging methods, medical images are becoming an important basis for clinical medical research, diagnosis, and treatment. Since low-quality medical images may affect the diagnostic accuracy of doctors and intelligent algorithms, quality assessment of medical images is one of the important prerequisite steps in clinical applications.

[0003] For example, Optical Coherence Tomography Angiography (OCTA), which is widely used in clinical diagnosis, obtains high-resolution volumetric blood flow data through motion contrast imaging technology to generate angiography images. Because it can clearly reveal important anatomical structures, it has great potential in accurately diagnosing various fundus-related diseases. Similarly, the angiography images generated by OCTA also have low-quality problems such as insufficient light, blurring, and low contrast.

[0004] Therefore, how to provide an image quality assessment method to achieve objective image quality evaluation and image quality classification has become a technical problem to be solved urgently. Summary of the Invention

[0005] The main purpose of the embodiments of this application is to propose an image quality assessment method, apparatus, electronic device, and computer-readable storage medium, which can achieve objective image quality evaluation and image quality classification.

[0006] To achieve the above object, a first aspect of the embodiments of this application proposes an image quality assessment method, the method including:

[0007] Obtain a plurality of images to be evaluated;

[0008] Input each of the images to be evaluated into a trained quality characterization extraction model to obtain a first quality feature of each of the images to be evaluated, where the quality characterization extraction model is trained by image samples of multiple first-level image qualities;

[0009] Determine a quality assessment value for each of the images to be evaluated according to the first quality feature, and based on the quality assessment value, perform at least one of the following steps:

[0010] Determine the images to be evaluated with a quality assessment value less than a preset threshold as first-level image quality;

[0011] Performing clustering processing on the to-be-evaluated images with a quality evaluation value greater than a preset threshold based on the first quality feature, and dividing the to-be-evaluated images with a quality evaluation value greater than the preset threshold into secondary image quality and tertiary image quality according to the clustering result.

[0012] According to the image quality evaluation method provided by some embodiments of the present application, the quality characterization extraction model includes an encoder and a plurality of decoders;

[0013] The step of inputting each to-be-evaluated image into the trained quality characterization extraction model to obtain the first quality feature of each to-be-evaluated image includes:

[0014] For each to-be-evaluated image, inputting the to-be-evaluated image into the encoder to obtain first image feature information of the to-be-evaluated image at multiple scales;

[0015] Performing pooling processing on the first image feature information at each scale respectively to obtain a plurality of first feature vectors;

[0016] Performing position encoding processing on the first image feature information at each scale respectively to obtain a plurality of first conditional vectors;

[0017] Inputting the first feature vector and the first conditional vector at each scale into the corresponding decoder respectively to obtain first spatial semantic features of the to-be-evaluated image at multiple scales;

[0018] Performing splicing processing on the multiple first spatial semantic features to obtain the first quality feature of the to-be-evaluated image.

[0019] According to the image quality evaluation method provided by some embodiments of the present application, the training process of the quality characterization extraction model includes:

[0020] Obtaining a training sample set, where the training sample set includes a plurality of image samples of primary image quality;

[0021] Inputting the image sample into the encoder to obtain second image feature information of the image sample at multiple scales;

[0022] Performing pooling processing on the second image feature information at each scale respectively to obtain a plurality of second feature vectors;

[0023] Performing position encoding processing on the second image feature information at each scale respectively to obtain a plurality of second conditional vectors;

[0024] Inputting the second feature vector and the second conditional vector at each scale into the corresponding decoder respectively to obtain second spatial semantic features of the to-be-evaluated image at multiple scales;

[0025] Determine a loss value according to the second feature vector, the second spatial semantic feature, and a preset loss function;

[0026] Train the decoder based on the loss value until a preset condition is met, and obtain the trained quality representation extraction model.

[0027] According to the image quality assessment method provided by some embodiments of the present application, the loss function is determined by the following formula:

[0028]

[0029] wherein, the L(θ) is the loss value, the D KL (·) is the KL divergence function, the p Z (·) is the density function, the is the second spatial semantic feature, the z is the second feature vector, the c is the second conditional vector, and the θ is the hyperparameter of the encoder.

[0030] According to the image quality assessment method provided by some embodiments of the present application, before clustering the to-be-evaluated images whose quality evaluation values are greater than a preset threshold based on the first quality feature, the method further includes:

[0031] Perform dimensionality reduction on the first quality feature to obtain a second quality feature;

[0032] The clustering the to-be-evaluated images whose quality evaluation values are greater than a preset threshold based on the first quality feature includes:

[0033] Cluster the to-be-evaluated images whose quality evaluation values are greater than a preset threshold based on the second quality feature.

[0034] According to the image quality assessment method provided by some embodiments of the present application, the performing dimensionality reduction on the first quality feature to obtain a second quality feature includes one of the following steps:

[0035] Perform dimensionality reduction on the first quality feature through non-negative matrix factorization to obtain a second quality feature;

[0036] Perform dimensionality reduction on the first quality feature through principal component analysis to obtain a second quality feature.

[0037] According to the image quality assessment method provided by some embodiments of the present application, the clustering the to-be-evaluated images whose quality evaluation values are greater than a preset threshold based on the first quality feature includes one of the following steps:

[0038] Perform clustering processing on the to-be-evaluated images whose quality evaluation values are greater than a preset threshold according to the first quality feature by K-means clustering;

[0039] Perform clustering processing on the to-be-evaluated images whose quality evaluation values are greater than a preset threshold according to the first quality feature by hierarchical clustering;

[0040] Perform clustering processing on the to-be-evaluated images whose quality evaluation values are greater than a preset threshold according to the first quality feature by Gaussian mixture model.

[0041] To achieve the above object, a second aspect of the embodiments of the present application proposes an image quality evaluation device, the device includes:

[0042] An image acquisition module, configured to acquire a plurality of to-be-evaluated images;

[0043] A quality characterization extraction module, configured to input each of the to-be-evaluated images into a trained quality characterization extraction model to obtain the first quality feature of each of the to-be-evaluated images, and the quality characterization extraction model is trained by image samples of multiple primary image qualities;

[0044] A quality evaluation module, configured to determine the quality evaluation value of each of the to-be-evaluated images according to the first quality feature, and based on the quality evaluation value, perform at least one of the following steps:

[0045] Determine the to-be-evaluated images whose quality evaluation values are less than the preset threshold as primary image quality;

[0046] Perform clustering processing on the to-be-evaluated images whose quality evaluation values are greater than the preset threshold based on the first quality feature, and divide the to-be-evaluated images whose quality evaluation values are greater than the preset threshold into secondary image quality and tertiary image quality according to the clustering result.

[0047] To achieve the above object, a third aspect of the embodiments of the present application proposes an electronic device, the electronic device includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the computer program is executed by the processor, the method described in the first aspect above is implemented.

[0048] To achieve the above object, a fourth aspect of the embodiments of the present application proposes a storage medium, the storage medium is a computer-readable storage medium for computer-readable storage, the storage medium stores one or more computer programs, and the one or more computer programs can be executed by one or more processors to implement the method described in the first aspect above.

[0049] The present application provides an image quality evaluation method, apparatus, electronic device, and computer-readable storage medium. The method first obtains a plurality of images to be evaluated, inputs each image to be evaluated into a trained quality characterization extraction model to obtain a first quality feature of each image to be evaluated, where the quality characterization extraction model is trained by image samples of multiple first-level image qualities. Then, a quality evaluation value of each image to be evaluated is determined according to the first quality feature. Next, the images to be evaluated with quality evaluation values less than a preset threshold are determined as first-level image qualities, or clustering processing is performed on the images to be evaluated with quality evaluation values greater than the preset threshold based on the first quality feature, and the images to be evaluated with quality evaluation values greater than the preset threshold are divided into second-level and third-level image qualities according to the clustering results. In the embodiments of the present application, the quality characterization extraction model is used to extract the quality features of the images to be evaluated to determine the quality evaluation values, and the quality evaluation values and the clustering method are used for image quality classification, which can achieve objective image quality evaluation and image quality classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 FIG. is a schematic flowchart of an image quality evaluation method provided by an embodiment of the present application;

[0051] Figure 2 FIG. is a schematic flowchart of an image quality evaluation method provided by another embodiment of the present application;

[0052] Figure 3 FIG. is a schematic flowchart of an image quality evaluation method provided by another embodiment of the present application;

[0053] Figure 4 FIG. is a schematic flowchart of an image quality evaluation method provided by another embodiment of the present application;

[0054] Figure 5 FIG. is a schematic structural diagram of a quality characterization extraction model provided by an embodiment of the present application;

[0055] Figure 6 FIG. is a schematic structural diagram of an image quality evaluation apparatus provided by an embodiment of the present application;

[0056] Figure 7 FIG. is a schematic structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0057] In order to make the objectives, technical solutions, and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0058] It should be understood that in the description of the embodiments of the present application, if there is a description of "first", "second", etc., it is only for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features or implicitly indicating the sequence relationship of the indicated technical features. "At least one" means one or more, and "a plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent the situation where A exists alone, A and B exist simultaneously, and B exists alone. Wherein A and B can be singular or plural. The character " / " generally represents an "or" relationship between the front and rear associated objects. "At least one of the following" and its similar expressions refer to any group of these items, including any group of single items or plural items. For example, at least one of a, b, and c can represent: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, and c can be single or multiple.

[0059] With the continuous development of medical imaging devices and the emergence of new imaging methods, medical images are becoming an important basis for clinical medical research, diagnosis, and treatment. Since low-quality medical images may affect the diagnostic accuracy of doctors and intelligent algorithms, the quality assessment of medical images is one of the important prerequisite steps in clinical applications.

[0060] For example, Optical Coherence Tomography Angiography (OCTA), which is widely used in clinical diagnosis, uses motion contrast imaging technology to obtain high-resolution volumetric blood flow data and generate angiography images. Because it can clearly reveal important anatomical structures, it has great potential in accurately diagnosing various fundus-related diseases. Similarly, the angiography images generated by OCTA also have low-quality problems such as insufficient light, blurring, and low contrast.

[0061] Therefore, how to provide an image quality assessment method to achieve objective image quality evaluation and image quality classification has become a technical problem to be solved urgently.

[0062] Based on this, the embodiments of the present application propose an image quality assessment method, device, electronic device, and computer-readable storage medium, which can achieve objective image quality evaluation and image quality classification.

[0063] Please refer to Figure 1 , Figure 1 which shows a schematic flowchart of an image quality assessment method provided by the embodiments of the present application. As Figure 1 shown, the image quality assessment method includes, but is not limited to, steps S110 to S150:

[0064] Step S110, obtain multiple images to be evaluated;

[0065] Step S120, input each of the images to be evaluated into a trained quality characterization extraction model to obtain a first quality feature of each of the images to be evaluated, where the quality characterization extraction model is trained by image samples of multiple first-level image qualities;

[0066] Step S130, determine a quality evaluation value of each of the images to be evaluated according to the first quality feature, and based on the quality evaluation value, perform at least one of the following steps:

[0067] Step S140, determine the images to be evaluated with a quality evaluation value less than a preset threshold as first-level image quality;

[0068] Step S150, perform clustering processing on the images to be evaluated with a quality evaluation value greater than the preset threshold based on the first quality feature, and divide the images to be evaluated with a quality evaluation value greater than the preset threshold into second-level image quality and third-level image quality according to the clustering result.

[0069] By inputting the images to be evaluated into a trained quality characterization extraction model, a first quality feature of each image to be evaluated can be obtained through the quality characterization extraction model. Since in the training stage, the quality characterization extraction model is trained based on image samples of first-level image quality and the quality characterization extraction model has not observed image samples of non-first-level image quality, therefore, the images can be divided into first-level image quality and non-first-level image quality according to the quality evaluation value of the images to be evaluated.

[0070] It can be understood that since only image samples of the first image quality are used to train the quality characterization extraction model, the feature information extracted by the quality characterization extraction model from images of different quality levels is also different. Therefore, clustering processing can be performed on the images to be evaluated with a quality evaluation value greater than the preset threshold based on the first quality feature, so that the images to be evaluated with non-first-level image quality can be further divided into second-level image quality and third-level image quality according to the clustering result.

[0071] Exemplarily, in clinical practice, the quality of OCTA images is usually divided into three levels: excellent, gradable, and ungradable. These three types of images are usually processed in different ways: excellent images can be directly analyzed or applied to specific tasks such as vessel segmentation; gradable images can be subjected to image quality enhancement, such as denoising, light equalization, contrast enhancement, etc.; ungradable images are considered useless.

[0072] In this embodiment, the primary image quality is excellent, the secondary image quality is gradable, and the tertiary image quality is non-gradable. Since it is difficult to obtain low-quality image samples, training the quality characterization extraction model with excellent image samples can reduce the implementation difficulty of the image quality assessment method. During the training process, since the model has not observed low-quality samples, the low-quality scores of non-excellent images should be higher than those of excellent images, so as to distinguish excellent and non-excellent images according to the low-quality scores, that is, to determine the images to be evaluated with primary image quality and non-primary image quality (secondary image quality and tertiary image quality) according to the quality assessment value.

[0073] Since the quality characterization extraction model is trained with excellent images, the feature information extracted by the quality characterization extraction model from images of different quality levels is also different. Therefore, the characterizations of gradable images and non-gradable images are different, and then the images to be evaluated are further subdivided into gradable images and non-gradable images according to the feature information by means of clustering.

[0074] Taking the OCTA image quality assessment as an example, see Figure 4 , Figure 4 which shows a schematic flowchart of an image quality assessment method provided by an embodiment of the present application. As Figure 4 shown, multiple images to be evaluated are input into the quality characterization extraction model trained based on excellent image samples, so as to obtain the first quality feature F of each image to be evaluated through the quality characterization extraction model H , and thus determine the quality assessment value of each image to be evaluated according to the first quality feature F H , and divide excellent images and non-excellent images based on this quality assessment value. Further, since the quality characterization extraction model extracts different feature information from images of different quality levels, for non-excellent images, the non-excellent images are divided into two quality levels, gradable and non-gradable, by means of clustering.

[0075] In some embodiments, the quality characterization extraction model includes an encoder and multiple decoders. Please see Figure 2 , Figure 2 which shows a schematic flowchart of an image quality assessment method provided by an embodiment of the present application. As Figure 2 shown, inputting each of the images to be evaluated into the trained quality characterization extraction model to obtain the first quality feature of each of the images to be evaluated includes, but is not limited to, steps S210 to S250:

[0076] Step S210, for each of the images to be evaluated, input the image to be evaluated into the encoder to obtain the first image feature information of the image to be evaluated at multiple scales;

[0077] Step S220: Perform pooling processing on the first image feature information at each scale respectively to obtain multiple first feature vectors;

[0078] Step S230: Perform positional encoding processing on the first image feature information at each scale respectively to obtain multiple first conditional vectors;

[0079] Step S240: Input the first feature vector and the first conditional vector at each scale into the corresponding decoder respectively to obtain the first spatial semantic features of the image to be evaluated at multiple scales;

[0080] Step S250: Perform splicing processing on multiple first spatial semantic features to obtain the first quality feature of the image to be evaluated.

[0081] It should be understood that the encoder can be used as a multi-scale feature extractor. If it extracts image features at n scales, there are at least n independent decoders correspondingly.

[0082] It can be understood that due to the different sizes and shapes of image anomalies, the quality characterization extraction model adopts a multi-scale feature pooling strategy to provide different receptive fields, so as to capture the global and local semantic information of the image through different receptive fields. Specifically, please refer to Figure 5 , Figure 5 which shows a schematic structural diagram of a quality feature extraction model provided by an embodiment of the present application. As Figure 5 shown, the quality characterization extraction model includes an encoder and multiple decoders. For each image to be evaluated, input the image to be evaluated into the encoder in the quality characterization extraction model to obtain the image feature information {T1,..., T n} of the image to be evaluated at different scales through the encoder. Then perform pooling dimensionality reduction processing and traditional positional encoding (PE) processing on the image feature information {T1,..., T n} at each scale respectively to obtain the first feature vector {Z1,..., Z n} and the first conditional vector {C1,..., C n}. Input the first feature vector and the first conditional vector at each scale into the corresponding decoder respectively to obtain the first spatial semantic features {P1,..., P n} of each image to be evaluated at multiple scales. Then splice multiple first spatial semantic features to obtain the first quality feature F H of each image to be evaluated.

[0083] It should be noted that it can be through the formula:

[0084]

[0085]

[0086] Directly calculate the conditional vectors of each scale feature.

[0087] Extract multi-scale features of the image to be evaluated through the encoder, and generate conditional vectors containing spatial position information in the form of two-dimensional positional encoding on the multi-scale features. Combining with the feature vectors obtained after pooling and dimensionality reduction of the multi-scale features, the decoder generates general spatial semantic features with different scales that contain the feature vectors and spatial vectors. Combining the global and local semantic information and spatial position information of the image can enhance the feature representation of the low-quality regions of the image, improving the accuracy of the quality characterization extraction model in classifying high-quality and low-quality images.

[0088] In some embodiments, see Figure 3 , Figure 3 shows a schematic flowchart of an image quality assessment method provided by an embodiment of the present application. As Figure 3 shown, the training process of the quality characterization extraction model includes but is not limited to steps S310 to S370:

[0089] Step S310, obtain a training sample set, where the training sample set includes multiple image samples of primary image quality;

[0090] Step S320, input the image sample into the encoder to obtain second image feature information of the image sample at multiple scales;

[0091] Step S330, perform pooling processing on the second image feature information at each scale respectively to obtain multiple second feature vectors;

[0092] Step S340, perform positional encoding processing on the second image feature information at each scale respectively to obtain multiple second conditional vectors;

[0093] Step S350, input the second feature vector and the second conditional vector at each scale into the corresponding decoder respectively to obtain second spatial semantic features of the image to be evaluated at multiple scales;

[0094] Step S360, determine a loss value according to the second feature vector, the second spatial semantic feature, and a preset loss function;

[0095] Step S370, train the decoder based on the loss value until a preset condition is met to obtain the trained quality characterization extraction model.

[0096] It should be understood that the encoder can be pre-trained through a large natural image database (ImageNet). For each independent decoder, the model parameters of the decoder are corrected according to the output of the encoder. Specifically, the Kullback-Leibler divergence can be used as the loss function to train the model, and this loss function is determined by the following formula:

[0097]

[0098] wherein, the L(θ) is the loss value, the D KK (·) is the KL divergence function, the p Z (·) is the density function, the is the second spatial semantic feature, the z is the second feature vector, the c is the second conditional vector, and the θ is the hyperparameter of the encoder.

[0099] It should be noted that a multivariate Gaussian distribution can be used as the density function p Z , and its formula is:

[0100]

[0101] wherein, and are the hyperparameters in the multivariate Gaussian distribution .

[0102] It can be understood that in the image quality assessment method provided by the embodiments of the present application, the quality characterization extraction model is trained in an unsupervised learning manner based on the image samples of the first-level image quality, and the quality features of the image to be evaluated are obtained through the trained quality characterization extraction model. Then, the image quality grading is realized according to the quality features and the clustering method. Training the model in an unsupervised learning manner, on the one hand, reduces the labor cost of labeling the image quality of the training samples (high quality, medium quality, low quality, etc.), and on the other hand, when it is difficult to obtain image samples of non-first-level image quality, only the quality characterization extraction model needs to be trained based on the first-level image samples, which can reduce the implementation difficulty of the image quality assessment method and improve the applicability of the image quality assessment method.

[0103] In some embodiments, before clustering the images to be evaluated with a quality evaluation value greater than a preset threshold based on the first quality feature, the method further includes:

[0104] Performing dimensionality reduction processing on the first quality feature to obtain a second quality feature;

[0105] The clustering the images to be evaluated with a quality evaluation value greater than a preset threshold based on the first quality feature includes:

[0106] Perform clustering on the to-be-evaluated images whose quality evaluation values are greater than a preset threshold based on the second quality feature.

[0107] It can be understood that the first quality feature obtained by the quality characterization extraction model is relatively complex. If high-dimensional features are directly used for clustering, the clustering performance is not good. Therefore, as Figure 4 shown, before clustering, dimensionality reduction is performed on the first quality feature F H to remove noise and redundant features, obtaining the second quality feature F L , and clustering is performed on the to-be-evaluated images whose quality evaluation values are greater than the preset threshold based on the dimensionality-reduced features.

[0108] In some embodiments, the dimensionality reduction of the first quality feature to obtain the second quality feature includes one of the following steps:

[0109] Perform dimensionality reduction on the first quality feature through non-negative matrix factorization to obtain the second quality feature;

[0110] Perform dimensionality reduction on the first quality feature through principal component analysis to obtain the second quality feature.

[0111] It can be understood that compressing the first quality feature through non-negative matrix factorization (NMF) or principal component analysis (PCA) can improve the clustering performance, thereby improving the accuracy of classifying the quality of secondary images and tertiary images.

[0112] In some embodiments, the clustering of the to-be-evaluated images whose quality evaluation values are greater than the preset threshold based on the first quality feature includes one of the following steps:

[0113] Perform clustering on the to-be-evaluated images whose quality evaluation values are greater than the preset threshold according to the first quality feature through K-means clustering;

[0114] Perform clustering on the to-be-evaluated images whose quality evaluation values are greater than the preset threshold according to the first quality feature through hierarchical clustering;

[0115] Perform clustering on the to-be-evaluated images whose quality evaluation values are greater than the preset threshold according to the first quality feature through Gaussian mixture model.

[0116] It can be understood that, according to the feature distances between the first quality features of each image to be evaluated, clustering is performed on the images to be evaluated whose quality evaluation values are greater than a preset threshold through K-means clustering, hierarchical clustering, or Gaussian Mixture Model (GMM), so as to divide them into secondary image quality and tertiary image quality.

[0117] In a specific embodiment, clustering is performed on the images to be evaluated whose quality evaluation values are greater than a preset threshold according to the second quality features through K-means clustering, hierarchical clustering, or Gaussian Mixture Model (GMM).

[0118] This application proposes an image quality evaluation method. The method first obtains a plurality of images to be evaluated, inputs each image to be evaluated into a trained quality characterization extraction model to obtain the first quality feature of each image to be evaluated. The quality characterization extraction model is trained through image samples of multiple primary image qualities. Then, according to the first quality feature, the quality evaluation value of each image to be evaluated is determined. Then, the images to be evaluated whose quality evaluation values are less than the preset threshold are determined as primary image quality. Or, clustering processing is performed on the images to be evaluated whose quality evaluation values are greater than the preset threshold based on the first quality feature, and according to the clustering result, the images to be evaluated whose quality evaluation values are greater than the preset threshold are divided into secondary image quality and tertiary image quality. The embodiments of this application use the quality characterization extraction model to extract the quality features of the images to be evaluated to determine the quality evaluation value, and use the quality evaluation value and the clustering method to perform image quality classification, which can achieve objective image quality evaluation and image quality classification.

[0119] Please refer to Figure 6 , the embodiments of this application also provide an image quality evaluation device 100, and the image quality evaluation device 100 includes:

[0120] An image acquisition module 110, configured to acquire a plurality of images to be evaluated;

[0121] A quality characterization extraction module 120, which inputs each of the images to be evaluated into a trained quality characterization extraction model to obtain the first quality feature of each of the images to be evaluated. The quality characterization extraction model is trained through image samples of multiple primary image qualities;

[0122] A quality evaluation module 130, configured to determine the quality evaluation value of each of the images to be evaluated according to the first quality feature, and based on the quality evaluation value, perform at least one of the following steps:

[0123] Determine the images to be evaluated whose quality evaluation values are less than the preset threshold as primary image quality;

[0124] Perform clustering processing on the to-be-evaluated images whose quality evaluation values are greater than a preset threshold based on the first quality feature, and divide the to-be-evaluated images whose quality evaluation values are greater than the preset threshold into second-level image quality and third-level image quality according to the clustering results.

[0125] It should be noted that for the information interaction, execution process, etc. between the modules of the above device, since they are based on the same concept as the method embodiment of the present application, for their specific functions and the technical effects brought, please refer to the method embodiment part for details, and will not be elaborated here.

[0126] Please refer to Figure 7 , Figure 7 showing the hardware structure of an electronic device provided by an embodiment of the present application. The internal structure of the electronic device 200 includes but is not limited to:

[0127] A memory 210 for storing programs;

[0128] A processor 220 for executing the programs stored in the memory 210. When the processor 220 executes the programs stored in the memory 210, the processor 220 is used to execute the above image quality evaluation method.

[0129] The processor 220 and the memory 210 can be connected by a bus or other means.

[0130] The memory 210, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs, such as the image quality evaluation method described in any embodiment of the present invention. The processor 220 realizes the above image quality evaluation method by running the non-transitory software programs and instructions stored in the memory 210.

[0131] The memory 210 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store the execution of the above image quality evaluation method. In addition, the memory 210 may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some embodiments, the memory 210 may optionally include a memory remotely set relative to the processor 220, and these remote memories can be connected to the processor 220 through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0132] The non-transitory software program and instructions required to implement the above-described image quality assessment method are stored in the memory 210, and when executed by one or more processors 220, implement the image quality assessment method provided by any embodiment of the present invention.

[0133] An embodiment of the present application also provides a computer-readable storage medium storing computer-executable instructions for executing the above-described image quality assessment method.

[0134] In one embodiment, the storage medium stores computer-executable instructions that are executed by one or more control processors, for example, by one or more processors 220 in the above-described electronic device 200, enabling the one or more processors 220 to execute the image quality assessment method provided by any embodiment of the present invention.

[0135] The embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0136] Those of ordinary skill in the art can understand that all or some of the steps and systems disclosed in the above methods can be implemented as software, firmware, hardware, and appropriate combinations thereof. Some physical components or all physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include a computer storage medium (or non-transitory medium) and a communication medium (or transitory medium). As is well known to those of ordinary skill in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassette, tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, as is well known to those of ordinary skill in the art, a communication medium generally includes computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and can include any information delivery medium.

[0137] The above is a specific description of the preferred embodiment of the present invention. However, the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent deformations or substitutions without departing from the spirit of the present invention. These equivalent deformations or substitutions are all included within the scope defined by the claims of the present invention.

Claims

1. An image quality assessment method, characterized in that, The method includes: Obtaining a plurality of images to be evaluated; Inputting each of the images to be evaluated into a trained quality characterization extraction model to obtain a first quality feature of each of the images to be evaluated, where the quality characterization extraction model is trained by a plurality of image samples of primary image quality; Determining a quality evaluation value for each of the images to be evaluated according to the first quality feature, and based on the quality evaluation value, performing at least one of the following steps: Determining the images to be evaluated with a quality evaluation value less than a preset threshold as primary image quality; Performing clustering processing on the images to be evaluated with a quality evaluation value greater than the preset threshold based on the first quality feature, and dividing the images to be evaluated with a quality evaluation value greater than the preset threshold into secondary image quality and tertiary image quality according to the clustering result; Wherein, the training process of the quality characterization extraction model includes: Obtaining a training sample set, where the training sample set includes a plurality of image samples of primary image quality; Inputting the image samples into an encoder to obtain second image feature information of the image samples at multiple scales; Performing pooling processing on the second image feature information at each scale respectively to obtain a plurality of second feature vectors; Performing position encoding processing on the second image feature information at each scale respectively to obtain a plurality of second conditional vectors; Inputting the second feature vectors and the second conditional vectors at each scale into corresponding decoders respectively to obtain second spatial semantic features of the image to be evaluated at multiple scales; Determining a loss value according to the second feature vectors, the second spatial semantic features and a preset loss function; wherein, the loss function is determined by the following formula: Among them, the is the loss value, the is the KL divergence function, the is the density function, the is the second spatial semantic feature, the is the second feature vector, the is the second conditional vector, and the is the hyperparameter of the encoder; Training the decoder based on the loss value until a preset condition is satisfied to obtain the trained quality characterization extraction model.

2. The method according to claim 1, characterized in that, The quality characterization extraction model includes an encoder and a plurality of decoders; The step of inputting each of the images to be evaluated into the trained quality characterization extraction model to obtain a first quality feature of each of the images to be evaluated includes: For each of the images to be evaluated, inputting the image to be evaluated into the encoder to obtain first image feature information of the image to be evaluated at multiple scales; Performing pooling processing on the first image feature information at each scale respectively to obtain a plurality of first feature vectors; Performing position encoding processing on the first image feature information at each scale respectively to obtain a plurality of first conditional vectors; Inputting the first feature vectors and the first conditional vectors at each scale into corresponding decoders respectively to obtain first spatial semantic features of the image to be evaluated at multiple scales; Performing splicing processing on the multiple first spatial semantic features to obtain the first quality feature of the image to be evaluated.

3. The method according to claim 1, characterized in that Before performing clustering processing on the images to be evaluated with a quality evaluation value greater than the preset threshold based on the first quality feature, the method further includes: Performing dimensionality reduction processing on the first quality feature to obtain a second quality feature; Performing clustering processing on the to-be-evaluated images with a quality evaluation value greater than a preset threshold based on the first quality feature includes: Performing clustering processing on the to-be-evaluated images with a quality evaluation value greater than a preset threshold based on the second quality feature.

4. The method according to claim 3, wherein Performing dimensionality reduction processing on the first quality feature to obtain the second quality feature includes one of the following steps: Performing dimensionality reduction processing on the first quality feature through non-negative matrix factorization to obtain the second quality feature; Performing dimensionality reduction processing on the first quality feature through principal component analysis to obtain the second quality feature.

5. The method according to claim 1, wherein Performing clustering processing on the to-be-evaluated images with a quality evaluation value greater than a preset threshold based on the first quality feature includes one of the following steps: Performing clustering processing on the to-be-evaluated images with a quality evaluation value greater than a preset threshold according to the first quality feature through K-means clustering; Performing clustering processing on the to-be-evaluated images with a quality evaluation value greater than a preset threshold according to the first quality feature through hierarchical clustering; Performing clustering processing on the to-be-evaluated images with a quality evaluation value greater than a preset threshold according to the first quality feature through Gaussian mixture model.

6. An image quality evaluation device, characterized in that, The device includes: An image acquisition module for acquiring a plurality of to-be-evaluated images; A quality characterization extraction module that inputs each of the to-be-evaluated images into a trained quality characterization extraction model to obtain the first quality feature of each of the to-be-evaluated images, and the quality characterization extraction model is trained by image samples of multiple primary image qualities; A quality evaluation module for determining the quality evaluation value of each of the to-be-evaluated images according to the first quality feature, and based on the quality evaluation value, performing at least one of the following steps: Determining the to-be-evaluated images with a quality evaluation value less than the preset threshold as primary image quality; Performing clustering processing on the to-be-evaluated images with a quality evaluation value greater than the preset threshold based on the first quality feature, and classifying the to-be-evaluated images with a quality evaluation value greater than the preset threshold into secondary image quality and tertiary image quality according to the clustering result; Wherein, the training process of the quality characterization extraction model includes: Obtaining a training sample set, and the training sample set includes image samples of multiple primary image qualities; Inputting the image samples into an encoder to obtain second image feature information of the image samples at multiple scales; Performing pooling processing on the second image feature information at each scale respectively to obtain a plurality of second feature vectors; Performing position encoding processing on the second image feature information at each scale respectively to obtain a plurality of second conditional vectors; Inputting the second feature vector and the second conditional vector at each scale into the corresponding decoder respectively to obtain second spatial semantic features of the to-be-evaluated images at multiple scales; Determining a loss value according to the second feature vector, the second spatial semantic feature and a preset loss function; wherein, the loss function is determined by the following formula: Among them, the is the loss value, the is the KL divergence function, the is the density function, the is the second spatial semantic feature, the is the second feature vector, the is the second conditional vector, and the is the hyperparameter of the encoder; Training the decoder based on the loss value until a preset condition is satisfied to obtain the trained quality characterization extraction model.

7. An electronic device, characterized in that, Includes: At least one processor; And, A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program, and the computer program is executed by the at least one processor so that the at least one processor can execute the method according to any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images