An unsupervised image quality evaluation method, system, electronic device and storage medium based on denoising diffusion probabilistic model

Abstract

The present invention discloses an unsupervised image quality evaluation method, system, electronic device and storage medium based on a denoising diffusion probability model, including: obtaining an input image, inputting the input image into a pre-trained denoising diffusion probability model to obtain a corresponding reconstructed image; respectively performing first preprocessing on the input image and the reconstructed image to obtain corresponding first feature vectors and reconstructed feature vectors, and calculating a first score related to the quality of the input image according to the first feature vectors and the reconstructed feature vectors; analyzing the corresponding structural information of the input image and the reconstructed image, and calculating the degree of structural information preservation by using the corresponding structural information of the input image and the reconstructed image to obtain a second score; performing second preprocessing on the input image to obtain a second feature vector, and inputting the second feature vector into a pre-trained regression model to obtain a third score. This method does not require manual annotation, improving the comprehensiveness and accuracy of evaluation.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly relates to an unsupervised image quality evaluation method, system, electronic device and storage medium based on a denoising diffusion probability model. Background Art

[0002] Currently, image quality assessment is an important research field aiming to automatically evaluate the quality of images. However, due to the diverse content and distortion of images, the quality assessment of unlabeled images remains a difficult task. Current image quality assessment methods mainly rely on supervised learning, requiring a large amount of labeled data, which is costly and difficult to scale to large-scale unlabeled datasets. Traditional unsupervised methods lack robustness in complex image scenarios and cannot effectively distinguish quality differences. Summary of the Invention

[0003] To solve the technical problems in the background art, the present invention proposes an unsupervised image quality evaluation method based on a denoising diffusion probability model.

[0004] In a first aspect, an unsupervised image quality evaluation method based on a denoising diffusion probability model proposed by the present invention includes:

[0005] Obtain an input image, and input the input image into a pre-trained denoising diffusion probability model to obtain a corresponding reconstructed image;

[0006] Perform first preprocessing on the input image and the reconstructed image respectively to obtain corresponding first feature vectors and reconstructed feature vectors, and calculate a first score related to the quality of the input image according to the first feature vectors and the reconstructed feature vectors;

[0007] Analyze the corresponding structural information of the input image and the reconstructed image, and calculate the degree of structural information preservation using the corresponding structural information of the input image and the reconstructed image to obtain a second score;

[0008] Perform second preprocessing on the input image to obtain a second feature vector, and input the second feature vector into a pre-trained regression model to obtain a third score;

[0009] Calculate the quality score of the input image based on the first score, the second score and the third score.

[0010] Preferably, calculating the quality score of the input image based on the first score, the second score and the third score specifically includes:

[0011] Perform inverse normalization on the first score and the third score, and then calculate the quality score of the input image according to the following formula by combining the inversely normalized first score and third score with the second score, ;

[0012] Among them, is the first score after reverse normalization; is the third score after reverse normalization; is the second score; is the quality score of the input image; is the weight of the first score after reverse normalization; is the weight of the second score; is the weight of the third score after reverse normalization; , and satisfy .

[0013] Preferably, the first preprocessing specifically includes:

[0014] Performing denoising, cropping, grayscale transformation, and normalization processing on the input image and the reconstructed image respectively to obtain the input image and the reconstructed image after eliminating noise interference and standardizing the data format;

[0015] Inputting the input image and the reconstructed image after eliminating noise interference and standardizing the data format into the pre-trained convolutional neural network to extract feature vectors, so as to obtain the corresponding first feature vector and reconstructed feature vector.

[0016] Preferably, calculating the first score related to the quality of the input image according to the first feature vector and the reconstructed feature vector specifically includes:

[0017] Calculating the reconstruction error of the image in the denoising diffusion probability model according to the first feature vector and the reconstructed feature vector;

[0018] Taking the reconstruction error as the first score for evaluating the image quality to obtain the first score related to the quality of the input image.

[0019] Preferably, the structural information includes but is not limited to average brightness, image variance, image covariance, and comparison terms of brightness; calculating the degree of preservation of structural information by using the structural information corresponding to the input image and the reconstructed image to obtain the second score, specifically: calculating the degree of preservation of the structural information of the reconstructed image relative to the input image according to the following formula, and taking the degree of preservation of the structural information as the second score; ;

[0020] Among them, is the average brightness of the input image; is the average brightness of the reconstructed image; is the variance of the input image; is the variance of the reconstructed image; is the covariance of the input image and the reconstructed image; and are small constants added to avoid a zero denominator, where and are adjustable parameters, is the range of pixel values. For example, for an 8-bit image, , is the comparison term for the brightness of the input image; is the comparison term for the brightness of the reconstructed image; is the second score.

[0021] Preferably, the second preprocessing specifically includes:

[0022] After performing operations to adjust the image size and brightness balance on the input image, the adjusted input image is segmented into multiple image blocks;

[0023] Statistical features are extracted one by one from the multiple image blocks, and the extracted statistical features are subjected to evaluation feature transformation and normalization processing to obtain corresponding multiple evaluation feature vectors;

[0024] The multiple evaluation feature vectors corresponding to the multiple image blocks are subjected to feature vector splicing to obtain the second feature vector of the input image.

[0025] Preferably, the pre-trained regression model is specifically a support vector machine SVM; the inputting the second feature vector into the pre-trained regression model to obtain a third score specifically includes:

[0026] The second feature vector is input into the support vector machine SVM for regression to obtain an evaluation result of the image quality of the input image, and the evaluation result of the image quality of the input image is used as the third score.

[0027] Preferably, the process of constructing the denoising diffusion probability model specifically includes:

[0028] Construct a denoising diffusion probability model, which includes forward diffusion and backward denoising;

[0029] The forward diffusion includes: simulating the process of data being gradually covered by noise, and iteratively adding noise to the given input image by sampling from the Gaussian distribution ;

[0030] The backward denoising includes: simulating the process of recovering the original data from the noisy data, and performing data recovery on the input image after adding noise to generate a corresponding reconstructed image.

[0031] Preferably, the training process of the denoising diffusion probability model specifically includes:

[0032] Obtain an image dataset, and input the image dataset into a pre-trained convolutional neural network CNN to extract the preliminary feature vectors of the images , where represents the feature vector of each image;

[0033] Input the preliminary feature vectors into the constructed denoising diffusion probabilistic model to obtain a corresponding reconstructed image dataset;

[0034] Based on the reconstructed image dataset, the image dataset, and the loss function Iteratively optimize the weight parameters of the constructed denoising diffusion probabilistic model to obtain a trained denoising diffusion probabilistic model;

[0035] The loss function Specifically: ;

[0036] where is the reconstructed image, is the input image; represents the loss function in the form of mean squared error MSE; represents the Euclidean distance; is the expectation calculation of the reconstructed image and the input image.

[0037] Preferably, it further includes:

[0038] S6. Obtain a dataset of images to be measured, evaluate multiple quality scores corresponding to multiple images to be measured in the dataset of images to be measured according to steps S1 - S5, and output the multiple images to be measured after sorting them in descending order of quality scores.

[0039] In a second aspect, an unsupervised image quality evaluation system based on a denoising diffusion probabilistic model proposed by the present invention includes:

[0040] A data acquisition module, configured to acquire an input image, and input the input image into a pre-trained denoising diffusion probabilistic model to obtain a corresponding reconstructed image;

[0041] A first processing module, configured to perform first preprocessing on the input image and the reconstructed image respectively to obtain corresponding first feature vectors and reconstructed feature vectors, and calculate a first score related to the quality of the input image according to the first feature vectors and the reconstructed feature vectors;

[0042] A second processing module, configured to analyze the corresponding structural information of the input image and the reconstructed image, and calculate the degree of structural information preservation by using the corresponding structural information of the input image and the reconstructed image to obtain a second score;

[0043] A third processing module, configured to perform a second preprocessing on the input image to obtain a second feature vector, and input the second feature vector into a pre-trained regression model to obtain a third score;

[0044] An output module, configured to calculate a quality score of the input image based on the first score, the second score, and the third score.

[0045] In a third aspect, the present application provides an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0046] The memory stores computer-executable instructions;

[0047] The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of the first aspects.

[0048] In a fourth aspect, the present application provides a computer-readable storage medium, in which computer-executable instructions are stored, and when the computer-executable instructions are executed by a processor, they are used to implement the method according to any one of the first aspects.

[0049] In a fifth aspect, the present application provides a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the method according to any one of the first aspects.

[0050] In the present invention, the proposed unsupervised image quality evaluation method based on the denoising diffusion probability model extracts features through pre-training a convolutional neural network CNN, combines the forward noise addition and backward reconstruction processes of the diffusion model, quantifies the reconstruction error, structural similarity, and spatial domain statistical features, and fuses them to generate a comprehensive quality score. This method does not require manual annotation, improving the comprehensiveness and accuracy of the evaluation. By analyzing the diffusion process, stability is embedded, facilitating adaptation to complex scenarios and diverse distortion types. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a schematic structural diagram of the working process of an unsupervised image quality evaluation method based on the denoising diffusion probability model proposed by the present invention;

[0052] Figure 2 It is a schematic structural diagram of an implementation process of an unsupervised image quality evaluation method based on the denoising diffusion probability model proposed by the present invention;

[0053] Figure 3 It is a schematic structural diagram of the electronic device provided by the embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0054] Referring to Figure 1 and Figure 2, an unsupervised image quality evaluation method based on the denoising diffusion probabilistic model proposed by the present invention includes the following steps:

[0055] S1. Obtain the input image, and input the input image into a pre-trained denoising diffusion probabilistic model to obtain the corresponding reconstructed image.

[0056] In this embodiment, the construction process of the denoising diffusion probabilistic model (Denoising Diffusion Probabilistic Model, DDPM) specifically includes: constructing a denoising diffusion probabilistic model, which includes forward diffusion and backward denoising; the forward diffusion includes: simulating the process of data being gradually covered by noise, and iteratively adding noise to the given input image by sampling from a Gaussian distribution ; the backward denoising includes: simulating the process of recovering the original data from the noisy data, and performing data recovery on the input image after adding noise to generate the corresponding reconstructed image.

[0057] Specifically, since the denoising diffusion probabilistic model DDPM has the following two advantages. The first is good robustness, because higher-quality images have stable representations in the embedding space of a given image classification model and are less affected by the perturbations introduced by the forward diffusion process compared to lower-quality images. The second is the difference in reconstruction quality. Compared with low-quality images, high-quality samples are easier to reconstruct from partially corrupted (noisy) data with incomplete identity information, and the difference between the input embedding and the denoised sample is smaller. Therefore, the method proposed in this application analyzes the embedding stability of given images by perturbing them through forward and backward diffusion processes, and then quantifies the results of quality estimation. By simultaneously using the forward (noise) and backward (reconstruction) diffusion steps, the quality of a given input sample is accurately evaluated.

[0058] Construct a denoising diffusion probabilistic model, which represents a special type of generative model that learns to model the (image) data distribution through two types of processes, namely the forward (noise) process and the backward (denoising) process. The forward diffusion process simulates the process of data being gradually covered by noise, and iteratively adds noise to the given input image by sampling from a Gaussian distribution The entire forward process can be presented as a Markov chain: ; ;

[0059] where is the time step selected from the sequence , is the variance parameter, indicating the moment in the forward process How much noise is added to the image sample. Input sample to any sample The process is expressed as: ;

[0060] where .

[0061] The reverse diffusion process, which simulates the process of recovering the original data from the noisy data, where is the conditional distribution of and under the condition of . The formula is as follows: ;

[0062] where is the data at the known time and the initial data under the condition that, from time recover to time of the data . The right side indicates that this probability distribution is a multivariate Gaussian distribution. Where is the number of time steps from the sequence ; The vector is the data at the previous moment to be recovered, representing the state of the data at that moment; The vector represents the noisy data at the current moment; is the original noise-free data, which is the data that we ultimately want to recover; is the mean function of the Gaussian distribution, which is a function of (the current noisy data), (the original data) and the time . This mean function determines the central tendency when recovering from to ; is the variance function of the Gaussian distribution, which is a function of , and the time . It measures the uncertainty in the recovery process; is the identity matrix. Multiplying it by means that the covariance matrix is a diagonal matrix, indicating that the dimensions of the data are independent of each other. The identity matrix, multiplying it by means that the covariance matrix is a diagonal matrix, indicating that the dimensions of the data are independent of each other.

[0063] where is the number of time steps from the sequence , , ;

[0064] is the mean function of the Gaussian distribution, which is a function of (the current noisy data), (the original data) and the time . This mean function determines the central tendency when restoring from to ; The vector is the noisy data at time , representing the state of the data at that time; The vector represents the original noise-free data, which is the data that we ultimately want to restore; and are parameters related to the diffusion process, usually related to the process of gradually adding noise. Their values are between 0 and 1, reflecting the degree of noise addition at different times. corresponds to time , corresponds to time ; is the cumulative product form from the initial time to time , that is, , used to describe the comprehensive impact of noise accumulation from the start to time ; is also a parameter in the diffusion process, usually related to , indicating information such as the amount of noise added at time , and its value is also between 0 and 1; represents the parameters of the model, which are learned during the training of the diffusion model and are used to adjust the calculation method of the mean to better restore the data.

[0065] In this embodiment, the training process of the denoising diffusion probability model specifically includes: obtaining an image dataset, inputting the image dataset into a pre-trained convolutional neural network CNN to extract the preliminary feature vectors of each image, where represents the feature vector of each image; inputting the preliminary feature vectors into the constructed denoising diffusion probability model to obtain a corresponding reconstructed image dataset; iteratively optimizing the weight parameters of the constructed denoising diffusion probability model based on the reconstructed image dataset, the image dataset, and the loss function to obtain a trained denoising diffusion probability model.

[0066] In this embodiment, the loss function​ Specifically: ;

[0067] Among them, is the reconstructed image; is the input image; represents the loss function in the form of mean square error MSE; represents the Euclidean distance; is the expected calculation of the reconstructed image and the input image, is the information of the state image at time

[0068] S2. Perform first preprocessing on the input image and the reconstructed image respectively to obtain corresponding first feature vectors and reconstructed feature vectors, and calculate a first score related to the quality of the input image according to the first feature vectors and the reconstructed feature vectors.

[0069] In this embodiment, the first preprocessing specifically includes: performing denoising, cropping, grayscale transformation, and normalization processing on the input image and the reconstructed image respectively to obtain the input image and the reconstructed image after eliminating noise interference and standardizing the data format; respectively inputting the input image and the reconstructed image after eliminating noise interference and standardizing the data format into a pre-trained convolutional neural network for feature vector extraction to obtain corresponding first feature vectors and reconstructed feature vectors.

[0070] In this embodiment, calculating a first score related to the quality of the input image according to the first feature vectors and the reconstructed feature vectors specifically includes: calculating the reconstruction error of the image in the denoising diffusion probability model according to the first feature vectors and the reconstructed feature vectors; using the reconstruction error as the first score for evaluating the image quality to obtain a first score related to the quality of the input image.

[0071] Specifically, since the denoising diffusion process of low-quality images is more sensitive to image perturbations introduced by the forward and backward diffusion processes than that of high-quality images. Therefore, the quality of the image is evaluated by calculating the reconstruction error of the image in the denoising diffusion model. Specifically, the quality score of each image is calculated by the following formula : ;

[0072] Among them, is the feature vector reconstructed by the image during the denoising diffusion process. By obtaining the intermediate feature state through feature extraction, the image quality evaluation index is calculated, is the original feature vector of the image .

[0073] S3. Analyze the structural information corresponding to the input image and the reconstructed image, and calculate the degree of structural information preservation using the structural information corresponding to the input image and the reconstructed image, so as to obtain a second score.

[0074] In this embodiment, the structural information includes but is not limited to average brightness, image variance, image covariance, and comparison terms of brightness; calculating the degree of structural information preservation using the structural information corresponding to the input image and the reconstructed image to obtain a second score, specifically: calculating the degree of preservation of the structural information of the reconstructed image relative to the input image according to the following formula, and taking the degree of preservation of the structural information as the second score; ;

[0075] Where, is the average brightness of the input image; is the average brightness of the reconstructed image; is the variance of the input image; is the variance of the reconstructed image; is the covariance of the input image and the reconstructed image; and are small constants added to avoid a zero denominator, where and are adjustable parameters, is the range of pixel values. For example, for an 8-bit image, , is the comparison term of the brightness of the input image; is the comparison term of the brightness of the reconstructed image; is the second score.

[0076] Specifically, the Structural Similarity (SSIM) is used to evaluate the degree of structural information preservation of the image, which takes into account the brightness, contrast, and structural information of the image. The range of the index is from -1 to 1, where 1 indicates that the two images are exactly the same, and 0 indicates that they are completely different.

[0077] S4. Perform a second preprocessing on the input image to obtain a second feature vector, and input the second feature vector into a pre-trained regression model to obtain a third score.

[0078] In this embodiment, the second preprocessing specifically includes: after performing operations to adjust the image size and brightness balance on the input image, dividing the adjusted input image into multiple image blocks; extracting statistical features from each of the multiple image blocks one by one, and performing evaluation feature transformation and normalization processing on the extracted statistical features to obtain corresponding multiple evaluation feature vectors; performing feature vector splicing on the multiple evaluation feature vectors corresponding to the multiple image blocks to obtain the second feature vector of the input image.

[0079] Specifically, for the input image, the spatial domain image quality assessment algorithm Brisque is used to evaluate the image, and the specific steps are as follows:

[0080] (1) Feature normalization is specifically: normalizing the transformed features and scaling them to the range between 0 and 1.

[0081] Calculation of local luminance normalization (Mean Subtracted Contrast Normalized, MSCN) coefficient: ;

[0082] where, is the calculated local luminance normalization (MSCN) coefficient, is the pixel value of the image at the coordinate , is the local mean, is the local standard deviation, is a constant (usually taken as 1) for numerical stability.

[0083] Statistical feature extraction is specifically: extracting multiple statistical features for each image block, including the mean, standard deviation, etc.

[0084] Among them, the formula for calculating the local mean: ;

[0085] represents the local mean at the coordinate in the image. Here, is the position coordinate of the pixel point in the two-dimensional plane of the image; represents the row index; column index; and are parameters used to define the neighborhood range, determines the width of the neighborhood in the horizontal direction, determines the height of the neighborhood in the vertical direction; is the weight coefficient, corresponding to the weight at the position of the coordinate in the neighborhood; represents the pixel value at the coordinate in the image, that is, the pixel value at the position offset from the central pixel ; through double summation, the pixels in the entire neighborhood centered on are traversed, and their weighted sum is calculated according to the weights.

[0086] (2) Standard deviation of statistical features, the calculation formula is as follows: ;

[0087] Among them, is the Gaussian kernel, represents the standard deviation of the neighborhood at the coordinate in the image; is the position of the pixel point in the two-dimensional plane of the image; and are used to define the neighborhood range, determines the width of the neighborhood in the horizontal direction, determines the height of the neighborhood in the vertical direction; is the weight coefficient, corresponding to the weight at the position with coordinates in the neighborhood; refers to the pixel value at the coordinate in the image, that is, the pixel value at the position offset from the central pixel in the neighborhood; is the local mean at the coordinate in the image, usually obtained by weighted summation of the pixel values in the neighborhood of this point, reflecting the average pixel value situation in the local area.

[0088] (3)Model the MSCN coefficient histogram using the Generalized Gaussian Distribution (GGD), and extract the features of the fitted Gaussian distribution as the features for image quality assessment. The Generalized Gaussian Distribution (GGD) model is as follows: ;

[0089] Among them, , For , is the value of the probability density function of the generalized Gaussian distribution at , representing the relative likelihood when the random variable takes this value; is the random variable, representing the data to be analyzed, which can be various data types such as image feature values, signal values, etc.; is the shape parameter, which determines the shape of the distribution. When , the generalized Gaussian distribution degenerates into the standard Gaussian distribution; when , the tail of the distribution is thicker than the standard Gaussian distribution, meaning that the probability of extreme values appearing is relatively large; when , the tail of the distribution is thinner than the standard Gaussian distribution, and the probability of extreme values appearing is smaller. is the variance parameter, which measures the degree of dispersion of the data and reflects the dispersion of the data around the mean. The larger the variance, the more dispersed the data; the smaller the variance, the more concentrated the data; associated with and related parameters , which participates in adjusting the scale of the distribution. is the gamma function to ensure the normalization of the probability density function (i.e., the area under the entire distribution is 1).

[0090] (4) Calculation of mass fraction result: Use the pre-trained regression model to obtain the image quality score.

[0091] In this embodiment, the pre-trained regression model is specifically a support vector machine SVM; input the second feature vector into the pre-trained regression model to obtain the third score, specifically including:

[0092] Input the second feature vector into the support vector machine SVM for regression to obtain the evaluation result of the image quality of the input image, and use the evaluation result of the image quality of the input image as the third score.

[0093] In this embodiment, the specific processing process of the regression model is as follows:

[0094] Feature vector input: The second feature vector of the image is input into the regression model;

[0095] Model prediction: SVM obtains the score by calculating the dot product of the second feature vector and the weight vector plus the bias term; the formula for calculating the score is: ;

[0096] where is the score value, is the second feature vector, and also dimensional vector, that is , is the weight vector, and also dimensional vector, that is , representing the parameters learned by SVM training, is the bias term, which is a scalar.

[0097] Result output: Determine the category to which the image quality belongs according to the score, and the category division formula is expressed as: ;

[0098] where is the category to which the image quality belongs, and the integer 1, 2, 3 respectively correspond to the category scores in different score intervals.

[0099] S5. Calculate the quality score of the input image based on the first score, the second score, and the third score.

[0100] In this embodiment, it specifically includes:

[0101] The first score and the third score are reverse normalized, and then the reverse normalized first score and the third score and the second score are used to calculate the quality score of the input image according to the following formula: ;

[0102] in, It is the first score after reverse normalization; The third score after reverse normalization; Rating for the second; Score the quality of the input image; is the weight of the first score after reverse normalization; The weight of the second score; is the weight of the third score after reverse normalization; , and satisfy .

[0103] It should be noted that due to the reconstruction error of the denoising diffusion model The smaller the better, structural similarity The larger the index, the better. The spatial domain image quality assessment results The smaller the better, so you need to and Perform reverse normalization so that all indicator scores are as high as possible, reverse normalization processing: ;

[0104] when Take 0.35, Take the sum of 0.4 When the value is 0.25, the quality score formula is: .

[0105] S6. Obtain a data set of images to be tested, evaluate multiple quality scores corresponding to multiple images to be tested in the data set of images to be tested according to steps S1-S5, and sort the multiple images to be tested in descending order of quality scores and then output them.

[0106] In this embodiment, the images in the image data set to be tested are sorted according to the calculated quality scores, and the scoring results and the processed high-quality images are output in a standardized format for easy storage or further application.

[0107] Specifically, according to the calculated quality scores, the image dataset to be tested is sorted and screened to identify high-quality and low-quality images.

[0108] Reference Figure 1 and Figure 2, an unsupervised image quality evaluation system based on a denoising diffusion probability model, comprising:

[0109] A data acquisition module for acquiring an input image and inputting the input image into a pre-trained denoising diffusion probability model to obtain a corresponding reconstructed image;

[0110] A first processing module for respectively performing first preprocessing on the input image and the reconstructed image to obtain corresponding first feature vectors and reconstructed feature vectors, and calculating a first score related to the quality of the input image according to the first feature vectors and the reconstructed feature vectors;

[0111] A second processing module for analyzing the corresponding structural information of the input image and the reconstructed image, and calculating the degree of structural information preservation by using the corresponding structural information of the input image and the reconstructed image to obtain a second score;

[0112] A third processing module for performing second preprocessing on the input image to obtain second feature vectors, and inputting the second feature vectors into a pre-trained regression model to obtain a third score;

[0113] An output module for calculating the quality score of the input image based on the first score, the second score, and the third score.

[0114] Figure 3 It is a schematic structural diagram of the electronic device provided in the embodiment of the present application. Please refer to Figure 3 , the electronic device 20 may include: a memory 21 and a processor 22. Exemplarily, the memory 21 and the processor 22 are interconnected with each other through a bus 23.

[0115] The memory 21 is used to store computer execution instructions;

[0116] The processor 22 is used to execute the computer execution instructions stored in the memory, so as to enable the control device 20 to execute the method shown in the above method embodiment.

[0117] The electronic device provided in the embodiment of the present application can execute the technical solution shown in the above method embodiment, and its implementation principle and beneficial effects are similar, and will not be elaborated here.

[0118] The embodiment of the present application provides a computer-readable storage medium, in which computer execution instructions are stored, and when the computer execution instructions are executed by a processor, they are used to implement the technical solution shown in the above method embodiment, and its implementation principle and beneficial effects are similar, and will not be elaborated here.

[0119] An embodiment of the present application may further provide a computer program product, including a computer program. When the computer program is executed by a processor, the technical solutions shown in the above method embodiments can be implemented. The implementation principle and beneficial effects are similar and will not be elaborated here.

[0120] It should be noted that for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the present application is not limited by the described action sequence, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to the present application.

[0121] Furthermore, it should be noted that although the steps in the flowchart are sequentially displayed according to the indication of the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps has no strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same moment, but can be executed at different moments. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least a part of other steps or sub-steps or stages of other steps.

[0122] It should be understood that the above device embodiments are illustrative, and the devices of the present application can also be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical function division, and there can be other division methods in actual implementation. For example, multiple units, modules or components can be combined, or can be integrated into another system, or some features can be ignored or not executed.

[0123] In addition, without special description, in each embodiment of the present application, each functional unit / module can be integrated in one unit / module, or each unit / module can exist physically alone, or two or more units / modules can be integrated together. The above integrated unit / module can be implemented in the form of hardware or in the form of a software program module.

[0124] When the integrated unit / module is implemented in the form of hardware, the hardware can be a digital circuit, an analog circuit, etc. Unless otherwise specified, the processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP for short), a digital signal processor (DSP for short), an application specific integrated circuit (ASIC for short), a field programmable gate array (FPGA for short), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. Unless otherwise specified, the storage unit can be any suitable magnetic storage medium or magneto-optical storage medium, such as a resistive random access memory (RRAM), a dynamic random access memory (DRAM), a static random access memory (SRAM), an enhanced dynamic random access memory (EDRAM), a high-bandwidth memory (HBM), a hybrid memory cube (HMC), etc.

[0125] When the integrated unit / module is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes: various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disc.

[0126] The above are only the preferred specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution and inventive concept of the present invention, making equivalent substitutions or changes should be covered by the protection scope of the present invention.

Claims

1. An unsupervised image quality assessment method based on a denoising diffusion probability model, characterized in that: The following steps are involved: S1. Obtain an input image and input the input image into a pre-trained denoising diffusion probability model to obtain a corresponding reconstructed image; S2, performing first preprocessing on the input image and the reconstructed image respectively to obtain corresponding first feature vectors and reconstructed feature vectors, and calculating a first score related to the quality of the input image according to the first feature vectors and the reconstructed feature vectors; S3, analyzing the structural information corresponding to the input image and the reconstructed image, and calculating the degree of structural information preservation using the structural information corresponding to the input image and the reconstructed image to obtain a second score; S4, performing a second preprocessing on the input image to obtain a second feature vector, and inputting the second feature vector into a pre-trained regression model to obtain a third score; S5. Calculate a quality score of the input image based on the first score, the second score, and the third score.

2. The unsupervised image quality assessment method based on denoising diffusion probability model according to claim 1 is characterized in that: The calculating the quality score of the input image based on the first score, the second score and the third score specifically includes: The first score and the third score are reverse normalized, and then the reverse normalized first score and the third score and the second score are used to calculate the quality score of the input image according to the following formula: ; in, It is the first score after reverse normalization; The third score after reverse normalization; Rating for the second; Score the quality of the input image; is the weight of the first score after reverse normalization; The weight of the second score; is the weight of the third score after reverse normalization; , and satisfy .

3. The unsupervised image quality assessment method based on denoising diffusion probability model according to claim 1, characterized in that: The first preprocessing specifically includes: The input image and the reconstructed image are subjected to denoising, cropping, grayscale conversion, and normalization processing respectively, so as to obtain the input image and the reconstructed image after eliminating noise interference and standardizing the data format; The input image and the reconstructed image after noise interference is eliminated and the data format is standardized are respectively input into the pre-trained convolutional neural network to extract feature vectors to obtain the corresponding first feature vector and reconstructed feature vector.

4. The unsupervised image quality assessment method based on denoising diffusion probability model according to claim 1, characterized in that: The step of calculating a first score related to the quality of the input image according to the first feature vector and the reconstructed feature vector specifically includes: Calculate the reconstruction error of the image in the denoising diffusion probability model according to the first eigenvector and the reconstruction eigenvector; The reconstruction error is used as a first score for evaluating image quality, so as to obtain a first score related to the quality of the input image.

5. The unsupervised image quality assessment method based on denoising diffusion probability model according to claim 1, characterized in that: The structural information includes average brightness, image variance, image covariance and brightness comparison items; the structural information preservation degree is calculated using the structural information corresponding to the input image and the reconstructed image to obtain the second score, specifically: the preservation degree of the structural information of the reconstructed image relative to the input image is calculated according to the following formula, and the preservation degree of the structural information is used as the second score; ; in, is the average brightness of the input image; is the average brightness of the reconstructed image; is the variance of the input image; is the variance of the reconstructed image; is the covariance of the input image and the reconstructed image; and is a small constant added to avoid the denominator being zero, where and is an adjustable parameter, is the range of pixel values, for 8-bit images, , is the comparison item of the brightness of the input image; is a comparison term for the brightness of the reconstructed image; Rated second.

6. The unsupervised image quality assessment method based on denoising diffusion probability model according to claim 1, characterized in that: The second preprocessing specifically includes: After adjusting the image size and brightness balance of the input image, the adjusted input image is divided into a plurality of image blocks; Extract statistical features from multiple image blocks one by one, and perform evaluation feature conversion and normalization processing on the extracted statistical features to obtain corresponding multiple evaluation feature vectors; Multiple evaluation feature vectors corresponding to multiple image blocks are concatenated to obtain a second feature vector of the input image.

7. The unsupervised image quality assessment method based on denoising diffusion probability model according to claim 1 is characterized in that: The pre-trained regression model is specifically a support vector machine (SVM); and inputting the second feature vector into the pre-trained regression model to obtain the third score specifically includes: The second eigenvector is input into a support vector machine (SVM) for regression to obtain an evaluation result of the image quality of the input image, and the evaluation result of the image quality of the input image is used as the third score.

8. The unsupervised image quality assessment method based on denoising diffusion probability model according to claim 1, characterized in that: The denoising diffusion probability model construction process specifically includes: Constructing a denoising diffusion probability model, wherein the denoising diffusion probability model includes forward diffusion and backward denoising; The forward diffusion includes: simulating the process of data being gradually covered by noise by Midsampling, iteratively adds noise to a given input image; The backward denoising includes: simulating the process of restoring original data from noisy data, and performing data restoration on the input image after adding noise to generate a corresponding reconstructed image.

9. The unsupervised image quality assessment method based on denoising diffusion probability model according to claim 8, characterized in that: The training process of the denoising diffusion probability model specifically includes: Obtain an image dataset and input the image dataset into a pre-trained convolutional neural network (CNN) to extract the preliminary feature vector of the image. ,in, A feature vector representing each image; The initial feature vector Input into the constructed denoising diffusion probability model to obtain a one-to-one corresponding reconstructed image data set; Based on reconstructed image dataset, image dataset and loss function Iteratively optimize the weight parameters of the constructed denoising diffusion probability model to obtain a trained denoising diffusion probability model; The loss function Specifically: ; in, is the reconstructed image, is the input image; Represents the loss function in the form of mean square error MSE; represents the Euclidean distance; is the expected computation of the reconstructed image and the input image; yes Information about the image at the moment.

10. The unsupervised image quality assessment method based on denoising diffusion probability model according to claim 1, characterized in that: Also includes: S6. Obtain a data set of images to be tested, evaluate multiple quality scores corresponding to multiple images to be tested in the data set of images to be tested according to steps S1-S5, and sort the multiple images to be tested in descending order of quality scores and then output them.

11. An unsupervised image quality assessment system based on a denoising diffusion probability model, characterized in that: include: A data acquisition module is used to acquire an input image and input the input image into a pre-trained denoising diffusion probability model to obtain a corresponding reconstructed image; A first processing module, configured to perform first preprocessing on the input image and the reconstructed image respectively to obtain corresponding first feature vectors and reconstructed feature vectors, and calculate a first score related to the quality of the input image according to the first feature vectors and the reconstructed feature vectors; A second processing module is used to analyze the structural information corresponding to the input image and the reconstructed image, and calculate the degree of structural information preservation using the structural information corresponding to the input image and the reconstructed image to obtain a second score; A third processing module, configured to perform a second preprocessing on the input image to obtain a second feature vector, and input the second feature vector into a pre-trained regression model to obtain a third score; The output module is used to calculate a quality score of the input image based on the first score, the second score and the third score.

12. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 10 when executed by a processor.

14. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 10 when being executed by a processor.

Images