Unsupervised image quality evaluation method and system based on de-noising diffusion probability model, electronic equipment and storage medium

Abstract

The invention discloses an unsupervised image quality evaluation method and system based on a de-noising diffusion probability model, electronic equipment and a storage medium, and the method comprises the steps: obtaining an input image, and inputting the input image into a pre-trained de-noising diffusion probability model to obtain a corresponding reconstructed image; performing first preprocessing on the input image and the reconstructed image to obtain a corresponding first feature vector and a reconstructed feature vector, and calculating a first score related to the quality of the input image according to the first feature vector and the reconstructed feature vector; analyzing structure information corresponding to the input image and the reconstructed image, and calculating a structure information retention degree by using the structure information corresponding to the input image and the reconstructed image to obtain a second score; and performing second preprocessing on the input image to obtain a second feature vector, and inputting the second feature vector into a pre-trained regression model to obtain a third score. According to the method, manual labeling is not needed, and evaluation comprehensiveness and accuracy are improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an unsupervised image quality assessment method, system, electronic device and storage medium based on a denoising diffusion probability model. Background Art

[0002] Image quality assessment is an important research area that aims to automatically evaluate the quality of images. However, image quality assessment without labeled labels remains a difficult task due to the different content and distortion of images. Current image quality assessment methods mainly rely on supervised learning, which requires a large amount of labeled data, is costly and difficult to extend to large-scale unlabeled datasets. Traditional unsupervised methods are not robust enough in complex image scenarios and cannot effectively distinguish quality differences. Summary of the invention

[0003] In order to solve the technical problems existing in the background technology, the present invention proposes an unsupervised image quality evaluation method based on a denoising diffusion probability model.

[0004] In a first aspect, the present invention proposes an unsupervised image quality assessment method based on a denoising diffusion probability model, comprising: Obtain an input image, and input the input image into a pre-trained denoising diffusion probability model to obtain a corresponding reconstructed image; Performing first preprocessing on the input image and the reconstructed image respectively to obtain corresponding first feature vectors and reconstructed feature vectors, and calculating a first score related to the quality of the input image according to the first feature vectors and the reconstructed feature vectors; Analyzing the structural information corresponding to the input image and the reconstructed image, and calculating the degree of structural information preservation using the structural information corresponding to the input image and the reconstructed image to obtain a second score; Performing a second preprocessing on the input image to obtain a second feature vector, and inputting the second feature vector into a pre-trained regression model to obtain a third score; A quality score of the input image is calculated based on the first score, the second score, and the third score.

[0005] Preferably, calculating the quality score of the input image based on the first score, the second score and the third score specifically includes: The first score and the third score are reverse normalized, and then the reverse normalized first score and the third score and the second score are used to calculate the quality score of the input image according to the following formula: ; in, It is the first score after reverse normalization; The third score is after reverse normalization; Rating for the second; Score the quality of the input image; is the weight of the first score after reverse normalization; The weight of the second score; is the weight of the third score after reverse normalization; , and satisfy .

[0006] Preferably, the first preprocessing specifically includes: The input image and the reconstructed image are subjected to denoising, cropping, grayscale conversion and normalization processing respectively, so as to obtain the input image and the reconstructed image after eliminating noise interference and standardizing the data format; The input image and the reconstructed image after noise interference is eliminated and the data format is standardized are respectively input into the pre-trained convolutional neural network to extract feature vectors to obtain the corresponding first feature vector and reconstructed feature vector.

[0007] Preferably, calculating a first score related to the quality of the input image according to the first feature vector and the reconstructed feature vector specifically includes: Calculate the reconstruction error of the image in the denoising diffusion probability model according to the first eigenvector and the reconstruction eigenvector; The reconstruction error is used as a first score for evaluating image quality, so as to obtain a first score related to the quality of the input image.

[0008] Preferably, the structural information includes but is not limited to average brightness, image variance, image covariance and brightness comparison items; the method of calculating the degree of structural information preservation using the structural information corresponding to the input image and the reconstructed image to obtain the second score is specifically as follows: calculating the degree of preservation of the structural information of the reconstructed image relative to the input image according to the following formula, and taking the degree of preservation of the structural information as the second score; ; in, is the average brightness of the input image; is the average brightness of the reconstructed image; is the variance of the input image; is the variance of the reconstructed image; is the covariance of the input image and the reconstructed image; and is a small constant added to avoid the denominator being zero, where and is an adjustable parameter, is the range of pixel values, for example, for an 8-bit image, , is the comparison item of the brightness of the input image; is a comparison term for the brightness of the reconstructed image; Rated second.

[0009] Preferably, the second preprocessing specifically includes: After adjusting the image size and brightness balance of the input image, the adjusted input image is divided into a plurality of image blocks; Extract statistical features from multiple image blocks one by one, and perform evaluation feature conversion and normalization processing on the extracted statistical features to obtain corresponding multiple evaluation feature vectors; The plurality of evaluation feature vectors corresponding to the plurality of image blocks are concatenated to obtain a second feature vector of the input image.

[0010] Preferably, the pre-trained regression model is specifically a support vector machine (SVM); and inputting the second feature vector into the pre-trained regression model to obtain the third score specifically includes: The second eigenvector is input into a support vector machine (SVM) for regression to obtain an evaluation result of the image quality of the input image, and the evaluation result of the image quality of the input image is used as the third score.

[0011] Preferably, the denoising diffusion probability model construction process specifically includes: Constructing a denoising diffusion probability model, wherein the denoising diffusion probability model includes forward diffusion and backward denoising; The forward diffusion includes: simulating the process of data being gradually covered by noise by Midsampling, iteratively adds noise to a given input image; The backward denoising includes: simulating the process of restoring original data from noisy data, performing data restoration on the input image after adding noise to generate a corresponding reconstructed image.

[0012] Preferably, the training process of the denoising diffusion probability model specifically includes: Obtain an image dataset and input the image dataset into a pre-trained convolutional neural network (CNN) to extract the preliminary feature vector of the image. ,in, A feature vector representing each image; The initial feature vector Input into the constructed denoising diffusion probability model to obtain a one-to-one corresponding reconstructed image data set; Based on reconstructed image dataset, image dataset and loss function Iteratively optimize the weight parameters of the constructed denoising diffusion probability model to obtain a trained denoising diffusion probability model; The loss function Specifically: ; in, is the reconstructed image, is the input image; Represents the loss function in the form of mean square error MSE; represents the Euclidean distance; is the desired computation of the reconstructed image and the input image.

[0013] Preferably, it also includes: S6. Obtain a data set of images to be tested, evaluate multiple quality scores corresponding to multiple images to be tested in the data set of images to be tested according to steps S1-S5, and sort the multiple images to be tested in descending order of quality scores and then output them.

[0014] In a second aspect, the present invention proposes an unsupervised image quality assessment system based on a denoising diffusion probability model, comprising: A data acquisition module is used to acquire an input image and input the input image into a pre-trained denoising diffusion probability model to obtain a corresponding reconstructed image; A first processing module, configured to perform first preprocessing on the input image and the reconstructed image respectively to obtain corresponding first feature vectors and reconstructed feature vectors, and calculate a first score related to the quality of the input image according to the first feature vectors and the reconstructed feature vectors; A second processing module is used to analyze the structural information corresponding to the input image and the reconstructed image, and calculate the degree of structural information preservation using the structural information corresponding to the input image and the reconstructed image to obtain a second score; A third processing module, configured to perform a second preprocessing on the input image to obtain a second feature vector, and input the second feature vector into a pre-trained regression model to obtain a third score; The output module is used to calculate a quality score of the input image based on the first score, the second score and the third score.

[0015] In a third aspect, the present application provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method as described in any one of the first aspects.

[0016] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method as described in any one of the first aspects.

[0017] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the method as described in any one of the first aspects.

[0018] In the present invention, the proposed unsupervised image quality assessment method based on the denoising diffusion probability model extracts features through a pre-trained convolutional neural network (CNN), combines the forward noise addition and reverse reconstruction process of the diffusion model, quantifies the reconstruction error, structural similarity and spatial domain statistical features, and fuses them to generate a comprehensive quality score. This method does not require manual labeling, which improves the comprehensiveness and accuracy of the evaluation. The embedding stability is analyzed through the diffusion process, which is convenient for adapting to complex scenes and various distortion types. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A schematic diagram of the work flow structure of an unsupervised image quality assessment method based on a denoising diffusion probability model proposed in the present invention; Figure 2 A schematic diagram of an implementation process of an unsupervised image quality assessment method based on a denoising diffusion probability model proposed by the present invention; Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0020] Reference Figure 1 and Figure 2 The present invention proposes an unsupervised image quality assessment method based on a denoising diffusion probability model, comprising the following steps: S1. Obtain an input image and input the input image into a pre-trained denoising diffusion probability model to obtain a corresponding reconstructed image.

[0021] In this embodiment, the denoising diffusion probability model (DDPM) construction process specifically includes: constructing a denoising diffusion probability model, the denoising diffusion probability model includes forward diffusion and backward denoising; forward diffusion includes: simulating the process of data being gradually covered by noise, by The method further comprises the following steps: sampling, iteratively adding noise to a given input image; backward denoising includes: simulating the process of recovering the original data from the noisy data, performing data recovery on the input image after adding the noise, so as to generate the corresponding reconstructed image.

[0022] Specifically, the denoising diffusion probability model DDPM has the following two advantages. The first is good robustness, because higher quality images have a stable representation in the embedding space of a given image classification model and are less affected by the perturbations introduced by the forward diffusion process than lower quality images. The second is the difference in reconstruction quality. Compared with low-quality images, high-quality samples are easier to reconstruct from partially damaged (noise) data with incomplete identity information, and the difference between the input embedding and the denoised samples is smaller. Therefore, the present application proposes a method to analyze the embedding stability of given images by perturbing them through forward and backward diffusion processes, and then quantify the results of quality estimation. By simultaneously utilizing the forward (noise) and backward (reconstruction) diffusion steps, the quality of a given input sample is accurately evaluated.

[0023] Construct a denoising diffusion probability model. The denoising diffusion probability model represents a special type of generative model that learns to model the distribution of (image) data through two types of processes, namely the forward (noise) process and the backward (denoising) process. The forward diffusion process simulates the process in which data is gradually covered by noise by learning from a Gaussian distribution. Iteratively adds noise to a given input image , the entire forward process can be presented as a Markov chain: ; in, From the sequence The number of time steps chosen in is the variance parameter, which indicates the time at which the forward process How much noise is added to the image samples. Input samples To any sample The process is expressed as: ; in, .

[0024] The reverse diffusion process simulates the process of recovering the original data from the noisy data. is given and Under the conditions, The conditional distribution of . The formula is as follows: ; in, At a known moment Data and initial data Under the condition, from the moment Restore to time Data The probability distribution of , the right side shows that this probability distribution is a multivariate Gaussian distribution. Among them, From the sequence The number of time steps in ; The vector is the last moment to be restored The data represents the state of the data at that moment; Vector represents the current time Noisy data; It is the original noise-free data, which is the data that we ultimately want to recover; is the mean function of the Gaussian distribution, which is about (currently noisy data), (raw data) and time The function of this mean function determines the Restore to central tendency of the time; is the variance function of the Gaussian distribution, which is about , and time A function of , which measures the uncertainty in the recovery process; is the identity matrix, and Multiplication indicates that the covariance matrix is ​​a diagonal matrix, which means that the dimensions of the data are independent of each other. It is the identity matrix, and Multiplication indicates that the covariance matrix is ​​a diagonal matrix, which means that the dimensions of the data are independent of each other.

[0025] in, From the sequence The number of time steps in , ; is the mean function of the Gaussian distribution, which is about (currently noisy data), (raw data) and time The function of this mean function determines the Restore to central tendency of the time; Vector is the moment The noisy data represents the state of the data at that moment; The vector represents the original noise-free data, which is the data that we ultimately want to recover; and These are parameters related to the diffusion process, usually related to the gradual addition of noise. Their values ​​are between 0 and 1, reflecting the degree of noise addition at different times. Corresponding time , Corresponding time ; yes From the initial moment to the moment The cumulative product form is , used to describe the time from the beginning to the The combined effects of noise accumulation; It is also a parameter in the diffusion process, usually Related, indicating that at the time The amount of noise added and other related information also have values ​​between 0 and 1; Represents the parameters of the model, which are learned during the training of the diffusion model and are used to adjust the calculation method of the mean to better recover the data.

[0026] In this embodiment, the training process of the denoising diffusion probability model specifically includes: obtaining an image data set, inputting the image data set into a pre-trained convolutional neural network CNN to extract a preliminary feature vector of the image ,in, Represents the feature vector of each image; the preliminary feature vector Input into the constructed denoising diffusion probability model to obtain a one-to-one corresponding reconstructed image dataset; based on the reconstructed image dataset, image dataset and loss function The weight parameters of the constructed denoising diffusion probability model are iteratively optimized to obtain a trained denoising diffusion probability model.

[0027] In this embodiment, the loss function Specifically: ; in, is the reconstructed image; is the input image; Represents the loss function in the form of mean square error MSE; represents the Euclidean distance; is the expected computation of the reconstructed image and the input image, yes Information about the image at the moment.

[0028] S2. Perform first preprocessing on the input image and the reconstructed image respectively to obtain corresponding first feature vectors and reconstructed feature vectors, and calculate a first score related to the quality of the input image according to the first feature vectors and the reconstructed feature vectors.

[0029] In this embodiment, the first preprocessing specifically includes: denoising, cropping, grayscale conversion, and normalization of the input image and the reconstructed image, respectively, to obtain the input image and the reconstructed image after noise interference is eliminated and the data format is standardized; the input image and the reconstructed image after noise interference is eliminated and the data format is standardized are respectively input into a pre-trained convolutional neural network to extract feature vectors, to obtain the corresponding first feature vector and the reconstructed feature vector.

[0030] In this embodiment, a first score related to the quality of the input image is calculated based on the first eigenvector and the reconstructed eigenvector, specifically including: calculating the reconstruction error of the image in the denoising diffusion probability model based on the first eigenvector and the reconstructed eigenvector; using the reconstruction error as the first score for evaluating the image quality to obtain a first score related to the quality of the input image.

[0031] Specifically, since the denoising diffusion process of low-quality images is more sensitive to the image perturbations introduced by the forward and backward diffusion processes than that of high-quality images, the image quality is evaluated by calculating the reconstruction error of the image in the denoising diffusion model. Specifically, the quality score of each image is calculated by the following formula : ; in, is an image The feature vector reconstructed in the denoising diffusion process. The intermediate feature state obtained through feature extraction is used to calculate the image quality evaluation index. is an image The original feature vector.

[0032] S3. Analyze the structural information corresponding to the input image and the reconstructed image, and calculate the degree of structural information preservation using the structural information corresponding to the input image and the reconstructed image to obtain a second score.

[0033] In this embodiment, the structural information includes but is not limited to average brightness, image variance, image covariance, and brightness comparison items; the degree of structural information preservation is calculated using the structural information corresponding to the input image and the reconstructed image to obtain a second score, specifically: the degree of preservation of the structural information of the reconstructed image relative to the input image is calculated according to the following formula, and the degree of preservation of the structural information is used as the second score; ; in, is the average brightness of the input image; is the average brightness of the reconstructed image; is the variance of the input image; is the variance of the reconstructed image; is the covariance of the input image and the reconstructed image; and is a small constant added to avoid the denominator being zero, where and is an adjustable parameter, is the range of pixel values, for example, for an 8-bit image, , is the comparison item of the brightness of the input image; is a comparison term for the brightness of the reconstructed image; Rated second.

[0034] Specifically, Structural Similarity (SSIM) is used to evaluate the degree of preservation of the structural information of the image, taking into account the brightness, contrast and structural information of the image. The index ranges from -1 to 1, where 1 means the two images are identical and 0 means they are completely different.

[0035] S4. Perform a second preprocessing on the input image to obtain a second feature vector, and input the second feature vector into a pre-trained regression model to obtain a third score.

[0036] In this embodiment, the second preprocessing specifically includes: after adjusting the image size and brightness balance of the input image, dividing the adjusted input image into multiple image blocks; extracting statistical features of the multiple image blocks one by one, and performing evaluation feature conversion and normalization processing on the extracted statistical features to obtain corresponding multiple evaluation feature vectors; and performing feature vector splicing on the multiple evaluation feature vectors corresponding to the multiple image blocks to obtain the second feature vector of the input image.

[0037] Specifically, the spatial domain image quality assessment algorithm Brisque is used to evaluate the input image. The specific steps are as follows: (1) Feature normalization is to normalize the transformed features and scale them to the range between 0 and 1.

[0038] Local brightness normalization (MSCN) coefficient calculation: ; in, is the calculated local brightness normalization (MSCN) coefficient, is the image at coordinates The pixel value at is the local mean, is the local standard deviation, is a constant (usually 1) used for numerical stability.

[0039] Statistical feature extraction specifically includes: extracting multiple statistical features from each image block, including mean, standard deviation, etc.

[0040] Among them, the local mean calculation formula is: ; Represents the coordinates in the image The local mean at It is the position coordinate of the pixel point in the two-dimensional plane of the image; Represents the row index; Column index; and is a parameter used to define the neighborhood range, Determines the width of the neighborhood in the horizontal direction, The height of the neighborhood in the vertical direction is determined; is the weight coefficient, and the corresponding coordinates in the neighborhood are The weight of the position; The coordinates in the image are The pixel value of the neighborhood relative to the central pixel Offset The pixel value at the position; through double summation, traversing The pixels within the entire neighborhood centered at , and calculate their weighted sum according to the weights.

[0041] (2) Statistical characteristic standard deviation, calculated as follows: ; in, is the Gaussian kernel, Represents the coordinates in the image The standard deviation of the neighborhood is the position of the pixel in the two-dimensional plane of the image; and Used to define the neighborhood range. Determine the width of the neighborhood in the horizontal direction, Determine the height of the neighborhood in the vertical direction; is the weight coefficient, and the corresponding coordinates in the neighborhood are The weight of the position; The coordinates in the image are The pixel value of the neighborhood relative to the center pixel Offset The pixel value at the position; is the coordinate in the image The local mean at a point is usually obtained by weighted summing of the pixel values ​​in the neighborhood of the point, reflecting the average pixel value in the local area.

[0042] (3) Use the generalized Gaussian distribution (GGD) to model the MSCN coefficient histogram and extract the features of the fitted Gaussian distribution as the image quality assessment. The feature generalized Gaussian distribution (GGD) model is as follows: ; in, , for , is the probability density function of the generalized Gaussian distribution in The value at which the random variable The relative probability of taking this value; It is a random variable, representing the data to be analyzed, which can be various data types such as image feature values, signal values, etc. The shape parameter determines the shape of the distribution. When , the generalized Gaussian distribution degenerates into the standard Gaussian distribution; when When , the tail of the distribution is thicker than that of the standard Gaussian distribution, which means that the probability of extreme values ​​is relatively large; when When , the tail of the distribution is thinner than that of the standard Gaussian distribution, and the probability of extreme values ​​appearing is smaller. is the variance parameter, which measures the degree of dispersion of the data and reflects the dispersion of the data around the mean. The larger the variance, the more dispersed the data; the smaller the variance, the more concentrated the data; and and Related parameters, , which is involved in adjusting the scale of the distribution. is a gamma function to ensure the normalization of the probability density function (i.e., the area under the entire distribution is 1).

[0043] (4) Quality score calculation: Use the pre-trained regression model to obtain the image quality score.

[0044] In this embodiment, the pre-trained regression model is specifically a support vector machine SVM; inputting the second feature vector into the pre-trained regression model to obtain the third score specifically includes: The second eigenvector is input into a support vector machine (SVM) for regression to obtain an evaluation result of the image quality of the input image, and the evaluation result of the image quality of the input image is used as the third score.

[0045] In this embodiment, the specific processing process of the regression model is as follows: Feature vector input: The second feature vector of the image is input to the regression model; Model prediction: SVM calculates the score by calculating the dot product of the second eigenvector and the weight vector plus the bias term; the formula for calculating the score is: ; in, is the rating value, is the second eigenvector, and dimensional vector, i.e. , is the weight vector, and dimensional vector, i.e. , represents the parameters learned by SVM training, is the bias term, which is a scalar.

[0046] Result output: The image quality category is determined based on the score. The category classification formula is expressed as: ; in, It is the category to which the image quality belongs, and the integers 1, 2, and 3 correspond to the category scores in different scoring intervals.

[0047] S5. Calculate a quality score of the input image based on the first score, the second score, and the third score.

[0048] In this embodiment, it specifically includes: The first score and the third score are reverse normalized, and then the reverse normalized first score and the third score and the second score are used to calculate the quality score of the input image according to the following formula: ; in, It is the first score after reverse normalization; The third score is after reverse normalization; Rating for the second; Score the quality of the input image; is the weight of the first score after reverse normalization; The weight of the second score; is the weight of the third score after reverse normalization; , and satisfy .

[0049] It should be noted that due to the reconstruction error of the denoising diffusion model The smaller the better, structural similarity The larger the index, the better. The spatial domain image quality assessment results The smaller the better, so you need to and Perform reverse normalization so that all indicator scores are as high as possible, reverse normalization processing: ; when Take 0.35, Take the sum of 0.4 When the value is 0.25, the quality score formula is: .

[0050] S6. Obtain a data set of images to be tested, evaluate multiple quality scores corresponding to multiple images to be tested in the data set of images to be tested according to steps S1-S5, and sort the multiple images to be tested in descending order of quality scores and then output them.

[0051] In this embodiment, the images in the image data set to be tested are sorted according to the calculated quality scores, and the scoring results and the processed high-quality images are output in a standardized format for easy storage or further application.

[0052] Specifically, according to the calculated quality scores, the image dataset to be tested is sorted and screened to identify high-quality and low-quality images.

[0053] Reference Figure 1 and Figure 2 , an unsupervised image quality assessment system based on denoising diffusion probability model, comprising: A data acquisition module is used to acquire an input image and input the input image into a pre-trained denoising diffusion probability model to obtain a corresponding reconstructed image; A first processing module, configured to perform first preprocessing on the input image and the reconstructed image respectively to obtain corresponding first feature vectors and reconstructed feature vectors, and calculate a first score related to the quality of the input image according to the first feature vectors and the reconstructed feature vectors; A second processing module is used to analyze the structural information corresponding to the input image and the reconstructed image, and calculate the degree of structural information preservation using the structural information corresponding to the input image and the reconstructed image to obtain a second score; A third processing module, configured to perform a second preprocessing on the input image to obtain a second feature vector, and input the second feature vector into a pre-trained regression model to obtain a third score; The output module is used to calculate a quality score of the input image based on the first score, the second score and the third score.

[0054] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 3 The electronic device 20 may include: a memory 21 and a processor 22. Exemplarily, the memory 21 and the processor 22 are interconnected via a bus 23.

[0055] The memory 21 is used to store computer-executable instructions; The processor 22 is used to execute the computer-executable instructions stored in the memory, so as to enable the control device 20 to execute the method shown in the above method embodiment.

[0056] The electronic device provided in the embodiment of the present application can execute the technical solution shown in the above method embodiment, and its implementation principle and beneficial effects are similar, which will not be repeated here.

[0057] An embodiment of the present application provides a computer-readable storage medium, in which computer execution instructions are stored. When the computer execution instructions are executed by a processor, they are used to implement the technical solution shown in the above method embodiment. The implementation principles and beneficial effects are similar and will not be repeated here.

[0058] The embodiment of the present application may also provide a computer program product, including a computer program. When the computer program is executed by a processor, it can implement the technical solution shown in the above method embodiment. Its implementation principle and beneficial effects are similar and will not be repeated here.

[0059] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application.

[0060] It should be further noted that, although the various steps in the flowchart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowchart may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0061] It should be understood that the above-mentioned device embodiments are only illustrative, and the device of the present application can also be implemented in other ways. For example, the division of units / modules in the above-mentioned embodiments is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units, modules or components can be combined, or can be integrated into another system, or some features can be ignored or not executed.

[0062] In addition, unless otherwise specified, each functional unit / module in each embodiment of the present application may be integrated into one unit / module, each unit / module may exist physically separately, or two or more units / modules may be integrated together. The above-mentioned integrated unit / module may be implemented in the form of hardware or in the form of a software program module.

[0063] If the integrated unit / module is implemented in the form of hardware, the hardware may be a digital circuit, an analog circuit, etc. If not otherwise specified, the processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, or discrete hardware components. If not otherwise specified, the storage unit may be any appropriate magnetic storage medium or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random-Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), etc.

[0064] If the integrated unit / module is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a memory, including a number of instructions for a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: U disk, read-only memory (ROM), random access memory (RAM), mobile hard disk, disk or optical disk and other media that can store program codes.

[0065] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. An unsupervised image quality assessment method based on a denoising diffusion probability model, characterized in that: The following steps are involved: S1. Obtain an input image and input the input image into a pre-trained denoising diffusion probability model to obtain a corresponding reconstructed image; S2, performing first preprocessing on the input image and the reconstructed image respectively to obtain corresponding first feature vectors and reconstructed feature vectors, and calculating a first score related to the quality of the input image according to the first feature vectors and the reconstructed feature vectors; S3, analyzing the structural information corresponding to the input image and the reconstructed image, and calculating the degree of structural information preservation using the structural information corresponding to the input image and the reconstructed image to obtain a second score; S4, performing a second preprocessing on the input image to obtain a second feature vector, and inputting the second feature vector into a pre-trained regression model to obtain a third score; S5. Calculate a quality score of the input image based on the first score, the second score, and the third score.

2. The unsupervised image quality assessment method based on denoising diffusion probability model according to claim 1 is characterized in that: The calculating the quality score of the input image based on the first score, the second score and the third score specifically includes: The first score and the third score are reverse normalized, and then the reverse normalized first score and the third score and the second score are used to calculate the quality score of the input image according to the following formula: ; in, It is the first score after reverse normalization; The third score after reverse normalization; Rating for the second; Score the quality of the input image; is the weight of the first score after reverse normalization; The weight of the second score; is the weight of the third score after reverse normalization; , and satisfy .

3. The unsupervised image quality assessment method based on denoising diffusion probability model according to claim 1, characterized in that: The first preprocessing specifically includes: The input image and the reconstructed image are subjected to denoising, cropping, grayscale conversion and normalization processing respectively, so as to obtain the input image and the reconstructed image after eliminating noise interference and standardizing the data format; The input image and the reconstructed image after noise interference is eliminated and the data format is standardized are respectively input into the pre-trained convolutional neural network to extract feature vectors to obtain the corresponding first feature vector and reconstructed feature vector.

4. The unsupervised image quality assessment method based on denoising diffusion probability model according to claim 1, characterized in that: The step of calculating a first score related to the quality of the input image according to the first feature vector and the reconstructed feature vector specifically includes: Calculate the reconstruction error of the image in the denoising diffusion probability model according to the first eigenvector and the reconstruction eigenvector; The reconstruction error is used as a first score for evaluating image quality, so as to obtain a first score related to the quality of the input image.

5. The unsupervised image quality assessment method based on denoising diffusion probability model according to claim 1, characterized in that: The structural information includes but is not limited to average brightness, image variance, image covariance and brightness comparison items; the structural information preservation degree is calculated using the structural information corresponding to the input image and the reconstructed image to obtain the second score, specifically: the preservation degree of the structural information of the reconstructed image relative to the input image is calculated according to the following formula, and the preservation degree of the structural information is used as the second score; ; in, is the average brightness of the input image; is the average brightness of the reconstructed image; is the variance of the input image; is the variance of the reconstructed image; is the covariance of the input image and the reconstructed image; and is a small constant added to avoid the denominator being zero, where and is an adjustable parameter, is the range of pixel values, for example, for an 8-bit image, , is the comparison item of the brightness of the input image; is a comparison term for the brightness of the reconstructed image; Rated second.

6. The unsupervised image quality assessment method based on denoising diffusion probability model according to claim 1, characterized in that: The second preprocessing specifically includes: After adjusting the image size and brightness balance of the input image, the adjusted input image is divided into a plurality of image blocks; Extract statistical features from multiple image blocks one by one, and perform evaluation feature conversion and normalization processing on the extracted statistical features to obtain corresponding multiple evaluation feature vectors; The plurality of evaluation feature vectors corresponding to the plurality of image blocks are concatenated to obtain a second feature vector of the input image.

7. The unsupervised image quality assessment method based on denoising diffusion probability model according to claim 1, characterized in that: The pre-trained regression model is specifically a support vector machine (SVM); and inputting the second feature vector into the pre-trained regression model to obtain the third score specifically includes: The second eigenvector is input into a support vector machine (SVM) for regression to obtain an evaluation result of the image quality of the input image, and the evaluation result of the image quality of the input image is used as the third score.

8. The unsupervised image quality assessment method based on denoising diffusion probability model according to claim 1, characterized in that: The denoising diffusion probability model construction process specifically includes: Constructing a denoising diffusion probability model, wherein the denoising diffusion probability model includes forward diffusion and backward denoising; The forward diffusion includes: simulating the process of data being gradually covered by noise by Midsampling, iteratively adds noise to a given input image; The backward denoising includes: simulating the process of restoring original data from noisy data, performing data restoration on the input image after adding noise to generate a corresponding reconstructed image.

9. The unsupervised image quality assessment method based on denoising diffusion probability model according to claim 8, characterized in that: The training process of the denoising diffusion probability model specifically includes: Obtain an image dataset and input the image dataset into a pre-trained convolutional neural network (CNN) to extract the preliminary feature vector of the image. ,in, A feature vector representing each image; The initial feature vector Input into the constructed denoising diffusion probability model to obtain a one-to-one corresponding reconstructed image data set; Based on reconstructed image dataset, image dataset and loss function Iteratively optimize the weight parameters of the constructed denoising diffusion probability model to obtain a trained denoising diffusion probability model; The loss function Specifically: ; in, is the reconstructed image, is the input image; Represents the loss function in the form of mean square error MSE; represents the Euclidean distance; is the desired computation of the reconstructed image and the input image.

10. The unsupervised image quality assessment method based on denoising diffusion probability model according to claim 1, characterized in that: Also includes: S6. Obtain a data set of images to be tested, evaluate multiple quality scores corresponding to multiple images to be tested in the data set of images to be tested according to steps S1-S5, and sort the multiple images to be tested in descending order of quality scores and then output them.

11. An unsupervised image quality assessment system based on a denoising diffusion probability model, characterized in that: include: A data acquisition module is used to acquire an input image and input the input image into a pre-trained denoising diffusion probability model to obtain a corresponding reconstructed image; A first processing module, configured to perform first preprocessing on the input image and the reconstructed image respectively to obtain corresponding first feature vectors and reconstructed feature vectors, and calculate a first score related to the quality of the input image according to the first feature vectors and the reconstructed feature vectors; A second processing module is used to analyze the structural information corresponding to the input image and the reconstructed image, and calculate the degree of structural information preservation using the structural information corresponding to the input image and the reconstructed image to obtain a second score; A third processing module, configured to perform a second preprocessing on the input image to obtain a second feature vector, and input the second feature vector into a pre-trained regression model to obtain a third score; The output module is used to calculate a quality score of the input image based on the first score, the second score and the third score.

12. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 10 when executed by a processor.

14. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 10 when being executed by a processor.

Images