A no-reference image quality assessment method based on generated noise estimation
By using a method based on generative noise estimation and employing neural network and diffusion models to simulate image quality degradation, the problem of accuracy in evaluation of image quality in complex scenes without reference is solved, and more accurate image quality assessment is achieved.
Patent Information
- Application Number
- CN202510077898.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-17
- Publication Date
- 2025-11-11
- Estimated Expiration
- 2045-01-17
AI Technical Summary
Existing methods for evaluating the quality of no-reference images are not accurate enough in complex scenes and cannot effectively reflect human visual perception.
A generative noise estimation-based approach is adopted. The first neural network model is trained to generate images with different levels of noise, and the second neural network model is used for feature extraction and quality assessment to simulate the image quality degradation process. The loss function is optimized by combining a diffusion model and a variational autoencoder to generate pseudo-reference images.
It improves the accuracy of quality assessment for no-reference images, and can more comprehensively simulate the quality degradation of images in complex environments, reflecting human visual perception.
Smart Images
Figure CN119992299B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing technology, and more specifically to a method for evaluating the quality of no-reference images based on generation noise estimation. Background Technology
[0002] Image quality assessment is a fundamental problem in image processing and computer vision, widely applied in image / video coding, super-resolution reconstruction, and image / video visual quality enhancement. This task aims to objectively evaluate image perceptual quality by establishing a model that simulates the human visual system's perception mechanism and using digital computation to measure the overall image quality.
[0003] No-reference image quality assessment (also known as blind image quality assessment) is an important objective quality assessment method because it does not require the original reference image. This method has broad application prospects, especially in situations where the original image is unavailable, such as satellite imagery and surveillance video. No-reference image quality assessment can effectively evaluate image quality and help improve image processing algorithms and systems.
[0004] A deep learning-based no-reference image quality assessment method has been proposed. This method constructs a deep neural network to learn the visual features of an image to build an image quality assessment model, or directly learns a function expression of image quality from a distorted image end-to-end. However, the accuracy of the assessment may be affected by the lack of an original reference image. Furthermore, variations in image content also pose challenges to the assessment.
[0005] Therefore, improving the accuracy of evaluation in complex scenarios and exploring new feature extraction methods to better reflect human visual perception are problems that urgently need to be solved by those skilled in the art. Summary of the Invention
[0006] In view of this, the present invention provides a no-reference image quality assessment method based on generated noise estimation, which can accurately simulate the degradation of distorted images and learn image distortion, thereby improving the accuracy of the assessment.
[0007] To achieve the above objectives, the present invention adopts the following technical solution:
[0008] A method for evaluating the quality of a no-reference image based on generation noise estimation includes the following steps:
[0009] Train the first neural network model and the second neural network model.
[0010] After the distorted image is input into the trained first neural network model, different levels of noise images are generated.
[0011] The images with different levels of noise are input into a trained second neural network model to generate image quality scores.
[0012] The training steps of the first neural network model include: acquiring multiple distorted image samples; inputting the distorted image samples into an encoder to map the distorted image features into a latent space to form latent variables; sampling in the latent space and generating noise images of different levels through a decoder; inputting the distorted images into a diffusion model for degradation repair and generating a pseudo-reference image; superimposing the pseudo-reference image and the noise image, and calculating a loss function with the distorted image for parameter optimization.
[0013] Preferably, the first neural network model includes an acquisition module, a variational autoencoder, and a diffusion model. The variational autoencoder and the diffusion model are respectively connected to the acquisition module, and the distorted image is acquired through the acquisition module.
[0014] Preferably, the latent vector follows a normal distribution, and different levels of noise images are generated by adjusting the sampling range by setting different standard deviation parameters.
[0015] Preferably, the pseudo-reference image and the noisy image are superimposed, specifically by superimposing the noise images of different levels, which are linearly weighted according to the noise level, with the pseudo-reference image recovered by the diffusion model to obtain a noisy degraded image.
[0016] Preferably, the second neural network model includes an input layer, a feature extraction module, a pooling layer, and a fully connected layer connected in sequence; the input layer superimposes noise images of different levels and then inputs them to the feature extraction module.
[0017] Preferably, the feature extraction module is the feature extraction part of Vision Transformer; after feature extraction, multiple blocks are generated, which are then averaged through the pooling layer and combined into a global feature vector. Finally, the global feature vector is mapped to a single scalar through the fully connected layer to obtain the image quality score.
[0018] Preferably, the training steps of the second neural network model include:
[0019] Calculate the quality scores of the noisy and distorted images separately, calculate the model loss, and optimize the parameters.
[0020] A no-reference image quality assessment system based on generative noise estimation includes a model training module, an image quality assessment module, an interface module, and a storage module.
[0021] The interface module is used to receive training data or distorted data to be evaluated.
[0022] The model training module is used to construct the first neural network model and the second neural network model, and to train the model based on the training data;
[0023] The image quality evaluation module is used to perform corresponding image processing on the distorted data to be evaluated based on the trained neural network model, and obtain the image quality score.
[0024] The storage module is used to store the trained model parameters.
[0025] A computer-readable storage medium storing a computer program, which, when executed by the computer, is used for the image quality evaluation method described above.
[0026] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a no-reference image quality assessment method based on generated noise estimation, which can learn the noise distribution from the distorted image to generate a noisy image to simulate the image quality degradation due to noise. It can more comprehensively simulate the quality degradation process of the image in various complex environments, thereby improving the final assessment accuracy. Attached Figure Description
[0027] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0028] Figure 1 This is a schematic diagram of a no-reference image quality assessment method based on generation noise estimation provided by the present invention.
[0029] Figure 2 This is a schematic diagram of a no-reference image quality assessment system based on generator noise estimation, provided as an embodiment of the present invention. Detailed Implementation
[0030] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0031] Example 1
[0032] like Figure 1This invention discloses a method for evaluating the quality of a no-reference image based on generated noise estimation, comprising the following steps:
[0033] S1: Train the first neural network model and the second neural network model. The training steps for the first neural network model include:
[0034] S11: Obtain multiple distorted image samples.
[0035] S12: Input the distorted image samples into the encoder to map the distorted image features into the latent space, forming latent variables.
[0036] S13: Sampling is performed in the latent space, and different levels of noise images are generated by the decoder.
[0037] S14: Input the distorted image into the diffusion model for degradation repair and generate a pseudo-reference image.
[0038] S15 overlays the pseudo-reference image and the noisy image to obtain a pseudo-distorted image, and then calculates the loss function based on the pseudo-distorted image and the distorted image to optimize the parameters.
[0039] S2: After inputting the distorted image into the trained first neural network model, different levels of noise images are generated.
[0040] S3: Input images with different levels of noise into the trained second neural network model to generate image quality scores.
[0041] In this embodiment, the diffusion model uses a forward process to gradually add noise to the original distorted image until the image becomes a noise distribution, and then uses a reverse process to gradually remove noise, generating a pseudo-reference image that is as close as possible to the original image.
[0042] A noisy image matching the characteristics of the distorted image is generated based on the distorted image using an encoder-decoder architecture to simulate the interference that the image may face under different complex environments. This noisy image is then superimposed on a pseudo-reference image generated by a diffusion model to simulate image quality degradation. This simulation method can reflect the image's sensitivity to different types of noise. Finally, a second neural network model is used to evaluate the quality score.
[0043] To further implement the above technical solution, the distorted image samples during the training process are images under the influence of different types of distortion {I}. d The distorted image is then uniformly resized to a specific pixel size, such as 224×224, using image processing software or libraries. This step ensures consistency in subsequent processing and the validity of comparisons.
[0044] The distorted image is formed from a sharp reference image through degradation processing, a process that can be described as follows:
[0045] y = x + ε
[0046] Where y represents the distorted image, x represents the original image, and ε represents the combined distortion factor.
[0047] To further implement the above technical solution, the first neural network model includes an acquisition module, a variational autoencoder, and a diffusion model. The variational autoencoder and the diffusion model are respectively connected to the acquisition module, and the distorted image is acquired through the acquisition module.
[0048] Specifically, the image processing process of a variational autoencoder (VAE) can be described as follows:
[0049] I Ni =decoder(φ(encoder(I d )))
[0050] Among them, I Ni This represents the generated noisy image, where decoder is the decoder, φ is the latent space, and encoder is the encoder.
[0051] Furthermore, suppose the prior distribution of the VAE, i.e., the latent variable Z, follows a standard normal distribution Z ~ N(0,I), where I is the identity matrix, representing that the variance of each dimension is 1. During the generation phase, Z is randomly sampled from this distribution, as shown in the formula:
[0052] Z=μ+∈*σ
[0053] Here, ∈ represents random noise sampled from a standard normal distribution. By adjusting the standard deviation σ of the normal distribution, the sampling of the latent variable Z deviates from the original noise-free region. For example, when σ = 0.5, the decoder generates a result close to the original image with only slight distortion. When σ = 2, the decoder generates an image with significant noise, and image details may be significantly distorted. The sampled noise estimation data is input into the decoder, which attempts to map it back to the image space, thereby generating a series of images with different noise levels.
[0054] To further implement the above technical solution, the pseudo-reference image and the noisy image are superimposed. Specifically, the noise images of different levels are linearly weighted and then superimposed with the pseudo-reference image recovered by the diffusion model to obtain a noisy degraded image. The expression is as follows:
[0055] I dis' =I r +α*I noise
[0056] Among them, Idis' For degraded images, I r As a pseudo-reference image, I noise The image shows the noise image after linear superposition, where α represents the factor controlling the noise intensity.
[0057] In this embodiment, during linear superposition, i.e., linear weighting, the weight coefficient of each noise image is determined by its own noise level. Since this embodiment controls the noise level by changing the size of the standard deviation, the standard deviation of each noise image can be normalized to obtain the weight coefficient of each noise image.
[0058] To further implement the above technical solution, the second neural network model includes an input layer, a feature extraction module, a pooling layer, and a fully connected layer connected in sequence.
[0059] Specifically, the input layer linearly weights and superimposes noise images of different levels before inputting them into the feature extraction module. The feature extraction module uses the feature extraction part of Vision Transformer, which is the part remaining after removing the classification head. After the feature extraction module outputs multiple patches, they are averaged and combined into a global feature vector through the pooling layer. Finally, the global feature vector is mapped to a single scalar through a fully connected layer to obtain the image quality score.
[0060] To further implement the above technical solution and improve the network's resilience, I is calculated. dis' We use the mean absolute error (MAE) as the loss between the original distorted image and the reconstructed degraded image I. dis' The loss function between them can be described as:
[0061]
[0062] Where n represents the number of samples. Indicates a distorted image. This represents the corresponding pseudo-distorted image.
[0063] Furthermore, the mean absolute error is chosen as the loss function L for noisy image quality estimation. s :
[0064]
[0065] in, Indicates the MOS value of the distorted image. This represents the prediction score for a noisy image.
[0066] Example 2
[0067] like Figure 2Based on the same inventive concept, this invention discloses a referenceless image quality assessment system based on generative noise estimation. The system adopts the image quality assessment method in Embodiment 1 and includes: a model training module, an image quality assessment module, an interface module, and a storage module.
[0068] The interface module receives training data or distorted data to be evaluated. The model training module constructs a first neural network model and a second neural network model, and trains the models based on the training data. The image quality evaluation module performs corresponding image processing on the distorted data to be evaluated based on the trained neural network models to obtain image quality scores. The storage module stores the trained model parameters.
[0069] In this embodiment, the interface module includes an input interface and an output interface. The input interface provides a user interface for uploading images that need to be evaluated for quality. This can be a file upload button on a webpage or a file selection dialog box in a desktop application. The output interface displays the predicted quality score to the user. On a webpage, the score can be displayed using text boxes, progress bars, etc.; similarly, the evaluation results can be presented intuitively in a desktop application.
[0070] Example 3
[0071] Based on the same inventive concept, this invention discloses a computer-readable storage medium storing a computer program that, when executed by a computer, is used for the image quality evaluation method in Embodiment 1.
[0072] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.
[0073] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method for evaluating the quality of a no-reference image based on generative noise estimation, characterized in that, Includes the following steps: Train the first neural network model and the second neural network model; After the distorted image is input into the trained first neural network model, different levels of noise images are generated; The images with different levels of noise are input into a trained second neural network model to generate image quality scores. The training steps of the first neural network model include: Acquire multiple distorted image samples; The distorted image samples are input into the encoder to map the distorted image features into the latent space, forming latent variables; Sampling is performed in the latent space, and different levels of noise images are generated by the decoder; The distorted image is input into a diffusion model for degradation repair, and a pseudo-reference image is generated. The pseudo-reference image and the noisy image are superimposed, and the loss function is calculated and optimized by combining them with the distorted image.
2. The method for evaluating the quality of a no-reference image based on generative noise estimation according to claim 1, characterized in that, The first neural network model includes an acquisition module, a variational autoencoder, and a diffusion model. The variational autoencoder and the diffusion model are respectively connected to the acquisition module, and the distorted image is acquired through the acquisition module.
3. The method for evaluating the quality of a no-reference image based on generative noise estimation according to claim 2, characterized in that, The latent variables follow a normal distribution. By setting different standard deviation parameters to adjust the sampling range, different levels of noise images are generated.
4. The method for evaluating the quality of a no-reference image based on generative noise estimation according to claim 1, characterized in that, The pseudo-reference image and the noisy image are superimposed, specifically by superimposing the noise images of different levels, which are linearly weighted according to the noise level, with the pseudo-reference image recovered by the diffusion model, to obtain a noisy degraded image.
5. The method for evaluating the quality of a no-reference image based on generative noise estimation according to claim 1, characterized in that, The second neural network model includes an input layer, a feature extraction module, a pooling layer, and a fully connected layer connected in sequence; the input layer superimposes noise images of different levels and then inputs them into the feature extraction module.
6. The method for evaluating the quality of a no-reference image based on generative noise estimation according to claim 5, characterized in that, The feature extraction module is the feature extraction part of Vision Transformer. After feature extraction, multiple blocks are generated, which are then averaged through the pooling layer and combined into a global feature vector. Finally, the fully connected layer maps the global feature vector into a single scalar to obtain the image quality score.
7. The method for evaluating the quality of a no-reference image based on generative noise estimation according to claim 1, characterized in that, The training steps for the second neural network model include: Calculate the quality scores of the noisy and distorted images separately, calculate the model loss, and optimize the parameters.
8. A no-reference image quality assessment system based on generative noise estimation, characterized in that, The image quality assessment method according to any one of claims 1-7 is adopted, including a model training module, an image quality assessment module, an interface module, and a storage module; The interface module is used to receive training data or distorted data to be evaluated; The model training module is used to construct the first neural network model and the second neural network model, and to train the model based on the training data; The image quality evaluation module is used to perform corresponding image processing on the distorted data to be evaluated based on the trained neural network model, and obtain the image quality score. The storage module is used to store the trained model parameters.
9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by the computer, is used to implement the image quality evaluation method according to any one of claims 1-7.