A blind image quality evaluation method and device based on diffusion repair technology

Abstract

The application discloses a blind image quality evaluation method and equipment based on a diffusion repair technology, and the method comprises the following steps: acquiring a distorted image, and performing blind denoising preprocessing on the distorted image; using a pre-trained stable diffusion model to repair the preprocessed image, and generating a repaired image; using a full-reference image quality evaluation model to evaluate the quality of the repaired image, and obtaining a preliminary quality score; extracting image block features from the repaired image, designing a quality evaluation network based on a VGGNet, and embedding a channel attention module and a spatial attention module; assigning different weights to each image block, calculating the weight and score of the image block, and obtaining the quality score of the overall image. The application can effectively evaluate the image quality without a reference image, has wide application prospects and practical significance, and can be widely applied to the fields of image processing and evaluation.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing and evaluation, and in particular to a blind image quality evaluation method and device based on diffusion inpainting technology. BACKGROUND

[0002] In the digital era, images and videos have become the main medium for information dissemination. Image quality assessment (IQA) technology is crucial for ensuring the quality of multimedia content, optimizing image processing algorithms, and reducing data transmission and storage costs. Image quality assessment methods are mainly divided into full-reference image quality assessment (FR-IQA) and no-reference image quality assessment (NR-IQA).

[0003] Full-reference image quality assessment methods rely on the original undistorted reference image to evaluate the quality of the distorted image. These methods usually provide high accuracy, but in practical applications, reference images are often unavailable, limiting their application range. Therefore, no-reference image quality assessment methods, i.e., blind image quality assessment methods, have attracted widespread attention because they do not rely on reference images. These methods attempt to evaluate the quality of distorted images directly from the distorted images themselves, which is of great significance for real-time video stream quality monitoring, online image sharing platform automatic quality detection, and other scenarios.

[0004] Although no-reference image quality assessment methods have broad application prospects in theory, they still face many challenges in practical applications. Traditional no-reference image quality assessment methods are mainly based on natural scene statistics (NSS) and hand-designed image features. These methods often struggle to achieve satisfactory performance when dealing with complex scenes and multiple distortion types. In addition, these methods often lack in-depth consideration of the characteristics of the human visual system, such as different sensitivities to different image regions.

[0005] In recent years, with the development of deep learning technology, image quality assessment methods based on deep neural networks have made significant progress. These methods can automatically learn complex feature representations from a large amount of image data, improving the accuracy of evaluation. However, existing deep learning-based no-reference image quality assessment methods still have some limitations, such as strong dependence on training data, limited generalization ability for specific distortion types, and the use of generative models in image inpainting tasks that may produce poor quality inpainted images. SUMMARY

[0006] To at least partially solve one of the technical problems existing in the prior art, the purpose of the present application is to provide a blind image quality evaluation method, device and medium based on diffusion repair technology.

[0007] The first technical solution adopted by the present application is:

[0008] A blind image quality evaluation method based on diffusion repair technology, comprising the following steps:

[0009] Obtain a distorted image, and perform blind denoising preprocessing on the distorted image to reduce noise in the image and prepare the image for subsequent diffusion repair;

[0010] Repair the preprocessed image using a pre-trained stable diffusion model to generate a repaired image, which aims to restore the quality of the image and simulate the reference image in full-reference image quality evaluation;

[0011] Use a full-reference image quality evaluation model to evaluate the quality of the repaired image to obtain a preliminary quality score;

[0012] Extract image block features from the repaired image, design a quality evaluation network based on VGGNet, and embed channel attention modules and spatial attention modules to simulate the different sensitivities of the human visual system to different channels and image regions;

[0013] Assign different weights to each image block, calculate the weight and score of the image block, and obtain the overall image quality score.

[0014] Further, the Swin-Conv-UNet network is used for blind denoising preprocessing of the distorted image, which can effectively remove different types of noise to improve the effect of subsequent diffusion repair.

[0015] Further, the pre-trained stable diffusion model is a Stable Diffusion model, which can generate high-quality image repair results and effectively restore the original quality of the image.

[0016] Further, the full-reference image quality evaluation model is a WaDIQaM model, which can provide accurate image quality scores and provide a reference for subsequent diffusion repair.

[0017] Further, the quality evaluation network includes multiple convolution layers, pooling layers, fully connected layers, and channel and spatial attention modules, wherein the channel attention module is used to simulate the sensitivity of the human visual system to different channels, and the spatial attention module is used to simulate the sensitivity of the human visual system to different image regions.

[0018] Further, a double-branch network is used to assign weights to each image block, where one branch network calculates the weights of the image block, and the other branch network calculates the quality score of the image block.

[0019] Further, a loss function is designed to optimize the network parameters based on the difference between the predicted image quality score and the actual quality score, where the loss function is a mean square error loss function.

[0020] Further, the diffusion repair process includes the following steps:

[0021] A1, using a pre-trained stable diffusion model to iteratively repair the distorted image after blind denoising preprocessing, where each iteration includes denoising and detail enhancement of the image to gradually restore the quality of the image;

[0022] A2, in each iteration, by comparing the difference between the repaired image and the distorted image, adjusting the parameters of the diffusion model to optimize the repair effect;

[0023] A3, repeat steps A1 and A2 until a predetermined number of iterations is reached or the repair effect meets the preset conditions, and finally generate the repaired image;

[0024] A4, quality assessment is performed on the repaired image, and a preliminary quality score is calculated using a full-reference image quality evaluation model;

[0025] A5, the difference between the preliminary quality score and the actual quality score is taken as the loss function, and the parameters of the diffusion model are updated through the backpropagation algorithm to further improve the repair effect.

[0026] Further, the quality evaluation network is designed based on VGGNet, which extracts image block features from the repaired image and embeds channel attention modules and spatial attention modules, including:

[0027] The repaired image is divided into multiple image blocks;

[0028] For each image block, a quality evaluation network based on VGGNet is used to extract the features of the image block;

[0029] In the quality evaluation network, channel attention modules and spatial attention modules are embedded to simulate the different sensitivities of the human visual system to different channels and image regions;

[0030] Through the channel attention modules and spatial attention modules, different weights are assigned to each image block to highlight the key areas in the image;

[0031] The extracted image block features and assigned weights are taken as input to calculate the quality score of each image block through a fully connected layer.

[0032] Further, the different weights are assigned to each image block, the weight and score of the image block are calculated, and the quality score of the overall image is obtained, including:

[0033] For each image block, the local quality score of the image block is calculated according to its features and the assigned weight;

[0034] The global quality score of the entire image is calculated through the global branch network;

[0035] The local quality score and the global quality score are combined through a preset fusion measure to obtain the quality score of the overall image;

[0036] In order to improve the accuracy of the quality score, a post-processing step is introduced to fine-tune the preliminary quality score, and the fine-tuning strategy can be adaptive adjustment based on image content or other advanced image processing techniques;

[0037] Finally, the quality score of the overall image is output, which reflects the overall quality of the image and can be used for various image quality evaluation tasks.

[0038] Further, the blind image quality evaluation method further includes a post-processing step to fine-tune the preliminary quality score to improve the accuracy of the final quality score.

[0039] Further, the blind image quality evaluation method can automatically process a large number of images and can adapt to different image quality and distortion types.

[0040] The second technical solution adopted by the present application is:

[0041] An electronic device, the electronic device includes a processor and a memory, the memory has at least one instruction, at least one program, a code set or an instruction set stored therein, the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to realize a blind image quality evaluation method based on diffusion repair technology as described above.

[0042] The third technical solution adopted by the present application is:

[0043] A computer readable storage medium, the storage medium has at least one instruction, at least one program, a code set or an instruction set stored therein, the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to realize a blind image quality evaluation method based on diffusion repair technology as described above.

[0044] The fourth technical solution adopted by the present application is:

[0045] A computer program product or computer program comprises computer instructions stored in a computer-readable storage medium. The processor of the computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the above-mentioned method.

[0046] The application has the beneficial effects that: the application generates a pseudo-reference image by using a pre-trained stable diffusion model to repair a distorted image with high quality, and combines an attention mechanism simulating the human visual system to realize accurate, efficient and automatic blind image quality evaluation. This method not only improves the accuracy and robustness of evaluation, but also can work stably in various practical application scenarios, and is not limited by specific distortion types or image content. In addition, the network structure optimization and loss function design of the application further improve the performance of the model, so that it exhibits excellent generalization ability and robustness on multiple image quality evaluation databases, surpassing existing no-reference image quality evaluation methods, and has wide application prospects and practical significance. BRIEF DESCRIPTION OF DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following introduces the drawings of the related technical solutions in the embodiments of the application or the prior art. It should be understood that the drawings in the following introduction are only for the convenience of clearly describing some embodiments in the technical solutions of the application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0048] Figure 1 is a step flow chart of a blind image quality evaluation method based on diffusion repair technology in the embodiments of the application;

[0049] Figure 2 is a framework schematic diagram of the DiffBIQA model in the embodiments of the application, including an image restoration module and an image quality evaluation module;

[0050] Figure 3 is a structure schematic diagram of the spatial channel attention module in the embodiments of the application;

[0051] Figure 4 is a relationship schematic diagram of the repaired image MOS, the distorted image MOS and the image distortion level and type in the embodiments of the application. DETAILED DESCRIPTION

[0052] Embodiments of the present application are described below in detail with reference to the accompanying drawings, wherein the same or similar components are denoted by the same or similar reference numerals throughout the drawings. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and should not be understood as limiting the present application. For the step numbers in the following embodiments, they are only set for the convenience of illustrating the description, and the order between the steps is not limited in any way, and the execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0053] In the description of the present application, it should be understood that the orientation description, such as the orientation or position relationship indicated by the upper, lower, front, rear, left, right, etc. is based on the orientation or position relationship shown in the drawings, and is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the device or element indicated must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application.

[0054] In the description of the present application, the meaning of several is one or more, and the meaning of multiple is more than two. Greater than, less than, more than, etc. are understood as not including the number, and above, below, etc. are understood as including the number. If it is described as first, second, it is only for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features or the order of indicated technical features.

[0055] In the description of the present application, unless otherwise explicitly limited, the words such as setting, installing, connecting, etc. should be understood in a broad sense, and those skilled in the art can reasonably determine the specific meaning of the above words in the present application in combination with the specific content of the technical solution.

[0056] To solve the existing technical problems, the present application proposes a blind image quality evaluation method based on diffusion repair technology. This method uses a pre-trained diffusion model to repair distorted images and generate high-quality pseudo-reference images, and combines deep learning technology for image quality evaluation. This method not only generates high-quality repair images, but also simulates the characteristics of the human visual system, improving the accuracy and robustness of the evaluation. In addition, the test results on multiple image quality evaluation databases show that this method has excellent performance and generalization ability on different distortion types, surpassing other existing no-reference image quality evaluation methods.

[0057] Example 1

[0058] As Figure 1As shown, the embodiment provides a blind image quality evaluation method based on diffusion repair technology, which can effectively evaluate image quality without reference image, has wide application prospect and practical significance. The method generates a pseudo reference image by using an advanced diffusion model to repair the distorted image, and simulates the full reference image quality evaluation method to score, so as to realize the no-reference image quality evaluation. This method is closer to the observation characteristics of the human visual system, can accurately identify the key information in the image, and effectively solves the problem of inaccurate judgment of the key area of the distorted image in the previous method, including the following steps:

[0059] S1, acquiring a distorted image, blind denoising preprocessing is performed on the distorted image to reduce the noise in the image and prepare the image for the next diffusion repair.

[0060] As an implementation, in step S1, the blind denoising (Blind Image Denoisiong, BID) preprocessing method used is Swin-Conv-UNet network, which can effectively remove different types of noise to improve the effect of subsequent diffusion repair. The BID preprocessing module is used to remove the noise degradation information in the distorted image (I distorted ) and generate a conditional image.

[0061] I BID =BID(I distorted )

[0062] The loss function is:

[0063]

[0064] Where I reference is the reference image, I distorted is the low-quality image, and I BID is the intermediate image generated by the repair module.

[0065] S2, using a pre-trained stable diffusion model to repair the preprocessed image to generate a repaired image, this step aims to restore the quality of the image and simulate the reference image in the full reference image quality evaluation.

[0066] As an implementation, in step S2, the pre-trained stable diffusion model used is the Stable Diffusion model, which can generate high-quality image repair results and effectively restore the original quality of the image. Stable Diffusion pre-trains an autoencoder to convert an image x into a latent variable z through an encoder E and reconstruct it through a decoder D, and the diffusion and denoising process is performed in the latent space. In the diffusion process, the variance β tGaussian noise ∈ ~ N(0, I) is added to the encoded latent variable z = E(x) to produce a noisy latent variable:

[0067]

[0068] where ∈ ~ N(0, I), a t = 1 - β t , and When t is large enough, the variable z t in the latent space is almost a standard Gaussian distribution.

[0069] The network ∈ θ is learned by predicting the noise ∈ at randomly chosen time steps t based on c (as text prompt). The optimization of the latent diffusion model is defined as follows:

[0070]

[0071] where x and c are sampled from the dataset, z = E(x), t is uniformly sampled, ∈ is sampled from a standard Gaussian distribution.

[0072] This model applies the ControlNet strategy, first using a pre-trained VAE encoder to encode the recovered conditional image:

[0073] c BID = E(I BID )

[0074] For the conditional network, following the ControlNet approach, a trainable pre-trained UNet encoder and intermediate blocks are copied and a U-Net encoder and intermediate blocks (denoted as Fcond) are trained, this network receives the condition information and outputs the control signal. This copying strategy provides good weight initialization for the conditional network. Then, we concatenate the condition c BID and the noisy latent variable z t at time t as the input of Fcond, denoted as:

[0075] z′ t = cat(z t ,c BID )

[0076] where z t is the noisy latent variable at time t, and cat denotes the concatenation operation.

[0077] The final goal is to minimize the following latent diffusion objective:

[0078]

[0079] where ∈ θdenoising network, input is the z' after splicing t .

[0080] Finally, through the whole image inpainting module, our input I distorted will produce a high-quality version output I restorted . I restorted is input into the subsequent quality assessment network as a pseudo-reference image, together with I distorted , to obtain the predicted image quality score.

[0081] S3, using a full-reference image quality evaluation model to evaluate the quality of the repaired image, to obtain a preliminary quality score.

[0082] Exemplarily, in step S3, the full-reference image quality evaluation model adopted by the embodiment is the WaDIQaM model, which can provide accurate image quality scores to provide a reference for subsequent diffusion repair. Through this step, a preliminary quality score can be obtained, which reflects the overall quality of the repaired image.

[0083] S4, extracting image block features from the repaired image, designing a quality assessment network based on VGGNet, and embedding a channel attention module and a spatial attention module to simulate the different sensitivities of the human visual system to different channels and image regions.

[0084] As an implementation, in step S4, we extract image block features from the repaired image and process them through the quality assessment network designed based on VGGNet. The original VGGnet design input size is 224x224 pixels, in order to adapt to the smaller size (32x32 pixels) input in the present application, three initial layers (conv3-32, conv3-32, maxpool) are added at the architecture level. In view of the challenge that traditional CNN cannot identify key areas in the feature extraction process, an attention module (CBAM) is integrated immediately after the second convolutional layer. The model extracts features through a series of carefully designed convolution and pooling layers, and applies a simple and effective feature fusion strategy to the extracted feature vectors and their difference features, and then performs regression operation through two fully connected layers FC-512 and FC-1. In order to ensure that the output size is consistent with the input, all convolutional layers use 3x3 pixel kernels and are activated by ReLU function, supplemented by zero padding. In order to alleviate overfitting, dropout operation is used in the fully connected layer, and the specific ratio is set to 0.5. The network regression module of the embodiment of the present application introduces an innovative double-branch design, including a quality estimation branch to calculate the score y i of each image piece i iRun in parallel with the original quality regression branch and share the same dimension information. Activate a by ReLU operation i And add a small stabilizing term, the modified Ensure Always positive. Normalize the weights by:

[0085]

[0086] The global image quality estimate Is:

[0087]

[0088] This design not only considers global quality perception, but also captures subtle changes in local quality perception in the image. In addition, CBAM modules are integrated in both branches, further refining the model's ability to identify and accurately weigh focal areas, significantly improving the accuracy and reliability of the assessment.

[0089] S5, assign different weights to each image block, calculate the weight and score of the image block, and obtain the quality score of the overall image.

[0090] As an implementation, in step S5, we assign different weights to each image block and calculate the weight and score of the image block. This process is achieved through a double-branch network, where one branch calculates the weight of the image block and the other branch calculates the quality score of the image block. By integrating this information, we obtain the quality score of the overall image. In addition, the method also includes a loss function that optimizes network parameters based on the difference between the predicted image quality score and the actual quality score, where the loss function is the mean square error loss function.

[0091] The above method is further described in conjunction with the accompanying drawings and specific embodiment modes.

[0092] The present embodiment provides a blind image quality evaluation method based on diffusion repair technology, which is suitable for evaluating image quality including but not limited to noise, blur, compression distortion and the like, including:

[0093] Step one: image repair module. Blind denoising preprocessing is performed on the distorted image, and an advanced stable diffusion generation model is used to restore the distorted image.

[0094] Step two: image evaluation module. A full-reference image quality evaluation model is used to evaluate the quality of the repaired image to obtain a preliminary quality score. At the same time, image block features are extracted from the repaired image, and the weight and score of each image block are calculated. Finally, these information is integrated to obtain the quality score of the overall image.

[0095] Referring to Figure 2 The specific implementation process mainly includes two aspects of image restoration and image evaluation:

[0096] First, the distorted image is obtained as input and is input into the blind denoising network SCUNet for denoising preprocessing to obtain an intermediate image. Then, the intermediate image is input into the pre-trained Stable Diffusion model, and after the diffusion and reconstruction process, a restored image is generated. In this process, specific operations of the forward diffusion of the diffusion model and the reverse generation process are involved, such as noise addition and removal operations on latent variables in the latent space, and related parameter settings and adjustments. The diffusion model is composed of a forward diffusion process and a reverse generation process. In the forward diffusion process, the model gradually adds noise to the data until the data is completely or partially transformed into noise. In the reverse process, the model learns how to gradually recover the original data from the noisy data. In the embodiment of the present application, the Stable Diffusion pre-trains an autoencoder to convert an image x into a latent variable z through an encoder E and reconstruct it through a decoder D, and the diffusion and denoising process is performed in the latent space.

[0097] Second, the image obtained by the image restoration model is used as a reference to obtain the final score of the distorted image. In order to more accurately estimate the weight of each image block, the embodiment of the present application integrates an attention mechanism that emphasizes key features while suppressing irrelevant features, and applies a new attention module called "spatial channel attention module". The module includes channel attention and spatial attention, and uses information-rich features extracted by convolution operations to sequentially apply channel attention and spatial attention mechanisms, so that each branch can independently learn "what" and "where" important features in the channel and spatial dimensions.

[0098] For the input image features, first, processing is performed in the channel dimension. The input features are simultaneously input into the maximum pooling layer and the average pooling layer, and then the pooled features are input into a shared multi-layer perceptron network to generate the final channel attention feature map. Then, processing is performed in the spatial dimension. Average pooling and maximum pooling are performed along the channel dimension, and then the obtained feature maps are connected, and convolution operations are applied on the connected feature maps to generate the final spatial attention feature map. Finally, the spatial and channel attention are combined in a serial manner, and the input features are weighted to obtain the attention-enhanced feature map. The specific implementation of this step is described in detail in the attached Figure 3 .

[0099] .Similarly, the embodiment of the present application also introduces a new image quality evaluation network, which is improved based on the VGGnet architecture. The original VGGnet design input size is 224x224 pixels, in order to adapt to the smaller size (32x32 pixels) input in the present application, three initial layers (conv3-32, conv3-32, maxpool) are added at the architecture level. In view of the challenge that the traditional CNN cannot identify the key area in the feature extraction process, an attention module is integrated immediately after the second convolutional layer. The model extracts features through a series of carefully designed convolution and pooling layers, and applies a simple and effective feature fusion strategy to the extracted feature vectors and their difference features, and then performs regression operation through FC-512 and FC-1 two fully connected layers. In order to ensure that the output size is consistent with the input, all convolutional layers use 3x3 pixel kernels and are activated by ReLU function, supplemented by zero padding. In order to alleviate overfitting, the dropout operation is adopted in the fully connected layer, and the specific ratio is set to 0.5. The network regression module of the embodiment of the present application introduces an innovative double-branch design, including a quality estimation branch calculating the score y i and a weight estimation branch calculating the weight a i , which runs in parallel with the original quality regression branch and shares the same dimensional information.

[0100] Firstly, the feature map processed by the spatial channel attention module is input into the improved VGGnet architecture. From the first convolutional layer, the features are extracted through a series of carefully designed convolution and pooling layers, and the attention module is used to pay attention to the key area after the second convolutional layer. Then, the feature fusion operation is performed on the extracted feature vectors and their difference features to obtain the fused features. Then the fused features are input into FC-512 and FC-1 two fully connected layers for regression operation, respectively obtaining the score and weight of the image block, and finally combining to obtain the quality score of the image.

[0101] The model proposed in the present application uses the MSE loss function to optimize the quality evaluation prediction performance of the distorted image. Generally speaking, the blind image quality evaluation loss function will be designed to narrow the gap between the predicted score and the true label. But because our image repair module performs the same repair operation on all distorted images, and uses the full reference model to give the repair image score label, we hope to find the relative relationship between the score difference and the feature difference between the distorted image and the repair image. Appendix Figure 4 It is illustrated that the score difference between the distorted image and the repair image is different according to different distortion levels and different distortion types. This also proves the rationality of designing the loss function based on measuring this difference. The formula is as follows:

[0102]

[0103] diff2 = q t -qz

[0104] Loss = L MSE (diff1,diff2)

[0105] This loss function encourages the model to predict a quality score that is close to the repaired image quality score (q z ), and that is close to the true quality score (q t ). z

[0106] Embodiment 2

[0107] The embodiments of the present application also provide an electronic device, which comprises a processor and a memory, and the memory stores at least one instruction, at least one program, a code set or an instruction set, which are loaded and executed by the processor to implement a blind image quality evaluation method based on a diffusion repair technology as shown in Figure 1

[0108] It can be understood that the memory can include a random access memory (RAM) and can also include a read-only memory (ROM). Optionally, the memory includes a non-transitory computer-readable storage medium. The memory can be used to store instructions, programs, codes, code sets or instruction sets. The memory can include a program storage area and a data storage area, wherein the program storage area can store instructions for implementing an operating system, instructions for at least one function, instructions for implementing each of the above-mentioned method embodiments, etc.; and the data storage area can store data created according to the use of the server, etc.

[0109] ​​The processor can include one or more processing cores. The processor connects various parts within the entire server by various interfaces and lines, executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory, and calling data stored in the memory. Alternatively, the processor can be implemented in at least one of a hardware form of a digital signal processing (DSP), a field-programmable gate array (FPGA), a programmable logic array (PLA). The processor can be integrated with one or several combinations of a central processing unit (CPU) and a modem. Among them, the CPU mainly processes operating systems and application programs; the modem is used to process wireless communication. It can be understood that the above-mentioned modem can also not be integrated into the processor, but be realized by a single chip.

[0110] Since the electronic device is an electronic device corresponding to the blind image quality evaluation method based on the diffusion repair technology according to the embodiment of the application, and the principle of solving the problem of the electronic device is similar to that of the method, the implementation of the electronic device can be referred to the implementation process of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0111] Embodiment 3

[0112] The embodiment of the application also provides a computer readable storage medium, wherein at least one instruction, at least one program, a code set or an instruction set are stored in the storage medium, and the at least one instruction, the at least one program, the code set or the instruction set are loaded and executed by a processor to realize the blind image quality evaluation method based on the diffusion repair technology as shown in Figure 1

[0113] ​Those skilled in the art can understand that all or part of the steps of various methods in the above embodiments can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer readable storage medium, including Read-Only Memory (ROM), Random Access Memory (RAM), Programmable Read-only Memory (PROM), Erasable Programmable Read Only Memory (EPROM), One-time Programmable Read-Only Memory (OTPROM), Electrically-Erasable Programmable Read-Only Memory (EEPROM), Compact Disc Read-Only Memory (CD-ROM) or other optical disk storage, magnetic disk storage, magnetic tape storage, or any other medium that can be used to carry or store data which can be read by a computer.

[0114] Since the storage medium is a storage medium corresponding to the blind image quality evaluation method based on diffusion repair technology according to the embodiments of the present application, and the principle of solving the problem of the storage medium is similar to that of the method, the implementation of the storage medium can be referred to the implementation process of the above method embodiments, and the repeated parts will not be described again.

[0115] Embodiment 4

[0116] In some possible implementation manners, various aspects of the method according to the embodiments of the present application can also be implemented in the form of a program product, which includes program codes for causing a computer device to execute the steps of the blind image quality evaluation method based on diffusion repair technology according to various exemplary embodiments of the present application described above in the specification when the program product is run on the computer device. Wherein, the executable computer program codes or "codes" for executing various embodiments can be written in high-level programming languages such as C, C++, C#, Smalltalk, Java, JavaScript, Visual Basic, Structured Query Language (for example, Transact-SQL), Perl, or in various other programming languages.

[0117] It should be understood that various aspects of the application can be implemented in hardware, software, firmware or a combination of them. In the above embodiments, various steps or methods can be implemented in software or firmware which is stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, and in another embodiment, any of the following technologies, known in the art, or their combination, can be used: discrete logic circuitry having logic gates for implementing logic functions upon an application of data signals, application-specific integrated circuits having appropriate combinational logic gates, programmable gate arrays (PGA), field-programmable gate arrays (FPGA), and so on.

[0118] In the description of the specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Also, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples. In addition, a person skilled in the art can combine and combine the different embodiments or examples described in the specification and the features of the different embodiments or examples, without contradiction.

[0119] The above embodiments are only for the purpose of illustrating the technical concept and characteristics of the present application, and the purpose is to enable those skilled in the art to understand the content of the present application and to implement it, and cannot limit the protection scope of the present application. Any equivalent changes or modifications made according to the essence of the present application should be covered within the protection scope of the present application.

Claims

1. A blind image quality assessment method based on diffusion inpainting technique, characterized in that, The method comprises the following steps: Obtaining a distorted image, and performing blind denoising preprocessing on the distorted image by using a Swin-Conv-UNet network; Using a pre-trained stable diffusion model to repair the preprocessed image and generate a repaired image; the pre-trained stable diffusion model is a Stable Diffusion model; Using a full-reference image quality evaluation model to evaluate the quality of the repaired image and obtain a preliminary quality score; Extracting image block features from the repaired image, designing a quality evaluation network based on a VGGNet, and embedding a channel attention module and a spatial attention module; Assigning different weights to each image block, calculating the weight and score of the image block, and obtaining the quality score of the overall image; The quality evaluation network based on the VGGNet is designed and the channel attention module and the spatial attention module are embedded, comprising: On the basis of the VGGnet, three initial layers are added, including two convolutional layers and a maximum pooling layer; a CBAM attention module is integrated after the second convolutional layer; The network regression module introduces a double-branch design, including a quality estimation branch for computing the score of each image block and a weight estimation branch for computing the weight of each image block , the weight estimation branch runs in parallel with the original quality estimation branch and shares the same dimension information; activated by a ReLU operation , and a small stabilizing term is added , the modified , ensures always positive.

2. The blind image quality assessment method based on diffusion inpainting technique according to claim 1, characterized in that, The full-reference image quality evaluation model is a WaDIQaM model, which can provide accurate image quality scores and provide a reference for subsequent diffusion repair.

3. The blind image quality assessment method based on diffusion inpainting technique according to claim 1, characterized in that, The quality evaluation network comprises multiple convolutional layers, pooling layers, fully connected layers, and channel and spatial attention modules, wherein the channel attention module is used to simulate the sensitivity of the human visual system to different channels, and the spatial attention module is used to simulate the sensitivity of the human visual system to different image regions.

4. The blind image quality assessment method based on diffusion inpainting technique according to claim 1, characterized in that, The diffusion repair process comprises the following steps: A1, using a pre-trained stable diffusion model to iteratively repair the distorted image after blind denoising preprocessing, wherein each iteration includes denoising and detail enhancement of the image to gradually restore the quality of the image; A2, in each iteration, by comparing the differences between the repaired image and the distorted image, adjusting the parameters of the diffusion model to optimize the repair effect; A3, repeating steps A1 and A2 until a predetermined number of iterations is reached or the repair effect meets the preset conditions, and finally generating a repaired image; A4, evaluating the quality of the repaired image and calculating a preliminary quality score using a full-reference image quality evaluation model; A5, taking the difference between the preliminary quality score and the actual quality score as a loss function, and updating the parameters of the diffusion model through a backpropagation algorithm to further improve the repair effect.

5. The blind image quality assessment method based on diffusion inpainting technique according to claim 1, characterized in that, The quality evaluation network based on the VGGNet is designed, image block features are extracted from the repaired image, and a channel attention module and a spatial attention module are embedded, comprising: Divide the repaired image into multiple image blocks; For each image block, use the quality evaluation network designed based on the VGGNet to extract the features of the image block; In the quality evaluation network, embed the channel attention module and the spatial attention module to simulate the different sensitivities of the human visual system to different channels and image regions, respectively; Through the channel attention module and the spatial attention module, different weights are assigned to each image block to highlight the key areas in the image; The extracted image block features and the assigned weights are taken as inputs to calculate a quality score of each image block through a fully connected layer.

6. The blind image quality assessment method based on diffusion inpainting technique according to claim 1, characterized in that, The method comprises the following steps: For each image block, a local quality score of the image block is calculated according to the features and the assigned weights of the image block; A global quality score of the whole image is calculated through a global branch network; The local quality score and the global quality score are combined through a preset fusion measurement to obtain a quality score of the whole image.

7. An electronic device, comprising: The electronic device comprises a processor and a memory, and the memory stores at least one program, which is loaded and executed by the processor to implement the method according to any one of claims 1 to 6.

Images