Image quality processing system, method, device and medium

Abstract

The present invention discloses an image quality processing system, method, device and medium. The system includes a high-quality data supply unit, a generator, and a discriminator. The generator is designed based on a generative adversarial network framework and embedded in a convolutional neural network. The generator reduces a high-quality image data set to a low-dimensional space, adds noise, and degrades to generate a simulated low-quality image data set, and transmits the generated simulated low-quality image data set to the discriminator. The discriminator, which is trained by an adversarial loss function, discriminates and compares the simulated low-quality image data set generated by the generator and the high-quality image data set provided by the data supply unit, and outputs a high-low mixed quality image data set. This solution uses a variational autoencoder to model the distribution of high-quality image data, solves the problem of unbalanced training samples, and realizes image quality processing acceleration and real-time control. It is low-cost and highly versatile.

Description

Technical Field

[0001] The present invention relates to image data processing technology, and in particular to an image quality processing system, method, device and medium. Background Art

[0002] In the image data processing system, the computer vision technology is used to optimize the performance of the quality control link in the image data processing process, including but not limited to medical images, remote sensing images, microscope slice images, natural images, etc. Image quality control, including the inspection of whether the image quality meets the requirements and the inspection of pre-processing quality, plays an important role in the subsequent batch analysis of image data.

[0003] Due to the lack of unified quantitative evaluation indicators for whether images are of high quality in practical applications, image quality control and screening currently rely mainly on manual completion or on specific indicators defined under specific tasks; the former brings high time and manpower costs, and the "specific indicators" in the latter are a task-related indirect indication of image quality, which fails to provide a guidance method with good generalization performance for image quality control.

[0004] In recent years, with the development of machine learning / deep learning technology, some studies have proposed image quality control methods based on adaptive learning algorithms. Such algorithms are data-driven and rely on iterative training of large amounts of data, and have achieved certain quality control effects. However, in actual applications, there is often a large difference in the amount of high-quality and low-quality data, which will affect the performance of the above-mentioned image quality control methods based on data-driven algorithms. At present, in the field of machine learning / deep learning, the solution to unbalanced samples is adaptive weighting, and its actual effect is usually unstable, so it is impossible to determine whether the model is optimal.

[0005] Taking brain magnetic resonance structural imaging data as an example, the degradation of magnetic resonance image quality comes from multiple factors in the spatial domain and the transform domain, which is difficult to model explicitly. Compared with the amount of high-quality data, the amount of data in the actual low-quality image data set is small, which is insufficient to provide the classifier with the distribution information of the real low-quality image data. Therefore, if the supervised learning method in conventional machine learning is used to train a classification model to distinguish high / low quality images, the performance of the model will be limited due to the limitations of the training data.

[0006] Therefore, in the field of medical imaging, there is an urgent need for an image quality control system that is high-speed, versatile, and independent of manually defined features for use in clinical and scientific research. Summary of the invention

[0007] In order to overcome the deficiencies of the prior art, an object of the present invention is to provide an image quality processing system, method, device and medium, which can solve the above-mentioned problems.

[0008] The purpose of the present invention is achieved by the following technical solutions:

[0009] An image quality processing system, the system includes a high-quality data supply unit, a generator, and a discriminator, wherein the high-quality data supply unit is arranged at the logical upstream of the generator and the discriminator, and the discriminator is arranged at the logical downstream of the generator; the high-quality data supply unit provides a high-quality image data set to the generator and the discriminator; the generator is designed based on a generative adversarial network framework and embedded in a convolutional neural network; the generator reduces the high-quality image data set to a low-dimensional space, adds noise, and degrades it to generate a simulated low-quality image data set, and transmits the generated simulated low-quality image data set to the discriminator; the discriminator, which is model-trained using an adversarial loss function, discriminates and compares the simulated low-quality image data set generated by the generator and the high-quality image data set provided by the data supply unit, and outputs a high-low mixed quality image data set.

[0010] Preferably, the generator adopts a variational autoencoder, which includes an encoder and a decoder; wherein the encoder encodes and reduces the dimension of the high-quality image data set, and sets a difference constraint on the encoder to avoid the result approaching zero; the decoder collects samples from the encoder, doubles the size of the feature map and halves the number of channels to obtain a picture with the same size as the original input high-quality image and after noise processing as the generated simulated low-quality image.

[0011] Preferably, the encoder and decoder both adopt a deep convolutional neural network, wherein the network structure of the encoder includes multiple layers of convolutional layers, batch normalization layers, activation function layers, and pooling layers, and after each group of convolutional layers, batch normalization layers, activation function layers, and pooling layers, the size of the feature map is halved, the number of channels is doubled, and an n-dimensional vector is output, where n is a positive integer greater than 2; after processing through a layer of fully connected layers, a 2-dimensional vector distribution is output; n values ​​are sampled from the n-dimensional vector distribution to form an n-dimensional vector as the input of the decoder; the network structure of the decoder includes multiple layers of deconvolutional layers, batch normalization layers, ReLU activation function layers, and convolutional layers, and after each group of deconvolutional layers, batch normalization layers, activation function layers, and convolutional layers, the size of the feature map is doubled, the number of channels is halved, and a picture with the same size as the original input high-quality image and processed by noise is obtained as a simulated low-quality image.

[0012] Preferably, the difference constraint adopts the JS divergence function; and the activation function layer adopts the ReLU activation function layer.

[0013] Preferably, the network structure of the encoder includes 5 convolution layers with a convolution kernel size of 3×3×3 and a step size of 1, and each convolution layer includes a batch normalization layer, a ReLU activation function layer, and a maximum pooling layer with a pooling kernel size of 2×2×2 and a step size of 2, and after processing, an n=128-dimensional vector is output; the network structure of the decoder includes 5 deconvolution layers with a convolution kernel size of 2×2×2 and a step size of 2, and each deconvolution layer includes a batch normalization layer, a ReLU activation function layer, and a convolution layer with a convolution kernel size of 3×3×3 and a step size of 1.

[0014] Preferably, the 2D vector output by the fully connected layer is distributed as a normal distribution with a mean of μ and an equation of σ.

[0015] Preferably, the discriminator structure adopts the VGG-16 network model, including the JS divergence loss function L JS And generate adversarial loss function L GAN , where the JS divergence loss function is:

[0016]

[0017] Where P is the distribution of the encoder, U is the standard normal distribution, and KL(·||·) is the KL divergence;

[0018] For the generated adversarial loss function L GAN , then the generated adversarial loss function is:

[0019] L GAN =-log(Dis(x))-log(1-Dis(x'))…………………………Formula-2,

[0020] Where x is a sample extracted from a real high-quality image dataset, x' is a noisy sample generated by the decoder, and Dis(·) is the output of the discriminator between 0 and 1. The closer it is to 1, the more the discriminator tends to judge the input image as high quality, and the closer it is to 0, the more the discriminator tends to judge the input image as low quality.

[0021] A method for image quality processing according to the aforementioned system, the method comprising:

[0022] S1. Construct a generator and a discriminator, wherein the generator adopts a variational autoencoder including an encoder and a decoder; the discriminator adopts a network model cross-trained by a generative adversarial function, and the discriminator improves the high / low quality discrimination performance of the image data through adversarial training. After the model converges, the obtained discriminator is used as the image data quality discriminator;

[0023] S2. Provide a high-quality image data set, the high-quality data supply unit provides a high-quality image data set to the encoder and the discriminator;

[0024] S3. Generate a simulated low-quality image dataset. The generator reduces the high-quality image dataset to a low-dimensional space, adds noise, and degrades it to generate a simulated low-quality image dataset, specifically including:

[0025] S31. Dimensionality reduction and noise addition for high-quality images. The encoder reduces the dimension of high-quality data to a low-dimensional space, outputs an n-dimensional vector distribution with half the size of the feature map and double the number of channels, and adds noise to the high-quality data in the low-dimensional space.

[0026] S32. Degraded simulated low-quality image data sets are generated, and the noisy image is mapped back to the original image space by the decoder, and the feature map size is doubled and the number of channels is halved to obtain a degraded simulated low-quality image;

[0027] S4. Output a mixed high- and low-quality image dataset. The generator transmits the generated simulated low-quality image dataset to the discriminator. The image quality discriminator compares the simulated low-quality image dataset transmitted by the generator and the high-quality image dataset provided by the data supply unit, and outputs a mixed high- and low-quality image dataset.

[0028] Preferably, in step S1, the cross-training strategy of the network model of the discriminator includes:

[0029] When training the variational autoencoder, the parameters of the discriminator are fixed and the variational autoencoder is adjusted. When training the variational autoencoder, the overall loss function of the model is

[0030] L VAE =L JS -L GAN …………………………Formula-3;

[0031] When training the discriminator, the parameters of the variational autoencoder are fixed and the parameters of the discriminator are adjusted. When training the discriminator, the overall loss function of the model is

[0032] L VAE =L JS +L GAN ……………………………………Formula-4;

[0033] Where, L JS is the JS divergence loss function, L GAN To generate adversarial loss function;

[0034] Through cross-training, the noise generation ability of the variational autoencoder and the discriminator's ability to distinguish between high-quality and low-quality samples are simultaneously enhanced. After the model converges, an image quality discriminator for image quality assessment is obtained.

[0035] Preferably, the model is optimized using an adaptive moment estimation algorithm, the learning rate is set to 1e-4, and the model parameters are updated by a gradient descent method.

[0036] The present invention also provides a device, comprising:

[0037] one or more processors;

[0038] A memory for storing one or more programs;

[0039] When one or more programs are executed by one or more processors, the one or more processors implement the aforementioned image quality processing method.

[0040] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the aforementioned image quality processing method is implemented.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] 1. Accelerate the processing speed of image quality control, realize real-time image quality control, and greatly reduce the labor cost of operation;

[0043] 2. Provide a highly versatile image quality control strategy that has good portability for different application scenarios.

[0044] 3. Use variational autoencoders to model the distribution of high-quality image data to solve the problem of unbalanced training samples faced by current quality control methods based on machine learning / deep learning in practical applications.

[0045] 4. This invention combines the variational autoencoder with the generative adversarial network model for the first time and applies it to the field of image quality control, providing a new solution for image quality control. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A schematic diagram of the training process of the image data quality control model based on the variational autoencoder-deep generative adversarial network of the present invention;

[0047] Figure 2 is a flow chart of the image quality processing method;

[0048] Figure 3 The present invention is a schematic diagram of a device. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] Technical terms used in the present invention are explained as follows.

[0051] VAE: Variational autoencoder;

[0052] Enc: Encoder;

[0053] Dec: decoder;

[0054] Dis: Discriminator;

[0055] GAN: Generative adversarial network;

[0056] CNN: Convolution neural network;

[0057] Gen: Generator.

[0058] Embodiment 1

[0059] An image quality processing system, see Figure 1 ,The system includes a high-quality data supply unit, a generator Gen, and a discriminator Dis.

[0060] Logical connection relationship: the high-quality data supply unit is set at the logical upstream of the generator and the discriminator, and provides high-quality image data sets to the generator and the discriminator; the discriminator is set at the logical downstream of the generator, and the generator is designed based on the generative adversarial network framework and embedded in a convolutional neural network; the generator reduces the high-quality image data set to a low-dimensional space, adds noise, and degrades it to generate a simulated low-quality image data set, and transmits the generated simulated low-quality image data set to the discriminator.

[0061] Furthermore, the discriminator trained with the adversarial loss function performs discrimination and comparison on the simulated low-quality image dataset generated by the generator and the high-quality image dataset provided by the data supply unit, and outputs a high- and low-mixed quality image dataset.

[0062] The present invention is based on a priori information: the distribution of high-quality image data sets obeys normal distribution, which is specifically as follows.

[0063] A high-quality image dataset that meets the requirements should be normally distributed in the spatial domain, the transform domain, and the low-dimensional manifold of the above space. Therefore, a large amount of high-quality data can be used to model the distribution of high-quality data using a variational autoencoder (VAE).

[0064] At the same time, the encoder in VAE can reduce the dimensionality of high-quality data to a low-dimensional space, add noise to the high-quality data in the low-dimensional space, and the noisy image is mapped back to the original image space by the decoder to obtain a degraded simulated low-quality image.

[0065] In order to make the simulated low-quality images as realistic as possible and use the simulated low-quality images and real high-quality images to obtain a classifier for high / low quality image discrimination, the idea of ​​zero-sum game is used, and the design of the generative adversarial network (GAN) framework is referred to. The Convolutional Neural Network (CNN) classifier is embedded in the VAE, that is, the VAE is regarded as the generator in the GAN, and the CNN is used as the discriminator in the GAN.

[0066] During the optimization process of the model, the reconstruction error loss function in the original VAE is replaced by the adversarial loss function of GAN. Through adversarial training, the discriminator's performance in distinguishing high / low quality of image data is improved. After the model converges, the obtained discriminator is used as an image data quality discriminator and can be used in the image quality control process.

[0067] The model training process is as follows Figure 1As shown. The high-quality image is first encoded (dimensionality reduction) by the encoder of VAE. Encoder is a deep convolutional neural network. Its network structure mainly includes 5 convolution layers with a convolution kernel size of 3×3×3 and a stride of 1. Each convolution layer includes a batch normalization layer, a ReLU activation function layer, and a maximum pooling layer with a pooling kernel size of 2×2×2 and a stride of 2. After each set of convolution layers, batch normalization layers, activation function layers, and pooling layers, the feature map size is halved and the number of channels is doubled. Finally, a 128-dimensional vector is output. After a fully connected layer, a 2-dimensional vector is output, which is set to the mean μ and variance σ of the normal distribution respectively. According to the above assumptions, the distribution of high-quality image data sets follows a normal distribution, which can theoretically be mapped to a standard normal distribution using an encoder; in addition, in order to make the final VAE-generated image a noisy image, the variance σ calculated by the encoder should be avoided from approaching 0 during the training process. Therefore, we use the JS divergence between the normal distribution N(μ,σ) obtained by the Encoder and the standard normal distribution N(0,1) to constrain the difference between the two normal distributions. 128 values ​​are sampled from the normal distribution N(μ,σ) to form a 128-dimensional vector as the input of the VAE decoder.

[0068] Decoder is a deep convolutional neural network. Its network structure mainly includes 5 deconvolution layers with a kernel size of 2×2×2 and a stride of 2. Each deconvolution layer is followed by a batch normalization layer, a ReLU activation function layer, and a convolution layer with a kernel size of 3×3×3 and a stride of 1. After each set of deconvolution layers, batch normalization layers, activation function layers, and convolution layers, the size of the feature map is doubled and the number of channels is halved. Finally, a noise-processed image with the same size as the original input high-quality image is obtained as a simulated low-quality image.

[0069] In order to make the simulated low-quality images as realistic as possible and use the simulated low-quality images and real high-quality images to obtain a classifier for high / low quality image discrimination, the idea of ​​zero-sum game is used, and the design of the GAN framework is referred to. The Convolution Neural Network (CNN) classifier is embedded in the VAE, that is, the VAE is regarded as the generator in the GAN, and the CNN is used as the discriminator in the GAN.

[0070] The Discriminator uses the VGG-16 network model. During the optimization of the model, the GAN adversarial loss function replaces the reconstruction error loss function in the original VAE, and the discriminator improves its high / low quality discrimination performance of image data through adversarial training.

[0071] According to the above design, the loss function of the model consists of two parts: JS divergence loss function, including JS divergence loss function L JS And generate adversarial loss function L GAN Specifically, let the distribution obtained by the Encoder be P and the standard normal distribution be U, then the JS divergence loss function is:

[0072]

[0073] Where P is the distribution of the encoder, U is the standard normal distribution, and KL(·||·) is the KL divergence;

[0074] For the generated adversarial loss function L GAN , then the generated adversarial loss function is:

[0075] L GAN =-log(Dis(x))-log(1-Dis(x'))…………………………Formula-2,

[0076] Where x is a sample extracted from a real high-quality image dataset, x' is a noisy sample generated by the decoder, and Dis(·) is the output of the discriminator between 0 and 1. The closer it is to 1, the more the discriminator tends to judge the input image as high quality, and the closer it is to 0, the more the discriminator tends to judge the input image as low quality.

[0077] The cross-training strategy of the discriminator network model includes:

[0078] When training the variational autoencoder, the parameters of the discriminator are fixed and the variational autoencoder is adjusted. When training the variational autoencoder, the overall loss function of the model is

[0079] L VAE =L JS -L GAN …………………………Formula-3;

[0080] When training the discriminator, the parameters of the variational autoencoder are fixed and the parameters of the discriminator are adjusted. When training the discriminator, the overall loss function of the model is

[0081] L VAE =L JS +L GAN …………………………Formula-4;

[0082] Where, L JS is the JS divergence loss function, L GAN To generate adversarial loss function;

[0083] Through cross-training, the noise generation ability of the variational autoencoder and the discriminator's ability to distinguish between high-quality and low-quality samples are simultaneously enhanced. After the model converges, an image quality discriminator for image quality assessment is obtained.

[0084] Furthermore, the model is optimized using the adaptive moment estimation algorithm, the learning rate is set to 1e-4, and the model parameters are updated by the gradient descent method.

[0085] Embodiment 2

[0086] According to the image quality processing method of the aforementioned system, see Figure 2 , the method comprises the following steps.

[0087] S1. Construct a generator and a discriminator, wherein the generator adopts a variational autoencoder including an encoder and a decoder; the discriminator adopts a network model cross-trained by a generative adversarial function, and the discriminator is trained to improve the high / low quality discrimination performance of image data through adversarial training. After the model converges, the obtained discriminator is used as the image data quality discriminator.

[0088] S2. Provide a high-quality image dataset. The high-quality data supply unit provides a high-quality image dataset to the encoder and the discriminator.

[0089] S3. Generate a simulated low-quality image dataset. The generator reduces the high-quality image dataset to a low-dimensional space, adds noise, and degrades it to generate a simulated low-quality image dataset, specifically including:

[0090] S31. Dimensionality reduction and noise addition for high-quality images. The encoder reduces the dimension of high-quality data to a low-dimensional space, outputs an n-dimensional vector distribution with half the size of the feature map and double the number of channels, and adds noise to the high-quality data in the low-dimensional space.

[0091] S32. Degraded low-quality image dataset is generated. The noisy image is mapped back to the original image space by the decoder. The feature map size is doubled and the number of channels is halved to obtain a degraded simulated low-quality image.

[0092] S4. Output a mixed high- and low-quality image dataset. The generator transmits the generated simulated low-quality image dataset to the discriminator. The image quality discriminator compares the simulated low-quality image dataset transmitted by the generator and the high-quality image dataset provided by the data supply unit, and outputs a mixed high- and low-quality image dataset.

[0093] In general, the present invention is an image quality control method based on variational autoencoder-deep generative adversarial network (VAE-GAN).

[0094] Embodiment 3

[0095] A device, see Figure 3 The device includes: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the aforementioned image quality processing method.

[0096] Specifically, the device includes a processor 10, a memory 11, a communication module 12, an input device 13 and an output device 14; the number of processors 10 in the device can be one or more. Figure 3 A processor 10 is taken as an example; the processor 10, memory 11, communication module 12, input device 13 and output device 14 in the device can be connected by a bus or other means. Figure 3 The example of connecting through bus is taken in the following.

[0097] The memory 11 is a computer-readable storage medium that can be used to store software programs, computer executable programs and modules, such as a module corresponding to an image quality processing method in this embodiment. The processor 10 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory 11, that is, implements the above-mentioned image quality processing method.

[0098] The memory 11 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the device, etc. In addition, the memory 11 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 11 may further include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0099] The communication module 12 is used to establish a connection with the display screen and implement data interaction with the display screen. The input device 13 can be used to receive input digital or character information and generate key signal input related to user settings and function control of the device.

[0100] A device provided in this embodiment can execute an image quality processing method provided in any embodiment of the present invention, with specific corresponding functions and beneficial effects.

[0101] Embodiment 4

[0102] A computer-readable storage medium stores a computer program, which implements the aforementioned image quality processing method when executed by a processor. The computer executable instructions are used to execute an image quality processing method when executed by a computer processor. The computer executable instructions are not limited to the method operations described above, but can also execute related operations in an image quality processing method provided by any embodiment of the present invention.

[0103] Conversion Example

[0104] a. In terms of network structure, the network structures of the Discriminator, Encoder, and Decoder in VAE are not limited to the structures described above, and may include but are not limited to the downsampling branches of VGG, ResNet, DenseNet, and VNet;

[0105] b. In VAE, the way to add noise to the image obtained by the encoder is not limited to the normal distribution mentioned above, and can be modified accordingly according to the definition of noise and high-quality images in specific scenarios;

[0106] c. Loss function part: The difference constraint between distributions can adopt other differentiable metrics besides JS divergence; the logarithmic loss function in the loss function of GAN can be composed of other classification loss functions, such as cross entropy loss function, Focalloss, etc.

[0107] d. The present invention is explained using brain magnetic resonance structural images as an example, but its applicable objects include but are not limited to other modalities of medical images, remote sensing images, microscope slice images, natural images, etc.

[0108] e. The present invention uses a variational autoencoder to model the distribution of high-quality image data to solve the problem of unbalanced training samples faced by current quality control methods based on machine learning / deep learning in practical applications.

[0109] f. This invention combines the variational autoencoder with the generative adversarial network model for the first time and applies it to the field of image quality control, providing a new solution for image quality control.

[0110] System Verification

[0111] The trained image quality discriminator was built and tested using Pytorch on an Intel(R)Xeon(R)CPU E5-2670 v2 2.50GHz processor equipped with a CentOs 6.5 system. The model can be judged on the quality of a single image within 0.1 second. The model was tested on 3,200 brain magnetic resonance imaging data from a hospital (3,000 high-quality data and 200 low-quality data), and the sensitivity (i.e., the proportion of high-quality data correctly identified as high-quality data) was about 99%, and the specificity (i.e., the proportion of low-quality data correctly identified as low-quality data) was about 95%.

[0112] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image quality processing system, characterized in that: The system includes a high-quality data supply unit, a generator, and a discriminator, wherein the high-quality data supply unit is arranged at the logical upstream of the generator and the discriminator, and the discriminator is arranged at the logical downstream of the generator; The high-quality data supply unit provides a high-quality image data set to the generator and the discriminator; The generator is designed based on a generative adversarial network framework and is embedded in a convolutional neural network; the generator adopts a variational autoencoder, which includes an encoder and a decoder; wherein the encoder performs encoding and dimensionality reduction on a high-quality image data set, and sets a difference constraint on the encoder to prevent the result from approaching zero; the decoder collects samples from the encoder, doubles the feature map size and halves the number of channels to obtain a noise-processed picture with the same size as the original input high-quality image as a simulated low-quality image produced; the generator reduces the high-quality image data set to a low-dimensional space, adds noise, and degrades it to generate a simulated low-quality image data set, and transmits the generated simulated low-quality image data set to the discriminator; The discriminator trained with the adversarial loss function performs discrimination and comparison on the simulated low-quality image dataset generated by the generator and the high-quality image dataset provided by the data supply unit, and outputs a mixed high-quality and low-quality image dataset; the discriminator structure adopts the VGG-16 network model, including the JS divergence loss function L JS And generate adversarial loss function L GAN , where the JS divergence loss function is: Where P is the distribution of the encoder, U is the standard normal distribution, and KL(·||·) is the KL divergence; For the generated adversarial loss function L GAN , then the generated adversarial loss function is: L GAN = -log(Dis(x)) - log(1 - Dis(x')) …………………………… Equation - 2 Where x is a sample extracted from a real high-quality image dataset, x' is a noisy sample generated by the decoder, and Dis(·) is the output of the discriminator between 0 and 1. The closer it is to 1, the more the discriminator tends to judge the input image as high quality, and the closer it is to 0, the more the discriminator tends to judge the input image as low quality.

2. The image quality processing system according to claim 1, characterized in that: The encoder and decoder both use deep convolutional neural networks, wherein the network structure of the encoder includes multiple convolutional layers, batch normalization layers, activation function layers, and pooling layers. After each set of convolutional layers, batch normalization layers, activation function layers, and pooling layers, the size of the feature map is halved, the number of channels is doubled, and an n-dimensional vector is output, where n is a positive integer greater than 2; after a layer of fully connected layers, a 2-dimensional vector distribution is output; Sample n values ​​from the n-dimensional vector distribution to form an n-dimensional vector as the input of the decoder; The network structure of the decoder includes multiple layers of deconvolution layers, batch normalization layers, ReLU activation function layers, and convolution layers. After each set of deconvolution layers, batch normalization layers, activation function layers, and convolution layers, the size of the feature map is doubled and the number of channels is halved, resulting in a noise-processed image with the same size as the original input high-quality image, which is used to simulate a low-quality image.

3. The image quality processing system according to claim 2, characterized in that: in, The network structure of the encoder includes 5 convolution layers with a convolution kernel size of 3×3×3 and a step size of 1, each convolution layer includes a batch normalization layer, a ReLU activation function layer, and a maximum pooling layer with a pooling kernel size of 2×2×2 and a step size of 2, and outputs an n=128-dimensional vector after processing; the network structure of the decoder includes 5 deconvolution layers with a convolution kernel size of 2×2×2 and a step size of 2, each deconvolution layer includes a batch normalization layer, a ReLU activation function layer, and a convolution layer with a convolution kernel size of 3×3×3 and a step size of 1.

4. The image quality processing system according to claim 2, characterized in that: The 2D vector output by the fully connected layer is a normal distribution with a mean of μ and an equation of σ.

5. The image quality processing method of any one of claims 1 to 4, characterized in that the method include: S1. Construct a generator and a discriminator, wherein the generator adopts a variational autoencoder including an encoder and a decoder; the discriminator adopts a network model cross-trained by a generative adversarial function, and the discriminator improves the high / low quality discrimination performance of the image data through adversarial training. After the model converges, the obtained discriminator is used as the image data quality discriminator; S2. Provide a high-quality image data set, the high-quality data supply unit provides a high-quality image data set to the encoder and the discriminator; S3. Generate a simulated low-quality image dataset. The generator reduces the high-quality image dataset to a low-dimensional space, adds noise, and degrades it to generate a simulated low-quality image dataset, specifically including: S31. Dimensionality reduction and noise addition for high-quality images. The encoder reduces the dimension of high-quality data to a low-dimensional space, outputs an n-dimensional vector distribution with half the size of the feature map and double the number of channels, and adds noise to the high-quality data in the low-dimensional space. S32. Degraded simulated low-quality image data sets are generated, and the noisy image is mapped back to the original image space by the decoder, and the feature map size is doubled and the number of channels is halved to obtain a degraded simulated low-quality image; S4. Output a high- and low-mixed quality image dataset. The generator transmits the generated simulated low-quality image dataset to the discriminator. The image quality discriminator compares the simulated low-quality image dataset transmitted by the generator and the high-quality image dataset provided by the data supply unit, and outputs a high- and low-mixed quality image dataset.

6. The method according to claim 5, characterized in that In step S1, the cross-training strategy of the discriminator network model includes: When training the variational autoencoder, the parameters of the discriminator are fixed and the variational autoencoder is adjusted. When training the variational autoencoder, the overall loss function of the model is L VAE =L JS -L GAN …………………………Formula-3, When training the discriminator, the parameters of the variational autoencoder are fixed and the parameters of the discriminator are adjusted. When training the discriminator, the overall loss function of the model is L VAE =L JS +L GAN …………………………Formula-4, Where, L JS is the JS divergence loss function, L GAN To generate adversarial loss function; Through cross-training, the noise generation ability of the variational autoencoder and the discriminator's ability to distinguish between high-quality and low-quality samples are simultaneously enhanced. After the model converges, an image quality discriminator for image quality assessment is obtained.

7. A device, characterized in that the device include: one or more processors; A memory for storing one or more programs; When one or more programs are executed by one or more processors, the one or more processors implement the image quality processing method as claimed in claim 5 or 6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the image quality processing method according to claim 5 or 6 is implemented.

Images