Picture quality evaluation method and apparatus

Abstract

The present disclosure relates to the technical field of picture processing, and provides a picture quality evaluation method and device. The method comprises: constructing a quality classification network and a quality regression network, and constructing a picture quality evaluation model by using a residual neural network, the quality classification network and the quality regression network; inputting a training picture in a training data set into the picture quality evaluation model, outputting a probability value of the training picture for each preset quality score by the quality classification network, and outputting a first quality score of the training picture by the quality regression network; calculating a total loss value corresponding to the training picture by a total loss function according to a plurality of preset quality scores, the probability value of the training picture for each preset quality score, a labeled quality score of the training picture and the first quality score; updating model parameters of the picture quality evaluation model by using the total loss value to complete training of the picture quality evaluation model; and performing picture quality evaluation by using the picture quality evaluation model after the training is completed.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of picture processing, and particularly relates to a picture quality evaluation method and device. BACKGROUND

[0002] In some scenarios of picture processing, it is often necessary to estimate the quality of a picture, and then use high-quality pictures to improve the efficiency of processing. For example, in face recognition, high-quality pictures are first selected, and then a face recognition model is trained using the high-quality pictures, so as to finally improve the accuracy of the face recognition model. At present, a trained model is commonly used to estimate the quality of a picture. The training of such a picture quality estimation model generally directly uses a square loss, and ignores the mining of difficult samples and important samples, resulting in inaccurate final picture quality estimation results.

[0003] In the process of implementing the present disclosure, the inventors have found that the related art has at least the following technical problem: low accuracy of picture quality evaluation. SUMMARY

[0004] Therefore, the embodiments of the present disclosure provide a picture quality evaluation method and device, an electronic device and a computer readable storage medium, to solve the problem of low accuracy of picture quality evaluation in the prior art.

[0005] In a first aspect, the embodiments of the present disclosure provide a picture quality evaluation method, comprising: constructing a quality classification network and a quality regression network, and constructing a picture quality evaluation model by using a residual neural network, the quality classification network and the quality regression network; inputting a training picture in a training data set into the picture quality evaluation model, outputting a probability value of the training picture being each preset quality score by the quality classification network, and outputting a first quality score of the training picture by the quality regression network; calculating a total loss value corresponding to the training picture by a total loss function according to a plurality of preset quality scores, the probability value of the training picture being each preset quality score, a labeled quality score of the training picture and the first quality score; updating model parameters of the picture quality evaluation model by using the total loss value, so as to complete the training of the picture quality evaluation model; and performing picture quality evaluation by using the picture quality evaluation model after the training is completed.

[0006] In a second aspect, the embodiment of the present disclosure provides a picture quality evaluation device, comprising: a construction module configured to construct a quality classification network and a quality regression network, and construct a picture quality evaluation model by using a residual neural network, the quality classification network and the quality regression network; an output module configured to input a training picture in a training data set into the picture quality evaluation model, output a probability value of the training picture for each preset quality score by using the quality classification network, and output a first quality score of the training picture by using the quality regression network; a calculation module configured to calculate a total loss value corresponding to the training picture by using a total loss function according to the plurality of preset quality scores, the probability value of the training picture for each preset quality score, a labeled quality score of the training picture and the first quality score; a training module configured to update a model parameter of the picture quality evaluation model by using the total loss value, so as to complete training of the picture quality evaluation model; and an evaluation module configured to perform picture quality evaluation by using the picture quality evaluation model after the training is completed.

[0007] In a third aspect, the embodiment of the present disclosure provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the above method when executing the computer program.

[0008] In a fourth aspect, the embodiment of the present disclosure provides a computer readable storage medium, which stores a computer program, and the computer program implements the steps of the above method when executed by a processor.

[0009] Compared with the prior art, the embodiment of the present disclosure has the beneficial effects that: the quality classification network and the quality regression network are constructed, the picture quality evaluation model is constructed by using the residual neural network, the quality classification network and the quality regression network, the training picture in the training data set is input into the picture quality evaluation model, the probability value of the training picture for each preset quality score is output by using the quality classification network, the first quality score of the training picture is output by using the quality regression network, the total loss value corresponding to the training picture is calculated by using the total loss function according to the plurality of preset quality scores, the probability value of the training picture for each preset quality score, the labeled quality score of the training picture and the first quality score, the model parameter of the picture quality evaluation model is updated by using the total loss value, so as to complete the training of the picture quality evaluation model, and the picture quality evaluation is performed by using the picture quality evaluation model after the training is completed. Therefore, by using the above technical means, the problem of low accuracy of evaluating picture quality in the prior art can be solved, and the accuracy of evaluating picture quality is improved. BRIEF DESCRIPTION OF DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without creative effort based on these drawings.

[0011] Figure 1 is a scene schematic diagram of an application scenario of the embodiments of the present disclosure.

[0012] Figure 2 is a flow schematic diagram of a picture quality evaluation method provided by the embodiments of the present disclosure.

[0013] Figure 3 is a structural schematic diagram of a picture quality evaluation device provided by the embodiments of the present disclosure.

[0014] Figure 4 is a structural schematic diagram of an electronic device provided by the embodiments of the present disclosure. DETAILED DESCRIPTION

[0015] In the following description, specific details are set forth in order to provide a thorough understanding of the embodiments of the present disclosure. However, persons skilled in the art should understand that the present disclosure can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present disclosure with unnecessary details.

[0016] A picture quality evaluation method and device according to the embodiments of the present disclosure will be described in detail below with reference to the drawings.

[0017] Figure 1 is a scene schematic diagram of an application scenario of the embodiments of the present disclosure. The application scenario can include terminal devices 101, 102 and 103, a server 104, and a network 105.

[0018] The terminal devices 101, 102 and 103 can be hardware or software. When the terminal devices 101, 102 and 103 are hardware, they can be various electronic devices with a display screen and supporting communication with the server 104, including but not limited to a smart phone, a tablet computer, a laptop computer and a desktop computer, etc.; when the terminal devices 101, 102 and 103 are software, they can be installed in the electronic devices as above. The terminal devices 101, 102 and 103 can be implemented as multiple software or software modules, or as a single software or software module, and the embodiments of the present disclosure do not make any limitation in this aspect. Further, various applications can be installed on the terminal devices 101, 102 and 103, such as a data processing application, an instant messaging tool, a social platform software, a search application, a shopping application, etc.

[0019] The server 104 can be a server providing various services, for example, a background server receiving a request sent by a terminal device establishing a communication connection therewith. The background server can receive and analyze the request sent by the terminal device, and generate a processing result. The server 104 can be a single server, a server cluster composed of several servers, or a cloud computing service center, and the embodiments of the present disclosure do not make any limitation in this aspect.

[0020] It should be noted that the server 104 can be hardware or software. When the server 104 is hardware, it can be various electronic devices providing various services for the terminal devices 101, 102 and 103. When the server 104 is software, it can be multiple software or software modules providing various services for the terminal devices 101, 102 and 103, or a single software or software module providing various services for the terminal devices 101, 102 and 103, and the embodiments of the present disclosure do not make any limitation in this aspect.

[0021] The network 105 can be a wired network connected by coaxial cables, twisted pairs and optical fibers, or a wireless network realizing interconnection of various communication devices without wiring, for example, Bluetooth, Near Field Communication (NFC), Infrared, etc., and the embodiments of the present disclosure do not make any limitation in this aspect.

[0022] A user can establish a communication connection with the server 104 via the network 105 through the terminal devices 101, 102 and 103 to receive or send information, etc. It should be noted that the specific types, quantities and combinations of the terminal devices 101, 102 and 103, the server 104 and the network 105 can be adjusted according to actual needs of an application scenario, and the embodiments of the present disclosure do not make any limitation in this aspect.

[0023] Figure 2is a flowchart of a picture quality evaluation method provided by an embodiment of the disclosure. Figure 2 The picture quality evaluation method can be executed by a computer or a server of Figure 1 or software on the computer or the server. As shown in Figure 2 , the picture quality evaluation method comprises the following steps.

[0024] S201, constructing a quality classification network and a quality regression network, and constructing a picture quality evaluation model by using a residual neural network, the quality classification network and the quality regression network;

[0025] S202, inputting a training picture in a training data set into the picture quality evaluation model, outputting a probability value of the training picture for each preset quality score by the quality classification network, and outputting a first quality score of the training picture by the quality regression network;

[0026] S203, calculating a total loss value corresponding to the training picture by a total loss function according to the plurality of preset quality scores, the probability value of the training picture for each preset quality score, a labeled quality score of the training picture and the first quality score;

[0027] S204, updating a model parameter of the picture quality evaluation model by using the total loss value, so as to complete training of the picture quality evaluation model;

[0028] S205, performing picture quality evaluation by using the picture quality evaluation model after the training is completed.

[0029] The residual neural network can be regarded as a backbone network, the quality classification network and the quality regression network are connected in parallel after the backbone network, and the picture quality evaluation model is obtained. Of course, the picture quality evaluation model can have other networks in addition to the residual neural network, the quality classification network and the quality regression network, for example, the picture quality evaluation model is connected with a layer of network at the end, and the layer of network is used for weighted summation of results finally output by the quality classification network and the quality regression network. The quality classification network outputs the probability value of the training picture for each preset quality score, and the quality classification network can also calculate a second quality score of the training picture according to the plurality of preset quality scores and the probability value of the training picture for each preset quality score, so the quality classification network outputs the second quality score of the training picture. The layer of network connected with the picture quality evaluation model at the end is actually weighted summation of the first quality score and the second quality score. The training data set should include a large number of training pictures, for the convenience of understanding, the training picture can be regarded as one. The labeled quality score of the training picture is a label of the training picture.

[0030] According to the technical scheme provided by the embodiment of the present disclosure, the quality classification network and the quality regression network are constructed, and the picture quality evaluation model is constructed by using the residual neural network, the quality classification network and the quality regression network; the training picture in the training data set is input into the picture quality evaluation model, the probability value of the training picture being each preset quality score is output by the quality classification network, and the first quality score of the training picture is output by the quality regression network; according to the plurality of preset quality scores, the probability value of the training picture being each preset quality score, the labeled quality score of the training picture and the first quality score, the total loss value corresponding to the training picture is calculated by the total loss function; the model parameters of the picture quality evaluation model are updated by using the total loss value, so as to complete the training of the picture quality evaluation model; the picture quality evaluation model after the training is completed is used for picture quality evaluation, therefore, by using the above technical means, the problem of low accuracy of evaluating picture quality in the prior art can be solved, and the accuracy of evaluating picture quality is improved.

[0031] Optionally, the internal structure of the quality classification network comprises, in sequence, a plurality of convolution layers, a batch normalization layer, an activation layer, an average pooling layer, a full connection layer, an activation layer, a full connection layer, and a softmax layer. For example, the internal structure of the quality classification network comprises, in sequence, two convolution layers with a 3x3 kernel and a channel number of 512, one batch normalization layer, one prelu activation layer, one average pooling layer, one full connection layer with a dimension of (512, 256), one prelu activation layer, one full connection layer with a dimension of (256, 101), and a softmax operation connected at the end.

[0032] Optionally, the internal structure of the quality regression network comprises, in sequence, a depth separable convolution layer, a batch normalization layer, an activation layer, a normal convolution layer, an activation layer, an average pooling layer, a full connection layer, an activation layer, a full connection layer, and a sigmoid layer. For example, the internal structure of the quality regression network comprises, in sequence, one depth separable convolution layer with a 3x3 kernel and a channel number of 512, one batch normalization layer, one gelu activation layer, one normal convolution layer with a 1x1 kernel and a channel number of 512, one relu activation layer, one average pooling operation, one full connection layer with a dimension of (512, 256), one relu activation layer, one full connection layer with a dimension of (256, 1), and a sigmoid layer connected at the end.

[0033] In step S203, the total loss value corresponding to the training picture is calculated by a total loss function according to the plurality of preset quality scores, the probability value of the training picture for each preset quality score, the labeled quality score of the training picture and the first quality score, including: the first loss value corresponding to the training picture is calculated by a first loss function according to the plurality of preset quality scores, the probability value of the training picture for each preset quality score and the labeled quality score of the training picture; the probability values of two preset quality scores adjacent to the labeled quality score of the training picture are determined, and the second loss value corresponding to the training picture is calculated by a second loss function according to the plurality of preset quality scores, the probability value of the training picture for each preset quality score, the labeled quality score of the training picture and the probability values of the two preset quality scores adjacent to the labeled quality score of the training picture; the third loss value corresponding to the training picture is calculated by a third loss function according to the labeled quality score of the training picture and the first quality score; and the total loss value corresponding to the training picture is calculated according to the first loss value, the second loss value and the third loss value of the training picture, and the total loss function includes the first loss function, the second loss function and the third loss function.

[0034] The first loss value loss1 corresponding to the training picture is calculated by the first loss function:

[0035]

[0036] i is the serial number of each preset quality score, q i is the i-th preset quality score, p i is the probability value of the training picture for the i-th preset quality score, y is the labeled quality score of the training picture, a is a prediction parameter, || is an absolute value symbol, and ∑ is a summation symbol.

[0037] a is generally set to 10.

[0038] In this embodiment, there are 101 preset quality scores, which are from 0 to 100. The serial number of the preset quality score 0 is 0, the serial number of the preset quality score 100 is 100, the probability value of the training picture for the 0th preset quality score, that is, 0, is 0.005 (which means that the probability value of the training picture quality score is 0.005) …… the probability value of the training picture for the 100th preset quality score, that is, 100, is 0.005. The sum of the probability values corresponding to the 101 preset quality scores is 1.

[0039] Because the gap |y-q i is larger, the exponential term will also be amplified, and the overall loss1 will be sharply amplified, so the first loss value can also be called a distance exponential amplification loss.

[0040] The probability values of two preset quality scores adjacent to the labeled quality score of the training picture are determined, the second loss value corresponding to the training picture is calculated by a second loss function according to the plurality of preset quality scores, the probability value of the training picture for each preset quality score, the labeled quality score of the training picture, and the probability values of the two preset quality scores adjacent to the labeled quality score of the training picture, including: the second quality score of the training picture is calculated according to the plurality of preset quality scores and the probability value of the training picture for each preset quality score; the probability difficulty weight coefficient of the training picture is calculated according to the probability values of the two preset quality scores adjacent to the labeled quality score of the training picture; the variance difficulty weight coefficient of the training picture is calculated according to the plurality of preset quality scores, the probability value of the training picture for each preset quality score, and the labeled quality score of the training picture; the second quality score of the training picture, the probability difficulty weight coefficient, and the variance difficulty weight coefficient are used to calculate the second loss value corresponding to the training picture by the second loss function.

[0041] The second loss value loss2 corresponding to the training picture is calculated by the second loss function:

[0042]

[0043]

[0044] h1=1-(p l +p r )

[0045]

[0046] i is the serial number of each preset quality score, q i is the i th preset quality score, p i is the probability value of the training picture for the i th preset quality score, r is the second quality score of the training picture, ∑ is the summation symbol, p l and p r are the probability values of the two preset quality scores adjacent to the labeled quality score of the training picture, h1 is the probability difficulty weight coefficient of the training picture, y is the labeled quality score of the training picture, || is the absolute value symbol, h2 is the variance difficulty weight coefficient of the training picture.

[0047] For example, the labeled quality score of the training picture is 56, and the two preset quality scores adjacent to the labeled quality score of the training picture are 55 and 57.

[0048] The third loss value loss3 corresponding to the training picture is calculated by the third loss function:

[0049] loss3=(y-z) 2

[0050] y is the labeled quality score of the training picture, and z is the first quality score of the training picture.

[0051] Total loss value loss = loss1 + loss2 + loss3

[0052] The total loss value loss corresponding to the training picture is calculated by the total loss function:

[0053]

[0054] In step 205, the picture quality evaluation model after training is used for picture quality evaluation, including: obtaining a target picture to be evaluated for quality; inputting the target picture into the picture quality evaluation model after training, and outputting a third quality score of the target picture.

[0055] The target picture is input into the picture quality evaluation model after training, the fourth quality score of the training picture is output through the quality regression network in the picture quality evaluation model after training, the fifth quality score of the training picture is output through the quality classification network in the picture quality evaluation model after training, and the third quality score is obtained by weighting and summing the fourth quality score and the fifth quality score.

[0056] Wherein, the fifth quality score of the target picture is calculated according to the plurality of preset quality scores and the probability value of each preset quality score for the target picture.

[0057] All the optional technical solutions can be combined to form optional embodiments of the present application, which will not be repeated here.

[0058] The following is an embodiment of the device of the present disclosure, which can be used to execute the method embodiment of the present disclosure. For details not disclosed in the device embodiment of the present disclosure, please refer to the method embodiment of the present disclosure.

[0059] Figure 3 is a schematic diagram of a picture quality evaluation device provided by an embodiment of the present disclosure. As shown in Figure 3 The picture quality evaluation device includes:

[0060] The construction module 301 is configured to construct the quality classification network and the quality regression network, and construct the picture quality evaluation model by using the residual neural network, the quality classification network and the quality regression network;

[0061] The output module 302 is configured to input the training picture in the training data set into the picture quality evaluation model, output the probability value of each preset quality score of the training picture through the quality classification network, and output the first quality score of the training picture through the quality regression network;

[0062] The calculation module 303 is configured to calculate the total loss value corresponding to the training image through the total loss function based on multiple preset quality scores, the probability value of the training image for each preset quality score, the labeled quality score of the training image and the first quality score;

[0063] Training module 304 is configured to update the model parameters of the image quality assessment model using the total loss value in order to complete the training of the image quality assessment model.

[0064] Evaluation module 305 is configured to perform image quality evaluation using the trained image quality evaluation model.

[0065] This can be viewed as using a residual neural network as the backbone, with quality classification and quality regression networks connected in parallel within the backbone to obtain the image quality assessment model. Of course, besides the residual neural network, quality classification network, and quality regression network, the image quality assessment model can also have other networks. For example, the final layer of the image quality assessment model can be used to perform a weighted sum of the final outputs of the quality classification and quality regression networks. The quality classification network outputs a probability value for each preset quality score for the training image. It can also calculate a second quality score for the training image based on multiple preset quality scores and the probability values ​​of the training image for each preset quality score. Therefore, the quality classification network outputs the second quality score of the training image. The final layer of the image quality assessment model is actually a weighted sum of the first and second quality scores. The training dataset should include a large number of training images; for ease of understanding, each training image can be considered a single image. The labeled quality score of the training image is its label.

[0066] According to the technical solution provided in this disclosure, a quality classification network and a quality regression network are constructed, and an image quality assessment model is built using a residual neural network, the quality classification network, and the quality regression network. Training images from the training dataset are input into the image quality assessment model. The quality classification network outputs the probability value of each preset quality score for the training images, and the quality regression network outputs the first quality score of the training images. Based on multiple preset quality scores, the probability value of each preset quality score for the training images, the labeled quality score of the training images, and the first quality score, a total loss value corresponding to the training images is calculated using a total loss function. The model parameters of the image quality assessment model are updated using the total loss value to complete the training of the image quality assessment model. The trained image quality assessment model is then used to perform image quality assessment. Therefore, by adopting the above technical means, the problem of low accuracy in image quality assessment in the prior art can be solved, thereby improving the accuracy of image quality assessment.

[0067] Optionally, the quality classification network internal structure in turn includes: a plurality of convolution layers, a batch normalization layer, an activation layer, an average pooling layer, a full connection layer, an activation layer, a full connection layer, and a softmax layer. For example, the quality classification network internal structure in turn includes 2 convolution layers with a 3x3 kernel and 512 channels, 1 batch normalization layer, 1 prelu activation layer, 1 average pooling layer, 1 full connection layer with a dimension of (512, 256), 1 prelu activation layer, 1 full connection layer with a dimension of (256, 101), and a subsequent softmax operation.

[0068] Optionally, the quality regression network internal structure in turn includes: a depth separable convolution layer, a batch normalization layer, an activation layer, a normal convolution layer, an activation layer, an average pooling layer, a full connection layer, an activation layer, a full connection layer, and a sigmoid layer. For example, the quality regression network internal structure in turn includes 1 depth separable convolution layer with a 3x3 kernel and 512 channels, 1 batch normalization layer, 1 gelu activation layer, 1 normal convolution layer with a 1x1 kernel and 512 channels, 1 relu activation layer, 1 average pooling operation, 1 full connection layer with a dimension of (512, 256), 1 relu activation layer, 1 full connection layer with a dimension of (256, 1), and a subsequent sigmoid layer.

[0069] Optionally, the computing module 303 is further configured to calculate, according to the plurality of preset quality scores, the probability value of the training picture for each preset quality score, and the labeled quality score of the training picture, a first loss value corresponding to the training picture by using a first loss function; determine the probability values of two preset quality scores adjacent to the labeled quality score of the training picture, and calculate, according to the plurality of preset quality scores, the probability value of the training picture for each preset quality score, the labeled quality score of the training picture, and the probability values of the two preset quality scores adjacent to the labeled quality score of the training picture, a second loss value corresponding to the training picture by using a second loss function; calculate, according to the labeled quality score of the training picture and the first quality score, a third loss value corresponding to the training picture by using a third loss function; and calculate a total loss value corresponding to the training picture according to the first loss value, the second loss value, and the third loss value of the training picture, wherein the total loss function includes the first loss function, the second loss function, and the third loss function.

[0070] Optionally, the computing module 303 is further configured to calculate, according to the plurality of preset quality scores, the probability value of the training picture for each preset quality score, and the labeled quality score of the training picture, a first loss value corresponding to the training picture by using a first loss function; determine the probability values of two preset quality scores adjacent to the labeled quality score of the training picture, and calculate, according to the plurality of preset quality scores, the probability value of the training picture for each preset quality score, the labeled quality score of the training picture, and the probability values of the two preset quality scores adjacent to the labeled quality score of the training picture, a second loss value corresponding to the training picture by using a second loss function; calculate, according to the labeled quality score of the training picture and the first quality score, a third loss value corresponding to the training picture by using a third loss function; and calculate a total loss value corresponding to the training picture according to the first loss value, the second loss value, and the third loss value of the training picture, wherein the total loss function includes the first loss function, the second loss function, and the third loss function.

[0071]

[0072] i is the serial number of each preset quality score, q i is the i-th preset quality score, p iis the probability value of the training picture being the i-th preset quality score, y is the labeled quality score of the training picture, a is a prediction parameter, || is an absolute value symbol, and å is a summation symbol.

[0073] a is generally set to 10.

[0074] In this embodiment, 101 preset quality scores are set, from 0 to 100. The serial number of the preset quality score 0 is 0, the serial number of the preset quality score 100 is 100, the probability value of the training picture being the 0-th preset quality score, that is, 0, is 0.005 (this sentence means that the probability value of the training picture being the quality score 0 is 0.005), and the probability value of the training picture being the 100-th preset quality score, that is, 100, is 0.005. The sum of the probability values corresponding to the 101 preset quality scores is 1.

[0075] Because the gap |y-q i is larger, the exponential term will also be amplified, and the overall loss1 will be sharply amplified, so the first loss value can also be referred to as a distance exponential amplification loss.

[0076] Optionally, the calculation module 303 is further configured to calculate a second quality score of the training picture according to the plurality of preset quality scores and the probability value of the training picture being each preset quality score; calculate a probability difficulty weight coefficient of the training picture according to the probability values of two preset quality scores adjacent to the labeled quality score of the training picture; calculate a variance difficulty weight coefficient of the training picture according to the plurality of preset quality scores, the probability value of the training picture being each preset quality score, and the labeled quality score of the training picture; and calculate a second loss value corresponding to the training picture by a second loss function according to the second quality score, the probability difficulty weight coefficient, and the variance difficulty weight coefficient of the training picture.

[0077] Optionally, the calculation module 303 is further configured to calculate a second loss value corresponding to the training picture by a second loss function:

[0078]

[0079]

[0080] h1=1-(p l +p r )

[0081]

[0082] i is the serial number of each preset quality score, q i is the i-th preset quality score, p i is the probability value of the training picture being the i-th preset quality score, and r is the second quality score of the training picture, å is a summation symbol, and pl and p r is a probability value of two preset quality scores adjacent to the labeled quality score of the training picture, h1 is a probability difficulty weight coefficient of the training picture, y is the labeled quality score of the training picture, || is an absolute value symbol, h2 is a variance difficulty weight coefficient of the training picture.

[0083] For example, if the labeled quality score of the training picture is 56, the two preset quality scores adjacent to the labeled quality score of the training picture are 55 and 57.

[0084] Optionally, the calculation module 303 is further configured to calculate the third loss value loss3 corresponding to the training picture by a third loss function:

[0085] loss3 = (y - z) 2

[0086] y is the labeled quality score of the training picture, and z is the first quality score of the training picture.

[0087] The total loss value loss = loss1 + loss2 + loss3

[0088] Optionally, the calculation module 303 is further configured to calculate the total loss value loss corresponding to the training picture by a total loss function:

[0089]

[0090] Optionally, the evaluation module 305 is further configured to obtain a target picture to be evaluated; input the target picture into the picture quality evaluation model after training is completed, and output a third quality score of the target picture.

[0091] The target picture is input into the picture quality evaluation model after training is completed, the fourth quality score of the training picture is output through the quality regression network in the picture quality evaluation model after training is completed, the fifth quality score of the training picture is output through the quality classification network in the picture quality evaluation model after training is completed, and the third quality score is obtained by weighted sum of the fourth quality score and the fifth quality score.

[0092] Wherein, the fifth quality score of the target picture is calculated according to the probability value of each preset quality score and the target picture.

[0093] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present disclosure.

[0094] Figure 4 is a schematic diagram of the electronic device 4 provided by the embodiments of the present disclosure. As shown in Figure 4As shown, the electronic device 4 of this embodiment includes a processor 401, a memory 402, and a computer program 403 stored in the memory 402 and executable on the processor 401. The processor 401 implements the steps in each of the above method embodiments when executing the computer program 403. Alternatively, the processor 401 implements the functions of each module / unit in each of the above apparatus embodiments when executing the computer program 403.

[0095] The electronic device 4 can be a desktop computer, a notebook computer, a palm computer, a cloud server, or the like. The electronic device 4 can include but is not limited to the processor 401 and the memory 402. Those skilled in the art can understand that the electronic device 4 can include more or less components, or different components than those shown. Figure 4 The electronic device 4 is merely an example and does not constitute a limitation on the electronic device 4, which can include more or less components, or different components than those shown.

[0096] The processor 401 can be a central processing unit (CPU), or other general purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or the like.

[0097] The memory 402 can be an internal storage unit of the electronic device 4, such as a hard disk or a memory of the electronic device 4. The memory 402 can also be an external storage device of the electronic device 4, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, or the like. The memory 402 can also include both an internal storage unit and an external storage device of the electronic device 4. The memory 402 is used to store computer programs and other programs and data required by the electronic device.

[0098] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of software functional unit.

[0099] If the integrated module / unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, all or part of the processes in the above-mentioned embodiment methods can also be instructed by a computer program to related hardware, and the computer program can be stored in a computer readable storage medium. The computer program can be executed by a processor to implement the steps of each method embodiment described above. The computer program can include computer program code, which can be in the form of source code, object code, executable file or some intermediate form. The computer readable medium can include any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content included in the computer readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.

[0100] The above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than limit them; although the present disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the protection scope of the present disclosure.

Claims

1. A method for evaluating image quality, characterized in that, include: Construct a quality classification network and a quality regression network, and use a residual neural network, the quality classification network, and the quality regression network to build an image quality assessment model; The training images in the training dataset are input into the image quality assessment model. The quality classification network outputs the probability value of each training image for each preset quality score, and the quality regression network outputs the first quality score of the training image. Based on multiple preset quality scores, the probability value of each preset quality score for the training image, the labeled quality score of the training image, and the first quality score, the total loss value corresponding to the training image is calculated using the total loss function. The model parameters of the image quality assessment model are updated using the total loss value to complete the training of the image quality assessment model; Image quality assessment is performed using the trained image quality assessment model. Based on multiple preset quality scores, the probability value of each training image for each preset quality score, the labeled quality score of the training image, and a first quality score, the total loss value corresponding to the training image is calculated using a total loss function, including: Based on multiple preset quality scores, the probability value of each preset quality score for the training image, and the labeled quality score of the training image, a first loss value corresponding to the training image is calculated using a first loss function. Determine the probability values ​​of two preset quality scores adjacent to the labeled quality score of the training image. Based on multiple preset quality scores, the probability value of each preset quality score for the training image, the labeled quality score of the training image, and the probability values ​​of two preset quality scores adjacent to the labeled quality score of the training image, calculate the second loss value corresponding to the training image using a second loss function. Based on the labeled quality score and the first quality score of the training image, the third loss value corresponding to the training image is calculated using the third loss function; The total loss value corresponding to the training image is calculated based on the first loss value, the second loss value, and the third loss value corresponding to the training image. The total loss function includes: the first loss function, the second loss function, and the third loss function.

2. The method according to claim 1, characterized in that, The first loss value loss1 corresponding to the training image is calculated using the first loss function: i is the sequence number of each preset quality score, q i It is the i-th preset quality score, p i Let be the probability value of the training image being the i-th preset quality score, y be the labeled quality score of the training image, α be the prediction parameter, and || be the absolute value operator. This is the summation symbol.

3. The method according to claim 1, characterized in that, The step of determining the probability values ​​of two preset quality scores adjacent to the labeled quality score of the training image, and calculating the second loss value corresponding to the training image using a second loss function based on multiple preset quality scores, the probability value of the training image for each preset quality score, the labeled quality score of the training image, and the probability values ​​of two preset quality scores adjacent to the labeled quality score of the training image, includes: The second quality score of the training image is calculated based on multiple preset quality scores and the probability value of the training image for each preset quality score. The probability difficulty weight coefficient of the training image is calculated based on the probability values ​​of two preset quality scores adjacent to the labeled quality score of the training image. Based on multiple preset quality scores, the probability value of each training image for each preset quality score, and the labeled quality score of the training image, the variance difficulty weight coefficient of the training image is calculated. The second quality score, probability difficulty weight coefficient, and variance difficulty weight coefficient of the training image are used to calculate the second loss value corresponding to the training image through the second loss function.

4. The method according to claim 1, characterized in that, The second loss value loss2 corresponding to the training image is calculated using the second loss function: i is the sequence number of each preset quality score, q i It is the i-th preset quality score, p i Let r be the probability value of the training image being the i-th preset quality score, and let r be the second quality score of the training image. p is the summation symbol. l and p r y is the probability value of two preset quality scores adjacent to the labeled quality score of the training image, h1 is the probability difficulty weight coefficient of the training image, y is the labeled quality score of the training image, || is the absolute value symbol, and h2 is the variance difficulty weight coefficient of the training image.

5. The method according to claim 1, characterized in that, The third loss value (loss3) corresponding to the training image is calculated using the third loss function. y is the labeled quality score of the training image, and z is the first quality score of the training image.

6. The method according to claim 1, characterized in that, Image quality assessment is performed using the trained image quality assessment model, including: Obtain the target image whose quality needs to be evaluated; The target image is input into the trained image quality assessment model, which outputs the third quality score of the target image.

7. An image quality assessment device, characterized in that, include: The construction module is configured to build a quality classification network and a quality regression network, and to construct an image quality assessment model using a residual neural network, the quality classification network and the quality regression network. The output module is configured to input training images from the training dataset into the image quality assessment model, output the probability value of each preset quality score for the training image through the quality classification network, and output the first quality score of the training image through the quality regression network. The calculation module is configured to calculate the total loss value corresponding to the training image based on multiple preset quality scores, the probability value of the training image for each preset quality score, the labeled quality score of the training image and the first quality score, through the total loss function. The training module is configured to update the model parameters of the image quality assessment model using the total loss value, so as to complete the training of the image quality assessment model; The evaluation module is configured to perform image quality evaluation using the trained image quality evaluation model. The calculation module is specifically configured to: calculate the first loss value corresponding to the training image through a first loss function based on multiple preset quality scores, the probability value of the training image for each preset quality score, and the labeled quality score of the training image; Determine the probability values ​​of two preset quality scores adjacent to the labeled quality score of the training image. Based on multiple preset quality scores, the probability value of each preset quality score for the training image, the labeled quality score of the training image, and the probability values ​​of the two preset quality scores adjacent to the labeled quality score of the training image, calculate the second loss value corresponding to the training image using a second loss function. Based on the labeled quality score and the first quality score of the training image, calculate the third loss value corresponding to the training image using a third loss function. The total loss value corresponding to the training image is calculated based on the first loss value, the second loss value, and the third loss value corresponding to the training image. The total loss function includes: the first loss function, the second loss function, and the third loss function.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 6.

Images