Image quality assessment method and device

Abstract

The present disclosure relates to the field of image processing technology, and provides an image quality assessment method and device. The method includes: obtaining a training data set, wherein the training data set contains multiple images with different degrees of distortion; constructing a twin model including two parallel branch networks, wherein each branch network includes: a basic image quality assessment network and a privacy image quality assessment network; using the training data set to train the twin model, wherein the two branch networks share weights during the training of the twin model; and using the trained twin model to provide image quality assessment services. The above-mentioned technical means can solve the problem in the prior art that the current method for assessing image quality cannot process privacy-containing images separately and is inefficient.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to an image quality assessment method and device. Background Art

[0002] Currently, the way to score pictures or evaluate the quality of pictures is often done manually, which is a lot of work for people. There is no fixed standard for manual evaluation, and different people have different scoring methods. At the same time, privacy-related factors (mainly private objects) are not included in the evaluation criteria for picture scoring, and private objects often have different scales, making them difficult to identify.

[0003] In the process of realizing the concept of the present disclosure, the inventors discovered that there are at least the following technical problems in the related art: the current method of evaluating image quality cannot process privacy-related images separately and is inefficient. Summary of the invention

[0004] In view of this, the embodiments of the present disclosure provide a method, device, electronic device and computer-readable storage medium for image quality assessment to solve the problems of low accuracy, missed detection and false detection in the image quality assessment model based on machine learning algorithm in the prior art.

[0005] According to a first aspect of an embodiment of the present disclosure, a method for image quality assessment is provided, comprising: obtaining a training data set, wherein the training data set contains multiple images with different degrees of distortion; constructing a twin model comprising two parallel branch networks, wherein each branch network comprises: a basic image quality assessment network and a privacy image quality assessment network; training the twin model using the training data set, wherein the two branch networks share weights during the training of the twin model; and providing image quality assessment services using the trained twin model.

[0006] According to a second aspect of an embodiment of the present disclosure, a picture quality assessment device is provided, comprising: an acquisition module, configured to acquire a training data set, wherein the training data set contains multiple pictures with different degrees of distortion; a construction module, configured to construct a twin model comprising two parallel branch networks, wherein each branch network comprises: a basic picture quality assessment network and a privacy picture quality assessment network; a training module, configured to train the twin model using the training data set, wherein the two branch networks share weights during the training of the twin model; and a service module, configured to provide picture quality assessment services using the trained twin model.

[0007] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0008] According to a fourth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, which stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0009] Compared with the prior art, the embodiments of the present disclosure have the following beneficial effects: because the embodiments of the present disclosure obtain a training data set, wherein the training data set contains multiple images with different degrees of distortion; construct a twin model including two parallel branch networks, wherein each branch network includes: a basic image quality assessment network and a privacy image quality assessment network; use the training data set to train the twin model, wherein the two branch networks share weights during the training of the twin model; use the trained twin model to provide image quality assessment services. Therefore, the above-mentioned technical means can be used to solve the problem in the prior art that the current method of evaluating image quality cannot process privacy-containing images separately and is inefficient, thereby improving the accuracy and efficiency of image quality assessment. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0011] Figure 1 is a schematic diagram of an application scenario of an embodiment of the present disclosure;

[0012] Figure 2 is a flowchart of a method for evaluating image quality provided by an embodiment of the present disclosure;

[0013] Figure 3 is a structural schematic diagram of a picture quality assessment device provided by an embodiment of the present disclosure;

[0014] Figure 4 It is a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0015] In the following description, specific details such as specific system structures and technologies are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present disclosure. However, it should be clear to those skilled in the art that the present disclosure may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obstructing the description of the present disclosure with unnecessary details.

[0016] A method and device for evaluating image quality according to an embodiment of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0017] Figure 1 1 is a schematic diagram of an application scenario of an embodiment of the present disclosure. The application scenario may include terminal devices 1, 2 and 3, a server 4 and a network 5.

[0018] Terminal devices 1, 2 and 3 can be hardware or software. When terminal devices 1, 2 and 3 are hardware, they can be various electronic devices with display screens and supporting communication with server 4, including but not limited to smart phones, tablet computers, laptop portable computers and desktop computers, etc.; when terminal devices 1, 2 and 3 are software, they can be installed in the above electronic devices. Terminal devices 1, 2 and 3 can be implemented as multiple software or software modules, or as a single software or software module, and the embodiments of the present disclosure are not limited to this. Furthermore, various applications can be installed on terminal devices 1, 2 and 3, such as data processing applications, instant messaging tools, social platform software, search applications, shopping applications, etc.

[0019] The server 4 may be a server that provides various services, for example, a background server that receives a request sent by a terminal device that establishes a communication connection with the server, and the background server may receive and analyze the request sent by the terminal device, and generate a processing result. The server 4 may be a single server, or a server cluster composed of several servers, or a cloud computing service center, which is not limited in the embodiments of the present disclosure.

[0020] It should be noted that the server 4 can be hardware or software. When the server 4 is hardware, it can be various electronic devices that provide various services for the terminal devices 1, 2, and 3. When the server 4 is software, it can be multiple software or software modules that provide various services for the terminal devices 1, 2, and 3, or it can be a single software or software module that provides various services for the terminal devices 1, 2, and 3, and the embodiments of the present disclosure are not limited to this.

[0021] The network 5 can be a wired network connected by coaxial cable, twisted pair and optical fiber, or it can be a wireless network that can interconnect various communication devices without wiring, such as Bluetooth, Near Field Communication (NFC), infrared, etc., which is not limited in the embodiments of the present disclosure.

[0022] The user can establish a communication connection with the server 4 via the network 5 through the terminal devices 1, 2 and 3 to receive or send information, etc. It should be noted that the specific types, quantities and combinations of the terminal devices 1, 2 and 3, the server 4 and the network 5 can be adjusted according to the actual needs of the application scenario, and the embodiments of the present disclosure are not limited to this.

[0023] Figure 2 It is a flowchart of a method for evaluating image quality provided by an embodiment of the present disclosure. Figure 2 The image quality assessment method can be Figure 1 The terminal device or server executes. Figure 2 As shown, the picture quality assessment method includes:

[0024] S201, obtaining a training data set, wherein the training data set contains a plurality of images with different distortion levels;

[0025] S202, constructing a twin model including two parallel branch networks, wherein each branch network includes: a basic image quality assessment network and a privacy image quality assessment network;

[0026] S203, training the twin model using the training data set, wherein the two branch networks share weights during the training of the twin model;

[0027] S204, using the trained twin model to provide image quality assessment services.

[0028] The basic image quality assessment network and the privacy image quality assessment network can be any neural network model, such as the faster-rcnn model. The two parallel branch networks in the twin model are the same neural network model. The disclosed embodiment uses a twin model instead of a branch network because the two branch networks share weights during the training of the twin model, the twin model has higher accuracy, and the model training is faster. The basic image quality assessment network and the privacy image quality assessment network in each branch network are also parallel or parallel.

[0029] According to the technical solution provided by the embodiment of the present disclosure, a training data set is obtained, wherein the training data set contains multiple pictures with different degrees of distortion; a twin model including two parallel branch networks is constructed, wherein each branch network includes: a basic picture quality assessment network and a privacy picture quality assessment network; the twin model is trained using the training data set, wherein the two branch networks share weights during the training of the twin model; and a picture quality assessment service is provided using the trained twin model. Therefore, the above-mentioned technical means can be used to solve the problem in the prior art that the current method for evaluating picture quality cannot process pictures containing privacy separately and is inefficient, thereby improving the accuracy and efficiency of picture quality assessment.

[0030] In step S203, the twin model is trained using the training data set, wherein the two branch networks share weights during the training of the twin model, including: performing a first training on the twin model using the training data set, wherein the two branch networks share weights in the first training; selecting any branch network from the twin model after the first training as the target branch network; performing a second training on the target branch network using the training data set; wherein the network structure of the basic image quality assessment network is: residual layer, convolution layer, pooling layer, convolution layer and pooling layer, and the network structure of the privacy image quality assessment network is: residual layer, convolution layer, pooling layer, convolution layer, pooling layer and spatial pyramid layer.

[0031] Sharing weights means sharing the parameters of the convolution kernel. Because the two branch networks share weights, the two branch networks in the twin model are the same after the first training. Any branch network is used as the target branch network, and the target branch network is trained for the second time to fine-tune the parameters of the target branch network. The trained twin model is used to provide image quality assessment services, and the target branch network is also used to provide image quality assessment services. In order to solve the problem of private objects of different sizes, a spatial pyramid is added to the privacy image quality assessment network. The spatial pyramid can perform deconvolution operations and can handle private objects of different sizes, thereby improving the recognition of private objects.

[0032] In step S203, the twin model is trained for the first time using the training data set, wherein the two branch networks share weights in the first training, including: dividing multiple pictures in the training data set into private pictures and basic pictures according to whether the pictures contain private objects; taking a private picture and a basic picture in the training data set as a group to obtain multiple groups; inputting the basic picture and the private picture in each group into a basic picture quality assessment network and a privacy picture quality assessment network in one branch network, respectively, to obtain multiple first losses and multiple second losses, respectively; inputting the basic picture and the private picture in each group into a basic picture quality assessment network and a privacy picture quality assessment network in another branch network, respectively, to obtain multiple third losses and multiple fourth losses, respectively; according to the first loss, second loss, third loss and fourth loss corresponding to each group, determining the gradient values ​​corresponding to the two branch networks according to the first preset rule; according to the gradient values, calculating the model parameters of the two branch networks using the back propagation algorithm, and updating the model parameters of the two branch networks using the stochastic gradient descent method.

[0033] The base image does not contain private objects, while the private image contains private objects. The base image quality assessment network in the branch network is trained with the base image and can be used to assess the quality of the base image. The privacy image quality assessment network in the branch network is trained with the privacy image and can be used to assess the quality of the privacy image. Of course, before using the target branch network to provide image quality assessment services, it should be determined whether the image to be assessed is a base image or a privacy image before using the base image quality assessment network and the privacy image quality assessment network. Of course, the image to be assessed can also be input into the base image quality assessment network and the privacy image quality assessment network at the same time, and the outputs of the two networks are weighted and summed, and the result is used as the assessment result of the image to be assessed.

[0034] Optionally, multiple groups can be equally divided into two data sets, and the two data sets are used to train the basic image quality assessment network and the privacy image quality assessment network respectively. This way, the amount of training can be reduced to half of the original amount, which is also one of the advantages of the twin model.

[0035] In step S203, according to the first loss, second loss, third loss and fourth loss corresponding to each group, the gradient values ​​corresponding to the two branch networks are determined according to the first preset rule, including: when the losses corresponding to the two branch networks are the same, the gradient values ​​corresponding to the two branch networks remain unchanged; when the losses corresponding to the two branch networks are different, the gradient value of the branch network with a larger corresponding loss is increased, and the gradient value of the branch network with a smaller corresponding loss is reduced; the losses include: the first loss, the second loss, the third loss and the fourth loss.

[0036] The first loss corresponds to the third loss, and the first loss and the third loss are the losses of the basic image quality assessment network. The second loss corresponds to the fourth loss, and the second loss and the fourth loss are the losses of the privacy image quality assessment network. For example, a set of corresponding images are input into the basic image quality assessment network and the privacy image quality assessment network, and four losses are obtained: the first loss, the second loss, the third loss, and the fourth loss. If the first loss is the same as the third loss, and the second loss is the same as the fourth loss, then the gradient values ​​corresponding to the two branch networks remain unchanged and can be 0. If the first loss is greater than the third loss, and the second loss is greater than the fourth loss, then the gradient value of the basic image quality assessment network of the branch network corresponding to the first loss is increased, and the gradient value of the privacy image quality assessment network of the branch network corresponding to the third loss is increased. The size of the increase or decrease is indicated according to the first preset rule.

[0037] Before executing step S201, that is, obtaining a training data set, wherein the training data set contains multiple images with different degrees of distortion, the method further includes: obtaining multiple original images, and determining whether each original image is overexposed or underexposed; extracting a brightness score of each original image, updating the brightness score of the overexposed original image using a first adjustment function, and updating the brightness score of the underexposed original image using a second adjustment function; generating multiple first images based on the updated brightness score of each original image; and constructing a training data set based on the multiple first images.

[0038] To determine whether each original image is overexposed or underexposed or to extract the brightness score of each original image, you can use some image processing software.

[0039] The first adjustment function can be formulated as follows:

[0040]

[0041] Where (i, j) represents the spatial index, g = 1, 2, ..., G, indicating the g-th level of the G-level distortion level; γ1, α1, β1, σ1 are hyperparameters, representing shape parameters, which are 6.00×10 -3 3.00×10 -2 , 3.00 and 2.50. L is the brightness score. L(i,j) indicates the brightness score is related to the spatial index. V(i,j) is the brightness score of the original overexposed image after the update.

[0042] The second adjustment function can be formulated as follows:

[0043]

[0044] The shape parameters γ2, α2, β2, and σ2 are -1.80×10 -3 , -1.50×10 -2 , -2.1×10 -5 and -3.1. Then the adjusted brightness component is converted back to image I t1 U(i,j) is the brightness score of the original underexposed image after the update.

[0045] Constructing a training data set based on multiple first images, including: processing the multiple first images according to a preset operator to obtain multiple second images; compressing the multiple second images according to a preset distortion level to obtain multiple third images; constructing a training data set based on the multiple first images, the multiple second images and the multiple third images.

[0046] Processing the multiple first images according to the preset operator may be processing the multiple first images through three preset operators, and the three preset operators may be respectively understood as: an operator corresponding to a high-pass filter, an operator corresponding to chromatic aberration processing, and an operator corresponding to a global contrast reduction operation. The training data set constructed according to the multiple first images, the multiple second images, and the multiple third images greatly enriches the images used in the training and improves the accuracy of the model. The multiple first images, the multiple second images, and the multiple third images may be understood as distorted images under various circumstances. The image obtained after processing the multiple first images through the three preset operators may be a method for improving image fusion to a new image.

[0047] Of course, the multiple first pictures, the multiple second pictures and the multiple third pictures are all labeled.

[0048] Before executing step S201, that is, obtaining a training data set, wherein the training data set contains multiple images with different degrees of distortion, the method further includes: obtaining multiple original images, and labeling the multiple original images; dividing the multiple original images into private images and basic images according to whether each original image contains private objects; performing image stitching processing on multiple groups of one or more private images and one or more basic images according to a second preset rule to obtain multiple stitched images; generating a label for each stitched image according to the labels of the private images and the basic images corresponding to each stitched image; and constructing a training data set according to the labeled multiple original images and the multiple stitched images after the labels are generated.

[0049] In addition to the above processing, the private image and the basic image can also be spliced ​​to obtain a spliced ​​image, and a training data set can be constructed based on the labeled original images and the labeled spliced ​​images. The image splicing processing of multiple groups of one or more private images and one or more basic images is performed according to the second preset rule, including:

[0050] X i It's a private picture, X j It is a normal picture without any private content. i With Y j is its corresponding score value. Then the image obtained by image splicing and its sample and corresponding label are It is expressed as follows:

[0051]

[0052]

[0053] M∈{0,1} W×HRepresents a binary mask for filling the removed area, ⊙ represents pixel-by-pixel multiplication, 1 represents a binary mask with all elements set to 1, and λ belongs to the Beta distribution: λ~Beta(α,α), where α=1 means λ belongs to the uniform distribution of (0,1).

[0054] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present application, which will not be described one by one here.

[0055] The following are embodiments of the device disclosed herein, which can be used to execute the method embodiments disclosed herein. For details not disclosed in the device embodiments disclosed herein, please refer to the method embodiments disclosed herein.

[0056] Figure 3 is a schematic diagram of a picture quality assessment device provided by an embodiment of the present disclosure. Figure 3 As shown, the picture quality assessment device includes:

[0057] An acquisition module 301 is configured to acquire a training data set, wherein the training data set contains a plurality of images with different distortion levels;

[0058] The construction module 302 is configured to construct a twin model including two parallel branch networks, wherein each branch network includes: a basic image quality assessment network and a privacy image quality assessment network;

[0059] A training module 303 is configured to train the twin model using the training data set, wherein the two branch networks share weights during the training of the twin model;

[0060] The service module 304 is configured to provide image quality assessment services using the trained twin model.

[0061] The basic image quality assessment network and the privacy image quality assessment network can be any neural network model, such as a faster-rcnn model. The two parallel branch networks in the twin model are the same neural network model. The disclosed embodiment uses a twin model instead of a branch network because the two branch networks share weights during the training of the twin model, the twin model has higher accuracy, and the model training is faster. The basic image quality assessment network and the privacy image quality assessment network in each branch network are also parallel or parallel.

[0062] According to the technical solution provided by the embodiment of the present disclosure, a training data set is obtained, wherein the training data set contains multiple pictures with different degrees of distortion; a twin model including two parallel branch networks is constructed, wherein each branch network includes: a basic picture quality assessment network and a privacy picture quality assessment network; the twin model is trained using the training data set, wherein the two branch networks share weights during the training of the twin model; and a picture quality assessment service is provided using the trained twin model. Therefore, the above-mentioned technical means can be used to solve the problem in the prior art that the current method for evaluating picture quality cannot process pictures containing privacy separately and is inefficient, thereby improving the accuracy and efficiency of picture quality assessment.

[0063] Optionally, the training module 303 is also configured to perform a first training on the twin model using the training data set, wherein the two branch networks share weights in the first training; select any branch network from the twin model after the first training as the target branch network; and perform a second training on the target branch network using the training data set; wherein the network structure of the basic image quality assessment network is: residual layer, convolution layer, pooling layer, convolution layer and pooling layer, and the network structure of the privacy image quality assessment network is: residual layer, convolution layer, pooling layer, convolution layer, pooling layer and spatial pyramid layer.

[0064] Sharing weights means sharing the parameters of the convolution kernel. Because the two branch networks share weights, the two branch networks in the twin model are the same after the first training. Any branch network is used as the target branch network, and the target branch network is trained for the second time to fine-tune the parameters of the target branch network. The trained twin model is used to provide image quality assessment services, and the target branch network is also used to provide image quality assessment services. In order to solve the problem of private objects of different sizes, a spatial pyramid is added to the privacy image quality assessment network. The spatial pyramid can perform deconvolution operations and can handle private objects of different sizes, thereby improving the recognition of private objects.

[0065] Optionally, the training module 303 is also configured to divide multiple pictures in the training data set into private pictures and basic pictures according to whether the pictures contain private objects; take a private picture and a basic picture in the training data set as a group to obtain multiple groups; input the basic picture and the private picture in each group into the basic picture quality assessment network and the privacy picture quality assessment network in a branch network, respectively, to obtain multiple first losses and multiple second losses, respectively; input the basic picture and the private picture in each group into the basic picture quality assessment network and the privacy picture quality assessment network in another branch network, respectively, to obtain multiple third losses and multiple fourth losses, respectively; determine the gradient values ​​corresponding to the two branch networks according to the first loss, second loss, third loss and fourth loss corresponding to each group according to the first preset rule; calculate the model parameters of the two branch networks according to the gradient values ​​using the back propagation algorithm, and update the model parameters of the two branch networks using the stochastic gradient descent method.

[0066] The base image does not contain private objects, while the private image contains private objects. The base image quality assessment network in the branch network is trained with the base image and can be used to assess the quality of the base image. The privacy image quality assessment network in the branch network is trained with the privacy image and can be used to assess the quality of the privacy image. Of course, before using the target branch network to provide image quality assessment services, it should be determined whether the image to be assessed is a base image or a privacy image before using the base image quality assessment network and the privacy image quality assessment network. Of course, the image to be assessed can also be input into the base image quality assessment network and the privacy image quality assessment network at the same time, and the outputs of the two networks are weighted and summed, and the result is used as the assessment result of the image to be assessed.

[0067] Optionally, multiple groups can be equally divided into two data sets, and the two data sets are used to train the basic image quality assessment network and the privacy image quality assessment network respectively. This way, the amount of training can be reduced to half of the original amount, which is also one of the advantages of the twin model.

[0068] Optionally, the training module 303 is also configured to keep the gradient values ​​corresponding to the two branch networks unchanged when the losses corresponding to the two branch networks are the same; to increase the gradient value of the branch network with a larger loss and reduce the gradient value of the branch network with a smaller loss when the losses corresponding to the two branch networks are different; the losses include: a first loss, a second loss, a third loss and a fourth loss.

[0069] The first loss corresponds to the third loss, and the first loss and the third loss are the losses of the basic image quality assessment network. The second loss corresponds to the fourth loss, and the second loss and the fourth loss are the losses of the privacy image quality assessment network. For example, a set of corresponding images are input into the basic image quality assessment network and the privacy image quality assessment network, and four losses are obtained: the first loss, the second loss, the third loss, and the fourth loss. If the first loss is the same as the third loss, and the second loss is the same as the fourth loss, then the gradient values ​​corresponding to the two branch networks remain unchanged and can be 0. If the first loss is greater than the third loss, and the second loss is greater than the fourth loss, then the gradient value of the basic image quality assessment network of the branch network corresponding to the first loss is increased, and the gradient value of the privacy image quality assessment network of the branch network corresponding to the third loss is increased. The size of the increase or decrease is indicated according to the first preset rule.

[0070] Optionally, the acquisition module 301 is also configured to acquire multiple original images and determine whether each original image is overexposed or underexposed; extract the brightness score of each original image, update the brightness score of the overexposed original image using the first adjustment function, and update the brightness score of the underexposed original image using the second adjustment function; generate multiple first images based on the updated brightness score of each original image; and construct a training data set based on the multiple first images.

[0071] To determine whether each original image is overexposed or underexposed or to extract the brightness score of each original image, you can use some image processing software.

[0072] The first adjustment function can be formulated as follows:

[0073]

[0074] Where (i, j) represents the spatial index, g = 1, 2, ..., G, indicating the g-th level of the G-level distortion level; γ1, α1, β1, σ1 are hyperparameters, representing shape parameters, which are 6.00×10 -3 3.00×10 -2 , 3.00 and 2.50. L is the brightness score. L(i,j) indicates the brightness score is related to the spatial index. V(i,j) is the brightness score of the original overexposed image after the update.

[0075] The second adjustment function can be expressed as follows:

[0076]

[0077] The shape parameters γ2, α2, β2, and σ2 are -1.80×10 -3 , -1.50×10 -2 , -2.1×10-5 and -3.1. Then the adjusted brightness component is converted back to image I t1 U(i,j) is the brightness score of the original underexposed image after the update.

[0078] Optionally, the acquisition module 301 is also configured to process multiple first images according to a preset operator to obtain multiple second images; compress multiple second images according to a preset distortion level to obtain multiple third images; and construct a training data set based on the multiple first images, the multiple second images, and the multiple third images.

[0079] Processing the multiple first images according to the preset operator may be processing the multiple first images through three preset operators, and the three preset operators may be respectively understood as: an operator corresponding to a high-pass filter, an operator corresponding to chromatic aberration processing, and an operator corresponding to a global contrast reduction operation. The training data set constructed according to the multiple first images, the multiple second images, and the multiple third images greatly enriches the images used in the training and improves the accuracy of the model. The multiple first images, the multiple second images, and the multiple third images may be understood as distorted images under various circumstances. The image obtained after processing the multiple first images through the three preset operators may be a method for improving image fusion to a new image.

[0080] Of course, the multiple first pictures, the multiple second pictures and the multiple third pictures are all labeled.

[0081] Optionally, the acquisition module 301 is also configured to acquire multiple original pictures and label the multiple original pictures; divide the multiple original pictures into private pictures and basic pictures according to whether each original picture contains private items; perform image stitching processing on multiple groups of one or more private pictures and one or more basic pictures according to a second preset rule to obtain multiple stitched pictures; generate a label for each stitched picture according to the labels of the private pictures and the basic pictures corresponding to each stitched picture; and construct a training data set based on the labeled multiple original pictures and the multiple stitched pictures after the label generation.

[0082] In addition to the above processing, the private image and the basic image can also be spliced ​​to obtain a spliced ​​image, and a training data set can be constructed based on the labeled original images and the labeled spliced ​​images. The image splicing processing of multiple groups of one or more private images and one or more basic images is performed according to the second preset rule, including:

[0083] X i It's a private picture, X j It is a normal picture without any private content. i With Y j is its corresponding score value. Then the image obtained by image splicing and its sample and corresponding label are It is expressed as follows:

[0084]

[0085]

[0086] M∈{0,1} W×H Represents a binary mask for filling the removed area, ⊙ represents pixel-by-pixel multiplication, 1 represents a binary mask with all elements set to 1, and λ belongs to the Beta distribution: λ~Beta(α,α), where α=1 means λ belongs to the uniform distribution of (0,1).

[0087] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present disclosure.

[0088] Figure 4 Schematic diagram of an electronic device 4 provided in an embodiment of the present disclosure. Figure 4 As shown, the electronic device 4 of this embodiment includes: a processor 401, a memory 402, and a computer program 403 stored in the memory 402 and executable on the processor 401. When the processor 401 executes the computer program 403, the steps in the above-mentioned method embodiments are implemented. Alternatively, when the processor 401 executes the computer program 403, the functions of the modules / units in the above-mentioned device embodiments are implemented.

[0089] Exemplarily, the computer program 403 may be divided into one or more modules / units, which are stored in the memory 402 and executed by the processor 401 to complete the present disclosure. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program 403 in the electronic device 4.

[0090] The electronic device 4 may be a desktop computer, a notebook, a PDA, a cloud server, or other electronic device. The electronic device 4 may include, but is not limited to, a processor 401 and a memory 402. Those skilled in the art will appreciate that Figure 4 It is only an example of the electronic device 4 and does not constitute a limitation of the electronic device 4. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device may also include input and output devices, network access devices, buses, etc.

[0091] The processor 401 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0092] The memory 402 may be an internal storage unit of the electronic device 4, for example, a hard disk or memory of the electronic device 4. The memory 402 may also be an external storage device of the electronic device 4, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 4. Further, the memory 402 may also include both an internal storage unit and an external storage device of the electronic device 4. The memory 402 is used to store computer programs and other programs and data required by the electronic device. The memory 402 may also be used to temporarily store data that has been output or is to be output.

[0093] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0094] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0095] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this disclosure.

[0096] In the embodiments provided in the present disclosure, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation. Multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection of devices or units, which may be electrical, mechanical or other forms.

[0097] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0098] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0099] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present disclosure implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. The computer program may include computer program code, and the computer program code may be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electric carrier signals and telecommunication signals.

[0100] The above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them. Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should all be included in the protection scope of the present disclosure.

Claims

1. A method for evaluating image quality, characterized in that: include: Obtaining a training data set, wherein the training data set contains a plurality of images with different distortion levels; Constructing a twin model including two parallel branch networks, wherein each of the branch networks includes a parallel basic image quality assessment network and a privacy image quality assessment network; Training the twin model using the training data set, wherein the two branch networks share weights during the training of the twin model; Providing image quality assessment services using the trained twin model; The using the training data set to train the twin model includes a first training, wherein the two branch networks share weights in the first training, and the first training includes: According to whether the pictures contain private objects, the multiple pictures in the training data set are divided into private pictures and basic pictures; Taking a private picture and a basic picture in the training data set as a group, obtaining a plurality of groups; Inputting the basic image and the private image in each group into the basic image quality assessment network and the private image quality assessment network in one of the branch networks, respectively, to obtain a plurality of first losses and a plurality of second losses; Inputting the base image and the private image in each group into the base image quality assessment network and the private image quality assessment network in another branch network respectively, to obtain a plurality of third losses and a plurality of fourth losses respectively; According to the first loss, the second loss, the third loss and the fourth loss corresponding to each group, the gradient values ​​corresponding to the two branch networks are determined according to the first preset rule; including: when the losses corresponding to the two branch networks are the same, the gradient values ​​corresponding to the two branch networks remain unchanged; when the losses corresponding to the two branch networks are different, the gradient value of the branch network with a larger corresponding loss is increased, and the gradient value of the branch network with a smaller corresponding loss is reduced; wherein the losses corresponding to the two branch networks correspond to the first loss and the third loss, and the second loss and the fourth loss; According to the gradient value, the model parameters of the two branch networks are calculated using a back propagation algorithm, and the model parameters of the two branch networks are updated using a stochastic gradient descent method.

2. The method according to claim 1, characterized in that: The step of training the twin model using the training data set further includes: Select any one of the branch networks from the twin model after the first training as the target branch network; Using the training data set to perform a second training on the target branch network; Among them, the network structure of the basic image quality assessment network is: residual layer, convolution layer, pooling layer, convolution layer and pooling layer; the network structure of the privacy image quality assessment network is: residual layer, convolution layer, pooling layer, convolution layer, pooling layer and spatial pyramid layer.

3. The method according to claim 1, characterized in that The method further comprises: obtaining a training data set, wherein before the training data set contains a plurality of images with different distortion levels: Obtain multiple original images, and determine whether each original image is overexposed or underexposed; Extracting the brightness score of each original image, updating the brightness score of the overexposed original image using the first adjustment function, and updating the brightness score of the underexposed original image using the second adjustment function; Generate multiple first images according to the updated brightness score of each original image; The training data set is constructed according to the multiple first pictures.

4. The method according to claim 3, characterized in that The step of constructing the training data set according to the plurality of first images includes: Processing the plurality of first images according to a preset operator to obtain a plurality of second images; compressing the plurality of second images according to a preset distortion level to obtain a plurality of third images; The training data set is constructed according to a plurality of first pictures, a plurality of second pictures and a plurality of third pictures.

5. The method according to claim 1, characterized in that The method further comprises: obtaining a training data set, wherein before the training data set contains a plurality of images with different distortion levels: Get multiple original pictures and label them; According to whether each original image contains private objects, multiple original images are divided into private images and basic images; Performing image stitching processing on multiple groups of one or more private images and one or more basic images according to a second preset rule to obtain multiple stitched images; Generate a label for each stitched image based on the labels of the private image and the base image corresponding to each stitched image; The training data set is constructed based on the labeled original images and the labeled spliced ​​images.

6. A picture quality assessment device, characterized in that: include: An acquisition module is configured to acquire a training data set, wherein the training data set contains a plurality of images with different distortion levels; A construction module is configured to construct a twin model including two parallel branch networks, wherein each of the branch networks includes a parallel basic image quality assessment network and a privacy image quality assessment network; A training module is configured to train the twin model using the training data set, wherein the two branch networks share weights during the training of the twin model; A service module, configured to provide image quality assessment services using the trained twin model; The training module is specifically configured to perform a first training, wherein the two branch networks share weights in the first training; the first training includes: According to whether the pictures contain private objects, the multiple pictures in the training data set are divided into private pictures and basic pictures; A private picture and a basic picture in the training data set are taken as a group to obtain multiple groups; the basic picture and the private picture in each group are respectively input into the basic picture quality assessment network and the private picture quality assessment network in one of the branch networks to obtain multiple first losses and multiple second losses respectively; Inputting the base image and the private image in each group into the base image quality assessment network and the private image quality assessment network in another branch network respectively, to obtain a plurality of third losses and a plurality of fourth losses respectively; According to the first loss, the second loss, the third loss and the fourth loss corresponding to each group, the gradient values ​​corresponding to the two branch networks are determined according to the first preset rule; including: when the losses corresponding to the two branch networks are the same, the gradient values ​​corresponding to the two branch networks remain unchanged; when the losses corresponding to the two branch networks are different, the gradient value of the branch network with a larger corresponding loss is increased, and the gradient value of the branch network with a smaller corresponding loss is reduced; wherein the losses corresponding to the two branch networks correspond to the first loss and the third loss, and the second loss and the fourth loss; According to the gradient value, the model parameters of the two branch networks are calculated using a back propagation algorithm, and the model parameters of the two branch networks are updated using a stochastic gradient descent method.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images