Face retouching image quality evaluation method and device based on texture analysis, equipment and medium

Abstract

Embodiments of the present application disclose a face retouching image quality evaluation method and device based on texture analysis, equipment and medium. The present application relates to the field of artificial intelligence technology, which comprises: obtaining a face retouching image to be evaluated, and adjusting the face retouching image to be evaluated to a preset size to obtain a target face retouching image to be evaluated; inputting the target face retouching image to be evaluated into a preset image quality evaluation model, so that the image quality evaluation model identifies the target face retouching image to be evaluated, and generates a quality score corresponding to the target face retouching image to be evaluated; when the image quality evaluation model identifies the target face retouching image to be evaluated, feature extraction is first performed, then texture analysis is performed, and finally splicing and deformation are performed to obtain a feature vector, and the feature vector is input into a full connection layer to generate a quality score. The embodiments of the present application can improve the performance and accuracy of face retouching image quality evaluation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a face retouching image quality evaluation method and device based on texture analysis, equipment and medium. BACKGROUND

[0002] With the advent of the mobile Internet era, users no longer browse images constrained by time and space. A large number of face retouching images that cannot be distinguished as true or false have emerged. Users manually edit image content according to personal preferences using P-picture software, such as adjusting the skin state of a person and changing the shape of facial features. However, blindly pursuing image aesthetics and over-editing images can easily distort the original content information or generate false information. Therefore, how to design a reasonable quality evaluation method to effectively evaluate face retouching images is of great significance.

[0003] Image quality evaluation mainly relies on subjective experimental evaluation and objective algorithm evaluation. Subjective experimental evaluation assesses the visual experience effect of an image through questionnaires and scoring. However, it has the disadvantages of being time-consuming and labor-intensive, inconvenient to transplant and integrate, and is usually used to construct an image quality evaluation database. Objective evaluation methods automatically evaluate the visual experience effect of an image based on the analysis of image content and distortion characteristics. This method has the characteristics of strong stability, simple operation, and convenience. Although a large amount of work has been done on image quality evaluation based on objective evaluation methods in the prior art, there is little discussion on face retouching image quality evaluation. Therefore, the prior art method cannot accurately evaluate the quality of face retouching images. SUMMARY

[0004] The embodiments of the present application provide a face retouching image quality evaluation method and device based on texture analysis, and equipment and medium, aiming to solve the problem that the prior art method cannot accurately evaluate the quality of face retouching images.

[0005] In a first aspect, the embodiments of the present application provide a face retouching image quality evaluation method based on texture analysis, which includes:

[0006] Obtaining a face retouching image to be evaluated, and adjusting the face retouching image to be evaluated to a preset size to obtain a target face retouching image to be evaluated;

[0007] Inputting the target face retouching image to be evaluated into a preset image quality evaluation model, so that the image quality evaluation model identifies the target face retouching image to be evaluated, and generates a quality score corresponding to the target face retouching image to be evaluated, wherein the image quality evaluation model is obtained by training a quality evaluation model constructed using a training data set;

[0008] The image quality evaluation model is used for identifying the target face retouching image to be evaluated, and generating a quality score corresponding to the target face retouching image to be evaluated.

[0009] The target face retouching image to be evaluated is subjected to feature map extraction to obtain a first feature map, a second feature map, a third feature map and a fourth feature map, and the second feature map is subjected to texture analysis to obtain a texture feature map.

[0010] The texture feature map, the third feature map and the fourth feature map are subjected to splicing and deformation to obtain a feature vector, and the feature vector is input into a full connection layer to generate a quality score.

[0011] In a second aspect, an embodiment of the present application further provides a face retouching image quality evaluation device based on texture analysis, which comprises:

[0012] An acquisition adjustment unit is configured to acquire a face retouching image to be evaluated, and adjust the face retouching image to be evaluated to a preset size to obtain a target face retouching image to be evaluated;

[0013] A quality evaluation unit is configured to input the target face retouching image to be evaluated into a preset image quality evaluation model, so that the image quality evaluation model identifies the target face retouching image to be evaluated, and generates a quality score corresponding to the target face retouching image to be evaluated, wherein the image quality evaluation model is obtained by training a quality evaluation model constructed by using a training data set;

[0014] The image quality evaluation model is used for identifying the target face retouching image to be evaluated, and generating a quality score corresponding to the target face retouching image to be evaluated.

[0015] The target face retouching image to be evaluated is subjected to feature map extraction to obtain a first feature map, a second feature map, a third feature map and a fourth feature map, and the second feature map is subjected to texture analysis to obtain a texture feature map.

[0016] The texture feature map, the third feature map and the fourth feature map are subjected to splicing and deformation to obtain a feature vector, and the feature vector is input into a full connection layer to generate a quality score.

[0017] In a third aspect, an embodiment of the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the above method when executing the computer program.

[0018] In a fourth aspect, the embodiments of the present application further provide a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the method described above.

[0019] The embodiments of the present application provide a face retouching image quality evaluation method and device based on texture analysis, and a medium. The method comprises the following steps: obtaining a face retouching image to be evaluated, and adjusting the face retouching image to be evaluated to a preset size to obtain a target face retouching image to be evaluated; inputting the target face retouching image to be evaluated into a preset image quality evaluation model, so that the image quality evaluation model identifies the target face retouching image to be evaluated, and generates a quality score corresponding to the target face retouching image to be evaluated, wherein the image quality evaluation model is obtained by training a quality evaluation model constructed by using a training data set; and the identification of the target face retouching image to be evaluated by the image quality evaluation model and the generation of the quality score corresponding to the target face retouching image to be evaluated specifically comprise the following steps: extracting a first feature map, a second feature map, a third feature map and a fourth feature map from the target face retouching image to be evaluated, and performing texture analysis on the second feature map to obtain a texture feature map; splicing and deforming the texture feature map, the third feature map and the fourth feature map to obtain a feature vector, and inputting the feature vector into a full connection layer to generate the quality score. The technical scheme of the embodiments of the present application trains the quality evaluation model constructed by using the training data set to obtain the image quality evaluation model, which can improve the generalization of the image quality evaluation model; the trained image quality evaluation model identifies the input face retouching image to be evaluated, pays attention to low-level texture information to generate the texture feature map during the identification process, generates the quality score based on the texture feature map, and ensures that more attention is paid to the structural information of the face retouching image during the quality evaluation of the image, so that the performance and accuracy of the face retouching image quality evaluation can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical scheme of the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiment description. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0021] Figure 1 is a model overall schematic diagram of a face retouching image quality evaluation method based on texture analysis provided by the embodiments of the present application;

[0022] Figure 2 is a flowchart of a face retouching image quality evaluation method based on texture analysis provided by the embodiments of the present application;

[0023] Figure 3 A sub-process flow diagram of a face retouching image quality evaluation method based on texture analysis provided by an embodiment of the present application is shown in FIG. 6;

[0024] Figure 4 A texture enhancement module diagram of a face retouching image quality evaluation method based on texture analysis provided by an embodiment of the present application is shown in FIG. 7;

[0025] Figure 5 A dense convolution diagram of a face retouching image quality evaluation method based on texture analysis provided by an embodiment of the present application is shown in FIG. 8;

[0026] Figure 6 A sub-process flow diagram of a face retouching image quality evaluation method based on texture analysis provided by an embodiment of the present application is shown in FIG. 6;

[0027] Figure 7 A schematic block diagram of a face retouching image quality evaluation device based on texture analysis provided by an embodiment of the present application is shown in FIG. 9; and

[0028] Figure 8 A schematic block diagram of a computer device provided by an embodiment of the present application is shown in FIG. 10. DETAILED DESCRIPTION

[0029] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some embodiments of the present application, but not all embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0030] It should be understood that when used in the specification and the appended claims, the terms “comprise” and “include” indicate the presence of described features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0031] It should also be understood that the terms used in the present application specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the present application specification and the appended claims, the singular forms “a”, “an” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0032] It should also be further understood that the term "and / or" as used in the specification and in the claims, means any one of the associated listed items, or any combination of the associated listed items, and includes all possible combinations.

[0033] As used in the specification and in the claims, the term "if' can be interpreted as meaning "when" or "once" or "in response to a determination" or "in response to a detection" depending on the context. Similarly, the phrase "if determined" or "if detected [the described condition or event]" can be interpreted to mean "once determined" or "in response to a determination" or "once detected [the described condition or event]" or "in response to a detection [the described condition or event]" depending on the context.

[0034] Referring to Figure 1 , Figure 1 is a model overall structure schematic diagram of a face retouching image quality evaluation method based on texture analysis provided by an embodiment of the present application. The face retouching image quality evaluation method based on texture analysis provided by the embodiment of the present application can be applied to a constructed image quality evaluation model. For example, the face retouching image quality evaluation method based on texture analysis can be realized by configuring a software program corresponding to the image quality evaluation model, so as to improve the performance and accuracy of image quality evaluation.

[0035] Referring to Figure 2 , Figure 2 is a flowchart of a face retouching image quality evaluation method based on texture analysis provided by an embodiment of the present application. As shown in Figure 2 , the method comprises the following steps S110-S120.

[0036] S110, obtaining a face retouching image to be evaluated, and adjusting the face retouching image to be evaluated to a preset size to obtain a target face retouching image to be evaluated;

[0037] S120, inputting the target face retouching image to be evaluated into a preset image quality evaluation model, so that the image quality evaluation model identifies the target face retouching image to be evaluated, and generates a quality score corresponding to the target face retouching image to be evaluated, wherein the image quality evaluation model is obtained by training a constructed quality evaluation model by using a training data set.

[0038] In the embodiment of the present application, an input face retouching image to be evaluated is acquired, and the face retouching image to be evaluated is adjusted to a preset size, for example, the preset size is 512x512, to obtain a target face retouching image to be evaluated; the target face retouching image to be evaluated is input into a preset image quality evaluation model to generate a quality score corresponding to the target face retouching image to be evaluated, wherein, as shown in Figure 1 The image quality evaluation model includes a multi-scale feature extraction module (DFEM) and a multi-task learning module (MTLM), wherein the multi-scale feature extraction module takes a ResNet50 network as a basic network, and the ResNet50 network is a network trained by an ImageNet data set. It should be noted that in the embodiment, the multi-scale feature extraction module includes a texture enhancement module (TEM), which is used to make the network pay more attention to low-level texture information and improve the prediction performance. It should be noted that in the embodiment of the present application, the image quality evaluation model uses a training data set to train the constructed quality evaluation model to obtain an image quality evaluation model, which can improve the generalization of the image quality evaluation model; the input face retouching image to be evaluated is identified by the trained image quality evaluation model, and low-level texture information is paid attention to in the identification process to generate a texture feature map, and a quality score is generated based on the texture feature map, which ensures that more attention is paid to the structural information of the face retouching image when the image quality is evaluated, and the performance and accuracy of the face retouching image quality evaluation can be improved. The specific training steps will be described below.

[0039] Please refer to Figure 3 The image quality evaluation model identifies the target face retouching image to be evaluated and generates a quality score corresponding to the target face retouching image to be evaluated, which specifically includes the following steps S121-S122:

[0040] S121, feature maps of the target face retouching image to be evaluated are extracted to obtain a first feature map, a second feature map, a third feature map and a fourth feature map, and the second feature map is analyzed to obtain a texture feature map;

[0041] S122, the texture feature map, the third feature map and the fourth feature map are spliced and deformed to obtain a feature vector, and the feature vector is input into a full connection layer to generate a quality score.

[0042] In the embodiments of the present application, for the convenience of understanding, it is assumed that the first feature map, the second feature map, the third feature map and the fourth feature map are Layer1, Layer2, Layer3 and Layer4 respectively, the texture feature map is EnhancedTexture2, the target face retouch image to be evaluated is input into the multi-scale feature extraction module for feature map extraction to obtain Layer1, Layer2, Layer3 and Layer4, wherein Layer1 is a feature map with a size of 56x56, Layer2 is a feature map with a size of 28x28, Layer3 is a feature map with a size of 14x14, and Layer4 is a feature map with a size of 7x7. It can be known that Layer1, Layer2, Layer3 and Layer4 are feature maps with different spatial size dimensions. EnhancedTexture2 is obtained by performing texture analysis on Layer2 through the texture enhancement module. Specifically, the texture enhancement module first performs average pooling and up-sampling processing on the second feature map to obtain a pooled and sampled feature map. The difference between the second feature map and the pooled and sampled feature map is calculated to obtain a first texture feature map. Dense convolution is performed on the first texture feature map to obtain a second texture feature map, and maximum pooling is performed on the second texture feature map to obtain a first feature weight. Average pooling is performed on the second feature map to obtain a second feature weight, and the texture feature map is determined according to the first feature weight, the second feature weight and the second texture feature map. More specifically, the first feature weight and the second feature weight are spliced to obtain a third feature weight. The third feature weight is activated through an activation function to obtain a fourth feature weight. The product of the fourth feature weight and the second texture feature map is calculated to obtain the texture feature map.

[0043] Further, for the convenience of understanding, it is assumed that the average pooling is Avg pooling, the maximum pooling is Max pooling, and the dense convolution is Dense conv, as follows. Figure 4As shown, the texture enhancement module performs 2*2 average pooling on Layer2 (512, 28, 28) to obtain a feature map A1 (512, 14, 14), specifically, the 2*2 average pooling is as shown in formula (1), in which X is Layer2, A is the 2*2 average pooling on X, M and N are the size of Layer2, and after bilinear interpolation of A1 to realize up sampling, a pooling sampling feature map (512, 28, 28) is obtained, and Layer2 (512, 28, 28) is subtracted from the pooling sampling feature map (512, 28, 28) to obtain a first texture feature map Texture2_, and after a dense convolution (Dense Conv) operation is performed on Texture2_, a second texture feature map Texture2 is obtained, and Texture2 is as shown in formula (2); 28*28 average pooling is performed on Layer2 (specifically, see formula (3)) to obtain a first feature weight A2, 28*28 maximum pooling is performed on Texture2 to obtain a second feature weight M2 (see formula (4)), A2 and M2 are spliced to obtain a third feature weight of 1024 dimensions, and after being activated by an activation function (see formula (5)), a fourth feature weight (1024, 1, 1) is obtained, and the fourth feature weight is multiplied by Texture2 to obtain an enhanced texture feature map Enhanced Texture2. It should be noted that in the embodiment, as shown in formula (2), the dense convolution is composed of a series of convolution Conv, batch normalization BN and activation function RELU to establish the dense connection of the front and rear layers in the channel. Figure 5 As shown, the dense convolution is composed of a series of convolution Conv, batch normalization BN and activation function RELU to establish the dense connection of the front and rear layers in the channel.

[0044]

[0045] Texture2 = DenseConv (Layer2 - Upsample (AvgPooling1 (Layer2))) (2)

[0046]

[0047]

[0048]

[0049]

[0050] Further, the texture feature map Enhanced Texture2, the third feature map Layer3 and the fourth feature map Layer4 are spliced and deformed to obtain a feature vector Vc of 1*4096 dimensions, wherein the full connection layer includes a first full connection layer, a second full connection layer and a third full connection layer, the first full connection layer is an FC layer, the second full connection layer is an FC LBP layer, and the third full connection layer is an FC score layer. The feature vector Vc is subjected to the first full connection layer to obtain an initial feature vector V of 1*2048 dimensions, the front 1024 dimensions of the initial feature vector V are taken as an initial LBP feature vector V1, and the rear 1024 dimensions are taken as an initial score feature vector V2. V1 is input into the FC LBP layer to obtain an LBP feature, V1 and V2 are spliced to generate a target feature vector V3, and specifically, as shown in formula (7); the target feature vector V3 is input into the FC score layer to generate a one-dimensional quality score. It should be noted that, in the embodiment, the Local Binary Patter (LBP) is an operator for describing local texture features of an image, the LBP feature has advantages such as gray invariance and rotation invariance, it compares each pixel in the image with its neighborhood pixel values, and saves the comparison results as binary numbers as the encoding values of the center pixel, that is, LBP feature values. The LBP feature can help to extract texture structure information of the face retouching image, and further help to predict the quality score.

[0051] V3 = Concat(V1, V2) (7)

[0052] Please refer to Figure 6 , the constructed quality evaluation model is trained by using a training data set to obtain an image quality evaluation model, and the method specifically includes the following steps S131-S134:

[0053] S131, for each training batch image in the training data set, the image is input into the constructed quality evaluation model to output a predicted LBP feature and a predicted quality score;

[0054] S132, a first loss value is calculated by a first loss function according to the predicted LBP feature and a label LBP feature in the training data set;

[0055] S133, a second loss value is calculated by a second loss function according to the predicted quality score and a label quality score in the training data set;

[0056] S134, calculate the sum of the first loss value and the second loss value to obtain a total loss value, and iteratively update the quality evaluation model according to the total loss value until a preset training batch number is reached to obtain an image quality evaluation model.

[0057] In the embodiment of the present application, before training the constructed quality evaluation model, a preset number of image data are selected from a face retouch quality evaluation data set Retouch with labeled quality scores as a training data set, and the remaining image data are used as a test data set, wherein the preset number is 1280, the Retouch data set has a total of 1600 images, that is, there are 1280 image data in the training data set and 320 image data in the test data set, and the division ratio of the training data set to the test data set is 8:2. It should be noted that in this embodiment, the Retouch data set is collected from 200 high-definition original images in the FFHQ data set, 1600 images of 8 different tampering degrees are synthesized using various P software (for example, Photoshop, Meitu Xiu Xiu, Guangying Magic Hand), and 20 subjects are recruited in accordance with the scoring guidelines of the International Telecommunication Union. The scoring results of the 1600 image data are processed to obtain image label scores by eliminating abnormal data. The FFHQ data set consists of 70000 high-quality PNG images with a resolution of 1024x1024, and contains considerable differences in age, race and image background. It also has good coverage of accessories such as glasses, sunglasses and hats. Understandably, in the embodiment of the present application, the training data set is used for the training phase of the image quality evaluation model, and the test data set is used for testing the optimized image quality evaluation model. In other embodiments, the number of image data in the training data set and the test data set is not specifically limited.

[0058] Further, during the training of the quality evaluation model, each training batch image in the training data set is input into the constructed quality evaluation model to output a predicted LBP feature and a predicted quality score; a first loss value is calculated according to the predicted LBP feature and the label LBP feature in the training data set through a first loss function, wherein the first loss function is as shown in formula (8), and in formula (8), P ij is a label LBP, is a predicted LBP feature; a second loss value is calculated according to the predicted quality score and the label quality score in the training data set through a second loss function, wherein the second loss function is as shown in formula (9), and in formula (9), Y i is a label quality score, To predict the quality score, the sum of the first loss value and the second loss value is calculated to obtain a total loss value, specifically as shown in equation (10), and the quality evaluation model is iteratively updated according to the total loss value L until a preset training batch number is reached. Specifically, it is determined whether the total loss value is less than the previous total loss value. If the total loss value is not less than the previous total loss value, it indicates that the total loss value remains stable, and the trained quality evaluation model is taken as the image quality evaluation model. Otherwise, if the total loss value is less than the previous total loss value, it indicates that the total loss value is still decreasing, and the network parameters are continued to be set to optimize the network, and step S131 is returned to continue training the evaluation model. It should be noted that in this embodiment, the first loss function and the second loss function are both mean square error loss functions, which are used to constrain the difference between the label quality score and the predicted quality score.

[0059]

[0060]

[0061] L = L Score + L LBP (10)

[0062] Further, in the embodiment of the present application, after the preset training batch number is reached, the test data set is input into the trained quality evaluation model to obtain a predicted quality score. An index value is calculated according to the test quality score and the quality score in the test data set, wherein the index value is an SROCC coefficient, a PLCC coefficient, a KROCC coefficient, and an RMSE. Understandably, the closer the SROCC coefficient, the PLCC coefficient, and the KROCC coefficient are to 1, and the closer the RMSE coefficient is to 0, the better the performance of the image quality evaluation model, and the more accurate the image quality evaluation. It should be noted that in this embodiment, it has been verified that the SROCC in this embodiment is 0.9039, the PLCC is 0.9026, the KROCC is 0.7249, and the RMSE is 0.7444. Therefore, it can be known that the image quality evaluation model in this embodiment has a good evaluation effect on the face retouching image.

[0063] Further, in the embodiment of the present application, all images input into the image quality evaluation model need to be adjusted to a preset size, wherein the preset size is 512*512; the picture format is png format; the pictures in the training set are randomly horizontally flipped, and the pictures in the test set are not flipped, so as to improve the robustness of the image quality evaluation model. The image quality evaluation model is built under the PyTorch deep learning framework, and the Adam optimizer is used, the learning rate is set to 0.0001, the training batch size batchsize is set to 64, and the iteration number is set to 60. It should be noted that the network model in the embodiment is implemented on the PyTorch platform. Because the picture size in the data set is large, data preprocessing is performed on all pictures, and all pictures are uniformly cropped to 512*512. The training set is trained through the multi-scale feature extraction module and the multi-task learning module. The test set is tested using the trained network.

[0064] Figure 7 is a schematic block diagram of a face retouching image quality evaluation device 200 based on texture analysis provided by the embodiment of the present application. As shown in Figure 7 corresponding to the above face retouching image quality evaluation method based on texture analysis, the present application also provides a face retouching image quality evaluation device 200 based on texture analysis. The face retouching image quality evaluation device 200 based on texture analysis includes units for executing the above face retouching image quality evaluation method based on texture analysis, and the device can be configured in a computer device. Specifically, please refer to Figure 7 , the face retouching image quality evaluation device 200 based on texture analysis includes an acquisition adjustment unit 201 and a quality evaluation unit 202.

[0065] The acquisition adjustment unit 201 is configured to acquire a face retouching image to be evaluated, and adjust the face retouching image to be evaluated to a preset size to obtain a target face retouching image to be evaluated; the quality evaluation unit 202 is configured to input the target face retouching image to be evaluated into a preset image quality evaluation model, so that the image quality evaluation model identifies the target face retouching image to be evaluated, and generates a quality score corresponding to the target face retouching image to be evaluated, wherein the image quality evaluation model is obtained by training a quality evaluation model using a training data set; wherein the image quality evaluation model identifies the target face retouching image to be evaluated, and generates a quality score corresponding to the target face retouching image to be evaluated, and specifically includes a feature extraction unit and a splicing generation unit, wherein the feature extraction unit is configured to extract a first feature map, a second feature map, a third feature map and a fourth feature map from the target face retouching image to be evaluated, and perform texture analysis on the second feature map to obtain a texture feature map; and the splicing generation unit is configured to splice and deform the texture feature map, the third feature map and the fourth feature map to obtain a feature vector, and input the feature vector into a full connection layer to generate a quality score.

[0066] In some embodiments, such as the present embodiment, the feature map extraction unit includes a pooling up-sampling unit, a first calculation unit, a convolution pooling unit and a pooling determination unit.

[0067] The pooling up-sampling unit is configured to perform average pooling and up-sampling processing on the second feature map to obtain a pooled and sampled feature map; the first calculation unit is configured to calculate a difference between the second feature map and the pooled and sampled feature map to obtain a first texture feature map; the convolution pooling unit is configured to perform dense convolution on the first texture feature map to obtain a second texture feature map, and perform maximum pooling on the second texture feature map to obtain a first feature weight; and the pooling determination unit is configured to perform average pooling on the second feature map to obtain a second feature weight, and determine a texture feature map according to the first feature weight, the second feature weight and the second texture feature map.

[0068] In some embodiments, such as the present embodiment, the pooling determination unit includes a first splicing unit, an activation unit and a second calculation unit.

[0069] The first splicing unit is configured to splice the first feature weight and the second feature weight to obtain a third feature weight; the activation unit is configured to activate the third feature weight by an activation function to obtain a fourth feature weight; and the second calculation unit is configured to calculate a product of the fourth feature weight and the second texture feature map to obtain a texture feature map.

[0070] In some embodiments, such as the present embodiment, the splicing generation unit comprises a first as unit, a second splicing unit, and a generation unit.

[0071] The first as unit is configured to reduce the dimension of the feature vector to obtain an initial feature vector, take the vectors of the initial feature vector with the first preset number of dimensions as initial LBP feature vectors, and take all the vectors remaining after removing the vectors of the initial feature vector with the first preset number of dimensions as initial fractional feature vectors; the second splicing unit is configured to splice the initial LBP feature vectors and the initial fractional feature vectors to generate a target feature vector; and the generation unit is configured to input the target feature vector into the fully connected layer to generate a quality score.

[0072] In some embodiments, such as the present embodiment, training the constructed quality evaluation model using the training data set to obtain an image quality evaluation model comprises an input-output unit, a third calculation unit, a fourth calculation unit, and an iterative update unit.

[0073] The input-output unit is configured to input each training batch image in the training data set into the constructed quality evaluation model to output a predicted LBP feature and a predicted quality score; the third calculation unit is configured to calculate a first loss value according to the predicted LBP feature and a label LBP feature in the training data set by using a first loss function; the fourth calculation unit is configured to calculate a second loss value according to the predicted quality score and a label quality score in the training data set by using a second loss function; and the iterative update unit is configured to calculate a total loss value by summing the first loss value and the second loss value, and iteratively update the quality evaluation model according to the total loss value until a preset number of training batches is reached.

[0074] In some embodiments, such as the present embodiment, after training the constructed quality evaluation model using the training data set to obtain an image quality evaluation model, the method further comprises an input unit and a fifth calculation unit.

[0075] The input unit is configured to input a test data set into the trained quality evaluation model to obtain a test quality score; and the fifth calculation unit is configured to calculate an index value according to the test quality score and a quality score in the test data set.

[0076] The specific implementation of the face retouching image quality evaluation device 200 based on texture analysis according to the embodiments of the present application corresponds to the face retouching image quality evaluation method based on texture analysis described above, and will not be repeated here.

[0077] The face retouching image quality evaluation device based on texture analysis described above can be implemented in the form of a computer program, which can run on a computer system as shown in Figure 8The computer device shown is running.

[0078] Please refer to Figure 8 , Figure 8 is a schematic block diagram of a computer device provided by an embodiment of the present application. The computer device 300 is a server, specifically, the server can be a standalone server, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and basic cloud computing services such as big data and artificial intelligence platforms.

[0079] Please refer to Figure 8 , the computer device 300 includes a processor 302, a memory, and a network interface 305 connected through a system bus 301, wherein the memory can include a storage medium 303 and an internal memory 304.

[0080] The storage medium 303 can store an operating system 3031 and a computer program 3032. The computer program 3032, when executed, can cause the processor 302 to perform a face retouching image quality evaluation method based on texture analysis.

[0081] The processor 302 is configured to provide computing and control capabilities to support the operation of the entire computer device 300.

[0082] The internal memory 304 provides an environment for the execution of the computer program 3032 in the storage medium 303, and the computer program 3032, when executed by the processor 302, can cause the processor 302 to perform a face retouching image quality evaluation method based on texture analysis.

[0083] The network interface 305 is configured to perform network communication with other devices. Those skilled in the art can understand that Figure 8 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device 300 to which the scheme of the present application is applied. The specific computer device 300 can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0084] The processor 302 is configured to run the computer program 3032 stored in the memory to implement any embodiment of the face retouching image quality evaluation method based on texture analysis described above.

[0085] It should be understood that, in the embodiments of the present application, the processor 302 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0086] Those skilled in the art can understand that all or part of the processes in the method of the above-mentioned embodiments can be completed by instructing the relevant hardware by a computer program. The computer program can be stored in a storage medium, which is a computer-readable storage medium. The computer program is executed by at least one processor in the computer system to realize the process steps of the above-mentioned embodiments of the method.

[0087] Therefore, the present application also provides a storage medium. The storage medium can be a computer-readable storage medium. The storage medium stores a computer program. The computer program is executed by a processor to make the processor execute any embodiment of the above-mentioned face retouching image quality evaluation method based on texture analysis.

[0088] The storage medium can be a U disk, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk, and various computer-readable storage media that can store program codes.

[0089] Those skilled in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in the above description in general terms. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0090] In several embodiments provided by the present application, it should be understood that the disclosed apparatus and method can be implemented in other manners. For example, the described apparatus embodiments are merely schematic. For example, the division of the units is merely a logical function division. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In this way, the actual division of the units can be different from the division in the embodiment.

[0091] The steps in the method embodiments of the present application can be adjusted, combined and deleted in sequence according to actual needs. The units in the apparatus embodiments of the present application can be combined, divided and deleted according to actual needs. In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit.

[0092] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a storage medium. Based on such understanding, the technical solutions of the present application essentially or say the part that contributes to the prior art, or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a terminal or a network device, etc.) to execute all or part of the steps of the methods described in the embodiments of the present application.

[0093] In the above embodiments, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0094] Obviously, those skilled in the art can make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, these modifications and variations also belong to the scope of the claims of the present application and their equivalent technologies, and the present application is intended to include these modifications and variations.

[0095] The above description is merely specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any skilled person in the art can easily think of various equivalent modifications or replacements within the technical range disclosed by the present application, and these modifications or replacements should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A face retouching image quality evaluation method based on texture analysis, characterized by, The method comprises the following steps: obtaining a face retouching image to be evaluated, and adjusting the face retouching image to be evaluated to a preset size to obtain a target face retouching image to be evaluated; inputting the target face retouching image to be evaluated into a preset image quality evaluation model, so that the image quality evaluation model identifies the target face retouching image to be evaluated, and generates a quality score corresponding to the target face retouching image to be evaluated, wherein the image quality evaluation model is obtained by training a constructed quality evaluation model using a training data set; wherein the image quality evaluation model identifies the target face retouching image to be evaluated, and generates a quality score corresponding to the target face retouching image to be evaluated, specifically comprising: extracting a first feature map, a second feature map, a third feature map and a fourth feature map from the target face retouching image to be evaluated, and performing texture analysis on the second feature map to obtain a texture feature map; splicing and deforming the texture feature map, the third feature map and the fourth feature map to obtain a feature vector, and inputting the feature vector into a fully connected layer to generate a quality score; the texture analysis on the second feature map to obtain a texture feature map comprises: performing average pooling and up-sampling processing on the second feature map to obtain a pooled and sampled feature map; calculating the difference between the second feature map and the pooled and sampled feature map to obtain a first texture feature map; performing dense convolution on the first texture feature map to obtain a second texture feature map, and performing maximum pooling on the second texture feature map to obtain a first feature weight; performing average pooling on the second feature map to obtain a second feature weight, and determining a texture feature map according to the first feature weight, the second feature weight and the second texture feature map.

2. The method of claim 1, wherein, the determination of the texture feature map according to the first feature weight, the second feature weight and the second texture feature map comprises: splicing the first feature weight and the second feature weight to obtain a third feature weight; activating the third feature weight through an activation function to obtain a fourth feature weight; calculating the product of the fourth feature weight and the second texture feature map to obtain a texture feature map.

3. The method of claim 1, wherein, the inputting of the feature vector into the fully connected layer to generate a quality score comprises: dimensionality reduction of the feature vector to obtain an initial feature vector, the initial feature vector being used as an initial LBP feature vector, and the remaining vectors after removing the first preset number of vectors from the initial feature vector being used as an initial score feature vector; splicing the initial LBP feature vector and the initial score feature vector to generate a target feature vector; inputting the target feature vector into the fully connected layer to generate a quality score.

4. The method of claim 1, wherein, training the constructed quality evaluation model using a training data set to obtain an image quality evaluation model comprises: for each training batch image in the training data set, inputting the image into the constructed quality evaluation model to output a predicted LBP feature and a predicted quality score; calculating a first loss value through a first loss function according to the predicted LBP feature and a label LBP feature in the training data set; calculating a second loss value by a second loss function according to the predicted quality score and a label quality score in the training data set; determining an image quality evaluation model according to the first loss value and the second loss value.

5. The method of claim 4, wherein, After the image quality evaluation model is obtained by training the constructed quality evaluation model using the training data set, the method further includes: inputting a test data set into the trained quality evaluation model to obtain a test quality score; calculating an index value according to the test quality score and a quality score in the test data set.

6. The method of claim 5, wherein, The step of determining the image quality evaluation model according to the first loss value and the second loss value includes: calculating a total loss value by summing the first loss value and the second loss value, and iteratively updating the quality evaluation model according to the total loss value until a preset training batch number is reached, so as to obtain the image quality evaluation model.

7. A face retouching image quality evaluation device based on texture analysis, characterized by, It includes: an acquisition adjusting unit configured to acquire a face retouch image to be evaluated, and adjust the face retouch image to be evaluated to a preset size to obtain a target face retouch image to be evaluated; a quality evaluation unit configured to input the target face retouch image to be evaluated into a preset image quality evaluation model, so that the image quality evaluation model identifies the target face retouch image to be evaluated and generates a quality score corresponding to the target face retouch image to be evaluated, wherein the image quality evaluation model is obtained by training a constructed quality evaluation model using a training data set; The image quality evaluation model identifies the target face retouch image to be evaluated and generates a quality score corresponding to the target face retouch image to be evaluated, specifically including: a feature extraction unit configured to extract a first feature map, a second feature map, a third feature map and a fourth feature map from the target face retouch image to be evaluated, and perform texture analysis on the second feature map to obtain a texture feature map; a splicing generation unit configured to splice and deform the texture feature map, the third feature map and the fourth feature map to obtain a feature vector, and input the feature vector into a full connection layer to generate a quality score; The feature extraction unit includes: a pooling up-sampling unit configured to perform average pooling and up-sampling processing on the second feature map to obtain a pooled and sampled feature map; a first calculation unit configured to calculate a difference between the second feature map and the pooled and sampled feature map to obtain a first texture feature map; a convolution pooling unit configured to perform dense convolution on the first texture feature map to obtain a second texture feature map, and perform maximum pooling on the second texture feature map to obtain a first feature weight; a pooling determination unit configured to perform average pooling on the second feature map to obtain a second feature weight, and determine a texture feature map according to the first feature weight, the second feature weight and the second texture feature map.

8. A computer device, comprising: The computer device includes a memory and a processor, the memory stores a computer program, and the processor executes the computer program to implement the method in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, and the computer program is executed by the processor to implement the method in any one of claims 1-6.

Images