A method, system, device and medium for quality evaluation of face retouched images

Abstract

The present invention belongs to the technical field of image quality evaluation, and discloses a method, system, device and medium for evaluating the quality of face retouched images, including: obtaining face image data to be evaluated, where the face image data to be evaluated includes an original facial image, a corresponding face retouched image and retouched image quality expert scoring data; inputting the face image data to be evaluated into the retouched image quality evaluation model for quality evaluation to obtain a retouched image quality score; wherein, the retouched image quality evaluation model includes an image encoder, a text encoder, a quality regression module, a differential feature attention module, a text information module and a multi-scale feature fusion module. The technical solution of the present invention can effectively identify and evaluate minute changes in images, can significantly improve the performance of image quality evaluation, and has strong adaptability.

Description

Technical Field

[0001] The present invention belongs to the technical field of image quality evaluation, and particularly relates to a method, system, device and medium for evaluating the quality of face-retouched images. Background Art

[0002] Face retouching generally refers to the technology of using digital tools and techniques to modify or enhance facial features. It is usually used to improve an individual's appearance, such as facial correction, eye enlargement, smiling, acne removal from the skin, whitening, etc. However, excessive use of face retouching operations may mislead face recognition in certain scenarios, such as secure access, authentication, and emotion analysis. Therefore, it is particularly important to evaluate the quality of retouched face images. This not only helps to strike a balance between enhancing visual attractiveness and maintaining a natural appearance, but also ensures that in key applications such as face recognition, the retouching operations will not have a negative impact on the accuracy and reliability of the system. By designing an effective quality evaluation method for retouched face (RF) images, the degree of retouching operations can be better grasped, and the negative effects brought by excessive retouching can be avoided, thereby realizing a more efficient and secure face recognition application.

[0003] Existing RF image quality evaluation methods have designed a new CNN network for the distortion characteristics of RF images, and evaluate the quality of RF images by considering the texture features of the images and the high-level semantic information of geometric changes. However, such methods usually lack an in-depth understanding of the personalized differences between the original facial images and the retouched images. Further observations show that when using photo editing software to perform the same retouching operations on different facial images, the degree of retouching often shows personalized differences. For example, for two different facial images, although the same retouching procedures and parameters are used to adjust the eyes, nose, mouth, and overall facial shape, the retouched effects may be very different. This difference stems from the differences in individual facial features, such as eye size, width of facial structure, etc. Therefore, there is an urgent need for an RF image quality evaluation method that can adapt to these personalized differences. Summary of the Invention

[0004] The purpose of the present invention is to provide a method, system, device and medium for evaluating the quality of face-retouched images to solve the problems existing in the above-mentioned prior art.

[0005] To achieve the above object, the present invention provides a method for evaluating the quality of face-retouched images, including:

[0006] Obtain the face image data to be evaluated, where the face image data to be evaluated includes the original facial image and the corresponding face-retouched image and the retouched image quality expert scoring data;

[0007] Input the face image data to be evaluated into the retouched image quality evaluation model for quality evaluation to obtain the retouched image quality score; wherein, the retouched image quality evaluation model includes an image encoder, a text encoder, a quality regression module, a differential feature attention module, a text information module, and a multi-scale feature fusion module. The image encoder and the multi-scale feature fusion module are both connected to the quality regression module. The text encoder, the text information module, and the multi-scale feature fusion module are connected in sequence. The text information module and the image encoder are connected through the differential feature attention module.

[0008] Optionally, the training process of the retouched image quality evaluation model includes:

[0009] Obtain training data, where the training data includes face training image data and corresponding retouched image quality scores. The face training image data includes original facial images, corresponding face retouched images, and retouched image quality expert scoring data;

[0010] Construct an initial retouched image quality evaluation model, input the training data into the initial retouched image quality evaluation model for quality evaluation, and take the minimum loss between the initial training result after quality evaluation and the retouched image quality score corresponding to the face training image data as the goal for training to obtain the retouched image quality evaluation model.

[0011] Optionally, the processing process of the retouched image quality evaluation model includes:

[0012] Input the original facial image and the corresponding face retouched image into the image encoder for feature extraction to obtain the global features and local features of the original facial image and the corresponding face retouched image;

[0013] Take the difference between the global features of the original facial image and the corresponding face retouched image as the global difference feature;

[0014] Input the local features of the original facial image and the corresponding face retouched image into the differential feature attention module for feature subtraction to obtain local difference features;

[0015] Input the retouched image quality expert scoring data into the text encoder to output a text vector;

[0016] Input the text vector and the local difference features into the text information module for text enhancement to obtain the text-enhanced local difference features, and aggregate the text-enhanced local difference features through the multi-scale feature fusion module to obtain aggregated features;

[0017] Perform a pooling operation on the aggregated feature, and concatenate the pooled aggregated feature with the global difference feature to obtain a concatenated feature map. Predict a score for the concatenated feature map through the quality regression model to obtain the retouched image quality score.

[0018] Optionally, input the local features of the original facial image and the corresponding face retouched image into the difference feature attention module for feature subtraction, which specifically includes:

[0019] Perform convolution operations on the local features corresponding to the original facial image and the face retouched image respectively to obtain corresponding convolution feature maps.

[0020] Use the difference between the convolution feature maps corresponding to the original facial image and the face retouched image as the initial local difference feature.

[0021] Introduce an attention mechanism to make the initial local difference feature interact with the convolution feature maps corresponding to the original facial image and the face retouched image respectively to obtain corresponding difference-aware features.

[0022] Concatenate the difference-aware features corresponding to the original facial image and the face retouched image, and perform a parallel convolution operation to obtain the local difference feature.

[0023] Optionally, input the text vector and the local difference feature into the text information module for text enhancement, which specifically includes:

[0024] Reshape the shape of the text vector to obtain a reshaped vector.

[0025] Make the reshaped vector pass through a convolutional layer and an activation function in sequence to obtain channel weights.

[0026] Multiply the channel weights with the local difference feature in the channel direction to obtain the text-enhanced local difference feature.

[0027] Optionally, aggregate the text-enhanced local difference feature through the multi-scale feature fusion module, which specifically includes:

[0028] The multi-scale feature fusion module includes several branches. In each branch, use bilinear interpolation to downsample the text-enhanced local difference feature to obtain the downsampled local difference feature, and perform a 1×1 convolution operation on the downsampled local difference feature to obtain a transformed feature.

[0029] Connect the transformed features of each branch in the channel direction and perform 3×3 convolution processing to obtain the aggregated feature.

[0030] A quality evaluation system for face retouched images, comprising:

[0031] A data acquisition module for obtaining face image data to be evaluated, where the face image data to be evaluated includes an original facial image, a corresponding face retouched image, and retouched image quality expert scoring data;

[0032] A quality evaluation module for inputting the face image data to be evaluated into the retouched image quality evaluation model for quality evaluation to obtain a retouched image quality score; wherein, the retouched image quality evaluation model includes an image encoder, a text encoder, a quality regression module, a differential feature attention module, a text information module, and a multi-scale feature fusion module. The image encoder and the multi-scale feature fusion module are both connected to the quality regression module. The text encoder, the text information module, and the multi-scale feature fusion module are connected in sequence. The text information module and the image encoder are connected through the differential feature attention module.

[0033] An electronic device, comprising a memory and a processor. The memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the quality evaluation method for face retouched images as described above.

[0034] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the quality evaluation method for face retouched images as described above.

[0035] The technical effects of the present invention are as follows:

[0036] The present invention fully considers the characteristics of face retouched images, provides an image quality evaluation method based on differential perception and text prompts, and generates a quality score for the degree of face image retouching. The present invention can effectively identify and evaluate minute changes in images. By introducing DFAM, TIM, and MFFM, the performance of image quality evaluation can be significantly improved. Moreover, the present invention shows strong generalization ability under various image editing settings and has strong adaptability under different image editing settings. Description of the Drawings

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required to be used in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0038] The accompanying drawings, which form a part of this application, are used to provide a further understanding of this application. The schematic embodiments and descriptions thereof of this application are used to explain this application and do not constitute an improper limitation to this application. In the drawings:

[0039] Figure 1 It is the structure diagram of the face retouching image quality evaluation network in the embodiment of the present invention;

[0040] Figure 2 It is the structure diagram of the differential feature attention module in the embodiment of the present invention;

[0041] Figure 3 It is the structure diagram of the text-image feature interaction module in the embodiment of the present invention;

[0042] Figure 4 It is the structure diagram of the multi-scale feature fusion module in the embodiment of the present invention;

[0043] Figure 5 It is the implementation flowchart of the image quality evaluation in the embodiment of the present invention. Detailed implementation manners

[0044] Now, various exemplary implementation manners of the present invention will be described in detail. This detailed description should not be considered as a limitation to the present invention, but rather as a more detailed description of certain aspects, characteristics, and implementation schemes of the present invention.

[0045] It should be understood that the terms described in the present invention are only for describing specific implementation manners and are not used to limit the present invention. Additionally, for the numerical ranges in the present invention, it should be understood that each intermediate value between the upper and lower limits of the range is also specifically disclosed. Each intermediate value within any stated value or stated range, as well as each smaller range between any other stated value or intermediate value within the stated range, is also included in the present invention. The upper and lower limits of these smaller ranges can be independently included or excluded from the range.

[0046] Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the art to which the present invention pertains. Although the present invention only describes the preferred methods, any method similar or equivalent to those described herein can also be used in the implementation or testing of the present invention. All documents mentioned in this specification are incorporated by reference to disclose and describe the methods related to the documents. In case of conflict with any incorporated document, the content of this specification shall prevail.

[0047] Without departing from the scope or spirit of the present invention, various improvements and changes can be made to the specific implementation manners of the specification of the present invention, which are obvious to those skilled in the art. Other implementation manners obtained from the specification of the present invention are obvious to those skilled in the art. The specification and embodiments of this application are only exemplary.

[0048] As used herein, terms such as "comprising", "including", "having", "containing", etc. are all open-ended terms, meaning including but not limited to.

[0049] It should be noted that, without conflict, the embodiments in this application and the features in the embodiments can be combined with each other. The following will detail this application with reference to the accompanying drawings and in conjunction with the embodiments.

[0050] Embodiment 1

[0051] As Figure 1 - Figure 5 shown, in this embodiment, a method for evaluating the quality of a face retouched image is provided, including: obtaining face image data to be evaluated, where the face image data to be evaluated includes an original facial image, a corresponding face retouched image, and retouched image quality expert scoring data; inputting the face image data to be evaluated into the retouched image quality evaluation model for quality evaluation to obtain a retouched image quality score; where the retouched image quality evaluation model includes an image encoder, a text encoder, a quality regression module, a differential feature attention module, a text information module, and a multi-scale feature fusion module, the image encoder and the multi-scale feature fusion module are both connected to the quality regression module, the text encoder, the text information module, and the multi-scale feature fusion module are connected in sequence, and the text information module and the image encoder are connected through the differential feature attention module.

[0052] This embodiment proposes a novel full-reference IQA network DIRF-IQA for evaluating RF images. The network takes an original facial image, an RF image, and a text prompt as inputs and uses a contrastive language-image pre-training (CLIP) model to encode image and text information. To effectively capture the differences between the original facial image and the RF image and effectively utilize the text prompt information, DIRF-IQA integrates three key modules: a differential feature attention module (DFAM), a text-image feature interaction module (TIM), and a multi-scale feature fusion module (MFFM). Specifically, the original facial image and the RF image are processed by the image encoder to obtain multi-scale features, while the text prompt is encoded by the text encoder to generate a text vector. For the original facial image and the RF image, the features extracted from the last layer of the image encoder are subtracted to capture the global difference features between them. Next, DFAM uses a cross-attention block (CAB) and a self-attention block (SAB) to process the features of each layer to capture the local feature differences between the original facial image and the RF image. Then, the obtained local difference features are combined with the text vector through TIM to enhance the model's understanding and representation of the distortion in the RF image. Finally, MFFM is used to fuse the text-enhanced features obtained by TIM at different layers to predict the RF image quality.

[0053] The purpose of this embodiment is to propose a face retouching image quality evaluation method based on differential information perception and text prompts, so as to solve the problem in the prior art that it is impossible to effectively measure the deviation degree between the retouched face image and the real image, and to ensure that in key applications such as face recognition, identity verification, and emotion analysis, the retouching operation will not have a negative impact on the accuracy and reliability of the system. The specific objectives of this embodiment include:

[0054] Provide a method that can effectively capture the global and local difference features between the original facial image and the retouched image, and overcome the problem that existing methods rely on distortion prior experience. Combine text prompt information to enhance the model's ability to evaluate the quality of retouched images and ensure that the retouching operation will not affect the accuracy of applications such as face recognition. Through multi-scale feature fusion, improve the model's comprehensive utilization ability of different-level features and achieve efficient prediction of the quality of retouched images.

[0055] This embodiment proposes a face retouching image quality evaluation method based on differential information perception and text prompts. The specific technical solutions include the following steps:

[0056] 1. Dataset acquisition: In this embodiment, two publicly available face retouching image quality evaluation datasets are selected to train and test the model, namely MARFD and SZU-RFD. Among them, MARFD contains various types of face images, including natural face, made-up face, and facial photos retouched to different degrees. This dataset contains a total of 2,500 high-resolution face retouching photos, which are generated by retouching 250 original face images with 10 different degrees of retouching operations, such as whitening, skin smoothing, and facial feature trimming. The SZU-RFD dataset includes a large number of face images taken under different environments and lighting conditions. The dataset contains various common retouching operations, such as face slimming, acne removal, and teeth whitening, and multiple retouching effects of different degrees are provided for each operation. This dataset is processed by retouching 200 original face samples with 8 different levels of retouching operations, resulting in a total of 1,600 face retouching images. These two datasets are not only attached with labels of retouching types and degrees, but also include the scores of experts on the quality of retouched images. These scores provide a reliable reference standard for the training and evaluation of the model.

[0057] 2. Model Design: The face retouched image quality evaluation method based on differential information perception and text prompt proposed in this embodiment designs a full-reference image quality evaluation network DIRF-IQA. This network uses the Contrastive Language-Image Pretraining (CLIP) model as the network backbone to effectively extract multi-modal features of text and images. Specifically, the design of DIRF-IQA mainly includes three key modules: DFAM, TIM, and MFFM. The main function of DFAM is to capture the differential features between the original facial image and the retouched image. This module processes the features of each layer through the Cross-Attention Block (CAB) and the Self-Attention Block (SAB) to further explore the subtle differences between the original image and the retouched image in the local area. In view of the characteristics of image retouching, this embodiment designs a text prompt template for retouching operations, using text to represent the retouching operations experienced by the image. TIM combines the local differential features obtained by the DFAM module with the text vector and uses the cross-attention mechanism to realize the interaction between text and image features. This interaction method enables the model to better understand the retouching information described by the text prompt, thereby improving the evaluation accuracy of the retouched image. The main function of MFFM is to fuse the text-enhanced features obtained by the TIM module at different levels and comprehensively utilize the information advantages of the features of each layer. To sum up, the DIRF-IQA network proposed in this embodiment realizes the efficient evaluation of the quality of face retouched images by integrating three major modules: differential feature attention, text-image feature interaction, and multi-scale feature fusion. The novelty and efficiency of the model design provide reliable technical support for face image quality evaluation.

[0058] 3. Loss Function: To effectively train the DIRF-IQA network, this embodiment introduces the Mean Squared Error (MSE) loss to measure the difference between the image quality score predicted by the model and the actual score.

[0059] 4. Network Training and Testing: In the network training and testing phases, first, the two selected publicly available face retouching image quality evaluation datasets (MARFD and SZU-RFD) are randomly divided into a training set and a testing set at a ratio of 8:2. The division process for each dataset is repeated 10 times to ensure the stability and reliability of the results. All images are resized to 224×224 and subjected to normalization processing and random horizontal flipping to enhance data diversity. In the training phase, the model uses the mean squared error (MSE) as the loss function and performs gradient backpropagation to minimize the difference between the predicted quality score and the true quality score. In the testing phase, the weight parameters of the network when it reaches the best performance during training are imported into the model, and metrics such as the Spearman rank correlation coefficient (SRCC), Pearson linear correlation coefficient (PLCC), and Kendall rank correlation coefficient (KRCC) are calculated to evaluate the performance of the model. These metrics are all between 0 and 1, and the higher the value, the better the prediction performance.

[0060] This embodiment fully considers the characteristics of face retouching images and provides an image quality evaluation method based on differential perception and text prompts, which generates a quality score for the degree of face image retouching. Experimental results show that this embodiment exhibits superior performance in evaluating the quality of RF images and has a high degree of consistency with human subjective observation results. The specific beneficial effects can be summarized as follows:

[0061] 1. High Accuracy: This embodiment can effectively identify and evaluate subtle changes in images. Experimental results show that DIRF-IQA achieves high SRCC, PLCC, and KRCC values on both the MARFD and SZU-RFD databases, indicating that this method has strong predictive ability and reliability.

[0062] 2. Component Effectiveness: The introduction of DFAM, TIM, and MFFM in this embodiment can significantly improve the performance of image quality evaluation. Experimental results show that removing any one of them will lead to a decline in prediction performance, while retaining all three can obtain the best evaluation results.

[0063] 3. Strong Generalization Ability under Various Settings: Experimental results show that this method exhibits strong generalization ability under various image editing settings, indicating that the method of this embodiment has strong adaptability under different image editing settings.

[0064] Specific algorithm examples of this embodiment include: The following combines the accompanying drawings in the examples of this embodiment to explain the technical solutions in detail. This embodiment proposes a method for evaluating the quality of retouched face images based on differential perception and text prompts. The framework is as Figure 1As shown in the figure. For a face retouched image, first, it passes through the image encoder of CLIP to initially extract multi-level features of the image. The features of the last layer are subtracted to represent the global difference features of the image. The local features of the remaining layers are processed by DFAM to form local difference features through self-attention and cross-attention methods. The retouching operation text is encoded by the text encoder of CLIP, and then TIM enhances the local difference features, enabling the model to fuse text multi-modal information. Finally, MFFM fuses the text-enhanced features obtained by TIM at different layers, and splices the obtained fused features with the global difference features to predict the RF image quality.

[0065] (1) Select the test platform: To verify the effectiveness of this embodiment, DIRF-IQA is trained and tested on a server configured with NVIDIA GeForce GTX 3090 GPU and two Intel Xeon 6226R CPUs. The experiment uses the open-source Pytorch library and trains the network model using the AdamW optimizer.

[0066] (2) Dataset acquisition: In this embodiment, two RF IQA datasets are selected to train and test the performance of the proposed model, namely the MARFD and SZU-RFD datasets. Each dataset contains images with different facial retouching effects. We designed corresponding text prompts according to each different setting, which effectively distinguish different tools and summarize the information of various settings. We randomly divide the database into a training set and a test set according to the content of the original facial images, with a ratio of 8:2. For fair comparison, this embodiment repeats the random division of training-test data 10 times and takes the average of these divisions as the final result.

[0067] (3) Model design: The DIRF-IQA framework designed in this embodiment is as Figure 1 shown. DIRF-IQA mainly consists of a CLIP image encoder based on ResNet50, a text encoder based on Transformer, a difference feature attention module (DFAM), a text information module (TIM), and a multi-scale feature fusion module (MFFM). Taking the retouched (RF) image and the original (OF) image as inputs, DIRF-IQA first uses a shared CLIP image encoder to extract multi-scale local features and global features. The local features of the RF image and the OF image come from the first three layers of the image encoder, which are respectively represented as and Its global features come from the last layer of the image encoder, which are respectively represented as and By subtracting the global features of the OF image and the RF image, a global difference feature Meanwhile, the local features of the RF image and the OF image are processed by DFAM to generate local difference features Considering that text information can enrich feature representation, this embodiment designs a unique text prompt to optimize these local difference features. Specifically, the text prompt first generates a text vector V through the CLIP text encoder t . With the help of TIM, the local difference features are combined with V t to produce text-enhanced local difference features Finally, the text-enhanced local difference features generated are aggregated using MFFM, and the aggregated features are obtained through average pooling will and be concatenated and input into a regression module, which includes two fully connected layers for predicting the quality score of the RF image

[0068] Difference Feature Attention Module: Figure 2 Shows the architecture of the proposed difference feature attention module. First, the obtained local features F of the i-th layer i r and F i o are processed by a 1×1 convolutional layer to reduce the number of channels, then the features of the two halved channels are subtracted and gradually input into a 1×1 convolutional layer, a 3×3 convolutional layer, and a 1×1 convolutional layer to obtain Next, to capture detailed local differences and focus on key regions, F i o,r is input into a cross-attention block (CAB) and a self-attention block (SAB), and interacts with F i r and F i o to obtain difference-aware features F i r and F i o . Finally, the difference-aware features of the obtained RF image and OF image are concatenated, and a pyramid block with four parallel convolutional layers is used to obtain local difference features F i d . The above process can be summarized as:

[0069] F i o = φ s (φ c (F i o,r , F i o ))

[0070] F ir = φ s (φ c (F i o,r , F i r ))

[0071]

[0072] Among them, represents the concatenation operation, φ c and φ s represent the CAB and SAB operations respectively, and Ψ represents a pyramid block containing a 7×7 convolutional layer, a 5×5 convolutional layer, a 3×3 convolutional layer, and a 1×1 convolutional layer. Figure 2 The structures of CAB and SAB are also given. They are both designed based on the multi-head attention mechanism. Specifically, first, the two-dimensional feature map is reshaped into a one-dimensional vector, and the required query (Q), key (K), and value (V) vectors are output through a linear regression layer. For CAB, Q and K come from the difference feature F i o,r , and V comes from the local feature F i o or F i r of the current layer. For SAB, Q, K, and V all come from the output features of CAB. Given a triple vector (Q, K, V), the attention weights are calculated through the dot product of Q and K, and then the attention feature A is obtained by weighted summation of V based on the attention weights:

[0073]

[0074] where d represents the dimension of the feature vector. Finally, the attention feature A is added to the original input feature F in , and the output feature F out is obtained through processing by a fully connected layer with a residual connection:

[0075] F out = F in + A + f c (F in + A)

[0076] Text-Image Feature Interaction Module: In this embodiment, a text prompt is set for each RF image, which corresponds to the retouching settings applied to the OF image. The standard format is: "Tool is {x}, {degree and operation details}k}." Here, x is the ID of the photo editing tool, used to distinguish different editing tools, and k is the number of photo editing steps. The degree of retouching can be customized according to the setting range provided by the photo editing tool. The reason for setting this text prompt format is to record in detail the key factors affecting the quality of the RF image. The prompt describes the number of retouching steps and specific operation details, which directly reflect the degree of adjustment made to the image, such as changes in brightness, color, and facial features. By recording these factors in detail, this text prompt provides rich context, making it easier for the model to identify the impact of retouching and thus perform more accurate quality assessment. The text prompt of the RF image is processed by a text encoder to obtain a text vector V t . Next, through the TIM, text and image features are effectively integrated to improve the IQA performance. Figure 3 Figure 4 shows the architecture of the proposed TIM. First, the obtained V t is reshaped from shape (1, 1024) to (1024, 1, 1), and the number of channels is adjusted through two 1×1 convolutional layers. Next, the output features form channel weights W through the Sigmoid function c , and the local difference feature F i d is multiplied with W c in the channel direction to generate a text-enhanced local difference feature:

[0077]

[0078] Multi-scale Feature Fusion Module: In this embodiment, a multi-layer feature fusion module (MFFM) is designed to fuse text-enhanced local difference features from different layers. As Figure 4 shown, the MFFM consists of three parallel branches, which process features from three layers respectively. For the text-enhanced local difference feature , it is first downsampled using bilinear interpolation to make its spatial size become 7×7. Then, the number of channels is adjusted to 1024 through a 1×1 convolutional layer. Finally, the transformed features from the three layers are concatenated in the channel direction and processed through a 3×3 convolutional layer to obtain the local difference feature

[0079] (4) Loss Function: In the training stage, our goal is to minimize the mean squared error loss on the training set:

[0080]

[0081] where N represents the number of images in a mini - batch, and q i are the predicted quality score and the ground - truth quality score of the i - th retouched face image, respectively.

[0082] (5) Network training and testing: The experiment uses the open - source Pytorch library and trains the DIRF - IQA network model using the AdamW optimizer. The weight decay is set to 1×10 -4 . The training process is divided into two stages: In the first stage, the parameters of the CLIP image and text encoders are frozen, and only the rest of DIRF - IQA is trained for 20 epochs with an initial learning rate of 1×10 -4 , which is adjusted to 2×10 -5 at the 14th epoch; In the second stage, the parameters of the image and text encoders are unfrozen, and the entire network is fine - tuned with a learning rate of 5×10 -6 , and the learning rate is further reduced to 1×10 -6 at the 40th epoch. In addition, this embodiment uses an early - stopping strategy. When the model performance does not improve for 15 consecutive epochs, the training stops, and the weight parameters when the network performance is the best are saved. This embodiment uses SRCC, PLCC, and KRCC as performance metrics. The three metrics calculate the rank correlation between the predicted scores and the actual values. The higher the metric value, the better the performance of the network.

[0083] (7) Method performance: The SRCC, PLCC, and KRCC results of the test set prove the superiority of the network model designed in this embodiment. Compared with other image quality assessment algorithms, this algorithm achieves the most optimal PLCC, SRCC, and KRCC metrics in the quality prediction of two face retouching image datasets. In addition, the designed model performs well under various retouching settings, demonstrating strong generalization ability.

[0084] A quality assessment system for face - retouched images, comprising:

[0085] A data acquisition module for obtaining face image data to be evaluated, where the face image data to be evaluated includes the original facial image, the corresponding face - retouched image, and the retouched image quality expert scoring data;

[0086] A quality evaluation module is configured to input the face image data to be evaluated into the retouched image quality evaluation model for quality evaluation, so as to obtain a retouched image quality score. The retouched image quality evaluation model includes an image encoder, a text encoder, a quality regression module, a differential feature attention module, a text information module, and a multi-scale feature fusion module. The image encoder and the multi-scale feature fusion module are both connected to the quality regression module. The text encoder, the text information module, and the multi-scale feature fusion module are connected in sequence. The text information module and the image encoder are connected through the differential feature attention module.

[0087] An electronic device includes a memory and a processor. The memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the quality evaluation method for a face retouched image as described above.

[0088] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the quality evaluation method for a face retouched image as described above is implemented.

[0089] As mentioned above, the above are only the preferred specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed in the present application should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for evaluating the quality of a face retouched image, characterized in that, Including: Obtain face image data to be evaluated, where the face image data to be evaluated includes an original facial image, a corresponding face retouched image, and retouched image quality expert scoring data; Input the face image data to be evaluated into a retouched image quality evaluation model for quality evaluation to obtain a retouched image quality score; wherein, the retouched image quality evaluation model includes an image encoder, a text encoder, a quality regression module, a differential feature attention module, a text information module, and a multi-scale feature fusion module. The image encoder and the multi-scale feature fusion module are both connected to the quality regression module. The text encoder, the text information module, and the multi-scale feature fusion module are connected in sequence. The text information module and the image encoder are connected through the differential feature attention module; The processing process of the retouched image quality evaluation model includes: Input the original facial image and the corresponding face retouched image into the image encoder for feature extraction to obtain the global features and local features of the original facial image and the corresponding face retouched image; Use the difference between the global features of the original facial image and the corresponding face retouched image as the global differential feature; Input the local features of the original facial image and the corresponding face retouched image into the differential feature attention module for feature subtraction to obtain local differential features; Input the retouched image quality expert scoring data into the text encoder to output a text vector; Input the text vector and the local differential features into the text information module for text enhancement to obtain text-enhanced local differential features, and aggregate the text-enhanced local differential features through the multi-scale feature fusion module to obtain aggregated features; Perform a pooling operation on the aggregated features, splice the pooled aggregated features with the global differential features to obtain a spliced feature map, and predict the score of the spliced feature map through the quality regression module to obtain the retouched image quality score.

2. The quality evaluation method for a face retouched image according to claim 1, characterized in that, The training process of the retouched image quality evaluation model includes: Obtain training data, where the training data includes face training image data and corresponding retouched image quality scores. The face training image data includes an original facial image, a corresponding face retouched image, and retouched image quality expert scoring data; Construct an initial retouched image quality evaluation model, input the training data into the initial retouched image quality evaluation model for quality evaluation, and perform training with the goal of minimizing the loss between the initial training result after quality evaluation and the retouched image quality score corresponding to the face training image data to obtain the retouched image quality evaluation model.

3. A quality evaluation method for a face retouched image according to claim 1, characterized in that, Input the local features of the original facial image and the corresponding face retouched image into the differential feature attention module for feature subtraction to obtain local differential features, specifically including: Perform convolution operations on the local features corresponding to the original facial image and the face retouched image respectively to obtain corresponding convolution feature maps, Use the difference between the convolution feature maps corresponding to the original facial image and the face retouched image as the initial local differential feature; Introduce an attention mechanism to enable the initial local difference features to interact with the convolutional feature maps corresponding to the original facial image and the face retouched image respectively, and obtain corresponding difference-aware features; Concatenate the difference-aware features corresponding to the original facial image and the face retouched image, and perform a parallel convolution operation to obtain the local difference features.

4. The quality evaluation method for a face retouched image according to claim 1, characterized in that Input the text vector and the local difference features into the text information module for text enhancement, specifically including: Reshape the text vector to obtain a reshaped vector; Let the reshaped vector pass through a convolutional layer and an activation function in sequence to obtain channel weights; Multiply the channel weights with the local difference features in the channel direction to obtain the text-enhanced local difference features.

5. A method for evaluating the quality of a face-retouched image according to claim 1, characterized in that, Aggregate the text-enhanced local difference features through the multi-scale feature fusion module, specifically including: The multi-scale feature fusion module includes several branches. In each branch, use bilinear interpolation to downsample the text-enhanced local difference features to obtain the downsampled local difference features, and perform a 1×1 convolution operation on the downsampled local difference features to obtain transformed features; Connect the transformed features of each branch in the channel direction and perform 3×3 convolution processing to obtain the aggregated features.

6. A quality evaluation system for face retouched images, characterized in that, Include: A data acquisition module for obtaining face image data to be evaluated, where the face image data to be evaluated includes an original facial image and the corresponding face retouched image and retouched image quality expert scoring data; A quality evaluation module for inputting the face image data to be evaluated into a retouched image quality evaluation model for quality evaluation to obtain a retouched image quality score; wherein, the retouched image quality evaluation model includes an image encoder, a text encoder, a quality regression module, a differential feature attention module, a text information module, and a multi-scale feature fusion module. The image encoder and the multi-scale feature fusion module are both connected to the quality regression module. The text encoder, the text information module, and the multi-scale feature fusion module are connected in sequence. The text information module and the image encoder are connected through the differential feature attention module. The processing process of the retouched image quality evaluation model includes: inputting the original facial image and the corresponding face retouched image into the image encoder for feature extraction to obtain the global features and local features of the original facial image and the corresponding face retouched image; taking the difference between the global features of the original facial image and the corresponding face retouched image as the global differential feature; inputting the local features of the original facial image and the corresponding face retouched image into the differential feature attention module for feature subtraction to obtain local differential features; inputting the retouched image quality expert scoring data into the text encoder to output a text vector; inputting the text vector and the local differential features into the text information module for text enhancement to obtain text-enhanced local differential features, aggregating the text-enhanced local differential features through the multi-scale feature fusion module to obtain aggregated features; performing a pooling operation on the aggregated features, splicing the pooled aggregated features with the global differential features to obtain a spliced feature map, and predicting a score for the spliced feature map through the quality regression module to obtain the retouched image quality score.

7. An electronic device, characterized in that, It includes a memory and a processor. The memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute a method for evaluating the quality of a face retouched image according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, It stores a computer program, and when the computer program is executed by the processor, it implements a method for evaluating the quality of a face retouched image according to any one of claims 1-5.

Images