Model training method and device, image quality evaluation method and device and terminal equipment

Abstract

The invention provides a model training method, an image quality evaluation method, a model training device, an image quality evaluation device and terminal equipment. The construction method comprises the following steps: acquiring a plurality of first sample objects of different skin color types and a plurality of first sample distortion images subjected to artificial synthesis distortion processing; acquiring a plurality of second sample objects of different skin color types, and acquiring a plurality of second sample distortion images in different shooting environments; carrying out labeling processing on the sample distortion images to obtain corresponding sample skin brightness evaluation information; and inputting the plurality of first sample distortion images, the plurality of second sample distortion images and the corresponding sample skin brightness evaluation information into a neural network model based on comparative learning so as to train and obtain an image quality evaluation model for outputting the skin brightness in any image. According to the method and the device, the evaluation information of the skin brightness in the input image containing any skin color can be quickly predicted in a high-quality manner.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to a model training method, an image quality evaluation method, an apparatus, and a terminal device. Background Art

[0002] With the rapid development of imaging technology, people have higher and higher requirements for image quality, and the accuracy requirements for face detection are getting higher and higher. Therefore, the image quality evaluation methods are becoming more and more diverse, but how to evaluate images based on human vision remains a difficult point. Summary of the Invention

[0003] The technical problem to be solved by the present disclosure is to overcome the above-mentioned defects in the prior art and to provide a model training method, an image quality evaluation method, an apparatus and a terminal device.

[0004] The present disclosure solves the above technical problems through the following technical solutions:

[0005] The present disclosure provides a method for constructing an image quality assessment model, the method comprising:

[0006] Acquire a plurality of first sample objects of different skin color types, and obtain a plurality of distorted images of the first sample objects after artificially synthesizing distortion processing;

[0007] Acquire a plurality of second sample objects of different skin color types, and obtain a plurality of distorted images of the second sample objects by using an image acquisition device captured under different shooting environments;

[0008] performing labeling processing on each of the first sample distorted image and the second sample distorted image to obtain sample skin brightness evaluation information corresponding to each image;

[0009] Several of the first sample distorted images, the second sample distorted images, and the corresponding sample skin brightness evaluation information are input into a neural network model based on contrastive learning to train the image quality evaluation model for outputting the skin brightness evaluation information in any image.

[0010] Preferably, for the same training batch, the step of inputting a plurality of the first sample distorted images, the second sample distorted images, and the corresponding sample skin brightness evaluation information into a neural network model based on contrastive learning to train and obtain the image quality assessment model for outputting the skin brightness evaluation information in any image includes:

[0011] performing feature extraction processing on each of the first sample distorted images input into the neural network model based on contrastive learning to obtain first sample feature data;

[0012] performing feature extraction processing on each of the second sample distorted images input into the neural network model based on contrastive learning to obtain second sample feature data;

[0013] Inputting a plurality of the first sample feature data and the second sample feature data into the contrast loss function in the neural network model to calculate a loss value;

[0014] In response to the loss value meeting the preset model training condition, the model obtained in the current training batch is used as the trained image quality evaluation model; otherwise, the training of the next training batch is entered until the loss value meets the preset model training condition.

[0015] Preferably, the feature extraction processing step includes:

[0016] Perform filtering processing, downsampling processing, color space conversion processing, encoding processing, and feature extraction processing in sequence;

[0017] and / or,

[0018] Before the step of inputting the plurality of first sample distorted images, the second sample distorted images and the corresponding sample skin brightness evaluation information into the neural network model based on contrastive learning, the method further includes:

[0019] Oversampling processing is performed on the first sample distorted image and / or the second sample distorted image.

[0020] The present disclosure further provides an image quality evaluation method, which is implemented based on the image quality evaluation model obtained by the above-mentioned construction method, and includes:

[0021] Acquire target skin image;

[0022] Determining a corresponding target skin color type based on the target skin image;

[0023] The target skin image and the target skin color type are input into the image quality assessment model to output target assessment information representing the skin brightness in the target skin image.

[0024] Preferably, the step of acquiring the target skin image includes:

[0025] Get the initial image;

[0026] The initial image is segmented to obtain the target skin image.

[0027] Preferably, the step of segmenting the initial image to obtain the target skin image includes:

[0028] Using a fast segmentation convolutional neural network to segment the initial image to obtain the target skin image including different skin areas;

[0029] Wherein, the fast segmentation convolutional neural network includes a pyramid pooling module.

[0030] Preferably, after the step of acquiring the initial image and before the step of segmenting the initial image to obtain the target skin image, the method further includes:

[0031] Preprocessing the initial image to obtain the preprocessed initial image;

[0032] Wherein, the preprocessing includes at least one of color balancing processing, color space conversion processing, and denoising processing;

[0033] and / or,

[0034] The step of determining corresponding target skin color type information based on the target skin image includes:

[0035] Obtaining target skin color information corresponding to the target skin image;

[0036] Based on preset skin color information ranges corresponding to different skin color types, determining the target skin color information range into which the target skin color information falls, so as to determine the corresponding target skin color type;

[0037] and / or,

[0038] The initial image includes a face image, and the target skin image includes a face skin image.

[0039] The present disclosure also provides a device for constructing an image quality assessment model, the device comprising:

[0040] A first sample image acquisition module is configured to acquire a plurality of first sample objects of different skin color types and a plurality of first sample distorted images that have undergone artificial synthesis distortion processing;

[0041] A second sample image acquisition module is configured to acquire a plurality of second sample objects of different skin color types, and obtain a plurality of distorted images of the second sample objects by using an image acquisition device captured under different shooting environments;

[0042] a sample image annotation module, configured to perform an annotation process on each of the first sample distorted image and the second sample distorted image to obtain sample skin brightness evaluation information corresponding to each image;

[0043] A model construction module is used to input several of the first sample distorted images, the second sample distorted images and the corresponding sample skin brightness evaluation information into a neural network model based on contrastive learning, so as to train the image quality evaluation model for outputting the skin brightness evaluation information in any image.

[0044] Preferably, for the same training batch, the model building module includes:

[0045] a first feature acquisition unit, configured to perform feature extraction processing on each of the first sample distorted images input into the neural network model based on contrastive learning to obtain first sample feature data;

[0046] a second feature acquisition unit, configured to perform feature extraction processing on each of the second sample distorted images input into the neural network model based on contrastive learning to obtain second sample feature data;

[0047] a loss value calculation unit, configured to input a plurality of the first sample feature data and the second sample feature data into a contrast loss function in the neural network model to calculate a loss value;

[0048] A model training unit is used to, in response to the loss value satisfying the preset model training condition, use the model obtained in the current training batch as the trained image quality evaluation model; otherwise, enter the training of the next training batch until the loss value satisfies the preset model training condition.

[0049] Preferably, the feature extraction processing step includes:

[0050] Perform filtering processing, downsampling processing, color space conversion processing, encoding processing, and feature extraction processing in sequence;

[0051] and / or,

[0052] The construction device also includes:

[0053] An oversampling processing module is configured to perform oversampling processing on the first sample distorted image and / or the second sample distorted image.

[0054] The present disclosure also provides an image quality evaluation device, the evaluation device comprising:

[0055] A skin image acquisition module, used to acquire a target skin image;

[0056] a skin color type determination module, configured to determine a corresponding target skin color type based on the target skin image;

[0057] The evaluation information output module is used to input the target skin color type into the image quality evaluation model obtained based on the above-mentioned construction device to output target evaluation information representing the skin brightness in the target skin image.

[0058] Preferably, the skin image acquisition module includes:

[0059] An initial image acquisition unit, configured to acquire an initial image;

[0060] The image segmentation processing unit is used to perform segmentation processing on the initial image to obtain the target skin image.

[0061] Preferably, the image segmentation processing unit is further configured to perform segmentation processing on the initial image using a fast segmentation convolutional neural network, so as to segment and obtain the target skin image including different skin areas;

[0062] Wherein, the fast segmentation convolutional neural network includes a pyramid pooling module.

[0063] Preferably, the evaluation device further includes:

[0064] A preprocessing module, configured to preprocess the initial image to obtain the preprocessed initial image;

[0065] Wherein, the preprocessing includes at least one of color balancing processing, color space conversion processing, and denoising processing;

[0066] and / or,

[0067] The skin color type determination module includes:

[0068] a skin color information acquiring unit, configured to acquire target skin color information corresponding to the target skin image;

[0069] a skin color type determining unit, configured to determine, based on preset skin color information ranges corresponding to different skin color types, a target skin color information range into which the target skin color information falls, so as to determine the corresponding target skin color type;

[0070] and / or,

[0071] The initial image includes a face image, and the target skin image includes a face skin image.

[0072] The present disclosure also provides a terminal device, which includes the image quality evaluation device as described above.

[0073] The present disclosure also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and used to run on the processor, characterized in that when the processor executes the computer program, it implements the above-mentioned method for constructing an image quality evaluation model or the above-mentioned method for evaluating image quality.

[0074] The present disclosure also provides a computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, it implements the above-mentioned method for constructing an image quality assessment model or the above-mentioned method for assessing image quality.

[0075] The present disclosure also provides a computer program product, including a computer program, characterized in that when the computer program is executed by a processor, it implements the above-mentioned method for constructing an image quality assessment model or the above-mentioned method for assessing image quality.

[0076] On the basis of conforming to the common sense in this field, the above-mentioned preferred conditions can be arbitrarily combined to obtain the preferred embodiments of the present disclosure.

[0077] The positive progress of this disclosure is:

[0078] In the present disclosure, based on a sample image set composed of a mixture of synthetic distorted images and real distorted images, these sample images are annotated according to preset annotation rules, and a neural network model with reference-free contrastive learning is used for model training to obtain an image quality evaluation model for outputting skin brightness evaluation in any target image. While ensuring the efficiency of model training, the reliability of model acquisition is also guaranteed; and the evaluation method of the present disclosure is not highly dependent on skin color and illumination uniformity, and can be used to evaluate images containing various skin color states, thereby ensuring the accuracy and efficiency of skin brightness quality assessment in images, thereby solving the problem of no reference evaluation of facial brightness quality in images of various skin colors. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] Figure 1 This is a flowchart of a method for constructing an image quality assessment model according to Embodiment 1 of the present disclosure;

[0080] Figure 2 This is a flowchart of a method for constructing an image quality assessment model according to Embodiment 2 of the present disclosure;

[0081] Figure 3 A schematic diagram showing the relationship between the true score corresponding to the input image and the predicted score output by the model in Example 2 of the present disclosure;

[0082] Figure 4 This is a schematic diagram of feature extraction for a sample distorted image according to Embodiment 2 of the present disclosure;

[0083] Figure 5This is a flowchart of the image quality evaluation method of Example 3 of the present disclosure;

[0084] Figure 6 This is a flowchart of the image quality evaluation method of Example 4 of the present disclosure;

[0085] Figure 7 This is a module diagram of a device for constructing an image quality assessment model according to Embodiment 5 of the present disclosure;

[0086] Figure 8 This is a module diagram of a device for constructing an image quality assessment model according to Embodiment 6 of the present disclosure;

[0087] Figure 9 This is a module diagram of an image quality evaluation device according to Embodiment 7 of the present disclosure;

[0088] Figure 10 This is a module diagram of an image quality evaluation device according to Embodiment 8 of the present disclosure;

[0089] Figure 11 This is a schematic structural diagram of an electronic device according to embodiment 10 of the present disclosure. DETAILED DESCRIPTION

[0090] The present disclosure is further illustrated below by way of examples, but the present disclosure is not limited to the scope of the examples.

[0091] In the embodiments of the present disclosure, prefixes such as "first" and "second" are used only to distinguish different description objects, and have no limiting effect on the position, order, priority, quantity or content of the described objects. In the embodiments of the present disclosure, the use of prefixes such as ordinal numbers to distinguish description objects does not constitute a limitation on the described objects. For the statement of the described objects, please refer to the description in the context of the claims or embodiments, and no unnecessary limitations should be constituted due to the use of such prefixes. In addition, in the description of this embodiment, unless otherwise specified, the meaning of "plurality" is two or more.

[0092] In the embodiments of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0093] The global population has diverse skin tones, and cameras and other devices process exposure and color differently for different skin tones. When shooting with the same device in the same environment, darker skin tones often appear underexposed, while lighter skin tones appear normal. Therefore, it's desirable to accurately evaluate a wide range of skin tones. While subjective evaluation based on the human eye is the ultimate standard for image quality, evaluating large amounts of image data is time-consuming, tedious, and unsuitable for mainstream rapid evaluation.

[0094] As research continues to deepen, the accuracy of full-reference quality assessment methods is becoming increasingly higher. However, this method requires a large number of undistorted reference images, which makes it difficult to quickly evaluate the quality of a group of images. However, the no-reference image quality assessment method does not require any information from the undistorted reference image and can evaluate the quality of the distorted image based solely on the distorted image. This scheme is a no-reference image quality assessment scheme that can achieve fast and high-quality image quality assessment. Specifically:

[0095] Example 1

[0096] like Figure 1 As shown, the method for constructing the image quality assessment model in this embodiment includes:

[0097] S101, obtaining a plurality of first sample objects of different skin color types, and synthesizing a plurality of distorted images of the first sample after artificially synthesizing distortion processing;

[0098] Among them, different skin color types include yellow, black, white, etc.

[0099] The artificially distorted images can be obtained by crawling the Internet to obtain publicly available artificially distorted images, or by distorting images obtained by crawling the Internet.

[0100] S102, obtaining a plurality of second sample objects of different skin color types, and obtaining a plurality of second sample distorted images by using an image acquisition device to acquire the objects under different shooting environments;

[0101] Among them, people with different skin colors are photographed using multiple camera devices in different shooting environments to directly obtain real distorted images.

[0102] Of course, the first sample distorted image and the second sample distorted image are images containing human facial skin; of course, the first sample distorted image and the second sample distorted image may also be images containing skin of other parts of the human body according to actual scene requirements.

[0103] S103, labeling each of the first sample distorted image and the second sample distorted image to obtain sample skin brightness evaluation information corresponding to each image;

[0104] Among them, generally, experts and other professionals mark these sample distorted images with corresponding sample skin brightness evaluation information; specifically, the sample skin brightness evaluation information can be an evaluation annotation score, and different score values represent different brightness conditions; further, according to needs, the interval of the evaluation annotation score can be [-1, 1], and the step size of the evaluation annotation score is set to 0.1.

[0105] S104: Inputting a plurality of first sample distorted images, a second sample distorted images, and corresponding sample skin brightness evaluation information into a neural network model based on contrastive learning to train an image quality evaluation model for outputting skin brightness in any image.

[0106] In this scheme, based on a sample image set composed of a mixture of synthetic distorted images and real distorted images, these sample images are annotated according to preset annotation rules, and a neural network model with reference-free contrastive learning is used for model training to obtain an image quality evaluation model for outputting skin brightness evaluation in any target image. While ensuring the efficiency of model training, the reliability of model acquisition is also guaranteed, thereby ensuring the accuracy and efficiency of skin brightness quality assessment in images, thereby solving the problem of no reference evaluation of facial brightness quality in images of various skin colors.

[0107] Example 2

[0108] The method for constructing the image quality assessment model of this embodiment is a further improvement of the embodiment 1. Specifically:

[0109] In one feasible solution, for the same training batch, step S104 includes:

[0110] S1041. Perform feature extraction processing on each first sample distorted image input into the neural network model based on contrastive learning to obtain first sample feature data;

[0111] S1042, performing feature extraction processing on each second sample distorted image input into the neural network model based on contrastive learning to obtain second sample feature data;

[0112] S1043, inputting the plurality of first sample feature data and the second sample feature data into a contrast loss function in the neural network model to calculate a loss value;

[0113] S1044. In response to the loss value meeting the preset model training conditions, the model obtained in the current training batch is used as the trained image quality evaluation model; otherwise, the training of the next training batch is entered until the loss value meets the preset model training conditions.

[0114] The number of first and second sample distorted images can be as large as possible, such as more than 30,000, to ensure the robustness and stability of model training. Alternatively, the entire sample distorted image set can be divided into a training set and a test set in a certain ratio (e.g., 8:2), and then trained and tested using a neural network model based on contrastive learning.

[0115] Specifically, during the model training process, the root mean square error (RMSE) is used as the loss function. The RMSE calculation formula is as follows:

[0116]

[0117] Where N represents the total number of training sample images in each batch, observed represents the true score of any input image (i.e., the true skin brightness evaluation information), and predicted represents the predicted score output after the input image is input into the trained model (i.e., the predicted skin brightness evaluation information).

[0118] For example, Figure 3 As shown, the horizontal axis represents the true score of the input image, and the vertical axis represents the predicted score output by the model. This shows the correspondence between the true scores corresponding to different input images and the predicted scores output by the model during model training. In addition, the preset model training conditions are set for actual model training scenarios and are not further described here.

[0119] In this solution, during the model training process, the feature data corresponding to the synthetic distorted image and the real distorted image are obtained respectively, and the loss value is calculated in combination with the contrast loss function in the model to quickly and accurately determine whether the currently trained model is feasible, thereby ensuring the rationality and efficiency of the model training process.

[0120] In one feasible solution, the feature extraction process includes:

[0121] Filtering, downsampling, color space conversion, encoding, and feature extraction are performed in sequence; specifically, the input image is filtered by a filter, and the input data is encoded by an encoder.

[0122] In this scheme, if Figure 4As shown in the figure, by processing the input sample distorted images in sequence, the quality of the obtained feature data is effectively guaranteed, and then the accuracy of the loss value determined by the subsequent contrast loss function is guaranteed, thereby improving the quality and reliability of the entire model training.

[0123] After step S103 and before step S104, the method further includes:

[0124] Oversampling is performed on the first sample distorted image and / or the second sample distorted image.

[0125] In this scheme, before model training, the distorted input sample images are oversampled to eliminate the imbalance between images of different skin color types, ensuring the feasibility and quality of model predictions obtained in subsequent training.

[0126] Example 3

[0127] The image quality evaluation method of this embodiment is implemented based on the image quality evaluation model obtained by the above-mentioned construction method.

[0128] like Figure 5 As shown, the image quality evaluation method of this embodiment includes:

[0129] S201, obtaining a target skin image;

[0130] The target skin image includes but is not limited to a facial image; the target skin image can be a local skin image directly, or can be an image extracted from the entire large image.

[0131] S202, determining a corresponding target skin color type based on the target skin image;

[0132] Among them, the target skin color types include yellow, black, white, etc.

[0133] S203: Input the target skin image and the target skin color type into an image quality evaluation model to output target evaluation information representing the skin brightness in the target skin image.

[0134] In this solution, for any target skin image, the target evaluation information of the skin brightness, that is, the target prediction score, can be directly predicted through the image quality evaluation model, so that for input images containing any skin color, regardless of whether the face has yellow, black, or white skin color, the evaluation information of the skin brightness can be obtained quickly and with high quality; and the evaluation method disclosed in the present invention does not rely much on skin color and illumination uniformity, and can be used to evaluate images containing various skin color states.

[0135] Example 4

[0136] like Figure 6As shown, the image quality evaluation method of this embodiment is a further improvement of embodiment 3, specifically:

[0137] In one feasible solution, step S201 includes:

[0138] S2011. Obtain an initial image; wherein the initial image is generally an image containing a face.

[0139] S2012: Segment the initial image to obtain a target skin image.

[0140] In this solution, any initial image, whether it includes part of a person's face, the entire face of a person, or two or more faces, can be processed using image segmentation technology to directly extract the skin image corresponding to each facial region. This removes other interfering content from the image, leaving only the skin image of the skin region. This simplifies the computational complexity of subsequent image processing while ensuring the overall efficiency of image quality evaluation. Furthermore, when there are two or more faces in an image, simultaneous segmentation and extraction can be achieved, and the image quality evaluation model can then output evaluation information for the skin brightness corresponding to each face, further improving the efficiency of the image skin brightness evaluation process.

[0141] In one feasible solution, step S2012 includes:

[0142] The initial image is segmented using a fast segmentation convolutional neural network to obtain a target skin image including different skin areas;

[0143] Among them, the fast segmentation convolutional neural network includes a pyramid pooling module.

[0144] In this solution, Fast-SCNN (Fast Segmentation Convolutional Neural Network) is used to identify and segment the skin area of the face. In complex segmentation tasks, Fast-SCNN introduces PPM (Pyramid Pooling Module) to compensate for the shortcomings of global average pooling in capturing multi-scale contextual information. This pyramid pooling module effectively distinguishes similar parts in the image, such as left and right eyes or eyebrows, by processing global average pooling operations of different sizes in parallel and fusing the results with the original features. This not only preserves spatial information but also enhances the semantic representation of features, thereby effectively improving segmentation accuracy.

[0145] In one feasible solution, after step S2011 and before step S2012, the following steps are further included:

[0146] Preprocessing the initial image to obtain a preprocessed initial image;

[0147] Among them, preprocessing includes color balancing, color space conversion, denoising and other operations.

[0148] Preferably, the initial image is subjected to color balancing processing, color space conversion processing, and denoising processing in sequence.

[0149] Specifically, when the initial image contains a face, it is necessary to first perform color balancing on the face image to eliminate the influence of light source color changes on face area detection. Specifically, the intensity of the red, green, and blue components of the image is adjusted to restore the original color characteristics of the image scene, thereby completing the color balancing in the image.

[0150] The distribution of human skin color in the YCbCr color space is relatively concentrated (called the clustering property of skin color). The difference in skin color between different races is mainly caused by brightness and has nothing to do with color attributes. By utilizing this property, image pixels are divided into two categories: skin color pixels and non-skin color pixels, which can greatly improve the efficiency and accuracy of face detection. Specifically, the image in the RGB color space is converted to the YCbCr color space. For example, the corresponding conversion formula is as follows:

[0151] Y=0.257*R+0.504*G+0.098*B+16

[0152] Cb=-0.148*R-0.291*G+0.439*B+128

[0153] Cr=0.439*R-0.368*G-0.071*B+128

[0154] Wherein, Y represents brightness, Cr represents the red component of the light source, and Cb represents the blue component of the light source; each parameter of the above formula can be re-determined or adjusted according to actual conditions.

[0155] In addition, after color space conversion, facial images generally have noise. In this case, a two-dimensional median filter with a sliding window of 3×3 can be used to remove the noise.

[0156] In this scheme, by performing a series of preprocessing on the facial image in the initial image, the accuracy of the skin image is guaranteed, thereby ensuring the accuracy of the data input to the image quality evaluation model, and ultimately ensuring the reliability and accuracy of the image quality evaluation results output by the image quality evaluation model.

[0157] In one feasible solution, step S202 includes:

[0158] S2021. Obtain target skin color information corresponding to the target skin image;

[0159] S2022: Based on preset skin color information ranges corresponding to different skin color types, determine the target skin color information range within which the target skin color information falls, to determine the corresponding target skin color type;

[0160] Specifically, for the facial skin area, the facial skin color is divided into three categories: yellow, black, and white according to the skin color color characteristics of the YCbCr space; by pre-defining the skin color information range of yellow skin Y as (80, 200), the skin color information range of Cb as (130, 195), and the skin color information range of Cr as (130, 170); the skin color information range of black skin Y as (70, 160), the skin color information range of Cb as (150, 180), and the skin color information range of Cr as (145, 170); the skin color information range of white skin Y value as (50, 180), the skin color information range of Cb as (140, 190), and the skin color information range of Cr as (142, 190), it is possible to determine the skin color corresponding to the skin in the face image to be detected according to the extracted skin color information distribution range.

[0161] In this solution, the corresponding target skin color information is obtained based on the pixel information in the target skin image, and then the skin color information range into which the target skin color information falls is determined, so as to quickly and accurately determine the corresponding skin color type, thereby ensuring the overall quality and efficiency of image quality evaluation.

[0162] The implementation principle of the image quality evaluation method in this embodiment is described in detail below:

[0163] (1) obtaining an input image containing a human face, where the input image may be an image of any skin color, such as yellow, black, or white;

[0164] (2) Preprocessing the input image, including color balancing, color space conversion, and denoising;

[0165] (3) Fast-SCNN with pyramid pooling module PPM is used to segment the preprocessed image to obtain facial skin image;

[0166] (4) determining the skin color type based on the skin color information of the facial skin image;

[0167] (5) The facial skin image and the skin color type are input into a pre-trained image quality evaluation model to directly output target evaluation information representing the skin brightness in the target skin image.

[0168] Example 5

[0169] like Figure 7 As shown, the apparatus for constructing the image quality assessment model in this embodiment includes:

[0170] A first sample image acquisition module 1 is used to acquire a plurality of first sample objects of different skin color types and a plurality of first sample distorted images that have been artificially synthesized and distorted;

[0171] Among them, different skin color types include yellow, black, white, etc.

[0172] The artificially distorted images can be obtained by crawling the Internet to obtain publicly available artificially distorted images, or by distorting images obtained by crawling the Internet.

[0173] A second sample image acquisition module 2 is used to acquire a plurality of second sample objects of different skin color types, by using an image acquisition device to acquire a plurality of second sample distorted images under different shooting environments;

[0174] Among them, people with different skin colors are photographed using multiple camera devices in different shooting environments to directly obtain real distorted images.

[0175] Of course, the first sample distorted image and the second sample distorted image are images containing human facial skin; of course, the first sample distorted image and the second sample distorted image may also be images containing skin of other parts of the human body according to actual scene requirements.

[0176] The sample image annotation module 3 is used to perform annotation processing on each of the first sample distorted image and the second sample distorted image to obtain sample skin brightness evaluation information corresponding to each image;

[0177] Among them, generally, experts and other professionals mark these sample distorted images with corresponding sample skin brightness evaluation information; specifically, the sample skin brightness evaluation information can be an evaluation annotation score, and different score values represent different brightness conditions; further, according to needs, the interval of the evaluation annotation score can be [-1, 1], and the step size of the evaluation annotation score is set to 0.1.

[0178] The model construction module 4 is used to input a plurality of first sample distorted images, a second sample distorted images and the corresponding sample skin brightness evaluation information into a neural network model based on contrastive learning, so as to train an image quality evaluation model for outputting the skin brightness in any image.

[0179] In this scheme, based on a sample image set composed of a mixture of synthetic distorted images and real distorted images, these sample images are annotated according to preset annotation rules, and a neural network model with reference-free contrastive learning is used for model training to obtain an image quality evaluation model for outputting skin brightness evaluation in any target image. While ensuring the efficiency of model training, the reliability of model acquisition is also guaranteed, thereby ensuring the accuracy and efficiency of skin brightness quality assessment in images, thereby solving the problem of no reference evaluation of facial brightness quality in images of various skin colors.

[0180] Example 6

[0181] like Figure 8 As shown, the apparatus for constructing the image quality evaluation model of this embodiment is a further improvement of embodiment 5, specifically:

[0182] In one feasible solution, for the same training batch, the model building module 4 includes:

[0183] a first feature acquisition unit 5, configured to perform feature extraction processing on each first sample distorted image input into the neural network model based on contrastive learning to obtain first sample feature data;

[0184] a second feature acquisition unit 6, configured to perform feature extraction processing on each second sample distorted image input into the neural network model based on contrastive learning to obtain second sample feature data;

[0185] A loss value calculation unit 7 is used to input a plurality of first sample feature data and a second sample feature data into a contrast loss function in a neural network model to calculate a loss value;

[0186] The model training unit 8 is used to respond to the loss value meeting the preset model training conditions and use the model obtained in the current training batch as the trained image quality evaluation model; otherwise, enter the training of the next training batch until the loss value meets the preset model training conditions.

[0187] The number of first and second sample distorted images can be as large as possible, such as more than 30,000, to ensure the robustness and stability of model training. Alternatively, the entire sample distorted image set can be divided into a training set and a test set in a certain ratio (e.g., 8:2), and then trained and tested using a neural network model based on contrastive learning.

[0188] Specifically, during the model training process, the root mean square error (RMSE) is used as the loss function. The RMSE calculation formula is as follows:

[0189]

[0190] Where N represents the total number of training sample images in each batch, observed represents the true score of any input image (i.e., the true skin brightness evaluation information), and predicted represents the predicted score output after the input image is input into the trained model (i.e., the predicted skin brightness evaluation information).

[0191] For example, Figure 3 As shown, the horizontal axis represents the true score of the input image, and the vertical axis represents the predicted score output by the model. This shows the correspondence between the true scores corresponding to different input images and the predicted scores output by the model during model training. In addition, the preset model training conditions are set for actual model training scenarios and are not further described here.

[0192] In this solution, during the model training process, the feature data corresponding to the synthetic distorted image and the real distorted image are obtained respectively, and the loss value is calculated in combination with the contrast loss function in the model to quickly and accurately determine whether the currently trained model is feasible, thereby ensuring the rationality and efficiency of the model training process.

[0193] In one feasible solution, the feature extraction process includes:

[0194] Perform filtering processing, downsampling processing, color space conversion processing, encoding processing, and feature extraction processing in sequence;

[0195] In this solution, during the model training process, the feature data corresponding to the synthetic distorted image and the real distorted image are obtained respectively, and the loss value is calculated in combination with the contrast loss function in the model to quickly and accurately determine whether the currently trained model is feasible, thereby ensuring the rationality and efficiency of the model training process.

[0196] In one feasible solution, the construction device further comprises:

[0197] The oversampling processing module 9 is configured to perform oversampling processing on the first sample distorted image and / or the second sample distorted image.

[0198] In this solution, before model training, the distorted image of the input sample is oversampled to eliminate the imbalance between images of different skin color types, thereby ensuring the feasibility and quality of the model prediction obtained by subsequent training. For the system embodiment, since it basically corresponds to the method embodiment, the relevant parts can be referred to the partial description of the method embodiment. The system embodiment described above is merely schematic, wherein the units described as separate components may or may not be physically separated, and the components of the units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the disclosed solution.

[0199] Example 7

[0200] like Figure 9 As shown, the image quality evaluation device of this embodiment includes:

[0201] The skin image acquisition module 10 is used to acquire a target skin image;

[0202] The target skin image includes but is not limited to a facial image; the target skin image can be a local skin image directly, or can be an image extracted from the entire large image.

[0203] A skin color type determination module 11 is used to determine the corresponding target skin color type based on the target skin image;

[0204] The evaluation information output module 12 is used to input the target skin color type into the image quality evaluation model obtained based on the above-mentioned construction device to output target evaluation information representing the skin brightness in the target skin image.

[0205] In this solution, for any target skin image, the target evaluation information of the skin brightness, that is, the target prediction score, can be directly predicted through the image quality evaluation model, so that for input images containing any skin color, regardless of whether the face has yellow, black, or white skin color, the evaluation information of the skin brightness can be obtained quickly and with high quality; and the evaluation method disclosed in the present invention does not rely much on skin color and illumination uniformity, and can be used to evaluate images containing various skin color states.

[0206] Example 8

[0207] like Figure 10 As shown, the image quality evaluation device of this embodiment is a further improvement of embodiment 7, specifically:

[0208] In one feasible solution, the skin image acquisition module 10 includes:

[0209] An initial image acquisition unit 13, configured to acquire an initial image;

[0210] The image segmentation processing unit 14 is used to perform segmentation processing on the initial image to obtain a target skin image.

[0211] In this solution, any initial image, whether it includes part of a person's face, the entire face of a person, or two or more faces, can be processed using image segmentation technology to directly extract the skin image corresponding to each facial region. This removes other interfering content from the image, leaving only the skin image of the skin region. This simplifies the computational complexity of subsequent image processing while ensuring the overall efficiency of image quality evaluation. Furthermore, when there are two or more faces in an image, simultaneous segmentation and extraction can be achieved, and the image quality evaluation model can then output evaluation information for the skin brightness corresponding to each face, further improving the efficiency of the image skin brightness evaluation process.

[0212] In one feasible solution, the image segmentation processing unit 14 is further configured to perform segmentation processing on the initial image using a fast segmentation convolutional neural network to obtain a target skin image including different skin regions;

[0213] Among them, the fast segmentation convolutional neural network includes a pyramid pooling module.

[0214] In this solution, Fast-SCNN (Fast Segmentation Convolutional Neural Network) is used to identify and segment the skin area of the face. In complex segmentation tasks, Fast-SCNN introduces PPM (Pyramid Pooling Module) to compensate for the shortcomings of global average pooling in capturing multi-scale contextual information. This pyramid pooling module effectively distinguishes similar parts in the image, such as left and right eyes or eyebrows, by processing global average pooling operations of different sizes in parallel and fusing the results with the original features. This not only preserves spatial information but also enhances the semantic representation of features, thereby effectively improving segmentation accuracy.

[0215] In one feasible solution, the evaluation device further includes:

[0216] A preprocessing module 15 is used to preprocess the initial image to obtain a preprocessed initial image;

[0217] The preprocessing includes at least one of color balancing, color space conversion, and denoising.

[0218] Preferably, the initial image is subjected to color balancing processing, color space conversion processing, and denoising processing in sequence.

[0219] Specifically, when the initial image contains a face, it is necessary to first perform color balancing on the face image to eliminate the influence of light source color changes on face area detection. Specifically, the intensity of the red, green, and blue components of the image is adjusted to restore the original color characteristics of the image scene, thereby completing the color balancing in the image.

[0220] The distribution of human skin color in the YCbCr color space is relatively concentrated (called the clustering property of skin color). The difference in skin color between different races is mainly caused by brightness and has nothing to do with color attributes. By utilizing this property, image pixels are divided into two categories: skin color pixels and non-skin color pixels, which can greatly improve the efficiency and accuracy of face detection. Specifically, the image in the RGB color space is converted to the YCbCr color space. For example, the corresponding conversion formula is as follows:

[0221] Y=0.257*R+0.504*G+0.098*B+16

[0222] Cb=-0.148*R-0.291*G+0.439*B+128

[0223] Cr=0.439*R-0.368*G-0.071*B+128

[0224] Wherein, Y represents brightness, Cr represents the red component of the light source, and Cb represents the blue component of the light source; each parameter of the above formula can be re-determined or adjusted according to actual conditions.

[0225] In addition, after color space conversion, facial images generally have noise. In this case, a two-dimensional median filter with a sliding window of 3×3 can be used to remove the noise.

[0226] In this scheme, by performing a series of preprocessing on the facial image in the initial image, the accuracy of the skin image is guaranteed, thereby ensuring the accuracy of the data input to the image quality evaluation model, and ultimately ensuring the reliability and accuracy of the image quality evaluation results output by the image quality evaluation model.

[0227] In one embodiment, the skin color type determination module 11 includes:

[0228] a skin color information acquiring unit 16, configured to acquire target skin color information corresponding to the target skin image;

[0229] A skin color type determining unit 17 is configured to determine a target skin color information range within which the target skin color information falls based on preset skin color information ranges corresponding to different skin color types, so as to determine the corresponding target skin color type;

[0230] Specifically, for the facial skin area, the facial skin color is divided into three categories: yellow, black, and white according to the skin color color characteristics of the YCbCr space; by pre-defining the skin color information range of yellow skin Y as (80, 200), the skin color information range of Cb as (130, 195), and the skin color information range of Cr as (130, 170); the skin color information range of black skin Y as (70, 160), the skin color information range of Cb as (150, 180), and the skin color information range of Cr as (145, 170); the skin color information range of white skin Y value as (50, 180), the skin color information range of Cb as (140, 190), and the skin color information range of Cr as (142, 190), it is possible to determine the skin color corresponding to the skin in the face image to be detected according to the extracted skin color information distribution range.

[0231] In this solution, the corresponding target skin color information is obtained based on the pixel information in the target skin image, and then the skin color information range into which the target skin color information falls is determined, so as to quickly and accurately determine the corresponding skin color type, thereby ensuring the overall quality and efficiency of image quality evaluation.

[0232] Since the system embodiments generally correspond to the method embodiments, reference will be made to the description of the method embodiments for relevant details. The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components of the units may or may not be physical units, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the disclosed solution.

[0233] Example 9

[0234] The terminal device of this embodiment includes the image quality evaluation device as described above.

[0235] The terminal device includes but is not limited to a mobile terminal.

[0236] In this embodiment, the terminal device is integrated with the above-mentioned evaluation device. For any target skin image, the target evaluation information of the skin brightness, that is, the target prediction score, can be directly predicted through the image quality evaluation model. Therefore, for input images containing any skin color, regardless of whether the face has yellow, black, or white skin color, the evaluation information of the skin brightness can be obtained quickly and with high quality, effectively improving the image processing performance of the terminal device, thereby improving the overall product performance of the terminal device.

[0237] Example 10

[0238] Figure 11This is a structural diagram of an electronic device showing an example embodiment of the present disclosure. The electronic device includes a memory, a processor, and a computer program stored in the memory and used to run on the processor. When the processor executes the computer program, the method of any of the above embodiments is implemented. Figure 11 The electronic device 90 shown is only an example and should not limit the functionality and scope of use of the embodiments of the present disclosure.

[0239] like Figure 11 As shown, the electronic device 90 can be in the form of a general-purpose computing device, for example, it can be a server device. The components of the electronic device 90 may include but are not limited to: at least one processor 91, at least one memory 92, and a processor that connects different system components (including the memory 92 and the processor). Processor 91) bus 93.

[0240] The bus 93 includes a data bus, an address bus, and a control bus.

[0241] The memory 92 may include a volatile memory, such as a random access memory (RAM) 921 and / or a cache memory 922 , and may further include a read-only memory (ROM) 923 .

[0242] The memory 92 may also include a program tool 925 (or utility) having a set (at least one) of program modules 924, such program modules 924 including but not limited to: an operating system, one or more application programs, other program modules and program data, each of which or some combination may include an implementation of a network environment.

[0243] The processor 91 executes various functional applications and data processing by running the computer programs stored in the memory 92, such as the method provided in any of the above embodiments.

[0244] The electronic device 90 can also communicate with one or more external devices 94 (e.g., a keyboard, pointing device, etc.). Such communication can occur via an input / output (I / O) interface 95. Furthermore, the electronic device 90 can communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 96. As shown, the network adapter 96 communicates with other modules of the electronic device 90 via a bus 93. It should be understood that, although not shown, other hardware and / or software modules can be used in conjunction with the electronic device 90, including but not limited to microcode, device drivers, redundant processors, external disk drive arrays, RAID (RAID) systems, tape drives, and data backup storage systems.

[0245] It should be noted that although several units / modules or sub-units / modules of the electronic device are mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided and embodied by multiple units / modules.

[0246] Example 11

[0247] An embodiment of the present disclosure further provides a computer-readable storage medium having a computer program stored thereon, which implements the method provided in any of the above embodiments when the program is executed by a processor.

[0248] The readable storage medium may include, but is not limited to, a portable disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0249] Example 12

[0250] An embodiment of the present disclosure further provides a computer program product, including a computer program, which implements any of the above methods when executed by a processor.

[0251] The program code for executing the computer program product of the present disclosure may be written in any combination of one or more programming languages, and the program code may be executed entirely on the user device, partially on the user device, as a standalone software package, partially on the user device and partially on a remote device, or entirely on the remote device.

[0252] While specific embodiments of the present disclosure have been described above, those skilled in the art will appreciate that these are merely illustrative and that the scope of protection of the present disclosure is defined by the appended claims. Those skilled in the art may make various changes or modifications to these embodiments without departing from the principles and essence of the present disclosure, and such changes and modifications are intended to fall within the scope of protection of the present disclosure.

Claims

1. A method for constructing an image quality assessment model, characterized in that: The construction method comprises: Acquire a plurality of first sample objects of different skin color types, and obtain a plurality of distorted images of the first sample objects after artificially synthesizing distortion processing; Acquire a plurality of second sample objects of different skin color types, and obtain a plurality of distorted images of the second sample objects by using an image acquisition device captured under different shooting environments; performing labeling processing on each of the first sample distorted image and the second sample distorted image to obtain sample skin brightness evaluation information corresponding to each image; Several of the first sample distorted images, the second sample distorted images, and the corresponding sample skin brightness evaluation information are input into a neural network model based on contrastive learning to train the image quality evaluation model for outputting the skin brightness evaluation information in any image.

2. The method for constructing an image quality assessment model according to claim 1, wherein: For the same training batch, the step of inputting a plurality of the first sample distorted images, the second sample distorted images, and the corresponding sample skin brightness evaluation information into a neural network model based on contrastive learning to train and obtain the image quality assessment model for outputting the skin brightness evaluation information in any image includes: performing feature extraction processing on each of the first sample distorted images input into the neural network model based on contrastive learning to obtain first sample feature data; performing feature extraction processing on each of the second sample distorted images input into the neural network model based on contrastive learning to obtain second sample feature data; Inputting a plurality of the first sample feature data and the second sample feature data into the contrast loss function in the neural network model to calculate a loss value; In response to the loss value meeting the preset model training condition, the model obtained in the current training batch is used as the trained image quality evaluation model; otherwise, the training of the next training batch is entered until the loss value meets the preset model training condition.

3. The method for constructing an image quality assessment model according to claim 2, wherein: The steps of feature extraction processing include: Perform filtering processing, downsampling processing, color space conversion processing, encoding processing, and feature extraction processing in sequence; and / or, Before the step of inputting the plurality of first sample distorted images, the second sample distorted images and the corresponding sample skin brightness evaluation information into the neural network model based on contrastive learning, the method further includes: Oversampling processing is performed on the first sample distorted image and / or the second sample distorted image.

4. A method for evaluating image quality, characterized in that: The evaluation method is implemented based on the image quality evaluation model obtained by the construction method according to any one of claims 1 to 3, and the evaluation method includes: Acquire target skin image; Determining a corresponding target skin color type based on the target skin image; The target skin image and the target skin color type are input into the image quality assessment model to output target assessment information representing the skin brightness in the target skin image.

5. The image quality evaluation method according to claim 4, wherein: The step of obtaining the target skin image includes: Get the initial image; The initial image is segmented to obtain the target skin image.

6. The image quality evaluation method according to claim 5, wherein: The step of segmenting the initial image to obtain the target skin image includes: Using a fast segmentation convolutional neural network to segment the initial image to obtain the target skin image including different skin areas; Wherein, the fast segmentation convolutional neural network includes a pyramid pooling module.

7. The image quality evaluation method according to claim 5 or 6, wherein: After the step of acquiring the initial image and before the step of segmenting the initial image to obtain the target skin image, the method further includes: Preprocessing the initial image to obtain the preprocessed initial image; Wherein, the preprocessing includes at least one of color balancing processing, color space conversion processing, and denoising processing; and / or, The step of determining corresponding target skin color type information based on the target skin image includes: Obtaining target skin color information corresponding to the target skin image; Based on preset skin color information ranges corresponding to different skin color types, determining the target skin color information range into which the target skin color information falls, so as to determine the corresponding target skin color type; and / or, The initial image includes a face image, and the target skin image includes a face skin image.

8. A device for constructing an image quality assessment model, characterized in that: The construction device comprises: A first sample image acquisition module is configured to acquire a plurality of first sample objects of different skin color types and a plurality of first sample distorted images that have undergone artificial synthesis distortion processing; A second sample image acquisition module is configured to acquire a plurality of second sample objects of different skin color types, and obtain a plurality of distorted images of the second sample objects by using an image acquisition device captured under different shooting environments; a sample image annotation module, configured to perform an annotation process on each of the first sample distorted image and the second sample distorted image to obtain sample skin brightness evaluation information corresponding to each image; A model construction module is used to input several of the first sample distorted images, the second sample distorted images and the corresponding sample skin brightness evaluation information into a neural network model based on contrastive learning, so as to train the image quality evaluation model for outputting the skin brightness evaluation information in any image.

9. An image quality evaluation device, characterized in that: The evaluation device comprises: A skin image acquisition module, used to acquire a target skin image; a skin color type determination module, configured to determine a corresponding target skin color type based on the target skin image; An evaluation information output module is used to input the target skin color type into the image quality evaluation model obtained based on the construction device according to claim 8 to output target evaluation information representing the skin brightness in the target skin image.

10. A terminal device, characterized in that: The terminal device includes the image quality evaluation device according to claim 9.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and configured to run on the processor, wherein: When the processor executes the computer program, the method for constructing an image quality assessment model according to any one of claims 1 to 3 or the method for assessing image quality according to any one of claims 4 to 7 is implemented.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for constructing an image quality assessment model according to any one of claims 1 to 3 or the method for assessing image quality according to any one of claims 4 to 7 is implemented.

13. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method for constructing an image quality assessment model according to any one of claims 1 to 3 or the method for assessing image quality according to any one of claims 4 to 7 is implemented.

Images