An evaluation method, device, apparatus and computer readable storage medium

Abstract

The application discloses an evaluation method, device and equipment and a computer readable storage medium. The evaluation method comprises the following steps: acquiring a plurality of image recognition models, a set of images to be evaluated and a first dimension quality score of each image to be evaluated in the set of images to be evaluated, wherein the first dimension quality score is used to reflect the quality of visual sense of each image to be evaluated; performing feature extraction processing on each image to be evaluated by using each image recognition model, so as to obtain a feature vector of each image to be evaluated; determining a second dimension quality score of each image to be evaluated based on the feature vector of each image to be evaluated; and determining a quality evaluation result of each image to be evaluated based on the first dimension quality score and the second dimension quality score. In this way, a quality evaluation result with higher accuracy can be obtained by combining multiple dimensions, and the quality evaluation result is optimized.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and includes, but is not limited to, an evaluation method, apparatus, device, and computer-readable storage medium. Background Technology

[0002] With the rapid development of image processing technology, facial recognition has become an indispensable technology in the security field, with broad application prospects. Video capture is an important application scenario in the security field. Facial images captured by network cameras are often complex and diverse, affected by factors such as lighting, facial pose, occlusion, and rapid movement, resulting in inconsistent image quality. Low-quality facial images significantly reduce the success rate of facial recognition. Therefore, how to select high-quality images from a series of facial images for recognition, thereby improving the speed and accuracy of facial recognition, is a problem that urgently needs to be solved.

[0003] In related technologies, face quality evaluation often revolves around issues such as how to detect lighting, pose, occlusion, and motion blur. However, these technologies also have certain problems. For example, deep learning-based face quality evaluation methods require manual scoring, which is time-consuming, labor-intensive, and subjective. Moreover, since many factors influence face quality, this manual scoring method cannot comprehensively consider the impact of multiple factors, leading to inaccurate face image quality scores. Summary of the Invention

[0004] In view of the above, embodiments of this application provide an evaluation method, apparatus, device, and computer-readable storage medium.

[0005] The technical solution of this application embodiment is implemented as follows:

[0006] This application provides an evaluation method, the method comprising:

[0007] A plurality of image recognition models, a set of images to be evaluated, and a first-dimensional quality score for each image to be evaluated in the set of images to be evaluated are obtained, wherein the first-dimensional quality score is used to reflect the visual quality of each image to be evaluated.

[0008] Each image recognition model is used to perform feature extraction processing on each image to be evaluated, and feature vectors of each image to be evaluated are obtained.

[0009] The second dimension quality score of each image to be evaluated is determined based on the feature vector of each image to be evaluated.

[0010] Based on the first dimension quality score and the second dimension quality score, the quality assessment result of each image to be evaluated is determined.

[0011] This application provides an evaluation apparatus, the evaluation apparatus comprising:

[0012] The first acquisition module is used to acquire multiple image recognition models, a set of images to be evaluated, and a first-dimensional quality score for each image to be evaluated in the set of images to be evaluated, wherein the first-dimensional quality score is used to reflect the visual quality of each image to be evaluated.

[0013] The feature extraction module is used to perform feature extraction processing on each image to be evaluated using various image recognition models to obtain the feature vector of each image to be evaluated.

[0014] The first determining module is used to determine the second dimension quality score of each image to be evaluated based on the feature vector of each image to be evaluated.

[0015] The second determining module is used to determine the quality assessment result of each image to be evaluated based on the first dimension quality score and the second dimension quality score.

[0016] This application provides an evaluation device, the evaluation device comprising:

[0017] Processor; and

[0018] Memory for storing computer programs that can run on the processor;

[0019] The computer program, when executed by a processor, implements the aforementioned evaluation method.

[0020] This application provides a computer-readable storage medium storing computer-executable instructions configured to perform the above-described evaluation method.

[0021] This application provides an evaluation method, apparatus, device, and computer-readable storage medium. The evaluation method includes: firstly, acquiring multiple image recognition models, a set of images to be evaluated, and a first-dimensional quality score for each image to be evaluated in the set of images to be evaluated. The first-dimensional quality score reflects the visual quality of each image to be evaluated; if the visual quality is better, the first-dimensional quality score is correspondingly higher. Next, each image recognition model is used to perform feature extraction processing on each image to be evaluated, obtaining feature vectors for each image to be evaluated. A second-dimensional quality score is then determined based on the feature vectors of each image to be evaluated. Finally, the first-dimensional quality score and the second-dimensional quality score are used to jointly determine the quality evaluation result of each image to be evaluated. In this way, the quality evaluation result of each image to be evaluated is jointly determined by the first-dimensional quality score reflecting the actual visual quality and the second-dimensional quality score reflecting the image features. The determination process does not require manual intervention, improving evaluation efficiency, ensuring the objectivity of the quality evaluation result, and also improving the accuracy of the quality evaluation result. Attached Figure Description

[0022] In the accompanying drawings (which are not necessarily drawn to scale), similar reference numerals may describe similar parts in different views. The drawings illustrate, by way of example and not limitation, the various embodiments discussed herein.

[0023] Figure 1 This is a schematic diagram of a first implementation flow of the evaluation method provided in the embodiments of this application;

[0024] Figure 2 This is a schematic diagram illustrating an implementation process for determining a scoring result provided in an embodiment of this application;

[0025] Figure 3 This is a schematic diagram of a second implementation flow of the evaluation method provided in the embodiments of this application;

[0026] Figure 4 This is a schematic diagram illustrating an implementation flow of the model training method provided in an embodiment of this application;

[0027] Figure 5 This is a schematic diagram of an implementation model structure of the image quality scoring model provided in the embodiments of this application;

[0028] Figure 6 A schematic diagram of the composition structure of the evaluation device provided in the embodiments of this application;

[0029] Figure 7 This is a schematic diagram of the composition structure of an evaluation device provided in an embodiment of this application. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0031] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0032] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0033] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0034] To better understand the evaluation method provided in the embodiments of this application, the evaluation methods in the following related technologies and their shortcomings will be explained first.

[0035] The first related technology includes: acquiring the target image to be evaluated; and evaluating the quality of the target image using a quality scoring network. The quality scoring network includes a classification part and a regression part. The classification part determines the image category corresponding to the target image, which includes false positives and false negatives. The regression part determines the quality score corresponding to the target image. This solves the problem of how to effectively and accurately evaluate the quality of face images, thus achieving accurate and rapid face image quality evaluation.

[0036] The second related technology includes: acquiring the original image and preprocessing the original image information. The preprocessing includes: performing face detection on the original image to obtain the face image region, and pre-classifying the image recognition features, and segmenting the original image according to the classification category to obtain multiple sub-images; evaluating the features in each sub-image to obtain all feature scores in each sub-image; and obtaining a comprehensive score for the face image based on the feature scores in all sub-images. This solves the problem that traditional face image quality algorithms focus too much on the image quality of the face and lack accurate analysis of the face.

[0037] The third related technology includes: acquiring an image to be processed containing a face; detecting the face in the image to be processed to obtain the corresponding face image; inputting the face image into a trained feature extraction model based on a mobile face recognition network to extract features from the face image to obtain feature data; and inputting the feature data into consecutively set first and second fully connected layers for processing to obtain a face quality score for the face image. This approach enables feature extraction and quality scoring of face images within a single network, effectively reducing the model's size. Furthermore, it allows for rapid evaluation of face image quality while ensuring the accuracy of the evaluation results.

[0038] The fourth related technique includes: acquiring Q face images from the same person, where Q is an integer greater than 1; inputting the Q face images into the feature extraction network of a face recognition model to obtain Q test feature vectors, where the objective function of the feature extraction network includes a cosine similarity loss term; obtaining the cosine similarity between the q-th test feature vector and the p-th test feature vector to obtain Q first cosine similarities, where q is a positive integer and 1≤q≤Q, p=1,2,……,Q; generating a scoring label for the image corresponding to the q-th test feature vector based on the Q first cosine similarities, and the scoring label is used to determine whether to perform face recognition on the image corresponding to the q-th test feature vector. This achieves the goal of objective and accurate quality evaluation of face images, and is compatible with the face recognition model. However, this scheme, based on a single model, shows a strong correlation between face image evaluation and recognition models, resulting in poor generalization. It also assumes that high-quality images in different face groups are all optimal face images, which differs from real samples.

[0039] The above scheme relies on manual evaluation of image quality, which requires a lot of time and effort and is subject to a certain degree of subjectivity, resulting in inaccurate evaluation samples.

[0040] To address the problems existing in related technologies, this application provides a method for evaluating image quality. This method can be implemented by a computer program, which, when executed, performs the evaluation method provided in this application. In some embodiments, the computer program can be executed on a processor in an image quality evaluation device, which can be a server or a computer. Figure 1 An implementation flow of the image quality evaluation method provided in the embodiments of this application is as follows: Figure 1 As shown, the evaluation method includes:

[0041] Step S101: Obtain multiple image recognition models, a set of images to be evaluated, and the first-dimensional quality score of each image to be evaluated in the set of images to be evaluated.

[0042] Here, the set of images to be evaluated is a set of images containing the first target object. Each image to be evaluated can be captured by a camera, webcam, mobile terminal, etc. Furthermore, each image to be evaluated can also be obtained from an open-source library. For example, this open-source library may include a face database (Labled Faces in the Wild, LFW), a large-scale face dataset (Visual Geometry Group Face 2rd, VGGFACE2), and a large-scale face attribute dataset (Celeb Faces Attributes Dataset, CelebA). The image recognition model is used to identify the first target object contained in each image to be evaluated and can also extract features from the first target object to obtain the feature vector of each image to be evaluated.

[0043] In the embodiments of this application, the first dimension quality score is used to reflect the visual quality of each image to be evaluated. Based on this, the first dimension quality score of each image to be evaluated can be obtained by analyzing the visual quality of each image to be evaluated. When the actual visual quality is better, the first dimension quality score is higher.

[0044] In practice, if the lighting and resolution of the image to be evaluated are good, then the visual perception of the image to be evaluated is considered to be good. In addition, when the first target object is facing directly forward, the visual perception of the image to be evaluated is also considered to be good.

[0045] Step S102: Use each image recognition model to perform feature extraction processing on each image to be evaluated to obtain the feature vector of each image to be evaluated.

[0046] Here, each image to be evaluated can be input into a separate image recognition model. Each image recognition model first identifies the first target object in the image to be evaluated. Then, features are extracted from the first target object to obtain the feature vector of each image to be evaluated.

[0047] In actual implementation, the primary target object is the object of interest, such as a person's face, a pet's face, or the surface of an object.

[0048] In this embodiment, the number of image recognition models can be 2, 3, 4, etc. Taking 3 image recognition models as an example, these 3 models can be a model based on the FaceNet algorithm, a model based on the InsightFace algorithm, and a model based on the SynFace algorithm. In actual implementation, each image recognition model is used sequentially for feature extraction. That is, for a specific image to be evaluated, features are extracted using the three image recognition models respectively, resulting in three feature vectors, which correspond to the image recognition model.

[0049] Step S103: Determine the second dimension quality score of each image to be evaluated based on the feature vector of each image to be evaluated.

[0050] Here, the scoring results of each image recognition model can be determined first based on the feature vectors of each image to be evaluated, and then the second dimension quality score of each image to be evaluated can be determined based on the scoring results of each image recognition model.

[0051] In this embodiment of the application, the second dimension quality score reflects the quality of the image to be evaluated from the perspective of feature vectors.

[0052] In practice, by combining the scoring results of multiple image recognition models, a second-dimensional quality score is determined, which can overcome the model bias caused by a single model.

[0053] Step S104: Based on the first dimension quality score and the second dimension quality score, determine the quality assessment result of each image to be evaluated.

[0054] Here, the quality assessment result for each image to be evaluated is determined by combining the first-dimensional quality score and the second-dimensional quality score. This quality assessment result considers not only illumination, resolution, and the orientation of the first target object, but also the machine feature information of the first target object. This avoids model bias caused by a single image recognition model and also avoids interference from the low quality of the best face in the set of images to be evaluated.

[0055] In actual implementation, the first product of the first dimension quality score and the second dimension quality score can be determined and used as the quality assessment result. Alternatively, the second product of the first product and the first preset coefficient can be determined and used as the quality assessment result.

[0056] This application provides an evaluation method comprising: first, acquiring multiple image recognition models, a set of images to be evaluated, and a first-dimensional quality score for each image in the set of images to be evaluated. This first-dimensional quality score reflects the visual quality of each image; a higher visual quality score corresponds to a higher first-dimensional quality score. Next, each image recognition model is used to perform feature extraction processing on each image to be evaluated, obtaining feature vectors for each image. A second-dimensional quality score is then determined based on the feature vectors of each image. Finally, the first-dimensional and second-dimensional quality scores are used to jointly determine the quality evaluation result for each image. In this way, the quality evaluation result for each image is jointly determined by the first-dimensional quality score reflecting the actual visual quality and the second-dimensional quality score reflecting the image features. This determination process requires no manual intervention, improving evaluation efficiency, ensuring the objectivity of the quality evaluation result, and also increasing the accuracy of the quality evaluation result.

[0057] In some embodiments, the above step S103, "determining the second-dimensional quality score of each image to be evaluated based on the feature vector of each image to be evaluated," can be implemented by the following steps S1031 and S1032:

[0058] Step S1031: Based on the feature vectors corresponding to each image recognition model, determine the scoring results of each image recognition model.

[0059] In practice, step S1031 can be achieved through the following steps S311 and S312:

[0060] Step S311: Based on the feature vectors corresponding to each image recognition model, determine the i-th similarity set between the i-th image to be evaluated and the other images to be evaluated.

[0061] Here, for a given image recognition model, the feature vectors corresponding to the image recognition model are subjected to similarity processing to obtain the similarity set of each image to be evaluated.

[0062] In this embodiment of the application, taking the i-th image to be evaluated as an example, the similarity between the i-th image to be evaluated and the remaining images to be evaluated is determined and denoted as the i-th similarity. Here, i is a positive integer, i is greater than or equal to 1, and i is less than or equal to N, where N is the total number of images to be evaluated contained in the set of images to be evaluated; the remaining images to be evaluated are the images to be evaluated other than the i-th image to be evaluated.

[0063] In practical implementation, the similarity can be determined by calculating the angle between the i-th feature vector to be evaluated and the feature vectors of the other images to be evaluated, then determining the cosine value of the angle, and finally using the cosine value of the angle as the similarity. In some embodiments, the similarity between vectors can also be determined based on Euclidean distance, Manhattan distance, Chebyshev distance, Pearson correlation coefficient, etc.

[0064] For example, with i = 1 and N = 5, the similarity between the first image to be evaluated and the second image to be evaluated will be determined, which can be denoted as: S 12 The similarity between the first and third images to be evaluated can be determined and denoted as: S 13 The similarity between the first image to be evaluated and the fourth image to be evaluated can be denoted as: S 14 The similarity between the first image to be evaluated and the fifth image to be evaluated can be denoted as... S 15 Based on this, S 12 , S 13 , S 14 , S 15 This forms the first similarity set corresponding to the image recognition model. Thus, using the same method, the i-th similarity set corresponding to each image recognition model can be determined.

[0065] Step S312: Determine the score result of the i-th image to be evaluated based on the i-th similarity set.

[0066] Here, by performing mean and normalization processing on the i-th similarity set in sequence, the scoring results can be mapped to a set interval, such as the interval [0,1], thereby obtaining the scoring result of the i-th image to be evaluated.

[0067] In practical implementation, since there are multiple image recognition models, the i-th image to be evaluated will have multiple scores, and the number of scores for the i-th image to be evaluated is equal to the number of image recognition models. That is, the score for the i-th image to be recognized corresponds not only to the i-th image to be evaluated, but also to the image recognition model.

[0068] Step S1032: Based on the scoring results of each image recognition model, determine the second dimension quality score of each image to be evaluated.

[0069] Here, for any image to be evaluated, the score of that image corresponds to the image recognition model.

[0070] Based on this, in the embodiments of this application, the scoring results of each image recognition model can be weighted and averaged to obtain a weighted average result, which can be determined as the second dimension quality score of the image to be evaluated.

[0071] In some embodiments, the weighted average result can be multiplied by a second preset coefficient to obtain a third product, and this third product can be determined as the second-dimensional quality score of the image to be evaluated. Thus, the second-dimensional quality score of each image to be evaluated can be determined using the same method.

[0072] In this embodiment, through steps S1031 and S1032, the similarity set corresponding to each image recognition model is determined based on the feature vectors corresponding to each image recognition model, thereby determining the scoring result of each image recognition model. Then, based on the scoring results of each image recognition model, the second-dimensional quality score of each image to be evaluated is determined. By using multiple image recognition models, model bias caused by a single image recognition model can be avoided, thereby improving the accuracy of the second-dimensional quality score.

[0073] Based on the above embodiments, such as Figure 2 As shown, step S312, "determining the scoring result of the i-th image to be evaluated based on the i-th similarity set," can be achieved through the following steps S3121 to S3123:

[0074] Step S3121: Determine the mean similarity of the i-th similarity based on the i-th similarity set.

[0075] Here, a weighted average can be applied to the i-th similarity set to obtain the i-th mean similarity.

[0076] Step S3122: Obtain the mean similarity of each image to be evaluated.

[0077] Here, the above weighted average processing can be performed on the similarity set of each image to be evaluated to obtain the mean similarity of each image to be evaluated.

[0078] Step S3123: Normalize the mean similarity of each image to be evaluated and the mean similarity of the i-th image to obtain the score result of the i-th image to be evaluated.

[0079] Here, the mean similarity of each image to be evaluated can be sorted in order of magnitude to obtain sorted mean similarity scores. The sorting can be done in descending order or ascending order. Next, the maximum and minimum mean similarity scores are determined from the sorted mean similarity scores, and the similarity difference between the maximum and minimum similarity scores is also determined. Finally, the quotient of the mean similarity score of the i-th image and the similarity difference score is determined as the score result of the i-th image to be evaluated.

[0080] In some embodiments, the scoring process for the remaining images to be evaluated may refer to the scoring process for the i-th image to be evaluated described above.

[0081] In this embodiment, through steps S3121 to S3123 described above, the scoring result of the i-th image to be evaluated can be obtained through weighted averaging and normalization. This simplifies the scoring process and improves scoring efficiency.

[0082] In some embodiments, each image to be evaluated includes a first target object. Before performing the above step S101 of "obtaining the first dimension quality score of each image to be evaluated in the set of images to be evaluated", the first dimension quality score of each image to be evaluated will be determined first. Based on this, before performing the above step S101, the first dimension quality score of each image to be evaluated can be determined through the following steps S001 and S002.

[0083] Step S001: Collect image characteristic information of each image to be evaluated and behavioral features corresponding to the first target object.

[0084] Here, image characteristic information includes illumination intensity, image resolution, and image blur; behavioral characteristics include facial features and pose features.

[0085] In this embodiment of the application, when collecting each image to be evaluated, the image characteristic information of each image to be evaluated and the behavioral characteristics corresponding to the first target object can be collected simultaneously.

[0086] In practical implementation, illumination intensity, image resolution, and image blur can be represented by a single value or a range of values. For example, illumination intensity can be 200 cres.

[0087] Step S002: Analyze and process the image characteristic information and behavioral features of each image to be evaluated to obtain the first dimension quality score of each image to be evaluated.

[0088] Here, the image characteristics information such as illumination intensity, image resolution, and image blur can be associated with reference ranges, and the behavioral characteristics such as facial features and pose features can also be associated with reference features.

[0089] In this embodiment of the application, the scores of each image feature information and each behavioral feature are first determined, and the corresponding weight values ​​of each image feature information and each behavioral feature are also obtained; then, a weighted sum is performed based on the scores and the corresponding weight values, and the weighted sum is determined as the first dimension quality score.

[0090] In practical implementation, a higher score is awarded if the acquired image feature information falls within the corresponding reference range, and vice versa. Similarly, a higher score is awarded for acquired behavioral features with high similarity to reference behavioral features, and a lower score is awarded for acquired behavioral features with low similarity to reference behavioral features. Furthermore, weight values ​​are between 0 and 1, and the sum of all weight values ​​is 1. Therefore, all weight values ​​can be set to be equal. Alternatively, weight values ​​can be assigned based on the specific circumstances. Generally, higher weight values ​​can be assigned to pose features and image blur characteristics, while lower weight values ​​can be assigned to other features.

[0091] For example, taking image characteristic information including illumination intensity, image resolution, and image blur, and behavioral features including pose features as examples, the weight values ​​corresponding to illumination intensity, image resolution, image blur, and pose features can be equal, with each weight value being 0.25; the weight values ​​corresponding to illumination intensity, image resolution, image blur, and pose features can also be unequal, with the weight values ​​corresponding to image blur and pose features both being 0.3, and the weight values ​​corresponding to illumination intensity and image resolution both being 0.2.

[0092] In this embodiment of the application, by analyzing and processing the collected image characteristic information and behavioral features, the first dimension quality score of each image to be evaluated can be obtained, thereby avoiding interference caused by the low quality of the best image to be evaluated in the set of images to be evaluated and improving the objectivity of the evaluation.

[0093] Based on the above embodiments, the quality assessment results of each image to be evaluated can be determined through steps S101 to S104. In practical applications, on the one hand, the image quality assessment model is trained using the images to be evaluated and their quality assessment results to obtain a trained image quality scoring model. This trained image quality scoring model has advantages such as strong generalization and high accuracy, so that it can be used to accurately score the images to be evaluated in the future. On the other hand, the quality assessment results of each image to be evaluated can be directly used to select the image with better quality from the images to be evaluated, so as to use the image with better quality to be evaluated for subsequent recognition of the first target object. This can improve recognition efficiency and increase recognition accuracy.

[0094] The image quality assessment model is trained using the image to be evaluated and its quality assessment results, resulting in a well-trained image quality scoring model, such as... Figure 3 As shown, the evaluation method further includes the following steps S105 to S112:

[0095] Step S105: Obtain the image quality scoring model.

[0096] Here, the image quality scoring model is used to score the quality of the input image. The image quality scoring model can be a lightweight convolutional neural network, residual neural network, etc.

[0097] Step S106: Each image to be evaluated is determined as a training image, and the quality evaluation result of each image to be evaluated is determined as the label information of the training image.

[0098] Here, the number of training images is the same as the number of images to be evaluated, that is, there are multiple training images, and each training image includes the first target object.

[0099] Step S107: Use the image quality scoring model to perform prediction processing on the training images to obtain the predicted quality score of the training images.

[0100] Here, training images are sequentially input into the image quality scoring model so that the image quality scoring model can predict the quality score of the input training images and obtain the predicted quality score of the training images.

[0101] Step S108: Based on the label information and predicted quality score of the training images, train the image quality scoring model to obtain the trained image quality scoring model.

[0102] Here, we first determine the difference between the label information and the predicted quality score. If the difference is less than the difference threshold, it indicates that the image quality scoring model can predict the image quality score relatively accurately, and no further training is needed; this image quality scoring model is now considered well-trained. However, if the difference is greater than or equal to the difference threshold, it indicates that the image quality scoring model cannot accurately predict the image quality score and requires training. This is done by backpropagating the image quality scoring model using the difference information, adjusting the parameters in the model until the difference is less than the difference threshold, thus obtaining a well-trained image quality scoring model.

[0103] Step S109: Obtain the set of images to be scored.

[0104] Here, a set of images to be scored can be obtained by using cameras, webcams, mobile terminals, building equipment, etc. Each image in the set of images to be scored contains a second target object.

[0105] Step S110: The trained image quality scoring model is used to score each image in the set of images to be scored, and the target quality score of each image is obtained.

[0106] Here, each image to be scored in the set of images to be scored can be input into the trained image quality scoring model. That is, the trained image quality scoring model scores each image to be scored in turn to obtain the target quality score of each image to be scored.

[0107] Step S111: Based on the target quality score of each image to be scored, determine the target image with the highest score from among the images to be scored.

[0108] Here, the target quality scores can be sorted in ascending order, and the highest-scoring target quality score can be selected from the sorted scores. Alternatively, the highest-scoring target quality score can be obtained by pairwise comparison. Finally, the image corresponding to the highest-scoring target quality score is determined as the target image.

[0109] Step S112: Input the target image into the recognition system so that the recognition system can recognize the second target object based on the target image.

[0110] Here, the recognition system is used to identify a second target object in a target image. This recognition system can be deployed on an image quality assessment device, or it can be deployed on a terminal device other than the image quality assessment device.

[0111] In this embodiment, since the target image has a high quality score, it can be characterized by high pixel count, moderate lighting, the second target object's face facing forward, and obvious machine features of the second target object. Thus, the recognition system can accurately identify the second target object in the target image, improving recognition speed and accuracy.

[0112] In this embodiment, through steps S105 to S112, each image to be evaluated can be used as a training image, and the quality evaluation results of each image to be evaluated can be used as the label information of the training image. Then, the image quality scoring model is trained using the training images and their label information to obtain a trained quality scoring model. This trained image quality scoring model is then used to score images, thereby improving the accuracy of the quality scoring model. Based on this, in practical implementation, the trained image quality scoring model can be used to score each image to be scored, obtaining the target quality score for each image. Then, the target image with the highest score is selected. Finally, the target image is input into the recognition system to accurately identify the second target object in the target image, thereby improving the recognition efficiency.

[0113] Based on the above embodiments, this application further provides an image quality evaluation method, which can be a parameter-free face image quality evaluation method combining human visual perception and machine recognition accuracy. This method can be used in scenarios such as video capture and face security. The method includes:

[0114] Since data collection requires the use of facial recognition account (Identity Document, ID) information, we consider collecting open-source facial recognition datasets as training datasets for the facial quality assessment model.

[0115] Automated image data annotation, firstly, to avoid model bias caused by a single face recognition model, this application's embodiments consider using multiple face recognition models to extract facial feature vectors before calculating the similarity between images. Secondly, to avoid interference from low-quality best faces in the samples, referencing the ISO / IEC 29794-1:2016 standard, automated scoring of face images is performed using facial perception-based metrics (such as blurriness, facial pose, and pixel resolution). Finally, the final face image quality label is obtained through weighted score calculation.

[0116] The trained model is deployed to servers, smart terminals and other devices to assist in downstream tasks such as facial recognition.

[0117] In some embodiments, this application proposes a parameter-free face image quality assessment method that combines human visual perception with machine recognition accuracy, such as during the model training phase. Figure 4 As shown, it includes the following steps:

[0118] Step S401, Begin.

[0119] Step S402: Obtain training data.

[0120] Here, since face ID information is required, we consider collecting open-source face recognition datasets as training datasets for the face quality assessment model. In order to obtain training datasets with large differences in face image quality, we prefer to use a combination of the LFW, VGGFACE2, and CelebA datasets to form the training dataset.

[0121] Step S403: Automated labeling of training data.

[0122] In actual implementation, step S403 can be achieved through the following steps S4031 to S4033:

[0123] Step S4031: Calculate the quality score based on machine recognition accuracy.

[0124] To avoid model bias caused by a single face recognition model, this embodiment considers using multiple face recognition models to extract face feature vectors and then calculating the similarity between face images. FaceNet, InsightFace, and SynFace are preferred algorithms, and a model trained on a Res50 backbone network is used for feature extraction. The face recognition model corresponds to the image recognition model in the above embodiment.

[0125] In practical implementation, firstly, a face recognition model is used to extract feature vectors from face images. Secondly, the cosine distance between pairs of samples under the same face ID is calculated. These distances represent the similarity between each image and the other face images. Here, it is assumed that the higher the quality of two images under the same ID, the more similar the feature vectors extracted by the face recognition model, and vice versa. After calculation, the cosine similarity matrix under the same ID can be obtained, as shown in formula (1):

[0126] Formula (1);

[0127] in, Let N represent the cosine similarity between the i-th face image and the j-th face image under the same ID, and let N represent the number of face images under the same ID. In order to map the quality score to [0,1], the mean similarity between the i-th image and the remaining images can be determined first by the following formula (2).

[0128] Formula (2);

[0129] in, m i This represents the average similarity between the i-th image and the remaining images under the current ID.

[0130] Next, the similarity sequence among all images under the current ID can be represented by formula (3):

[0131] Formula (3);

[0132] in, M This represents the similarity sequence among all images under the current ID.

[0133] Finally, the quality score based on machine recognition accuracy is calculated using formula (4).

[0134] Formula (4);

[0135] in, mNorm i This represents the normalized similarity between the i-th image under the current ID and the other images.

[0136] In this embodiment, after obtaining the normalized similarity of the feature vectors extracted by the three models, the average of the normalized similarity is calculated to obtain a quality score based on machine recognition accuracy. This quality score based on machine recognition accuracy corresponds to the second dimension quality score in the above embodiment.

[0137] Step S4032: Calculate the quality score based on human visual perception.

[0138] In this embodiment, to avoid interference caused by the low quality of the best face in the sample, the face image is automatically scored using a scoring index based on facial perception (such as blurriness, face pose, pixel resolution, and lighting conditions) in accordance with the ISO / IEC 29794-1:2016 standard. The score for each index is between [0, 100]. After calculating the score for each index, the visual perception quality score is calculated using the following formula (5). The visual perception quality score corresponds to the first dimension quality score in the above embodiment.

[0139] Formula (5);

[0140] in, h i This represents the visual quality score of the i-th image. blur , pose , pixel , illu Let represent the scores for blur, face pose, pixel resolution, and lighting conditions of the i-th image, respectively. α 1, α 2, α 3, α4 represents the weights of the scores for blurriness, face pose, pixel resolution, and lighting conditions, respectively.

[0141] In practice, after scoring according to the above rules, images can be divided into two categories: low-quality face images and high-quality face images.

[0142] Step S4033: Obtain the face image quality rating label.

[0143] After obtaining the quality score based on machine recognition accuracy in S4031 and the quality score based on human visual perception in S4032, the two can be multiplied by formula (6) to obtain the final face image quality label. The final face image quality label corresponds to the quality evaluation result in the above embodiment.

[0144] Formula (6);

[0145] in, f i This represents the final face image quality label for the i-th image.

[0146] Step S404: Training the face quality assessment network.

[0147] Here, after obtaining the training image data from step S402 and the final face image quality labels from step S403, the face image quality scoring model is trained. The model structure is as follows: Figure 5 As shown, the backbone network can be selected from MobileNet, ResNet50, etc., for reference. Figure 5 The network includes convolutional layers 501, 502, 503, 504, and 505, and fully connected layers 506 and 507. To avoid overfitting, dropout layers or L2 regularization are added to the fully connected modules. Since the quality scoring task is a regression task, this embodiment uses mean squared error as the loss function for model training, calculated as shown in Equation 7.

[0148] Formula (7);

[0149] in, y A quality label representing a face image. The model prediction result is represented by N, which is the batch size during training. The gradient with respect to the existing neural network parameters is calculated based on the loss function described above. The parameters are then updated using stochastic gradient descent with momentum until the network converges, yielding the weights of the face quality assessment model. Here, the face quality assessment model corresponds to the image quality scoring model in the above embodiment.

[0150] Step S405, End.

[0151] The evaluation method provided in this application has the following advantages: First, it uses a parameter-free face quality assessment model. Compared to traditional face quality assessment algorithms based on manual annotation, this eliminates the need for extensive time and effort in annotation, avoiding the problem of decreased model accuracy due to inaccurate manually labeled samples. Second, it uses multiple face recognition models for quality assessment based on machine recognition accuracy. Compared to traditional algorithms based on a single face recognition model, this avoids model bias caused by a single model and enhances the generalization ability of the face quality assessment model. Third, it uses a parameter-free face image quality assessment method that combines human visual perception with machine recognition accuracy. This avoids the problem of inconsistent quality scores for high-quality images in different face groups, and optimizes model recognition accuracy by comprehensively considering both recognition accuracy and human visual perception in scoring images.

[0152] Based on the foregoing embodiments, this application provides an image quality evaluation device. The various modules and units included in the device can be implemented by a processor in a computer device; of course, they can also be implemented by corresponding logic circuits. In the implementation process, the processor can be a central processing unit (CPU), a microprocessor unit (MPU), a digital signal processor (DSP), or a field programmable gate array (FPGA), etc.

[0153] This application embodiment further provides an image quality evaluation device, Figure 6 This is a schematic diagram of the composition of the evaluation device provided in the embodiments of this application, such as... Figure 6 As shown, the evaluation device 600 includes:

[0154] The first acquisition module 601 is used to acquire multiple image recognition models, a set of images to be evaluated, and a first-dimensional quality score of each image to be evaluated in the set of images to be evaluated, wherein the first-dimensional quality score is used to reflect the visual quality of each image to be evaluated.

[0155] The feature extraction module 602 is used to perform feature extraction processing on each image to be evaluated using each image recognition model to obtain the feature vector of each image to be evaluated.

[0156] The first determining module 603 is used to determine the second dimension quality score of each image to be evaluated based on the feature vector of each image to be evaluated.

[0157] The second determining module 604 is used to determine the quality assessment result of each image to be evaluated based on the first dimension quality score and the second dimension quality score.

[0158] In some embodiments, the first determining module includes:

[0159] The first determining submodule is used to determine the scoring result of each image recognition model based on the feature vectors corresponding to each image recognition model.

[0160] The second determining submodule is used to determine the second dimension quality score of each image to be evaluated based on the scoring results of each image recognition model.

[0161] In some embodiments, the first determining submodule includes:

[0162] The first determining unit is used to determine the i-th similarity set between the i-th image to be evaluated and the remaining images to be evaluated based on the feature vectors corresponding to each image recognition model; wherein i is a positive integer, i is greater than or equal to 1, and i is less than or equal to N, and N is the total number of images to be evaluated contained in the set of images to be evaluated; the remaining images to be evaluated are images to be evaluated other than the i-th image to be evaluated.

[0163] The second determining unit is used to determine the scoring result of the i-th image to be evaluated based on the i-th similarity set.

[0164] In some embodiments, the second determining unit includes:

[0165] The first determining subunit is used to determine the i-th mean similarity based on the i-th similarity set;

[0166] A sub-unit is used to obtain the mean similarity of each image to be evaluated;

[0167] The normalization subunit is used to normalize the mean similarity of each image to be evaluated and the i-th mean similarity to obtain the score result of the i-th image to be evaluated.

[0168] In some embodiments, the evaluation apparatus 600 further includes:

[0169] The acquisition module is used to acquire image characteristic information of each image to be evaluated and behavioral features corresponding to the first target object. The image characteristic information includes illumination intensity, image resolution, and image blur. The behavioral features include facial features and pose features.

[0170] The analysis module is used to analyze and process the image characteristic information and behavioral features of each image to be evaluated, and obtain the first dimension quality score of each image to be evaluated.

[0171] In some embodiments, the evaluation apparatus 600 further includes:

[0172] The second acquisition module is used to acquire the image quality scoring model;

[0173] The third determining module is used to determine each image to be evaluated as a training image and to determine the quality evaluation result of each image to be evaluated as the label information of the training image.

[0174] The prediction module is used to perform prediction processing on the training image using the image quality scoring model to obtain the predicted quality score of the training image.

[0175] The training module is used to train the image quality scoring model based on the label information of the training images and the predicted quality scores of the training images, so as to obtain a trained image quality scoring model.

[0176] In some embodiments, the evaluation apparatus 600 further includes:

[0177] The third acquisition module is used to acquire a set of images to be scored, wherein each image in the set of images to be scored contains a second target object;

[0178] The scoring module is used to score each image in the set of images to be scored using the trained image quality scoring model, so as to obtain the target quality score of each image to be scored.

[0179] The fourth determining module is used to determine the target image with the highest score from the images to be scored based on the target quality scores of each image to be scored;

[0180] An input module is used to input the target image into the recognition system so that the recognition system can recognize the second target object based on the target image.

[0181] It should be noted that the description of the evaluation apparatus in this application is similar to the description of the method embodiments described above, and has similar beneficial effects. For technical details not disclosed in this apparatus embodiment, please refer to the description of the method embodiments in this application for understanding.

[0182] It should be noted that, in the embodiments of this application, if the above-described evaluation method is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to the related technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0183] Accordingly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the evaluation method provided in the above embodiments.

[0184] This application provides an image quality evaluation device. Figure 7 This is a schematic diagram of the composition structure of the evaluation device provided in the embodiments of this application, such as... Figure 7 As shown, the evaluation device 700 includes: a processor 701, at least one communication bus 702, a user interface 703, at least one external communication interface 704, and a memory 705. The communication bus 702 is configured to enable communication between these components. The user interface 703 may include a display screen, and the external communication interface 704 may include standard wired and wireless interfaces. The processor 701 is configured to execute a program of an evaluation method stored in the memory to implement the evaluation method provided in the above embodiment.

[0185] The descriptions of the above-described evaluation device and storage medium embodiments are similar to those of the above-described method embodiments, and have similar beneficial effects. For technical details not disclosed in the evaluation device and storage medium embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0186] It should be noted that the descriptions of the storage medium and evaluation device embodiments above are similar to the descriptions of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium and evaluation device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0187] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0188] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0189] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0190] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of the embodiments of this application, depending on actual needs.

[0191] In addition, each functional unit in the various embodiments of this application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0192] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0193] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an AC to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0194] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An evaluation method, characterized in that, The method includes: A plurality of image recognition models, a set of images to be evaluated, and a first-dimensional quality score for each image to be evaluated in the set of images to be evaluated are obtained, wherein the first-dimensional quality score is used to reflect the visual quality of each image to be evaluated. Each image recognition model is used to perform feature extraction processing on each image to be evaluated, and feature vectors of each image to be evaluated are obtained. Based on the feature vectors corresponding to each image recognition model, a similarity set between different images to be evaluated under the same model is determined. For each image to be evaluated, a mean similarity is determined based on the similarity set corresponding to the image to be evaluated. The mean similarities are normalized to obtain the scoring result of the evaluation image under the corresponding model. Based on the scoring results of each image recognition model, a second-dimensional quality score is determined for each image to be evaluated. Based on the first dimension quality score and the second dimension quality score, the quality assessment result of each image to be evaluated is determined.

2. The method according to claim 1, characterized in that, The step of determining the similarity set between different images to be evaluated under the same model based on the feature vectors corresponding to each image recognition model includes: Based on the feature vectors corresponding to each image recognition model, the i-th similarity set between the i-th image to be evaluated and the remaining images to be evaluated is determined; wherein, i is a positive integer, i is greater than or equal to 1, and i is less than or equal to N, and N is the total number of images to be evaluated contained in the set of images to be evaluated; the remaining images to be evaluated are the images to be evaluated other than the i-th image to be evaluated.

3. The method according to claim 1, characterized in that, Each image to be evaluated includes a first target object; the method further includes: The image characteristic information of each image to be evaluated and the behavioral features corresponding to the first target object are collected. The image characteristic information includes illumination intensity, image resolution, and image blur. The behavioral features include facial features and pose features. The image characteristic information and behavioral features of each image to be evaluated are analyzed and processed to obtain the first dimension quality score of each image to be evaluated.

4. The method according to any one of claims 1 to 3, characterized in that, The method further includes: Obtain the image quality scoring model; Each image to be evaluated is determined as a training image, and the quality evaluation result of each image to be evaluated is determined as the label information of the training image; The training images are predicted using the image quality scoring model to obtain a predicted quality score for the training images. Based on the label information of the training images and the predicted quality scores of the training images, the image quality scoring model is trained to obtain a trained image quality scoring model.

5. The method according to claim 4, characterized in that, The method further includes: Obtain a set of images to be scored, wherein each image in the set of images to be scored contains a second target object; The trained image quality scoring model is used to score each image in the set of images to be scored, thereby obtaining the target quality score of each image to be scored. Based on the target quality scores of each image to be scored, the target image with the highest score is determined from the images to be scored. The target image is input into the recognition system so that the recognition system can identify the second target object based on the target image.

6. An evaluation device, characterized in that, The evaluation device includes: The first acquisition module is used to acquire multiple image recognition models, a set of images to be evaluated, and a first-dimensional quality score for each image to be evaluated in the set of images to be evaluated, wherein the first-dimensional quality score is used to reflect the visual quality of each image to be evaluated. The feature extraction module is used to perform feature extraction processing on each image to be evaluated using various image recognition models to obtain the feature vector of each image to be evaluated. The first determining module is used to determine the similarity set between different images to be evaluated under the same model based on the feature vectors corresponding to each image recognition model, and for each image to be evaluated, determine the mean similarity based on the similarity set corresponding to the image to be evaluated, normalize each mean similarity to obtain the score result of the evaluation image under the corresponding model; and determine the second dimension quality score of each image to be evaluated based on the score results of each image recognition model. The second determining module is used to determine the quality assessment result of each image to be evaluated based on the first dimension quality score and the second dimension quality score.

7. An evaluation device, characterized in that, The evaluation equipment includes: Processor; and Memory for storing computer programs that can run on the processor; The computer program, when executed by a processor, implements the evaluation method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions configured to perform the evaluation method according to any one of claims 1 to 5.

Images