Image scoring method and electronic device

Abstract

The present invention provides an image scoring method and electronic device, the method comprising: obtaining an image to be scored, inputting the image to be scored into a preset image quality detection model, outputting quality data of a face image carried by the image to be scored through the preset image quality detection model, wherein the quality data corresponds to a characteristic attribute of the image to be scored; and determining a target score of the image to be scored according to the quality data of the image to be scored. The present invention outputs the quality data of the face image carried by the scored image through a preset image quality detection model, and then determines the target score of the image to be scored according to the quality data, so as to select the image according to the target score, so that the target score of the image to be scored can be automatically given, thereby improving the efficiency of image scoring and also improving the efficiency of image selection.

Description

Technical Field

[0001] The present invention relates to the technical field of picture processing, and in particular to an image scoring method and electronic equipment. Background Art

[0002] As the performance of cameras equipped in smart devices becomes higher and higher, more and more people use smart devices (such as smart phones, tablet computers or smart glasses, etc.) to take photos, and the same scene may be taken multiple times in the photos taken, so that the album contains multiple photos of repeated scenes, among which the most repeated photos are photos of people. However, multiple repeated photos will waste the storage space of smart terminal devices.

[0003] In order to solve the problem of wasting storage space by duplicate photos, it is currently common to use human eyes to identify whether the target object in the photo meets the requirements to determine a satisfactory photo. However, this method of identifying the target object by human eyes requires users to spend a lot of time to select photos, resulting in a waste of user time and inconvenience to users. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, the present invention aims to provide an image scoring method and an electronic device to solve the problem in the prior art that manual identification of photo defects is required.

[0005] In order to solve the above problems, the technical solution adopted in the present invention is:

[0006] An image scoring method, comprising:

[0007] Acquire an image to be rated, wherein the image to be rated carries a face image;

[0008] Inputting the image to be rated into a preset image quality detection model, and outputting quality data of the face image carried by the image to be rated through the preset image quality detection model, wherein the quality data is used to reflect the quality of the characteristic attributes of the image to be rated;

[0009] A target score for the image to be rated is determined according to the quality data of the image to be rated.

[0010] The image scoring method, wherein the face image includes a plurality of face images, the inputting of the image to be scored into a preset image quality detection model, and the outputting of the quality data of the face image carried by the image to be scored through the preset image quality detection model are specifically:

[0011] The image to be rated is input into a preset image quality detection model, and the quality data corresponding to each face image in the image to be rated is determined by the image quality detection model.

[0012] The image scoring method, wherein determining the score of the image to be scored according to the quality data of the image to be scored specifically comprises:

[0013] For each face image in the to-be-rated images, a weight coefficient corresponding to the face image is obtained, and a candidate score for the face image is determined according to the weight coefficient and quality data of the face image;

[0014] The target score corresponding to the image to be rated is determined according to the candidate score of each face image.

[0015] The image scoring method, wherein the weight coefficient is determined based on the score corresponding to the area occupied by the face image, the score corresponding to the position of the face, and the score corresponding to the orientation of the face.

[0016] The image scoring method, wherein the feature attributes include a smile feature, an eye closure feature, a facial occlusion feature, and a facial blur feature.

[0017] The image scoring method, wherein the training process of the image quality detection model specifically includes:

[0018] The preset network model generates predicted quality data corresponding to the training image according to the training image in the training sample set; wherein the training sample set includes a plurality of training sample groups, each group of training samples includes a training image and real quality data and real face key points corresponding to the training image, the real quality data includes a feature score group, and each feature score in the feature score group is a quality score corresponding to a feature attribute of the training image;

[0019] The preset network model modifies the model parameters according to the predicted quality data and the actual quality data corresponding to the training image, and continues to execute the step of generating the detection data corresponding to the training image according to the next set of training images in the training sample set until the training status of the preset network model meets the preset conditions to obtain the image quality detection model.

[0020] The image scoring method, wherein the preset network model includes: a feature extraction module and an upper network module; the preset network model generates prediction quality data corresponding to the training image according to the training image in the training sample set, including:

[0021] Inputting the training image in the training sample set into the feature extraction module, and obtaining the feature image corresponding to the training image through the feature extraction module;

[0022] The feature image is input into the upper network module, and the predicted quality data and predicted facial key points corresponding to the training image are output through the upper network module.

[0023] The image scoring method, wherein the upper network module includes a plurality of quality detection units and key point detection units arranged in parallel, and the feature attributes corresponding to the quality detection units are different from each other; the inputting of the feature image into the upper network module, and the outputting of the predicted quality data corresponding to the training image and the predicted face key points through the upper network module specifically include:

[0024] Inputting the feature image into each quality detection unit and the key point detection unit respectively;

[0025] Each detection unit generates a feature score corresponding to each detection unit, and the key point detection unit outputs the predicted face key points to obtain the predicted quality data corresponding to the training image.

[0026] The image scoring method, wherein the step of the preset network model correcting the model parameters according to the predicted quality data and the quality data corresponding to the training image comprises:

[0027] For each quality detection unit, a first loss value corresponding to the quality detection unit is calculated according to a predicted quality score corresponding to the quality detection unit and a true quality score corresponding to the quality detection unit, and a model parameter corresponding to the quality detection unit is corrected according to the first loss value;

[0028] Calculating a second loss value according to the predicted face key points and the real face key points, and correcting the model parameters corresponding to the key point detection unit according to the second loss value;

[0029] The module parameters corresponding to the feature extraction module are modified according to each first loss value and the second loss value.

[0030] The image scoring method, wherein the method further comprises:

[0031] Detecting model parameters of the key point detection unit;

[0032] When the model parameters of the key point detection unit meet the preset conditions, aligning the training samples in the training sample set to obtain an aligned training sample set, wherein the orientations of the face images in the training images in the aligned training samples are the same;

[0033] The preset network model is trained using the aligned training sample set.

[0034] The image scoring method, wherein when the number of images to be scored is at least two, the method further comprises:

[0035] Obtain the target score corresponding to each image to be rated;

[0036] Determine the recommended images corresponding to the at least two images to be rated according to the acquired target scores.

[0037] In the image scoring method, the at least two images to be scored are multiple images obtained by continuous shooting with an imager.

[0038] An electronic device includes a memory and a processor, wherein the memory stores a computer-readable program that can be executed by the processor, and the computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in any of the image scoring methods described above, or to implement the steps in any of the image recommendation methods described above.

[0039] A computer-readable storage medium storing one or more programs, wherein the one or more programs can be executed by one or more processors to implement the steps in any of the above-mentioned image scoring methods, or to implement the steps in any of the above-mentioned image recommendation methods.

[0040] Compared with the prior art, the present invention provides an image scoring method and electronic device, the method comprising: obtaining an image to be scored, inputting the image to be scored into a preset image quality detection model, outputting quality data of a face image carried by the image to be scored through the preset image quality detection model, wherein the quality data corresponds to a characteristic attribute of the image to be scored; and determining a target score of the image to be scored according to the quality data of the image to be scored. The present invention outputs the quality data of the face image carried by the scored image through a preset image quality detection model, and then determines the target score of the image to be scored according to the quality data, so as to select the image according to the target score, so that the target score of the image to be scored can be automatically given, thereby improving the efficiency of image scoring and also improving the efficiency of image selection. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0042] Figure 1is a flowchart of the steps of the image scoring method provided by the present invention;

[0043] Figure 2 It is a schematic diagram of the principle of the preset network model in the image scoring method provided by the present invention;

[0044] Figure 3 It is a flow chart of the training process of the image quality detection model in the image scoring method provided by the present invention;

[0045] Figure 4 A structural principle diagram of a terminal device provided by the present invention. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0047] Since the photo selection or recommendation methods in the prior art are all based on manual implementation by the user, and the method of manually selecting photos by human eye recognition is not only inefficient, but also takes a lot of time for the user. Moreover, with the popularization of portable device cameras such as mobile phone cameras and tablet cameras, a large number of duplicate or non-compliant photos stored in the devices are accumulating, which will cause certain troubles to the users. Based on the problem of poor intelligence of the device camera in automatically organizing photos, this embodiment provides a method that can automatically identify the attribute characteristics of the target object, and then select or recommend photos in the photo collection according to the identified attribute characteristics of the target object.

[0048] Various non-limiting embodiments of the present invention are described in detail below in conjunction with the accompanying drawings.

[0049] This embodiment provides an image scoring method, which can be performed by a scoring device, which can be implemented by software and applied to smart terminals such as smart phones, PCs, servers, tablet computers, or personal digital assistants. Figure 1 The image scoring method provided in this embodiment specifically includes:

[0050] S10: Obtain an image to be rated.

[0051] Specifically, the image to be rated is an image carrying a face, that is, the target object in the image to be rated includes a person. Among them, the acquisition method of the image to be rated may include: shooting through a terminal device (for example, a smart phone and a tablet computer) equipped with an imager, sending through an external device, and downloading through a network (for example, Baidu, etc.), and the image format of the image to be rated may include RGB image, Bayer image and raw image, etc. In addition, since the image to be rated needs to carry a face image, when the image to be rated is obtained, the image to be rated can be screened to determine whether the image to be rated carries a portrait. Therefore, in a possible implementation of this embodiment, the acquisition of the image to be rated may include: acquiring a first image to be rated, and determining whether the first image to be rated carries a face image, when the first image to be rated carries a face image, the first image to be rated is used as the image to be rated; when the first image to be rated does not carry a face image, it is prompted to re-acquire the image to be rated. At the same time, when prompting to re-acquire the image to be rated, the first image to be rated can be discarded, or the first image to be rated can be stored in a cache, etc.

[0052] S20, inputting the image to be rated into a preset image quality detection model, and outputting quality data of the face image carried by the image to be rated through the preset image quality detection model, wherein the quality data is used to reflect the quality of the characteristic attributes of the image to be rated.

[0053] Specifically, the quality data is the image quality evaluation data output by the image quality detection model, and the quality data corresponds to the characteristic attribute of the image to be rated, and is used to reflect the quality of the characteristic attribute of the image to be rated. In this embodiment, the quality data includes the score of at least one characteristic attribute of the image to be rated, wherein the characteristic attribute may include a smile feature, an eye closure feature, a facial occlusion feature, and a facial blur feature. In addition, the quality data may include one score or multiple scores. It is understandable that when the quality data includes one score, the image to be rated contains one characteristic attribute; when the quality data includes multiple scores, the image to be rated contains multiple characteristic attributes. At the same time, when the quality data includes multiple scores, the multiple scores correspond one-to-one to the multiple characteristic attributes contained in the image to be rated, that is, each characteristic attribute of the image to be rated corresponds to one score.

[0054] Furthermore, the image quality detection model is pre-trained, and its input data is the image to be rated, and its output data is the quality data corresponding to the image to be rated. Figure 3 As shown, in one implementation of this embodiment, the training process of the image quality detection model specifically includes:

[0055] H10, the preset network model generates prediction quality data corresponding to the training image and predicts facial key points according to the training image in the training sample set;

[0056] H20. The preset network model modifies the model parameters according to the predicted quality data, the predicted facial key points, the real quality data corresponding to the training image and the real facial key points corresponding to the training image, and continues to execute the step of generating the detection data corresponding to the training image according to the next set of training images in the training sample set, until the training status of the preset network model meets the preset conditions to obtain the image quality detection model.

[0057] Specifically, in step H10, the preset training sample set includes several training sample groups; each group of training samples includes training images, real quality data and real face key points, wherein the real quality data includes several feature scores, each of which is a quality score corresponding to a feature attribute of the training image. The predicted quality data includes several predicted feature scores, each of which is a predicted quality score corresponding to a feature attribute of the training image, and for each feature attribute of the training image, the quality score corresponding to the feature attribute corresponds to the predicted quality score corresponding to the feature attribute, and each predicted quality score is evaluated by the quality score. In addition, the training images in each group of training samples carry face images, and the training images only carry one face image, which reduces the time for distinguishing multiple faces in multiple face images, thereby improving the training speed of the preset network model, and also avoiding the confusion of multiple groups of quality scores, thereby improving the accuracy of the preset network model training.

[0058] Furthermore, the real face key points are the face key points of the face image in the pre-labeled training image, and the predicted face key points are the face key points of the face image in the training image output by the preset network model, wherein the number of face key points in the real face key points is equal to the number of face key points in the predicted face key points, and the face key points in the real face key points correspond to the face key points in the predicted face key points one by one. In addition, in a possible implementation of this embodiment, the real face key points and the predicted face key points may both include the position information of the facial key points, or both include the position information of 68 facial feature points, etc.

[0059] Further, in an implementation of this embodiment, the preset network model includes: a feature extraction module and an upper network module; the preset network model generates prediction quality data corresponding to the training image and predicts facial key points according to the training image in the training sample set, including:

[0060] L10, inputting the training image in the training sample set into the feature extraction module, and obtaining a feature image corresponding to the training image through the feature extraction module;

[0061] L20. Input the feature image into the upper network module, and output the predicted quality data and predicted facial key points corresponding to the training image through the upper network module.

[0062] Specifically, the feature extraction module is used to extract the feature image of the training image, and the feature image includes the basic features of the face image, wherein the basic features may include edges, colors, shapes, etc. It can be understood that the feature extraction module belongs to a shallow network module, and the shallow network model is used to extract the basic features of the face image. The upper network module is cascaded with the feature extraction module, and the upper network module is located behind the feature extraction module, and uses the output item of the feature extraction module as its own input item, that is, the input item of the upper network module is the output item of the feature extraction module.

[0063] Further, in one implementation of this embodiment, Figure 2 As shown, the upper network module includes a plurality of quality detection units and key point detection units, wherein the plurality of quality detection units and key point detection units are independent of each other and arranged in parallel, and the input items of each quality detection unit and key point detection unit are the feature images input by the feature extraction module. Accordingly, the inputting of the feature image into the upper network module, and the outputting of the predicted quality data corresponding to the training image and the predicted face key points by the upper network module specifically include:

[0064] L21, inputting the feature image into each quality detection unit and the key point detection unit respectively;

[0065] L22. Generate a feature score corresponding to each detection unit through each detection unit, and output the predicted face key points through the key point detection unit to obtain the predicted quality data corresponding to the training image.

[0066] Specifically, the plurality of quality detection units correspond one-to-one to the plurality of quality scores included in the predicted quality data, each quality detection unit is used to output a quality score, and the key point detection unit is used to input the predicted face key points. It can be understood that the plurality of quality detection units correspond one-to-one to the feature attributes included in the training image, each quality detection unit is used to input a quality score corresponding to a feature attribute, and the quality scores corresponding to each feature attribute form the predicted quality data.

[0067] Further, in an implementation of the present embodiment, the feature attributes may include a smile feature, an eye closure feature, a facial occlusion feature, and a facial fuzziness feature. Accordingly, the quality detection unit includes a smile detection unit, an eye closure detection unit, a facial occlusion detection unit, and a facial fuzziness detection unit, and outputs a smile feature quality score corresponding to the smile feature, an eye closure feature quality score corresponding to the eye closure feature, a facial occlusion feature quality score corresponding to the facial occlusion feature, and a facial fuzziness feature quality score corresponding to the facial fuzziness feature through the smile detection unit, the eye closure detection unit, the facial occlusion detection unit, and the facial fuzziness detection unit, respectively. In addition, in a possible implementation of the present embodiment, the quality scores corresponding to each attribute feature may be identified by 1 and 0. For example, for a smile feature, the smile feature quantity score in the quality data output by the preset network model may be 1 or 0. When the face image in the image to be rated carries a smile, the smile feature quantity score corresponding to the image to be rated is 1. When the face image in the image to be rated does not carry a smile, the smile feature quantity score corresponding to the image to be rated is 0. In addition, the quality scores corresponding to each attribute feature may be shown in the following table:

[0068] Blink Score Blink determination (1 for open eyes, 0 for closed eyes) Smile score Smile determination (1 means smiling, 0 means not smiling) Fuzzy score Fuzzy judgment (1 means clear, 0 means fuzzy) Occlusion score Occlusion determination (1 means occlusion, 0 means no occlusion) .

[0069] Further, in step H20, the correction of the preset network model refers to correcting the model parameters of the preset network model until the model parameters meet the preset conditions. The preset conditions include that the loss function value meets the preset requirements or the number of training times reaches the preset number of times. The preset requirements can be determined according to the accuracy of the image quality detection model, which is not described in detail here. The preset number of times can be the maximum number of training times of the preset network model, for example, 4000 times. Thus, when the preset network model outputs the predicted quality data, the loss function value of the preset network model is calculated according to the predicted quality data and the real quality data. After the loss function value is calculated, it is determined whether the loss function value meets the preset requirements; if the loss function value meets the preset requirements, the training is terminated; if the loss function value does not meet the preset requirements, it is determined whether the number of training times of the preset network model reaches the predicted number of times. If it does not reach the preset number of times, the network parameters of the preset network model are corrected according to the loss function value; if it reaches the preset number of times, the training is terminated. In this way, judging whether the training of the preset network model is terminated by the loss function value and the number of training times can avoid the training of the preset network model entering an infinite loop due to the failure of the loss function value to meet the preset requirements.

[0070] Furthermore, since the network parameters of the preset network model are modified when the training situation of the preset network model does not meet the preset conditions (for example, the loss function value does not meet the preset requirements and the number of training times does not reach the preset number of times), after the network parameters of the preset network model are corrected according to the loss function value, it is necessary to continue to train the preset network model, that is, continue to execute the step of inputting the training images in the training image set into the preset network model. Among them, the training images in the training image set that continue to be input into the preset network model can be training images that have not been input into the preset network model as input items. For example, all training images in the training image set have a unique image identifier (for example, an image number), and the image identifier of the training image input into the preset network model for the first training is different from the image identifier of the training image input into the preset network model for the second training, such as the image number of the training image input into the preset network model for the first training is 1, the image number of the training image input into the preset network model for the second training is 2, and the image number of the training image input into the preset network model for the Nth training is N. Of course, in practical applications, since the number of training images in the training image set is limited, in order to improve the training effect of the image quality detection model, the training images in the training image set can be sequentially input into the preset network model to train the preset network model. After all the training images in the training image set have been input into the preset network model, the operation of sequentially inputting the training images in the training image set into the preset network model can be continued, so that the training image group in the training image set is cyclically input into the preset network model. It should be noted that in the process of inputting the training images into the preset network model for training, the training images can be input in the order of the image numbers of the training images, or they can be input in a non-sequential order. Of course, the same training image can be reused to train the preset network model, or the same training image can be non-repeatedly used to train the preset network model. In this embodiment, the specific implementation method of "continuing to execute the step of inputting the training images in the training image set into the preset network model" is not limited.

[0071] Further, in an implementation of the present embodiment, the preset network model includes a feature extraction module and an upper network module, the upper network module includes a plurality of quality detection units and a key point detection unit, wherein the feature extraction module, the plurality of quality detection units and the key point detection unit are all independent network models. Thus, when the preset network model is corrected according to the predicted quality data, the predicted face key points, the real quality data and the real key points, the feature extraction module, the plurality of quality detection units and the key point detection unit are trained separately, wherein the plurality of quality detection units and the key point detection unit are sequenced based on the predicted items output by each of them and the real items corresponding to the predicted items, and the feature extraction module is corrected based on the predicted quality data, the predicted face key points, the real quality data and the real key points. Accordingly, in a possible implementation of the present embodiment, the preset network model corrects the model parameters according to the predicted quality data and the real quality data corresponding to the training image, which may specifically include:

[0072] For each quality detection unit, a first loss value corresponding to the quality detection unit is calculated according to a predicted quality score corresponding to the quality detection unit and a true quality score corresponding to the quality detection unit, and a model parameter corresponding to the quality detection unit is corrected according to the first loss value;

[0073] Calculating a second loss value according to the predicted face key points and the real face key points, and correcting the model parameters corresponding to the key point detection unit according to the second loss value;

[0074] The module parameters corresponding to the feature extraction module are modified according to each first loss value and the second loss value.

[0075] Specifically, when the correction is made based on the loss value, the gradient value is calculated by the loss value, and the network parameters corresponding to the loss value are updated and iterated by the reverse transmission network to update the network parameters corresponding to each loss value, wherein the loss value can be each first loss value, the second loss value and the third loss value, and the third loss value is calculated based on the first loss value and the second loss value, for example, the third loss value is the sum of each first loss value and the second loss value, or the third loss value is the average of each first loss value and the second loss value, and the third loss value is obtained by weighting each first loss value and the second loss value, wherein the weight coefficient of each first loss value and the second loss value can be preset. It can be understood that the correction process of the network model based on each first loss value, the second loss value and the third loss value can be the same. In this embodiment, by using the output items corresponding to each quality detection unit and the key point detection unit to determine the corresponding loss value, and using the corresponding loss value for separate training, it is possible to avoid repeated extraction and repeated training of feature images corresponding to each quality detection module and the key point detection module, which greatly reduces the amount of calculation during repeated network training.

[0076] Furthermore, in an implementation of this embodiment, during the training process of the preset network model, the method may further include:

[0077] Detecting model parameters of the key point detection unit;

[0078] When the model parameters of the key point detection unit meet the preset conditions, aligning the training samples in the training sample set to obtain an aligned training sample set;

[0079] The preset network model is trained using the aligned training sample set.

[0080] Specifically, the preset condition may include that the loss function value meets the preset requirement or the number of training times reaches the preset number of times. The preset condition may be determined according to the accuracy of the key point detection unit, which will not be described in detail here. The preset number of times may be the maximum number of training times of the key point detection unit, for example, 2000 times. Thus, when the key point detection unit outputs the predicted face key points, the second loss value of the key point detection unit is calculated according to the predicted face key points and the real face key points. After the second loss value is calculated, it is determined whether the second loss value meets the preset requirement; if the second loss value meets the preset requirement, the training is terminated; if the second loss value does not meet the preset requirement, it is determined whether the number of training times of the key point detection unit reaches the predicted number of times. If it does not reach the preset number of times, the network parameters of the key point detection unit are corrected according to the second loss value; if it reaches the preset number of times, the training is terminated. In this way, judging whether the training of the key point detection unit is terminated by the second loss value and the number of training times can avoid the training of the key point detection unit entering an infinite loop due to the failure of the second loss value to meet the preset requirement. Of course, it is worth noting that the key point detection unit is a subunit of the preset network model, and after the model parameters of the key point detection unit meet the preset conditions, the training samples need to be aligned, and the preset network model is trained using the aligned training samples. Therefore, the loss function requirement corresponding to the key point detection unit is lower than the loss function requirement corresponding to the preset network model, and the number of detections corresponding to the key point detection unit is less than the number of detections corresponding to the preset network model.

[0081] Further, in an implementation of the present embodiment, the alignment processing of each training sample in the training sample set refers to: for each training sample in the training sample, the training image in the training sample is aligned so that the face image carried in the training image faces a preset direction, for example, straight ahead. It can be understood that the alignment processing is to adjust the direction of the face image in the training image so that the direction of the face image is the preset direction. In addition, after the alignment processing is performed on each training sample image, the aligned face image is used as the training image to update the training sample set, and after the training sample set is updated, the updated training sample set is used as the training sample of the preset network model, wherein when the updated training sample set is used as the training sample, the model parameters of the preset network model are the model parameters obtained by training the unupdated training samples. In this way, by aligning the training sample set and continuing to train the preset network model with the aligned training sample set, the low accuracy of the image quality detection model obtained by training due to the face direction can be avoided.

[0082] Further, in an implementation of this embodiment, after the trained image quality detection model is obtained, the trained image quality detection model can be post-processed so that the output item of the post-processed image quality detection model is quality data. The post-processing method may include controlling the key point detection unit not to output and removing the key point detection unit. In this way, the output item of the image quality detection model can be quality data through post-processing, so that the image quality detection model can output the quality data corresponding to the image to be rated.

[0083] Further, it can be known from the training process of the above-mentioned image quality detection model that the image quality detection model includes a feature extraction module and an upper network model. Accordingly, the step of inputting the image to be rated into a preset image quality detection model and outputting the quality data of the face image carried by the image to be rated through the preset image quality detection model may specifically include:

[0084] S21, inputting the image to be rated into a feature extraction module, and obtaining a feature image corresponding to the training image through the feature extraction module;

[0085] S22, inputting the feature image into the upper network module, and outputting the quality data corresponding to the training image through the upper network module.

[0086] Specifically, the feature extraction module is used to extract the feature image of the image to be scored, and input the feature image to the upper network module, and the upper network module extracts the quality data of the image to be scored based on the feature image. Among them, the processing of the image to be processed by the feature extraction module of the image quality detection model is the same as the processing process of the training by the feature extraction module in the preset network model, so the specific execution process of step S21 can refer to step L10. Similarly, the processing of the feature image corresponding to the image to be scored by the upper network module of the image quality detection model is the same as the processing process of the feature image corresponding to the training image by the upper network module in the preset network model, so the specific execution process of step S22 can refer to step L20.

[0087] It is understandable that the network structure corresponding to the image quality detection model during the training process is the same as the network structure corresponding to the application process (determining the quality data of the image to be rated). For example, during the training process, the image quality detection model includes a feature extraction module and an upper network module, then correspondingly, when the quality data of the image to be rated is output through the image quality detection model, the image quality detection model also includes a feature extraction module and an upper network module.

[0088] In addition, it is worth explaining that the structure of the upper network module is basically the same as that of the upper network module in the preset network model, and the only difference is that the key point detection unit of the upper network model does not output or the key point detection unit is removed from the upper network model. Correspondingly, the inputting of the feature image into the upper network module and the outputting of the quality data corresponding to the training image through the upper network module are specifically: inputting the feature image into each quality detection unit respectively, outputting the corresponding quality score through each quality detection unit, and determining the quality score corresponding to the image to be scored according to the output quality scores. Among them, the processing process of each quality detection unit is the same as the processing process of each quality detection unit in the upper network module in the preset network model, and will not be repeated here.

[0089] S30: Determine a target score for the image to be rated according to the quality data of the image to be rated.

[0090] Specifically, the target score is a score corresponding to the image to be rated, which is used to reflect the quality of the image to be rated, wherein the image quality is judged based on whether the face image in the image is smiling, closing eyes, being blocked or blurred. For example, for two consecutive images, the difference between the first image in the two consecutive images and the second image in the two consecutive images is that the face image in the first image is in a smiling state, while the face image in the second image is not in a smiling state, then the image quality of the first image is higher than the image quality of the second image; for another example, for two consecutive images, the difference between the third image in the two consecutive images and the fourth image in the two consecutive images is that the face image in the third image is in a closed eye state, while the face image in the second image is not in a closed eye state, then the image quality of the first image is lower than the image quality of the second image. In addition, the target score is obtained based on the smile feature, the closed eye feature, the blurred feature and the blocked feature, so that the image quality is used to reflect the comprehensive effect of the smile feature, the closed eye feature, the blurred feature and the blocked feature, and when the target score is high, the image quality corresponding to the image to be rated is high; when the target score is low, the image quality corresponding to the image to be rated is low. The target score is calculated based on each quality score in the quality data. For example, the target score may be the sum of each quality score, or the average of each quality score, or the weighted sum of each quality score.

[0091] In one implementation of this embodiment, the target score is the weighted sum of each quality score. Thus, after obtaining the quality data of the image to be rated, each quality score in the quality data is extracted, and then the weighted sum of each quality score is performed, and the value obtained by the weighted sum is used as the target score of the image to be rated. In addition, when performing the weighted summation of each quality score, a different weight coefficient can be configured for each quality score, wherein the weight coefficient can be pre-set, and the weight coefficient corresponding to each quality score can be determined according to the importance of the attribute feature corresponding to each quality score in the face image, and the greater the importance of the attribute feature, the greater the weight coefficient corresponding to the attribute feature. Thus, the calculation formula corresponding to the target score of the image to be rated is:

[0092]

[0093] Among them, n is the number of feature attributes, k represents the kth feature attribute, and a k It is expressed as the quality score corresponding to the kth feature attribute. is the weight coefficient of the quality score corresponding to the kth feature attribute.

[0094] Further, in an implementation of this embodiment, the image to be rated may carry multiple face images, and the preset image quality detection model obtains the quality data of the image to be rated by determining the quality data corresponding to each face image in the image to be rated through the image quality detection model. It can be understood that when the image to be rated carries multiple face images, each face image can correspond to one quality data, and the quality scores included in the quality data corresponding to each face image correspond to each other. In addition, when multiple quality scores are obtained according to the image to be rated, when determining the target score according to the quality data, the target score of the image to be rated is determined according to the multiple quality scores. Among them, the method of determining the target score of the image to be rated according to the multiple quality scores may include the following methods: the first method is to calculate the candidate score according to each quality score, and then sum up the candidate scores to obtain the target score; the second method is to calculate the candidate score according to each quality score, and then use the average of the candidate scores as the target score; the third method is to calculate the candidate score according to each quality score, and then weight the candidate scores to obtain the target score.

[0095] Further, in an implementation of this embodiment, determining the score of the image to be rated according to the quality data of the image to be rated specifically includes:

[0096] For each face image in the to-be-rated images, a weight coefficient corresponding to the face image is obtained, and a candidate score for the face image is determined according to the weight coefficient and quality data of the face image;

[0097] The target score corresponding to the image to be rated is determined according to the candidate score of each face image.

[0098] Specifically, the weight coefficient is used to reflect the importance of the corresponding face image in the image to be rated, and the weight coefficient corresponding to each face image is determined according to the position and / or area occupied by the face image in the image to be rated. For example, if the weight coefficient is determined according to the corresponding area occupied, then the corresponding relationship between the weight coefficient and the area occupied by the face image in the image to be rated can be: when the area occupied by the face image is larger, the weight coefficient corresponding to the face image is higher; when the area occupied by the face image is smaller, the weight coefficient corresponding to the face image is lower.

[0099] Furthermore, in an implementation of this embodiment, the weight coefficient is determined based on the area occupied by the face image, the position of the face, and the orientation of the face, so that the weight coefficient includes the score corresponding to the area occupied by the face image, the score corresponding to the position of the face, and the score corresponding to the orientation of the face. Among them, the area occupied by the face image refers to the area occupied by the face image in the image to be rated, the face position refers to the position of the face image in the image to be rated, and the face orientation refers to the orientation of the face image in the image to be rated. Accordingly, the process of obtaining the weight coefficient can be:

[0100] For each face image, obtain the area occupied by the face image in the image to be rated, the location information, and the face orientation;

[0101] Determine a first score corresponding to the occupied area according to the occupied area, calculate a second score corresponding to the location according to the location information, and determine a third score corresponding to the face orientation according to the face orientation;

[0102] A weight coefficient corresponding to the facial image is calculated according to the first score, the second score, and the third score.

[0103] Specifically, the occupied area, location information and face orientation in the image to be rated can be obtained by preprocessing the image to be rated to obtain each face image carried by the image to be rated, and the occupied area, location information and face orientation corresponding to each face image. In addition, the first score can be determined according to the ratio of the occupied area to the image area of ​​the image to be rated. When the ratio of the occupied area to the image area of ​​the image to be rated is larger, the first score is larger, and when the ratio of the occupied area to the image area of ​​the image to be rated is smaller, the first score is smaller. The second score is determined according to the distance between the center point of the position of the face image and the center point of the image. When the distance between the center point and the center point of the image is larger, the second score is larger, and when the distance between the center point and the center point of the image is smaller, the second score is smaller. The third score is determined according to the angle between the face orientation and the front. When the angle is larger, the third score is larger, and when the angle is smaller, the third score is smaller.

[0104] Further, after obtaining the first score, the second score and the third score, the method of calculating the weight coefficient according to the first score, the second score and the third score includes: taking the sum of the first score, the second score and the third score as the weight coefficient; taking the average of the first score, the second score and the third score as the weight coefficient; and weighting the first score, the second score and the third score, and taking the weighted score as the weight coefficient, wherein the weight coefficient can be pre-set.

[0105] Further, after determining the weight coefficient of each facial image, the candidate score of each facial image is calculated according to the quality data and weight coefficient corresponding to each facial image, and then the target score is calculated according to the candidate score. The process of calculating the candidate score of each facial image according to the quality data and weight coefficient corresponding to each facial image can be: firstly, the target score corresponding to the facial image is calculated according to the quality data, and then the candidate score is calculated according to the target score and the weight coefficient, wherein the candidate score can be the sum of the weight coefficient and the target score, or the product of the weight coefficient and the target score, etc. In addition, after determining the candidate score of each facial image, the target score can be the sum of the candidate scores corresponding to each facial image, or the average of the candidate scores corresponding to each facial image, etc. In a possible implementation of this embodiment, the candidate score corresponding to each facial image is the product of the weight coefficient corresponding to the facial image and the target score, and the target score is the sum of the candidate scores.

[0106] Furthermore, in an implementation of this embodiment, the method may further include:

[0107] When there are at least two images to be rated, the method further includes:

[0108] Obtain the target score corresponding to each image to be rated;

[0109] Determine the recommended images corresponding to the at least two images to be rated according to the acquired target scores.

[0110] Specifically, each of the at least two images to be rated carries a face image, and the at least two images may be at least two continuous images, for example, the at least two images to be rated are continuous images obtained by continuously shooting the same scene by an imager, and for another example, the at least two continuous images may be multiple images obtained in a continuous shooting mode. In addition, for each of the at least two images to be rated, the image to be rated is input into an image quality detection model, and the target score of the image to be rated is output through the image quality detection model; after obtaining the target scores corresponding to all the images to be rated, after obtaining the target scores corresponding to each of the images to be rated, one of the at least two images to be rated is selected as a recommended image according to each target score, wherein the target score corresponding to the recommended image may be the maximum target score of the target scores corresponding to each of the images to be rated.

[0111] Based on the above-mentioned image scoring method, this embodiment provides a computer-readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the steps in the image processing method described in the above-mentioned embodiment.

[0112] Based on the above image scoring method, the present invention also provides a terminal device, such as Figure 4 As shown, it includes at least one processor (processor) 20; display panel 21; and memory (memory) 22, and may also include a communication interface (Communications Interface) 23 and a bus 24. Among them, the processor 20, the display panel 21, the memory 22 and the communication interface 23 can communicate with each other through the bus 24. The display panel 21 is configured to display a preset user guide interface in the initial setting mode. The communication interface 23 can transmit information. The processor 20 can call the logic instructions in the memory 22 to execute the method in the above embodiment.

[0113] In addition, the logic instructions in the memory 22 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0114] The memory 22 is a computer-readable storage medium that can be configured to store software programs, computer executable programs, such as program instructions or modules corresponding to the methods in the embodiments of the present disclosure. The processor 20 executes functional applications and data processing by running the software programs, instructions or modules stored in the memory 22, that is, implementing the methods in the above embodiments.

[0115] The memory 22 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory 22 may include a high-speed random access memory and may also include a non-volatile memory. For example, a variety of media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, may also be a transient storage medium.

[0116] In addition, the specific process of loading and executing the plurality of instructions in the storage medium and the processor in the terminal device has been described in detail in the method, and will not be described here one by one. It is understandable that for ordinary technicians in this field, equivalent replacement or change can be made according to the technical solution and inventive concept of the present invention, and all these changes or replacements should belong to the protection scope of the claims attached to the present invention.

Claims

1. An image scoring method, characterized in that: It includes: Acquire images to be rated, wherein the images to be rated are multiple images obtained by continuous shooting of an imager, each image to be rated carries a face image, and the face image includes multiple face images; Input the image to be rated into a preset image quality detection model, and output the quality data of the face image carried by the image to be rated through the preset image quality detection model, wherein the quality data is used to reflect the quality of the characteristic attributes of the image to be rated, and the image quality detection model includes a feature extraction module and an upper network module; the upper network module includes a plurality of quality detection units and key point detection units arranged in parallel; Determining a target score for the image to be rated according to the quality data of the image to be rated; The process of the image quality detection model includes: Detecting model parameters of the key point detection unit; When the model parameters of the key point detection unit meet the preset conditions, aligning the training samples in the training sample set to obtain an aligned training sample set, wherein the faces in the training images in the aligned training sample set are in the same orientation; The preset network model is trained using the aligned training sample set, wherein when the aligned training sample set is used as the training sample, the model parameters of the preset network model are the model parameters obtained by training using the unupdated training sample; Determining the target score of the image to be rated according to the quality data of the image to be rated specifically includes: For each face image in the to-be-rated images, a weight coefficient corresponding to the face image is obtained, and a candidate score for the face image is determined according to the weight coefficient and quality data of the face image; Determining a target score corresponding to the image to be rated according to the candidate scores of each face image; The weight coefficient is determined according to the score corresponding to the area occupied by the face image, the score corresponding to the position of the face, and the score corresponding to the orientation of the face.

2. The image scoring method according to claim 1, characterized in that: The step of inputting the image to be rated into a preset image quality detection model and outputting the quality data of the face image carried by the image to be rated through the preset image quality detection model is specifically: The image to be rated is input into a preset image quality detection model, and the quality data corresponding to each face image in the image to be rated is determined by the image quality detection model.

3. The image scoring method according to claim 1, characterized in that: The feature attributes include a smile feature, an eye-closing feature, a facial occlusion feature, and a facial blur feature.

4. The image scoring method according to any one of claims 1 to 3, characterized in that: The training process of the image quality detection model specifically includes: The preset network model generates predicted quality data corresponding to the training image according to the training image in the training sample set; wherein the training sample set includes a plurality of training sample groups, each group of training samples includes a training image and real quality data and real face key points corresponding to the training image, the real quality data includes a feature score group, and each feature score in the feature score group is a quality score corresponding to a feature attribute of the training image; The preset network model modifies the model parameters according to the predicted quality data and the actual quality data corresponding to the training image, and continues to execute the step of generating the detection data corresponding to the training image according to the next set of training images in the training sample set until the training status of the preset network model meets the preset conditions to obtain the image quality detection model.

5. The image scoring method according to claim 4, characterized in that: The step of generating prediction quality data corresponding to the training image by the preset network model according to the training image in the training sample set includes: Inputting the training image in the training sample set into the feature extraction module, and obtaining the feature image corresponding to the training image through the feature extraction module; The feature image is input into the upper network module, and the predicted quality data and predicted facial key points corresponding to the training image are output through the upper network module.

6. The image scoring method according to claim 5, characterized in that: The feature attributes corresponding to each quality detection unit are different from each other; the inputting the feature image into the upper network module, and outputting the predicted quality data corresponding to the training image and the predicted face key points through the upper network module specifically include: Inputting the feature image into each quality detection unit and the key point detection unit respectively; Each detection unit generates a feature score corresponding to each detection unit, and the key point detection unit outputs the predicted face key points to obtain the predicted quality data corresponding to the training image.

7. The image scoring method according to claim 6, characterized in that: The step of modifying the model parameters of the preset network model according to the predicted quality data and the quality data corresponding to the training image comprises: For each quality detection unit, a first loss value corresponding to the quality detection unit is calculated according to a predicted quality score corresponding to the quality detection unit and a true quality score corresponding to the quality detection unit, and a model parameter corresponding to the quality detection unit is corrected according to the first loss value; Calculating a second loss value according to the predicted face key points and the real face key points, and correcting the model parameters corresponding to the key point detection unit according to the second loss value; The module parameters corresponding to the feature extraction module are modified according to each first loss value and the second loss value.

8. The image scoring method according to claim 1, characterized in that: When there are at least two images to be rated, the method further includes: Obtain the target score corresponding to each image to be rated; Determine the recommended images corresponding to the at least two images to be rated according to the acquired target scores.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer-readable program executable by the processor, wherein: The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the image scoring method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the image scoring method according to any one of claims 1 to 8.

Images