Facial image acne grading method and device
By marking the sample facial images of the acne detection box category and location, the acne grading model of facial images was trained, and the problem of low generalization performance in the prior art was solved, achieving a more accurate and explainable acne grading effect.
Patent Information
- Application Number
- CN202310584535.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-23
- Publication Date
- 2024-11-26
AI Technical Summary
The prior art has low generalization performance in facial image grading for acne disease and cannot fully utilize the finer grained information on the face, resulting in poor grading effect.
By obtaining sample facial images marked with the categories and locations of the acne detection box, the image features are extracted using convolutional neural network, and a multi-task loss function is constructed based on preset grading standards to train the facial image acne grading model to achieve a finer face image in the facial image Full utilization of granularity information.
It improves the accuracy and interpretability of acne grading results, optimizes the model grading effect, and improves the accuracy of prediction.
Smart Images

Figure CN119028536A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of Internet medical technology, and in particular to a facial image acne grading method and device. Background Art
[0002] In the post-diagnosis phase of Internet medical treatment, when a patient is diagnosed with acne, the evaluation of the post-diagnosis effect of acne is an important part of post-diagnosis management. The evaluation of the post-diagnosis effect of acne includes the evaluation of the changes in the severity of the patient's acne based on the patient's facial image.
[0003] In the related art, the severity of acne is graded based on its area and color, but the generalization performance of this method is low; the end-to-end grading method using deep learning can solve the problem of low generalization performance, but it cannot make full use of the finer-grained information of the face, resulting in poor grading effect. Summary of the invention
[0004] In view of this, the embodiments of the present invention provide a facial image acne grading method and device, which can solve the problem of low generalization performance and improve the accuracy and interpretability of acne grading result prediction.
[0005] To achieve the above object, according to one aspect of an embodiment of the present invention, a method for training a facial image acne grading model is provided, comprising:
[0006] Acquire multiple sample facial images and annotation data of each of the sample facial images, wherein the sample facial images are annotated with the category and position of the acne detection frame, and the annotation data includes the classification of the acne detection frame;
[0007] A facial image acne grading model is trained based on the multiple sample facial images and the annotated data of each of the sample facial images.
[0008] Optionally, a plurality of sample facial images are obtained, including:
[0009] Inputting a plurality of original facial images into a target detection model to obtain a detection result of each of the original facial images, wherein the detection result includes a category and a position of an acne detection frame in the original facial image;
[0010] According to the category and position of the acne detection frame in the original facial image, a corresponding acne detection frame is added to the original facial image to obtain a sample facial image corresponding to the original facial image.
[0011] Optionally, the target detection model is obtained by the following method:
[0012] Acquire multiple facial images and a labeling result of each of the facial images, wherein the labeling result includes a category and a position of an acne detection frame on the facial image;
[0013] The target detection model is trained based on the multiple facial images and the annotation results of each facial image.
[0014] Optionally, training a facial image acne grading model according to the plurality of sample facial images and the annotated data of each of the sample facial images comprises:
[0015] Using a convolutional neural network to extract image features of each of the sample facial images;
[0016] Construct a loss function based on the preset classification criteria;
[0017] The loss function is iteratively trained according to the image features and annotation data of each sample facial image to obtain the facial image acne grading model.
[0018] Optionally, the preset grading standard includes a first grading standard and a second grading standard, and constructing a loss function based on the preset grading standard includes:
[0019] Constructing a first grading loss function based on the first grading standard;
[0020] Constructing a second grading loss function based on the second grading standard;
[0021] The loss function is constructed according to the first hierarchical loss function and the second hierarchical loss function.
[0022] Another aspect of an embodiment of the present invention provides a facial image acne grading method, comprising:
[0023] Obtain a facial image to be detected;
[0024] Determining the category and position of the acne detection frame in the facial image to be detected;
[0025] adding a corresponding acne detection frame on the facial image to be detected according to the category and position of the acne detection frame to update the facial image to be detected;
[0026] Inputting the updated facial image to be detected into the facial image acne grading model to obtain an acne grading result corresponding to the facial image to be detected;
[0027] Wherein, the facial image acne grading model is obtained according to a training method of a facial image acne grading model.
[0028] Optionally, determining the category and position of the acne detection frame in the facial image to be detected includes:
[0029] Determine the category and position of the acne detection frame in the facial image to be detected by using a target detection model;
[0030] The target detection model is obtained by the following method:
[0031] Acquire multiple facial images and a labeling result of each of the facial images, wherein the labeling result includes a category and a position of an acne detection frame on the facial image;
[0032] The target detection model is trained based on the multiple facial images and the annotation results of each facial image.
[0033] A third aspect of an embodiment of the present invention provides a training device for a facial image acne grading model, comprising:
[0034] A first acquisition module acquires a plurality of sample facial images and annotation data of each of the sample facial images, wherein the sample facial images are annotated with the category and position of the acne detection frame, and the annotation data is the classification of the acne detection frame;
[0035] The training module is configured to train a facial image acne grading model based on the multiple sample facial images and the annotation data of each sample facial image.
[0036] Optionally, the first acquisition module is further used to:
[0037] Inputting a plurality of original facial images into a target detection model to obtain a detection result of each of the original facial images, wherein the detection result includes a category and a position of an acne detection frame in the original facial image;
[0038] According to the category and position of the acne detection frame in the original facial image, a corresponding acne detection frame is added to the original facial image to obtain a sample facial image corresponding to the original facial image.
[0039] Optionally, the target detection model is obtained by the following method:
[0040] Acquire multiple facial images and a labeling result of each of the facial images, wherein the labeling result includes a category and a position of an acne detection frame on the facial image;
[0041] The target detection model is trained based on the multiple facial images and the annotation results of each facial image.
[0042] Optionally, the training module is further used to:
[0043] Using a convolutional neural network to extract image features of each of the sample facial images;
[0044] Construct a loss function based on the preset classification criteria;
[0045] The loss function is iteratively trained according to the image features and annotation data of each sample facial image to obtain the facial image acne grading model.
[0046] Optionally, the preset grading standard includes a first grading standard and a second grading standard, and the training module is further used to:
[0047] Constructing a first grading loss function based on the first grading standard;
[0048] Constructing a second grading loss function based on the second grading standard;
[0049] The loss function is constructed according to the first hierarchical loss function and the second hierarchical loss function.
[0050] A fourth aspect of the embodiments of the present invention provides a facial image acne grading device, comprising:
[0051] The second acquisition module acquires the facial image to be detected;
[0052] A first determination module determines the category and position of the acne detection frame in the facial image to be detected;
[0053] An updating module, which adds a corresponding acne detection frame to the facial image to be detected according to the category and position of the acne detection frame, so as to update the facial image to be detected;
[0054] A second determination module inputs the updated facial image to be detected into the facial image acne grading model to obtain an acne grading result corresponding to the facial image to be detected;
[0055] Wherein, the facial image acne grading model is obtained according to a training method of a facial image acne grading model.
[0056] Optionally, the first determining module is further configured to:
[0057] Determine the category and position of the acne detection frame in the facial image to be detected by using a target detection model;
[0058] The target detection model is obtained by the following method:
[0059] Acquire multiple facial images and a labeling result of each of the facial images, wherein the labeling result includes a category and a position of an acne detection frame on the facial image;
[0060] The target detection model is trained based on the multiple facial images and the annotation results of each facial image.
[0061] According to another aspect of an embodiment of the present invention, there is provided an electronic device, including:
[0062] one or more processors;
[0063] a storage device for storing one or more programs,
[0064] When the one or more programs are executed by the one or more processors, the one or more processors implement the facial image acne grading model training method or facial image acne grading method provided by the present invention.
[0065] According to another aspect of an embodiment of the present invention, a computer-readable medium is provided, on which a computer program is stored. When the program is executed by a processor, the facial image acne grading model training method or facial image acne grading method provided by the present invention is implemented.
[0066] An embodiment of the above invention has the following advantages or beneficial effects: a facial image acne grading model is trained using sample facial images with the category and position of the acne detection frame annotated and the annotated data of the sample facial images. The model training method has high generalization performance, and through the category and position of the acne detection frame in the sample facial image, it is possible to fully utilize more fine-grained information in the facial image for model training, thereby optimizing the model grading effect and improving the accuracy and interpretability of the model prediction.
[0067] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS
[0068] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.
[0069] Figure 1 is a schematic diagram of the main process of a method for training a facial image acne grading model according to an embodiment of the present invention;
[0070] Figure 2 is a schematic diagram of a sample facial image according to an embodiment of the present invention;
[0071] Figure 3 is a schematic diagram of the main process of another method for training a facial image acne grading model according to an embodiment of the present invention;
[0072] Figure 4is a schematic diagram of the main process of another method for training a facial image acne grading model according to an embodiment of the present invention;
[0073] Figure 5 is a schematic diagram of the main process of a facial image acne grading method according to an embodiment of the present invention;
[0074] Figure 6 is a schematic diagram of main modules of a training device for a facial image acne grading model according to an embodiment of the present invention;
[0075] Figure 7 is a schematic diagram of main modules of a facial image acne grading device according to an embodiment of the present invention;
[0076] Figure 8 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;
[0077] Fig. 9 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION
[0078] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.
[0079] It should be noted that in the technical solution of this disclosure, the collection, collection, update, analysis, processing, use, transmission, storage and other aspects of user personal information (such as user facial images) involved are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information, network security and national security.
[0080] Figure 1 FIG. 4 is a schematic diagram of the main process of a method for training a facial image acne grading model according to an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:
[0081] Step S101: obtaining a plurality of sample facial images and annotation data of each sample facial image, wherein the category and position of the acne detection frame are annotated on the sample facial image, and the annotation data includes the classification of the acne detection frame;
[0082] Step S102: training a facial image acne grading model based on a plurality of sample facial images and the annotated data of each sample facial image.
[0083] In an embodiment of the present invention, the sample facial image is a facial image of an acne patient with an acne detection frame labeled with a category and position of the acne detection frame, wherein the category of the acne detection frame indicates the category of acne corresponding to the acne detection frame, and the category of the acne detection frame may be at least one of skin lesion types such as comedones, papules, pustules, cysts, and acne marks; the position of the acne detection frame indicates the position of the acne corresponding to the acne detection frame, and the acne detection frame may be a frame of any shape, such as a rectangular frame, and the position of the acne detection frame may be the coordinates of the four vertices of the rectangular frame. The category and position of the acne in the sample facial image are labeled with the acne detection frame, thereby achieving refined positioning of different acne categories, using more fine-grained information of the facial image for model training, and improving the accuracy and interpretability of the model prediction.
[0084] The annotation data of each sample facial image includes the acne grade corresponding to each acne detection frame in the facial image, and the acne grade indicates the severity of the acne. The acne grade can be determined according to one or more grading standards. For example, the grading standard can include a first grading standard and a second grading standard, wherein the first grading standard can be to divide acne into 3 degrees and 4 grades according to the nature of the skin lesions, and the specific grading method is: mild (grade I): only comedones; moderate (grade II): inflammatory papules; moderate (grade III): pustules appear; severe (grade IV): nodules and cysts. The second grading standard may be the Pillsbury grading standard, and the specific grading method is: Mild Grade I: mainly comedones, a small number of papules and pustules, and a total number of skin lesions less than 30; Moderate Grade I: comedones and a moderate number of papules and pustules, with a total number of skin lesions of 31-50; Moderate Grade III: a large number of papules and pustules, a total number of skin lesions of 50-100, and a number of nodules less than 3; Severe Grade IV: nodular / cystic acne or conglobate acne, with a total number of skin lesions greater than 100, and greater than 3 nodules / cysts.
[0085] Figure 2 The figure is a schematic diagram of a sample facial image of an embodiment of the present invention. In the sample facial image, the acne detection frame is a rectangular frame, and the rectangular frame contains the acne in the sample facial image. The location of the acne can be determined according to the coordinates of the four vertices of the rectangular frame; the color of the rectangular frame is set to different colors according to the category of the acne, and different numbers can also be used in the middle of the rectangular frame to represent the category of the acne, and the color of the rectangular frame is set corresponding to the number. Among them, the category of acne corresponding to the pink rectangular frame is acne mark, which is represented by the number 1; the category of acne corresponding to the purple rectangular frame is blackhead, which is represented by the number 2; the category of acne corresponding to the green rectangular frame is papule, which is represented by the number 3; the category of acne corresponding to the brown rectangular frame is pustule, which is represented by the number 4 ( Figure 2 There is no pustule in the sample facial image shown); the category of acne corresponding to the yellow rectangular box is cyst, which is represented by the number 5. On this basis, the acne corresponding to each rectangular box can be classified to obtain the labeled data of the sample facial image, and the labeled data corresponds to the rectangular box.
[0086] In the embodiment of the present invention, Figure 3 As shown, multiple sample facial images are obtained, including:
[0087] Step S301: inputting a plurality of original facial images into a target detection model to obtain a detection result for each original facial image, wherein the detection result includes a category and a position of an acne detection frame in the original facial image;
[0088] Step S302: adding a corresponding acne detection frame to the original facial image according to the category and position of the acne detection frame in the original facial image to obtain a sample facial image corresponding to the original facial image.
[0089] In an embodiment of the present invention, the category and position of the acne detection frame in multiple sample facial images can be obtained through a target detection model. The original facial image is a facial image of an acne patient without the category and position of the acne detection frame marked. The multiple original facial images are input into the target detection model, and the category and position of the acne detection frame in the original facial image can be output. The target detection model can be implemented using a variety of target detection algorithm models, for example, the target detection model can be a Yolo series model based on a single-stage target detection algorithm, or a Faster RCNN model based on a two-stage target detection algorithm.
[0090] After obtaining the category and position of the acne detection frame in the original facial image, a sample facial image corresponding to the original facial image is obtained according to the original facial image and the category and position of the obtained acne detection frame, that is, the acne detection frame is drawn according to the position of the acne detection frame, and the category of the acne detection frame can be achieved by setting the frame to different colors.
[0091] In an embodiment of the present invention, the target detection model is obtained by the following method: obtaining multiple facial images and the annotation results of each facial image, the annotation results including the category and position of the acne detection frame on the facial image; training the target detection model according to the multiple facial images and the annotation results of each facial image. That is, a data set is constructed by multiple facial images and the annotation results corresponding to each facial image, and training is performed to obtain the target detection model.
[0092] In the embodiment of the present invention, Figure 4 As shown, based on multiple sample facial images and the annotated data of each sample facial image, a facial image acne grading model is trained, including:
[0093] Step S401: extracting image features of each sample facial image using a convolutional neural network;
[0094] Step S402: constructing a loss function based on a preset grading standard;
[0095] Step S403: According to the image features and annotation data of each sample facial image, the loss function is iteratively trained to obtain a facial image acne grading model.
[0096] In the embodiment of the present invention, the preset grading standard may be one or more. For example, the preset grading standard includes a first grading standard and a second grading standard. The loss function is constructed based on the preset grading standard, including:
[0097] Constructing a first grading loss function based on the first grading standard;
[0098] Constructing a second classification loss function based on the second classification standard;
[0099] A loss function is constructed according to the first hierarchical loss function and the second hierarchical loss function.
[0100] In the embodiment of the present invention, the network structure of the convolutional neural network is not specifically limited, for example, it may include at least one of the network structures such as ResNet (residual network), Inception (a convolutional neural network), such as Inception. The convolutional neural network may also include a feature pyramid network (FPN), etc. For example, RestNet may be used as a backbone network to extract multi-level features in a sample facial image, and then FPN may be used to fuse the extracted multi-level features, so that the feature information of the extracted image features is richer, thereby improving the accuracy of the model prediction. The convolutional neural network may also include a fully convolutional network, which is used to classify image features.
[0101] After using the convolutional neural network to extract the image features of the sample facial image, the image features and annotation data of each sample facial image in multiple sample facial images are used to iteratively train the loss function, which is constructed based on the preset grading standard. The actual acne grading results in the annotation data correspond to the preset grading standard. During training, the predicted acne grading results corresponding to the preset grading standard can be obtained, and the loss is calculated based on the predicted acne grading results and the actual acne grading results. The loss function is iteratively trained, and the parameters of the convolutional neural network are continuously updated using the reverse gradient descent optimization method until the loss no longer decreases, thereby obtaining a facial image acne grading model.
[0102] In an embodiment of the present invention, the preset grading standard may be one or more grading standards, and a multi-task loss function may be constructed based on the multiple grading standards to obtain multiple grading results. For example, the preset grading standard includes a first grading standard and a second grading standard, so that a first grading result corresponding to the first grading standard and a second grading result corresponding to the second grading standard can be obtained respectively. Among them, the annotated data may include a true first grading result and a true second grading result. A first grading loss function may be constructed based on the first grading standard, and a second grading loss function may be constructed based on the second grading standard. The first grading loss function and the second grading loss function have corresponding weight coefficients respectively. The first grading loss function and the second grading loss function are weighted and summed to construct a multi-task loss function. The first grading loss is determined by the predicted first grading result and the true first grading result, and the second grading loss is determined according to the predicted second grading result and the true second grading result. The multi-task loss function is trained according to the first grading loss and the second grading loss, and the parameters of the convolutional neural network are continuously updated by the reverse gradient descent optimization method to obtain a facial image acne grading model including two classification heads. By constructing a multi-task loss function, the model can learn more general classification features and improve the accuracy of the two grading results.
[0103] In an embodiment of the present invention, a facial image acne grading model is trained using sample facial images with the category and position of acne detection frames and the labeled data of the sample facial images. The model training method uses a deep learning algorithm to solve the problem of low generalization performance. Through the category and position of the acne detection frame in the sample facial image, the model training is fully utilized with more fine-grained information in the facial image, thereby optimizing the model grading effect and improving the accuracy and interpretability of the model prediction. Through the sample facial images with the category and position of the acne detection frame and the multi-task loss function constructed by multiple grading standards, a facial image acne grading model that can predict the grading results of different grading standards can be obtained, the accuracy of the grading results predicted by the model is improved, and the model is interpretable.
[0104] Another aspect of the embodiment of the present invention is as follows Figure 5 As shown, a facial image acne grading method is provided, comprising:
[0105] Step S501: Acquire a facial image to be detected;
[0106] Step S502: determining the category and position of the acne detection frame in the facial image to be detected;
[0107] Step S503: adding a corresponding acne detection frame to the facial image to be detected according to the category and position of the acne detection frame to update the facial image to be detected;
[0108] Step S504: inputting the updated facial image to be detected into the facial image acne grading model to obtain an acne grading result corresponding to the facial image to be detected;
[0109] The facial image acne grading model is obtained according to the training method of the facial image acne grading model.
[0110] In an embodiment of the present invention, the facial image to be detected may be a facial image uploaded or photographed by a user through a client. Then, the category and position of the acne detection frame in the facial image to be detected are determined, and then, according to the category and position of the acne detection frame, a corresponding acne detection frame is added or drawn on the facial image to be detected to update the facial image to be detected, and the updated facial image to be detected includes the category and position of the acne detection frame. Among them, different categories of acne detection frames can be drawn in different colors to distinguish different categories of acne.
[0111] In an embodiment of the present invention, a target detection model may be used to detect a facial image to be detected, and the category and position of an acne detection frame in the facial image to be detected may be obtained, wherein the target detection model is obtained by the following method:
[0112] Acquire multiple facial images and annotation results of each facial image, where the annotation results include the category and position of the acne detection frame on the facial image;
[0113] Based on multiple facial images and the annotation results of each facial image, an object detection model is trained.
[0114] In an embodiment of the present invention, the updated facial image to be detected is input into the facial image acne grading model to obtain the acne grading result corresponding to the facial image to be detected, including: using a convolutional neural network to extract the image features of the updated facial image to be detected, and determining the acne grading result according to the image features and the preset grading standard. That is, the facial image to be detected is predicted using the trained facial image acne grading model to obtain the corresponding acne grading result. The acne grading result corresponds to the preset grading standard, and the preset grading standard can be one or more, and the corresponding acne grading result is one or more.
[0115] In an embodiment of the present invention, the preset grading standard includes a first grading standard and a second grading standard, and the acne grading result includes a first grading result corresponding to the first grading standard and a second grading result corresponding to the second grading standard. That is, the facial image acne grading model can simultaneously obtain acne grading results corresponding to multiple different grading standards of the facial image to be detected. The facial image acne grading model is obtained by training by using sample facial images with the category and position of the acne detection frame and the annotation data of the sample facial images, constructing a multi-task loss function based on multiple grading standards, and adopting the reverse gradient descent optimization method.
[0116] The facial image acne grading method of the embodiment of the present invention obtains a facial image to be detected, then uses a target detection model to obtain the category and position of the acne detection frame in the facial image to be detected, then adds an acne detection frame to the facial image to be detected according to the category and position of the acne detection frame to update the facial image to be detected, and then inputs the updated facial image to be detected into the facial image acne grading model to obtain the acne grading result of the facial image to be detected. Among them, the facial image acne grading model is trained using sample facial images with the category and position of the acne detection frame annotated and the annotated data of the sample facial image. The facial image acne grading model is trained by constructing a multi-task loss function based on different grading standards, and can simultaneously predict the acne grading results corresponding to different grading standards; and the method uses a deep learning algorithm to solve the problem of low generalization performance. By drawing the category and position of the acne detection frame on the facial image, it is realized to fully utilize the more fine-grained information in the facial image for model training, thereby optimizing the model grading effect and improving the accuracy and interpretability of the model prediction.
[0117] In a third aspect of the embodiment of the present invention, Figure 6 As shown, a training device 600 for a facial image acne grading model is provided, comprising:
[0118] A first acquisition module 601 acquires a plurality of sample facial images and annotation data of each sample facial image, wherein the sample facial images are annotated with the category and position of the acne detection frame, and the annotation data is the classification of the acne detection frame;
[0119] The training module 602 is configured to train a facial image acne grading model based on a plurality of sample facial images and the annotation data of each sample facial image.
[0120] In the embodiment of the present invention, the first acquisition module 601 is further used to:
[0121] Inputting multiple original facial images into the target detection model to obtain a detection result for each original facial image, the detection result including the category and position of the acne detection frame in the original facial image;
[0122] According to the category and position of the acne detection frame in the original facial image, a corresponding acne detection frame is added to the original facial image to obtain a sample facial image corresponding to the original facial image.
[0123] In the embodiment of the present invention, in the first acquisition module 601, the target detection model is obtained by the following method:
[0124] Acquire multiple facial images and annotation results of each facial image, where the annotation results include the category and position of the acne detection frame on the facial image;
[0125] Based on multiple facial images and the annotation results of each facial image, an object detection model is trained.
[0126] In this embodiment of the present invention, the training module 602 is further used to:
[0127] A convolutional neural network is used to extract image features of each sample facial image;
[0128] Construct a loss function based on the preset classification criteria;
[0129] According to the image features and annotation data of each sample facial image, the loss function is iteratively trained to obtain the facial image acne grading model.
[0130] In the embodiment of the present invention, the preset classification standard includes a first classification standard and a second classification standard, and the training module 602 is further used to:
[0131] Constructing a first grading loss function based on the first grading standard;
[0132] Constructing a second classification loss function based on the second classification standard;
[0133] A loss function is constructed according to the first hierarchical loss function and the second hierarchical loss function.
[0134] In a fourth aspect of the embodiments of the present invention, Figure 7 As shown, a facial image acne grading device 700 is provided, comprising:
[0135] The second acquisition module 701 acquires a facial image to be detected;
[0136] A first determination module 702 determines the category and position of the acne detection frame in the facial image to be detected;
[0137] An updating module 703 adds a corresponding acne detection frame to the facial image to be detected according to the category and position of the acne detection frame to update the facial image to be detected;
[0138] The second determination module 704 inputs the updated facial image to be detected into the facial image acne grading model to obtain an acne grading result corresponding to the facial image to be detected;
[0139] The facial image acne grading model is obtained according to the training method of the facial image acne grading model.
[0140] In the embodiment of the present invention, the first determination module 702 is further used to: determine the category and position of the acne detection frame in the facial image to be detected by using the target detection model;
[0141] Among them, the target detection model is obtained by the following method:
[0142] Acquire multiple facial images and annotation results of each facial image, where the annotation results include the category and position of the acne detection frame on the facial image;
[0143] Based on multiple facial images and the annotation results of each facial image, an object detection model is trained.
[0144] According to another aspect of an embodiment of the present invention, an electronic device is provided, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the facial image acne grading model training method or facial image acne grading method provided by the present invention.
[0145] According to another aspect of an embodiment of the present invention, a computer-readable medium is provided, on which a computer program is stored. When the program is executed by a processor, the training method of the facial image acne grading model or the facial image acne grading method provided by the present invention is implemented.
[0146] Figure 8 An exemplary system architecture 800 is shown to which a facial image acne grading model training method or facial image acne grading model training device, facial image acne grading method or facial image acne grading device according to an embodiment of the present invention may be applied.
[0147] like Figure 8 As shown, system architecture 800 may include terminal devices 801, 802, 803, a network 804, and a server 805. Network 804 is used to provide a medium for communication links between terminal devices 801, 802, 803 and server 805. Network 804 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.
[0148] Users can use terminal devices 801, 802, and 803 to interact with server 805 through network 804 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 801, 802, and 803, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only examples).
[0149] The terminal devices 801 , 802 , and 803 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.
[0150] The server 805 may be a server that provides various services, such as a backend management server (only an example) that provides support for shopping websites browsed by users using the terminal devices 801, 802, and 803. The backend management server may analyze and process the received data such as product information query requests, and feed back the processing results (such as target push information, product information - only an example) to the terminal device.
[0151] It should be noted that the facial image acne grading model training method or facial image acne grading method provided in the embodiments of the present invention is generally executed by the server 805 , and accordingly, the facial image acne grading model training device or facial image acne grading device is generally set in the server 805 .
[0152] It should be understood that Figure 8 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to the implementation requirements.
[0153] Reference below Fig. 9 , which shows a schematic diagram of the structure of a computer system 900 of a terminal device suitable for implementing an embodiment of the present invention. Fig. 9 The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.
[0154] like Fig. 9 As shown, the computer system 900 includes a central processing unit (CPU) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage part 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the system 900 are also stored. The CPU 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.
[0155] The following components are connected to the I / O interface 905: an input section 906 including a keyboard, a mouse, etc.; an output section 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, a modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed, so that a computer program read therefrom is installed into the storage section 908 as needed.
[0156] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 909, and / or installed from the removable medium 911. When the computer program is executed by the central processing unit (CPU) 901, the above-mentioned functions defined in the system of the present invention are executed.
[0157] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.
[0158] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0159] The modules involved in the embodiments of the present invention may be implemented by software or hardware. The modules described may also be set in a processor, for example, it may be described as: a processor includes a first acquisition module and a training module. The names of these modules do not constitute a limitation on the modules themselves in some cases, for example, the first acquisition module may also be described as "acquiring multiple sample facial images and annotation data of each sample facial image".
[0160] As another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes: obtaining multiple sample facial images and annotation data of each sample facial image, the sample facial images are annotated with the category and position of the acne detection frame, and the annotation data includes the classification of the acne detection frame; according to the multiple sample facial images and the annotation data of each sample facial image, a facial image acne classification model is trained.
[0161] According to the technical solution of the embodiment of the present invention, the training method of the facial image acne grading model uses sample facial images with the category and position of the acne detection frame marked and the annotation data of the sample facial images to train the facial image acne grading model. The model training method uses a deep learning algorithm to solve the problem of low generalization performance. Through the category and position of the acne detection frame in the sample facial image, it is possible to fully utilize the more fine-grained information in the facial image for model training, thereby optimizing the model grading effect and improving the accuracy and interpretability of the model prediction. Through the sample facial images with the category and position of the acne detection frame marked, and the multi-task loss function constructed by multiple grading standards, a facial image acne grading model that can predict the grading results of different grading standards can be obtained, the accuracy of the grading results predicted by the model is improved, and the model is interpretable.
[0162] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A method for training a facial image acne grading model, characterized in that: include: Acquire multiple sample facial images and annotation data of each of the sample facial images, wherein the sample facial images are annotated with the category and position of the acne detection frame, and the annotation data includes the classification of the acne detection frame; A facial image acne grading model is trained based on the multiple sample facial images and the annotated data of each of the sample facial images.
2. The training method according to claim 1, characterized in that: Get multiple sample face images, including: Inputting a plurality of original facial images into a target detection model to obtain a detection result of each of the original facial images, wherein the detection result includes a category and a position of an acne detection frame in the original facial image; According to the category and position of the acne detection frame in the original facial image, a corresponding acne detection frame is added to the original facial image to obtain a sample facial image corresponding to the original facial image.
3. The training method according to claim 2, characterized in that: The target detection model is obtained by the following method: Acquire multiple facial images and a labeling result of each of the facial images, wherein the labeling result includes a category and a position of an acne detection frame on the facial image; The target detection model is trained based on the multiple facial images and the annotation results of each facial image.
4. The training method according to claim 1, characterized in that: According to the plurality of sample facial images and the annotated data of each of the sample facial images, a facial image acne grading model is trained, including: Using a convolutional neural network to extract image features of each of the sample facial images; Construct a loss function based on the preset grading criteria; The loss function is iteratively trained according to the image features and annotation data of each sample facial image to obtain the facial image acne grading model.
5. The training method according to claim 4, characterized in that: The preset classification standard includes a first classification standard and a second classification standard, and constructing a loss function based on the preset classification standard includes: Constructing a first grading loss function based on the first grading standard; Constructing a second grading loss function based on the second grading standard; The loss function is constructed according to the first hierarchical loss function and the second hierarchical loss function.
6. A facial image acne grading method, characterized in that: include: Obtain a facial image to be detected; Determining the category and position of the acne detection frame in the facial image to be detected; adding a corresponding acne detection frame on the facial image to be detected according to the category and position of the acne detection frame to update the facial image to be detected; Inputting the updated facial image to be detected into the facial image acne grading model to obtain an acne grading result corresponding to the facial image to be detected; Wherein, the facial image acne grading model is obtained according to any one of the methods of claims 1-5.
7. The method according to claim 6, characterized in that Determining the category and position of the acne detection frame in the facial image to be detected includes: Determine the category and position of the acne detection frame in the facial image to be detected by using a target detection model; The target detection model is obtained by the following method: Acquire multiple facial images and a labeling result of each of the facial images, wherein the labeling result includes a category and a position of an acne detection frame on the facial image; The target detection model is trained based on the multiple facial images and the annotation results of each facial image.
8. A training device for a facial image acne grading model, characterized in that: include: A first acquisition module acquires a plurality of sample facial images and annotation data of each of the sample facial images, wherein the sample facial images are annotated with the category and position of the acne detection frame, and the annotation data is the classification of the acne detection frame; The training module is configured to train a facial image acne grading model based on the multiple sample facial images and the annotation data of each sample facial image.
9. The training device according to claim 8, characterized in that The first acquisition module is further used for: Inputting a plurality of original facial images into a target detection model to obtain a detection result of each of the original facial images, wherein the detection result includes a category and a position of an acne detection frame in the original facial image; According to the category and position of the acne detection frame in the original facial image, a corresponding acne detection frame is added to the original facial image to obtain a sample facial image corresponding to the original facial image.
10. The training device according to claim 9, characterized in that The target detection model is obtained by the following method: Acquire multiple facial images and a labeling result of each of the facial images, wherein the labeling result includes a category and a position of an acne detection frame on the facial image; The target detection model is trained based on the multiple facial images and the annotation results of each facial image.
11. The training device according to claim 8, characterized in that The training module is further used to: Using a convolutional neural network to extract image features of each of the sample facial images; Construct a loss function based on the preset grading criteria; The loss function is iteratively trained according to the image features and annotation data of each sample facial image to obtain the facial image acne grading model.
12. The training device according to claim 11, characterized in that The preset classification standard includes a first classification standard and a second classification standard, and the training module is further used to: Constructing a first grading loss function based on the first grading standard; Constructing a second grading loss function based on the second grading standard; The loss function is constructed according to the first hierarchical loss function and the second hierarchical loss function.
13. A facial image acne grading device, characterized in that: include: The second acquisition module acquires the facial image to be detected; A first determination module determines the category and position of the acne detection frame in the facial image to be detected; An updating module, which adds a corresponding acne detection frame to the facial image to be detected according to the category and position of the acne detection frame, so as to update the facial image to be detected; A second determination module inputs the updated facial image to be detected into the facial image acne grading model to obtain an acne grading result corresponding to the facial image to be detected; Wherein, the facial image acne grading model is obtained according to the method according to any one of claims 1-5.
14. The device according to claim 13, characterized in that The first determining module is further used to: Determine the category and position of the acne detection frame in the facial image to be detected by using a target detection model; The target detection model is obtained by the following method: Acquire multiple facial images and a labeling result of each of the facial images, wherein the labeling result includes a category and a position of an acne detection frame on the facial image; The target detection model is trained based on the multiple facial images and the annotation results of each facial image.
15. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.
16. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.
