Chloasma severity assessment method based on neural network

Abstract

At present, diagnosis of the severity of chloasma mainly depends on experience judgment of professional doctors and is related to actual experience of the doctors, and evaluation results often have high subjectivity and are not accurate enough. The invention provides a chloasma severity assessment method based on a neural network, and the method comprises the steps: inputting a face image, firstly, detecting key points of all parts of a face through a face key point detection model, cutting out an area needing the chloasma severity analysis according to the key point information, and carrying out the chloasma severity assessment. According to the method, a chloasma area is cut, such as a left side cheek area, a right side cheek area, a forehead area or a lower jaw area, then the cut area image is loaded into a chloasma severity assessment model to obtain an assessment value, the chloasma area severity is scientifically, objectively and accurately assessed, the whole assessment process is automatically completed by a computer, manual intervention is not needed, and the method is efficient and reliable. And accurate auxiliary information can be provided for subsequent diagnosis and treatment of chloasma.

Description

Technical Field

[0001] The present invention relates to a method for evaluating the severity of melasma based on a neural network, belonging to the field of medical image processing. Background Art

[0002] In recent years, with the improvement of living standards, people have paid more and more attention to their appearance and skin condition, and the daily investment in nursing and maintenance has also shown an explosive growth trend year by year. However, due to reasons such as excessive skin care, improper use of cosmetics, and improper medical aesthetics, melasma, which was originally only common on the faces of middle-aged people, has become younger. Regarding the diagnosis of the severity of melasma, it mainly relies on the experience judgment of professional doctors. It is related to the actual experience of doctors, and the evaluation results are often highly subjective and inaccurate.

[0003] It is found in practice that currently, the evaluation of the severity of melasma mainly adopts an image processing method, calculating the color difference between the melasma area and the normal area to obtain the severity score. For example, the Chinese patent application number CN202110539513.X, with a publication date of June 15, 2021, discloses a method, system, and device for identifying melasma images based on deep learning. The binarized mask image obtained based on the melasma detection and segmentation model is dilated iteratively at least three times and subtracted from the original mask image as the mask of the normal skin area. The CIED2000 color difference calculation formula, which is the chromaticity calculation model for the melasma area, is used to calculate the pixel average value of the two mask areas in the Lab color space, and the result obtained is the evaluation score of the severity of the melasma area. The accuracy of this method depends on the recognition accuracy of the melasma detection and segmentation model. Since the size of the segmented area of the skin spots directly affects the calculation result of the color difference between the positive and abnormal areas, at the same time, for patients with more than 90% of the skin spots on the face, the method for obtaining the normal area is not very applicable, which has certain limitations. Summary of the Invention

[0004] The present invention proposes a method for evaluating the severity of melasma based on a neural network. By inputting a face image, first, a face key point detection model is used to detect the key points of each part of the face. According to the key point information, the area to be analyzed for the severity of melasma is cropped, such as the left cheek area, the right cheek area, the forehead area, or the jaw area. Then, the cropped area image is loaded into the melasma severity evaluation model to obtain an evaluation value.

[0005] (1) Face Key Point Detection Model The face key-point detection model is used to predict the coordinate information of key points such as eyebrows, eyes, nose, mouth, and face contour of a face. After post-processing, the key-point coordinates are selectively combined and connected, and then a binary mask image of the right cheek, left cheek, frontal forehead, and mandibular region can be obtained to achieve an accurate division of the face region. The face key-point detection model used in the present invention is an open-source lightweight face key-point detection model, such as the commonly used DAN-Deep Alignment Network face key-point detection model, MediaPipe 2D and 3D face and key-point detection models, InsightFace face and key-point detection models, dlib face and key-point detection algorithms, etc.

[0006] (2)Melasma area severity assessment model The present invention uses a classification model to evaluate the chroma of melasma. The model training, validation, and test data sets are labeled by professional dermatologists according to the 4 types of scoring indicators specified in the internationally commonly used modified MASI scoring calculation formula. Regarding the selection of the model, there are many classification algorithms currently, such as ResNet, ResNext, EfficientNet, Vit, and Swin Transformer, etc., which are relatively popular. Any one of the algorithms can be selected for the training of the melasma area severity assessment. The data set is divided into a training set, a test set, and a validation set according to the ratio of 7:2:1. The input size of the model is 224x224x3, and the number of model training categories is 4. By adding data augmentation methods such as horizontal flipping, random cropping, random blurring, and random exposure, setting the corresponding learning rate, number of training epochs, optimizer, and learning rate schedule, etc., the training of the melasma area severity assessment model is started. During the validation process, the category with the maximum output probability of the model is taken as the severity assessment value, and the model with the best performance on the validation set is retained for each round of training. The category with the maximum probability is taken as the severity assessment value to perform a performance test on the trained model. If the performance does not meet the requirements, then analyze the test results, adjust the training strategy and model parameters, and continue to train the model until a model with the melasma area severity assessment accuracy meeting the expectations is trained.

[0007] The severity scoring rules in the modified MASI medical standard scoring are shown in Table 1 as follows: Brief Description of the Drawings

[0008] Figure 1 It is a flowchart of the method for evaluating the severity of melasma in the present invention; Embodiment

[0009] The present invention will be further described below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and not to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms also fall within the scope defined by the appended claims of this application.

[0010] A method for evaluating the severity of melasma based on a neural network disclosed in this embodiment specifically includes the following steps:

[0011] Step 1, input a face image

[0012] Step 2, first detect the key points of each part of the face through a face key point detection model

[0013] Step 3, crop the area that needs to be analyzed for the severity of melasma according to the key point information, such as the left cheek area, the right cheek area, the forehead area or the jaw area

[0014] Step 4, load the cropped area image into the melasma severity evaluation model to obtain an evaluation value.

Claims

1. A method for evaluating the severity of melasma based on a neural network, characterized in that, Specifically, it includes the following steps: Step 1: Input a face image Step 2: First, detect the key points of each part of the face through a face key point detection model Step 3: According to the key point information, crop the area that needs to be analyzed for the severity of melasma, such as the left cheek area, the right cheek area, the forehead area, or the jaw area Step 4: Load the cropped area image into the melasma severity assessment model to obtain an assessment value.

2. The method for evaluating the severity of melasma based on a neural network according to claim 1, wherein It includes the following models: (1) Face key point detection model: The face key point detection model is used to predict the coordinate information of key points such as eyebrows, eyes, nose, mouth, and face contour of the face. After post-processing, the key point coordinates are selectively combined and connected, and a binary mask image of the right cheek, left cheek, frontal forehead, and jaw areas can be obtained to achieve an accurate division of the face area. The face key point detection model used in the present invention is an open-source lightweight face key point detection model. More commonly used ones are, for example, the DAN-Deep Alignment Network face key point detection model, the MediaPipe 2D and 3D face and key point detection models, the InsightFace face and key point detection model, the dlib face and key point detection algorithm, etc. (2) Melasma area severity assessment model: The present invention uses a classification model to achieve the assessment of the chroma of melasma. The model training, validation, and test data sets are labeled by professional dermatologists according to the 4 types of scoring indicators specified in the internationally commonly used modified MASI scoring calculation formula. Regarding the selection of the model, there are currently many classification algorithms such as ResNet, ResNext, EfficientNet, Vit, and SwinTransformer that are more popular. Any one of the algorithms can be selected for the training of the melasma area severity assessment. The data set is divided into a training set, a test set, and a validation set according to a ratio of 7:2:

1. The input size of the model is 224x224x3, and the number of model training categories is 4. By adding data augmentation methods such as horizontal flipping, random cropping, random blurring, and random exposure, setting corresponding learning rates, number of training rounds, optimizers, and learning rate schedules, etc., start the training of the melasma area severity assessment model. During the validation process, take the category with the highest output probability of the model as the severity assessment value, and retain the model with the best performance on the validation set for each round of training. Take the category with the highest probability as the severity assessment value to perform a performance test on the trained model. If the performance does not meet the requirements, analyze the test results, adjust the training strategy and model parameters, and continue to train the model until a model with the melasma area severity assessment accuracy meeting the expectations is trained.

3. The method for evaluating the severity of melasma based on a neural network according to claim 2, wherein, It includes the severity scoring rules in the modified MASI medical standard score, as shown in the table:

4. The method for evaluating the severity of melasma based on a neural network according to claim 1, characterized in that, The evaluation process is automatically completed by the computer without manual intervention, which is efficient and reliable.

Images