Skin typing method for small sample data autonomous learning

Abstract

The application discloses a skin type prediction method for small sample data autonomous learning, and relates to the technical fields of deep learning and medical beauty skin type. The initial model is obtained by a convolutional neural network through a small number of labeled samples, and the subsequent new samples are accepted through the initial model, and whether the new samples have labeling value is judged according to the prediction result of the model, so that only the samples determined to be inaccurate by the model are selected for labeling and retraining. Compared with the existing skin type method, a single model can be used to obtain a more perfect result, the model is continuously iteratively optimized by using the enhanced unit in the actual use process, and the idea of active learning is used to select the samples with greater uncertainty for labeling, so that the number of samples needing expert labeling can be greatly reduced, and the labor cost is reduced. According to the reliability module, when the samples are expanded, the skin type with insufficient reliability is preferred, so that the final typing effect is improved.

Description

Technical Field

[0001] This invention relates to the fields of deep learning and medical aesthetic skin typing technology, specifically to a skin typing method that autonomously learns from small sample data. Background Technology

[0002] In recent years, with the improvement of living standards and quality of life, people have paid more and more attention to their personal image and appearance. As a result, skin care has flourished, which has also brought about the continuous prosperity of the cosmetics and medical aesthetics industries. While people's awareness of skin care has increased, the abuse of skin care products has also occurred. Different skin types generally require different care methods and skin care products. Only by choosing the right skin care products for the skin type can the skin truly achieve the effect of beauty and whitening. Therefore, skin type classification has a great demand in the field of medical aesthetics.

[0003] Currently, skin typing methods mainly rely on manual identification and skin texture analyzers. Manual identification is extremely inefficient and wastes manpower, but it is generally quite accurate. Skin texture analyzers are more efficient. Simple skin texture analyzers, such as moisture test pens, have fewer functions and mainly determine skin hydration based on the difference in conductivity caused by different skin moisture levels. More complex instruments, such as the Visia skin analyzer and the DermavisionPRO skin analysis tool, mainly include some small sensors and high-definition cameras. Although they have a certain degree of accuracy, they mainly use high-definition images of the skin, such as 4K ultra-high-definition images, and apply traditional digital image processing methods to type the skin.

[0004] The emergence of machine learning has greatly promoted the development of image recognition and multi-classification applications, and has also brought new methods to the field of skin typing. However, the number of applications is relatively small, and the following problems exist:

[0005] 1. Existing deep learning-based skin classification methods mainly focus on a single detail for binary classification, such as whether the skin is oily, but cannot achieve multi-label classification, such as dry-sensitive skin;

[0006] 2. Skin samples are difficult to obtain. Existing skin typing schemes require a large number of samples at the initial stage of modeling. However, the current level of skin sample data collection is not high, and the number of samples is very limited. Existing schemes are not feasible for skin typing on small samples.

[0007] 3. Existing solutions require labeling a large number of samples before training the model. This process consumes a lot of manpower and resources, and manual operation may result in incorrect labeling. Summary of the Invention

[0008] The purpose of this invention is to provide a skin subtyping prediction method and system for autonomous learning based on small sample data. An initial model is obtained by training a convolutional neural network with a small number of labeled samples. The initial model is used to accept subsequent new samples, and the model's prediction results are used to determine whether the new samples have labeling value. Thus, only samples that the model judges inaccurately are selected for labeling and retraining, which can greatly reduce the workload while improving the model's prediction accuracy.

[0009] To achieve the above-mentioned technical objectives and effects, the present invention is implemented through the following technical solution:

[0010] A skin subtyping prediction method for autonomous learning based on small sample data is proposed. An initial model is obtained by training a convolutional neural network with a small number of labeled samples. The initial model is used to accept subsequent new samples, and the model's prediction results are used to determine whether the new samples have labeling value. Only samples that the model judges inaccurately are selected for labeling and retraining.

[0011] A skin subtyping prediction method based on autonomous learning for small sample data includes the following steps:

[0012] S1: Input a small number of labeled samples, preset the reliability of each subtype label, and use the labeled data to build an initial classification model with multiple output layers;

[0013] S2: Input a new unlabeled sample, use the model to predict, update the reliability module based on the prediction results, and label and classify the sample at the same time.

[0014] S3: If the input sample has an uncertain label, it enters the manual labeling queue to determine whether the model has reached the retraining threshold. If it meets the retraining threshold, it expands the sample according to a certain rule based on the labeled sample and the reliability module and enters S4. Otherwise, it determines whether the reliability module meets the final model standard. If it does not meet the standard, it enters S2. If it does meet the standard, it enters S5.

[0015] S4: Retrain the model by combining historical training samples and newly labeled expanded samples. After the model is trained, proceed to S2.

[0016] S5: The model's optimization channel is closed, and updates will no longer be performed. The model can be copied for mass production and application.

[0017] Furthermore, S1 specifically includes:

[0018] S101: Select a small number of skin sample images and manually label the samples for skin subtyping;

[0019] S102: Initialize the reliability assessment module, in which several mutually exclusive skin types, namely dry, oily, normal and combination, are group A, and the remaining skin types with independent probabilities, namely sensitive, aging and pigmentation, are group B, C and D respectively. Each group has an independent reliability and an initial value of 0.

[0020] S103: Standardize the format and size of the labeled images, and use geometric augmentation to increase the number of samples to three times the original size, and divide the images into training and test sets.

[0021] S104: Based on the VGG16 deep neural network structure, the dataset divided in S103 is used for modeling, and multiple output layers are set. Output layer O1 is the result of classifying several mutually exclusive skin types in group A, and output layer O2 is the result of classifying several skin types with independent probabilities in groups B, C, and D.

[0022] Furthermore, S2 specifically includes:

[0023] S201: Input new samples into the model in batches. These samples can come from pre-reserved or clinically used image resources. Calculate the probability of each attribute of the sample.

[0024] S202: Update the reliability assessment group according to the calculated attributes: If all attributes of O1 are below 0.5, it is considered unreliable, and the number of unreliable samples in group A increases by 1; if the results of several attributes of O2 are between 0.35 and 0.65, the reliability of that attribute is considered unreliable, and the number of unreliable samples in its corresponding group increases by 1; otherwise, the number of reliable samples increases by 1. Finally, calculate the reliability of its group based on the number of unreliable samples.

[0025] If all attributes of O1 are below 0.5, it means that the predictions of several mutually exclusive attributes did not yield high-confidence answers, and therefore the result is considered unreliable. Similarly, the predicted values ​​for the several probabilistically independent attributes of O2 are in the range of 0.5 ± 0.15, indicating that the result is ambiguous and is also considered unreliable. Subsequently, the reliability scores of groups A, B, C, and D will be updated respectively, calculated using the following formula:

[0026]

[0027] Where m is the minimum number of samples required for reliability assessment; for example, m=1000 means that at least 1000 most recently predicted samples must be used to calculate reliability; r i R represents the reliability of sample i. i ∈{0,1}, where 1 represents that sample i is reliable and 0 represents otherwise; n is the total number of samples that have been predicted; R is the reliability of this group;

[0028] From the above formula combined with the actual situation, it can be seen that in the initial application, the situation of n < m will occur. Therefore, the minimum value of n is set to m, and the missing samples are directly set to r i to be 0, which means that if the number of predicted samples far fails to meet the most basic requirements, the reliability is insufficient;

[0029] S203: If the result of the classification is reliable, assign the corresponding classification label to the sample, otherwise assign an uncertain label to the classification of the sample;

[0030] Furthermore, the said S3 specifically includes:

[0031] S301: If the sample passed in by S2 has an "uncertain" label, enter the manual annotation queue and wait for the domain expert to annotate;

[0032] S302: If the number of annotated samples meets the threshold for retraining, that is, the number of samples that have been manually annotated for the current model reaches the value N, and the reliability module does not meet the standard, enter S303, otherwise enter S304;

[0033] S303: According to the reliability of groups A, B, C, and D, use the geometric enhancement method to expand the number of newly annotated samples, and then enter step S4;

[0034] For the expansion of the number of new samples, first use the geometric enhancement method to expand the number of new samples to 3 times the original data. On this basis, according to the reliability ratio of each group, expand the samples additionally. For example, if the reliability ratio of A, B, C, and D is a:b:c:d, then the additional expansion multiple of group A is

[0035] where M is the additional expansion coefficient, and the value is {1, 2, 3}, which means an additional expansion of M times.

[0036] S304: If the reliability module meets the final model standard, enter S5, otherwise enter S2 to continue prediction.

[0037] Furthermore, the said S4 specifically includes:

[0038] S401: Integrate the historical sample data and the data after the expansion of new samples, and re-divide the training set and the test set;

[0039] S402: Retrain the model in the same way as training the model in S1. After the training is completed, enter S2;

[0040] Furthermore, the said S5 includes: The model closes the optimization channel and no longer updates, and the model can be copied for batch production and application.<​Another objective of this invention is to provide a prediction system based on a skin typing prediction method that autonomously learns from small sample data;

[0042] The prediction system includes: a model prototype unit, a model enhancement unit, and a final application unit;

[0043] The prototype unit of the model preprocesses a small amount of skin image sample data to realize a prediction model based on convolutional neural networks to perform multi-classification and multi-label classification of skin images, and at the same time establishes a reliability evaluation module for various classifications.

[0044] The model enhancement unit applies the preliminary model generated by the prototype unit and updates the reliability evaluation module. Specifically, it inputs a new skin image for prediction, updates the reliability based on the classification probability, manually labels samples that meet the labeling conditions, then retrains using historically labeled samples, and continuously iterates the process of this unit.

[0045] The final application unit is a model that, after the enhancement unit has iterated to a certain extent, no longer requires manual sample labeling and can be directly generalized and applied in batches. The input is a skin image, and the output is the skin type.

[0046] The beneficial effects of this invention are:

[0047] (1) In real-world applications, collecting skin sample data is very difficult, and the number of samples collected is generally very limited. Compared with existing skin subtyping modeling methods, this invention only requires a small number of labeled samples to obtain an initial model through convolutional neural network training, which can reduce the amount of initial samples required.

[0048] (2) This invention has multiple output layers, which can achieve the effect of multiple labels and multiple classifications. It is more comprehensive and accurate than the existing skin classification results that only have a single label, and solves the shortcomings of the existing skin classification methods that can only perform single classification (i.e., binary classification).

[0049] (3) After obtaining the initial model, this invention can receive subsequent new samples and determine whether the new samples have labeling value based on the model's prediction results. Thus, it only selects samples that the model judges inaccurately for labeling and retraining, which can greatly reduce the workload while improving the model's classification accuracy. In contrast, traditional skin typing models require a lot of manpower to manually label the collected image data during construction.

[0050] (4) This invention has the ability to absorb new knowledge while retaining and optimizing old knowledge. Based on the idea of ​​incremental learning, it integrates historical labeled samples and new labeled samples for retraining, thereby improving the performance of the model. Existing skin subtyping models are no longer modified or updated once they are initialized and trained. However, this invention can improve the level of the model up to a certain threshold in the continuous arrival of new samples. It has the ability to absorb new knowledge while retaining and optimizing old knowledge. It can strengthen the training effect of the model through continuous iterative training and improve the accuracy of subtyping prediction.

[0051] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0052] Figure 1 This is a flowchart illustrating the skin subtyping method for autonomous learning based on small sample data as described in an embodiment of the present invention.

[0053] Figure 2 This is a schematic diagram of the autonomous learning method of the skin typing method for autonomous learning based on small sample data as described in an embodiment of the present invention;

[0054] Figure 3 This is a schematic diagram of the prediction system structure of the skin subtyping method based on autonomous learning for small sample data as described in an embodiment of the present invention; Detailed Implementation

[0055] To more clearly illustrate the technical solutions of the embodiments of the present invention, the present invention will be described in detail below with reference to the accompanying drawings.

[0056] A skin subtyping prediction method for autonomous learning based on small sample data is proposed. An initial model is obtained by training a convolutional neural network with a small number of labeled samples. The initial model is used to accept subsequent new samples, and the model's prediction results are used to determine whether the new samples have labeling value. Only samples that the model judges inaccurately are selected for labeling and retraining.

[0057] The present invention will now be described in conjunction with specific embodiments:

[0058] Example 1

[0059] like Figure 1 and Figure 2 As shown:

[0060] A skin subtyping prediction method based on autonomous learning for small sample data includes the following steps:

[0061] S1: Input a small number of labeled samples, preset the reliability of each subtype label, and use the labeled data to build an initial classification model with multiple output layers;

[0062] In this embodiment, 600-800 skin sample images are selected, as follows:

[0063] S101: Select 600 skin sample images and manually label the samples for skin subtyping;

[0064] In this embodiment, the modeling is based on the idea of ​​active learning, which means that the optimization of the model depends on subsequent samples rather than a large number of initial samples. Therefore, in the initial modeling, only a small number of manually labeled samples are needed to train the basic model, and this advantage is obvious.

[0065] S102: Initialize the reliability assessment module, in which several mutually exclusive skin types, namely dry, oily, normal and combination, are group A, and the remaining skin types with independent probabilities, namely sensitive, aging and pigmentation, are group B, C and D respectively. Each group has an independent reliability and an initial value of 0.

[0066] In this embodiment, the reliability module mainly provides a benchmark for subsequent retraining and sample expansion. If the reliability of a certain classification is low, the focus is on expanding the samples of that category so that the retrained model performs better.

[0067] S103: Standardize the format and size of the labeled images, and use geometric augmentation to increase the number of samples to three times the original size, and divide the images into training and test sets.

[0068] In this embodiment, the labeled images are uniformly converted to a 227×227×3 format, the size is normalized, and geometric enhancement methods such as translation, rotation, and shearing are used to increase the number of samples to 3 times the original data. The training set and test set are divided according to the 7:3 principle.

[0069] In this embodiment, the unified format and size normalization are performed according to the standard neural network training model steps for data preprocessing. Geometric augmentation to expand the sample size is to compensate for the problem of insufficient initial data by appropriately expanding the sample size.

[0070] S104: Based on the VGG16 deep neural network structure, the dataset divided in S103 is used for modeling, and multiple output layers are set. Output layer O1 is the result of classifying several mutually exclusive skin types in group A, and output layer O2 is the result of classifying several skin types with independent probabilities in groups B, C, and D.

[0071] In this embodiment, the Sigmoid and Softmax activation functions are used in combination. The Sigmoid function maps the results to a range of 0 to 1, with each of the three labels corresponding to a different mapping, resulting in predicted values ​​for sensitivity, aging, and pigmentation, thus achieving multi-label classification. In contrast, existing skin type classification models use the Softmax function in their output layer. This function normalizes a numerical vector into a probability distribution vector, with the sum of all probabilities being 1. Therefore, it can only distinguish the oiliness of the skin and cannot label the skin with sensitivity, aging, pigmentation, or other related characteristics.

[0072] In this embodiment, the input image undergoes two convolutional and max-pooling processes, followed by a third to fifth convolutional process. The result of the fifth convolution is then max-pooled, followed by a fully connected layer. Dropout is used to randomly discard 25% of the neurons, and the fully connected and Dropout operations are repeated once more, before a final fully connected layer is formed. Simultaneously, an output layer O1 is set, using the Softmax activation function to calculate the probabilities of several mutually exclusive skin types. The results are distributed between 0 and 1, and the sum of the results for these categories is 1, thus achieving mutually exclusive classification and obtaining predicted values ​​for dry, oily, neutral, and mixed skin types. An output layer O2 is set, using the Sigmoid function for activation, mapping the results to between 0 and 1. The three labels correspond to three different mappings, obtaining predicted values ​​for sensitivity, aging, and pigmentation types, achieving multi-label classification.

[0073] In this embodiment, the VGG16 deep neural network used in this solution contains 13 convolutional layers and 3 fully connected layers. It is a deep learning network widely used in the field of image recognition. Utilizing the basic structure of VGG16, its output layer is improved. The improvement lies in the following: After processing by the Softmax function, the calculation results of the four types of dryness, oiliness, neutrality, and mixedness will be between 0 and 1, and the sum of the results of these categories is 1, thus achieving the purpose of mutually exclusive classification; After processing by the Sigmoid function, the output layer O2 can map the results to between 0 and 1. The three labels of sensitivity, aging, and pigmentation correspond to three mappings respectively. This enables the model to perform multi-label classification. Compared with the existing single-output layer solution, this advantage is obvious.

[0074] In this embodiment: Existing skin typing models require a large number of labeled samples when they are built. This step selects only a small number of labeled samples for training and initializing the neural network, which reduces the difficulty of collecting samples. At the same time, the Softmax and Sigmoid functions are used to achieve two types of classification: multi-class and multi-label classification, which solves the problem of the single result of existing skin typing and enables more comprehensive typing of skin type.

[0075] S2: Input a new unlabeled sample, use the model to predict, update the reliability module based on the prediction results, and label and classify the sample at the same time.

[0076] In this embodiment, step S2 is... Figure 2 This is reflected in, specifically including:

[0077] S201: Input new samples into the model in batches and calculate the probability of each attribute;

[0078] S202: Update the reliability assessment group according to the calculated attributes: If all attributes of O1 are below 0.5, it is considered to have insufficient confidence, and the number of unreliable samples in group A increases by 1; if the results of several attributes of O2 are between 0.35 and 0.65, the confidence of that attribute is considered insufficient, and the number of unreliable samples in its corresponding group increases by 1; otherwise, the number of reliable samples increases by 1. Finally, calculate the reliability of its group based on the number of unreliable samples.

[0079] If all attributes of O1 are below 0.5, it means that the predictions of several mutually exclusive attributes did not yield high-confidence answers, and therefore the result is considered unreliable. Similarly, the predicted values ​​for the several probabilistically independent attributes of O2 are in the range of 0.5 ± 0.15, indicating that the result is ambiguous and is also considered unreliable. Subsequently, the reliability scores of groups A, B, C, and D will be updated respectively, calculated using the following formula:

[0080]

[0081] Where m is the minimum number of samples required for reliability assessment; for example, m=1000 means that at least 1000 most recently predicted samples must be used to calculate reliability; r i R represents the reliability of sample i. i ∈{0,1}, where 1 represents that sample i is reliable and 0 represents otherwise; n is the total number of samples that have been predicted; R is the reliability of this group;

[0082] The initial value of n is set to 1000, and the corresponding r i If the number of predicted samples is far less than the most basic requirement m, the reliability is insufficient.

[0083] S203: If the typing result is reliable, then assign the corresponding typing label to the sample; otherwise, assign an uncertain label to the sample for that typing.

[0084] In this embodiment, the screening of new samples with high uncertainty can be completed automatically, and the process can be iterated continuously to update the reliability module, making the skin typing prediction results more accurate.

[0085] S3: If the input sample has an uncertain label, it enters the manual labeling queue to determine whether the model has reached the retraining threshold. If it meets the retraining threshold, it expands the sample according to a certain rule based on the labeled sample and the reliability module and enters S4. Otherwise, it determines whether the confidence module meets the final model standard. If it does not meet the standard, it enters S2. If it does meet the standard, it enters S5.

[0086] In this embodiment:

[0087] S301: If the sample passed in by S2 has an "uncertain" label, it will enter the manual annotation queue and wait for domain experts to annotate it;

[0088] In this embodiment, the core of active learning is to acquire sample data that is relatively "difficult" to classify through machine learning methods, have it manually verified and reviewed, and then train the manually labeled data again using a supervised or semi-supervised learning model to gradually improve the model's predictive performance, thus integrating human experience into the machine learning model. Existing skin type classification modeling schemes require a large amount of skin type data and pre-labeled classification labels to train the model, meaning that experts in the medical aesthetics field need to spend a lot of time labeling. However, this invention, based on the principle of active learning, only when the input sample label in S2 is "uncertain" does it indicate that the model's prediction results for this part of the sample are not reliable enough, and the model has difficulty distinguishing the relevant labels of the sample data, requiring manual labeling before entering the expert labeling queue. This can greatly reduce the burden of expert labeling, obtain more valuable labeled data with less cost, and further improve the algorithm's performance. Therefore, in terms of the amount of labeled samples required, active learning is inherently superior to traditional deep learning, and this advantage is obvious.

[0089] S302: If the number of labeled samples meets the retraining threshold, that is, the number of manually labeled samples for the current model reaches the value N, and the reliability module does not meet the standard, then proceed to S303; otherwise, proceed to S304.

[0090] In this embodiment, the value N can be determined according to the actual situation. If the retraining threshold requirement is high and there are many sample resources, N can be set to 800. If the requirement is low, it can be set to 500. The standard range of reliability is (0,1), which is also determined according to the actual situation. This standard should not be too low, otherwise the final model performance will be poor, while too high may make the final model difficult to achieve.

[0091] S303: Based on the reliability of groups A, B, C, and D, the number of newly labeled samples is expanded using geometric augmentation, and then proceed to step S4;

[0092] S304: If the reliability module meets the final model standard, proceed to S5; otherwise, proceed to S2 to continue prediction.

[0093] In this embodiment, the specific steps are as follows:

[0094] After entering the manual annotation queue, it will wait for annotation. If the sample in the queue has not been annotated for more than 24 hours during the task start period, the person in charge of annotation will be notified.

[0095] If the retraining threshold is met, i.e., the number of manually labeled samples for the current model reaches 500 and the reliability of each group does not reach 0.85, then proceed to S303; otherwise proceed to S304.

[0096] Based on the reliability of groups A, B, C, and D, the number of newly labeled samples is expanded using geometric augmentation, and then proceeds to step S4;

[0097] Regarding the expansion of the new sample size, geometric augmentation is first used to expand the new sample size to three times the original data. Based on this, additional samples are expanded according to the reliability ratios of each group. Let the reliability ratios of A, B, C, and D be a:b:c:d, then the additional expansion factor for group A is: The same logic applies to the other groups:

[0098] Here, 8 represents the product of the additional expansion coefficient M=2 and the number of groups 4, which means an additional expansion of 2 times.

[0099] If the reliability of each group in the reliability module reaches 0.85 and the model passes manual random verification, proceed to S5; otherwise, proceed to S2 to continue prediction.

[0100] In this embodiment, the manual verification scheme is as follows: randomly select 100 samples that the model has predicted but are not labeled as uncertain, and manually verify them. If the error is less than 10%, the manual verification is passed.

[0101] In this embodiment, based on the idea of ​​active learning, only samples with labeling value are manually labeled. This can greatly reduce the cost of manual labeling and solve the problem that existing skin subtyping models require a lot of manpower to label samples during initial training. At the same time, when expanding new samples, the number of groups with insufficient reliability can be increased according to the reliability ratio, so that subsequent retraining of the model will focus on subtypings with insufficient reliability.

[0102] S4: Retrain the model by combining historical training samples and newly labeled expanded samples. After the model is trained, proceed to S2.

[0103] In this embodiment:

[0104] S401: Integrate historical sample data and data after expanding with new samples, and re-divide the training set and test set;

[0105] In this embodiment, the training set and the test set are randomly re-divided according to a 7:3 ratio;

[0106] S402: Retrain the model according to the training method in S1. After training is complete, proceed to S2.

[0107] In this embodiment, as time goes by, more data gradually expands the labeled samples, increasing the dataset available for training. Therefore, the model is retrained using this data, allowing it to be continuously updated before reaching the formation threshold. In contrast, existing skin classification models are formed after the initial training and do not undergo any subsequent changes. Therefore, compared to existing solutions, this invention has the ability to absorb new knowledge while retaining and optimizing old knowledge.

[0108] In this embodiment, based on the idea of ​​incremental learning, historical labeled samples and new labeled samples are fused together for retraining, thereby improving the performance of the model. Existing skin classification models are no longer modified or updated once they are initialized and trained, while this invention can improve the accuracy of the model's prediction results by continuously training on new samples.

[0109] S5: The model's optimization channel is closed, and updates will no longer be performed. The model can be copied for mass production and application.

[0110] In this embodiment:

[0111] The initialization model requires only 600 labeled samples, while existing solutions require approximately 10,000 samples, which is 16.67 times that of this invention. Therefore, the smaller initial sample requirement is one of the advantages of this solution. Furthermore, the model can output skin type (dry, oily, normal, combination) and whether the skin is sensitive, aging, or has pigmentation issues, while existing solutions can only distinguish oiliness levels. Based on active learning, this invention selects valuable samples for manual labeling through continuous practice. Existing solutions require manual labeling of 10,000 samples before initializing the model, consuming significant manpower. This invention requires only about 3,000 manual labeling operations to meet the final application standards, saving approximately 70% of the labeling cost compared to existing solutions.

[0112] Example 2

[0113] A prediction system based on the skin typing prediction method for autonomous learning oriented to small sample data as described in Example 1;

[0114] The prediction system includes: a model prototype unit, a model enhancement unit, and a final application unit;

[0115] The prototype unit of the model preprocesses a small amount of skin image sample data to realize a prediction model based on convolutional neural networks to perform multi-classification and multi-label classification of skin images, and at the same time establishes a reliability evaluation module for various classifications.

[0116] The model enhancement unit applies the preliminary model generated by the prototype unit and updates the reliability evaluation module. Specifically, it inputs a new skin image for prediction, updates the reliability based on the classification probability, manually labels samples that meet the labeling conditions, then retrains using historically labeled samples, and continuously iterates the process of this unit.

[0117] The final application unit is a model that, after the enhancement unit has iterated to a certain extent, no longer requires manual sample labeling and can be directly generalized and applied in batches. The input is a skin image, and the output is the skin type.

[0118] By implementing a prototype unit with a small number of samples, this invention requires significantly less initial data compared to existing skin classification models. Once implemented, the model can perform multi-classification and multi-label classification of skin images. Compared to existing skin classification methods, it can obtain more comprehensive results using only a single model. Subsequently, in practical use, the model is continuously iterated and optimized using augmentation units. At the same time, the idea of ​​active learning can be used to select samples with greater uncertainty for annotation. Compared to existing skin classification models, it can greatly reduce the number of samples required for expert annotation, reducing labor costs. During retraining, the reliability module can make the model more inclined to include classifications with insufficient reliability when expanding samples, thereby improving the final classification effect.

[0119] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A skin subtyping prediction method based on autonomous learning for small sample data, characterized in that: An initial model is obtained by training a convolutional neural network with a small number of labeled samples. The initial model is then used to accept new samples. The model's prediction results are used to determine whether the new samples have labeling value. Only samples that the model does not accurately identify are labeled and then retrained. Specifically, the following steps are included: S1: Input a small number of labeled samples, preset the reliability of each subtype label, and use the labeled data to build an initial classification model with multiple output layers; S2: Input a new unlabeled sample, use the model to predict, update the reliability module based on the prediction results, and label and classify the sample at the same time. S3: If the input sample has an uncertain label, it enters the manual labeling queue to determine whether the model has reached the retraining threshold. If it meets the retraining threshold, it expands the sample according to a certain rule based on the labeled sample and the reliability module and enters S4. Otherwise, it determines whether the reliability module meets the final model standard. If it does not meet the standard, it enters S2. If it does meet the standard, it enters S5. S4: Retrain the model by combining historical training samples and newly labeled expanded samples. After the model is trained, proceed to S2. S5: The model's optimization channel is closed, and updates will cease. The model can be copied for mass production and application. S1 specifically includes: S101: Select a small number of skin sample images and manually label the samples for skin subtyping; S102: Initialize the reliability assessment module, in which several mutually exclusive skin types, namely dry, oily, normal and combination, are group A, and the remaining skin types with independent probabilities, namely sensitive, aging and pigmentation, are group B, C and D respectively. Each group has an independent reliability and an initial value of 0. S103: Standardize the format and size of the labeled images, and use geometric augmentation to increase the number of samples to three times the original size, and divide the images into training and test sets. S104: Based on the VGG16 deep neural network structure, the dataset divided in S103 is used for modeling, and multiple output layers are set up. Output layer O1 is the result of classifying several mutually exclusive skin types in group A, and output layer O2 is the result of classifying several skin types with independent probabilities in groups B, C, and D. S2 specifically includes: S201: Input new samples into the model in batches. These samples can come from pre-reserved or clinically used image resources. Calculate the probability of each attribute of the sample. S202: Update the reliability assessment group according to the calculated attributes: If all attributes of O1 are below 0.5, it is considered to be unreliable, and the number of unreliable samples in group A increases by 1; if the results of several attributes of O2 are between 0.35 and 0.65, it is considered that the reliability of that attribute is unreliable, and the number of unreliable samples in its corresponding group increases by 1; otherwise, the number of reliable samples increases by 1; finally, calculate the reliability of its group based on the number of unreliable samples. If the values ​​of all attributes for O1 are below 0.5, it means that the predictions of the mutually exclusive attributes did not yield high-confidence answers, and therefore the results are considered unreliable. Similarly, the predicted values ​​of the independent attributes for O2 are in the range of 0.5 ± 0.15, indicating that the results are ambiguous and are also considered unreliable. Subsequently, the reliability scores for groups A, B, C, and D will be updated respectively, calculated using the following formula: Among them, m is the minimum number of samples required for reliability evaluation. When m is 1000, it means that at least the last 1000 predicted samples are used to calculate the reliability; r i represents the reliability of sample i, r i ∈{0, 1}, where 1 means that sample i is reliable and 0 means otherwise; n is the total number of all samples that have been predicted currently; R is the reliability of this group. From the above formula combined with the actual situation, it can be seen that in the initial application, the situation of n < m may occur. Therefore, the minimum value of n is set to m, and for the missing samples, their r i is set to 0, which means that if the number of predicted samples far fails to meet the most basic requirements, the reliability is insufficient. S203: If the typing result is reliable, then label the sample with the corresponding typing label; otherwise, label the sample with an uncertain label for that typing.

2. The skin subtyping prediction method for autonomous learning based on small sample data as described in claim 1, characterized in that, S3 specifically includes: S301: If the sample passed in by S2 has an "uncertain" label, it will enter the manual annotation queue and wait for domain experts to annotate it; S302: If the number of labeled samples meets the retraining threshold, that is, the number of manually labeled samples for the current model reaches the value N, and the reliability module does not meet the standard, then proceed to S303; otherwise, proceed to S304. S303: Based on the reliability of groups A, B, C, and D, the number of newly labeled samples is expanded using geometric augmentation, and then proceed to step S4; To expand the number of new samples, geometric augmentation was first used to expand the sample size to three times the original data. Based on this, additional samples were expanded according to the reliability ratios of each group. The reliability ratios of groups A, B, C, and D are a:b:c:d. Therefore, the additional expansion factor for group A is calculated as follows: Where M is the coefficient of extra expansion, taking values ​​of {1, 2, 3}, meaning an extra expansion of M times; S304: If the reliability module meets the final model standard, proceed to S5; otherwise, proceed to S2 to continue prediction.

3. The skin subtyping prediction method for autonomous learning based on small sample data as described in claim 1, characterized in that, S4 specifically includes: S401: Integrate historical sample data and data after expanding with new samples, and re-divide the training set and test set; S402: Retrain the model using the same method as in S1. After training, proceed to S2.

4. A prediction system based on the skin typing prediction method for autonomous learning of small sample data as described in any one of claims 1-3, characterized in that: The prediction system comprises: a model prototype unit, a model enhancement unit, and a final application unit. The model prototype unit preprocesses a small amount of skin image sample data to implement a convolutional neural network-based prediction model for multi-class and multi-label classification of skin images, while simultaneously establishing a reliability evaluation module for each classification. The model enhancement unit applies the preliminary model generated by the prototype unit and updates the reliability evaluation module. Specifically, it inputs a new skin image for prediction, updates the reliability based on the classification probability, manually labels samples that meet the labeling criteria, and then retrains using historically labeled samples, continuously iterating the process of this unit. The final application unit, after the model has iterated to a certain extent in the enhancement unit, no longer requires manual sample labeling and can be directly generalized and applied in batches. The input is a skin image, and the output is the skin type to which the skin belongs.

Images