Facial acne grading method based on collaborative mapping multi-scale self-distillation

Abstract

The invention discloses a facial acne grading method based on collaborative mapping multi-scale self-distillation, is applied to the field of computers, and aims to solve the problem of low accuracy of acne severity grading in the prior art. According to the method, the convolutional neural network model is trained by adopting a self-distillation architecture, and a novel distillation module optimization training process is provided in combination with the characteristics of acnes, so that the model can notice the uniqueness of acne images, and the severity grading precision of the model for facial acnes is improved. The method can automatically identify the acne features and grade the severity of the acne features, assists doctors in diagnosis and treatment, and accelerates the healing period of the acne.

Description

Technical Field

[0001] The invention belongs to the field of computers, and in particular relates to a facial acne severity grading technology. Background Art

[0002] Facial acne is a common skin disease that not only occurs in teenagers, but also affects many adults. According to statistics, about 80% of teenagers have experienced facial acne, and 3% of men and 12% of women have not been cured of facial acne even in adulthood. The scars left by facial acne will affect the patient's self-confidence and cause trouble to their normal life. Therefore, reasonable and appropriate diagnosis and treatment of facial acne is very necessary.

[0003] One of the key steps in the diagnosis and treatment of facial acne is to grade the severity of acne. Doctors will prescribe medicine based on the severity of acne and customize a personalized treatment plan. Therefore, accurate acne grading results can greatly accelerate the patient's acne cure cycle. In the past, the severity of acne was often graded manually by doctors, and the accuracy of the grading strongly depended on the doctor's experience. In addition, the grading task itself is complex and time-consuming, which easily consumes the doctor's experience and prolongs the diagnosis and treatment cycle.

[0004] Due to the above shortcomings, computer-assisted diagnosis and treatment of facial acne grading has become the current mainstream grading approach. This process requires experienced doctors to use their professional knowledge to evaluate the images one by one, and then make a diagnosis based on the grading results. This not only requires doctors to have solid knowledge of dermatology, but also involves a lot of manpower costs. The most important thing is that the professional ability of doctors directly affects the diagnosis effect of patients, and the number of excellent doctors is limited, which cannot meet the increasing needs of patients in a timely manner. Therefore, computer-assisted grading models came into being, using computer technology to assist doctors in grading acne. This model can help doctors make more accurate judgments and develop more appropriate treatment plans to a certain extent. The self-distillation architecture can further improve the grading accuracy of the model through self-distillation, but the existing self-distillation model does not take into account the particularity of acne images, so the performance improvement is limited.

[0005] Existing acne auxiliary diagnosis and treatment technologies mainly rely on Convolutional Neural Networks (CNN) to extract features from acne images and use the extracted features for grade evaluation. However, in most cases, due to limitations such as model size, data size, and processing time, a single naive convolutional neural network model (such as ResNet) often has difficulty in achieving satisfactory results while processing quickly.

[0006] The knowledge distillation architecture is mainly used to solve the contradiction between speed and effect mentioned above. It uses a large model with good performance and large parameters as the teacher model, and passes its predictions or intermediate features as knowledge to a small model with small parameters to guide the training of the small model, so as to achieve the same excellent effect as the large model on the faster small model. However, such an architecture will bring additional computing overhead and increase storage costs. At the same time, different teacher-student combinations will also affect the final effect and make training inconvenient.

[0007] The self-distillation architecture is based on knowledge distillation and discards the additional teacher model. It only has a single model, that is, the model plays the role of both teacher and student. Compared with traditional knowledge distillation, self-distillation not only reduces the training cost, but also greatly speeds up the training speed. It can adapt to different data sets without the need to train a teacher model for each specific data set. Although the self-distillation architecture is also a single network model, it distills itself by adding bypasses and other means to extract image features at a deeper level. Therefore, the self-distillation architecture often performs better than the naive CNN.

[0008] When performing facial acne severity grading tasks, the only drawback is that the existing self-distillation architecture does not consider the following unique problems of acne images: 1. The features of adjacent classes of acne images are similar; 2. The features of acne images are unevenly distributed. Therefore, the current self-distillation method cannot fully extract and learn features of acne images, which affects the overall grading performance of the network for acne images.

[0009] Accurate grading of facial acne can provide strong guidance for doctors to prescribe treatment plans, and the computer-assisted automatic acne grading algorithm based on deep learning can greatly facilitate doctors' diagnosis and treatment, and can prevent the misjudgment of the severity of acne to the greatest extent. However, both the machine learning method based on manual feature extractors and the widely used naive convolutional neural network method are difficult to achieve satisfactory results. At the same time, due to the following characteristics of acne images: 1. Neighboring similarity of acne images. Acne images with similar severity levels often show great similarity. Such similarity will confuse the model, weaken its learning level, affect the training of the model, and lead to inaccurate prediction of the model; 2. Uneven distribution of features of acne images. The location of acne is uncertain, and the uncertainty of the location will lead to uneven distribution of features. When the acne distribution is relatively scattered, the model will focus on the features of a certain part of the image and ignore other parts, resulting in misjudgment of the severity of the acne image. Due to the above characteristics, the learning efficiency of the model is often greatly affected when using the self-distillation model for the task of acne grading, and the result is that the model performs poorly on acne images.Figure 1 Images of acne of varying severity are shown.

[0010] In summary, the shortcomings of the existing methods and the solutions of the method of the present invention are:

[0011] 1) The features extracted by the manual feature extractor are shallow features with poor effects;

[0012] 2) The traditional CNN model does not learn the extracted features well enough, and the model cannot fully learn from the extracted features, resulting in poor final classification effect;

[0013] 3) Other methods ignore the similarity of acne images, resulting in the model not having a specific module to distinguish acne images of adjacent severity levels, which weakens its learning of similar acne images of different severity levels;

[0014] 4) Other methods ignore the uneven distribution of features of acne images, resulting in the model focusing on only one or two areas when acne is dispersed, ignoring other areas, which weakens the model's learning of acne images. Summary of the invention

[0015] To solve the above technical problems, the present invention proposes a facial acne grading method based on collaborative mapping multi-scale self-distillation, and designs a series of novel feature extraction modules, so that the model can better perceive the feature areas that have a greater impact on the final grading results, thereby improving the prediction performance of the model.

[0016] The technical solution adopted by the present invention is: a facial acne grading method based on collaborative mapping multi-scale self-distillation, comprising:

[0017] S1, preprocessing the acne image data, and dividing the preprocessed acne image data into a training set and a test set;

[0018] S2. Construct a self-distillation model; the self-distillation model includes: a CNN module, a bottleneck module, a multi-mapper feature extraction module, a standardization module, a multi-scale feature capture module and 4 fully connected layers; the CNN module includes: a first CNN layer, a second CNN layer, a third CNN layer and a fourth CNN layer; the bottleneck module includes: a first bottleneck layer, a second bottleneck layer and a third bottleneck layer; the multi-mapper feature extraction module includes a first multi-mapper feature extraction layer, a second multi-mapper feature extraction layer and a third multi-mapper feature extraction layer; the standardization module includes a first feature standardization layer, a second feature standardization layer, a third feature standardization layer and a fourth feature standardization layer; the multi-scale feature capture module includes a first multi-scale feature capture layer, a second multi-scale feature capture layer, a third multi-scale feature capture layer and a fourth multi-scale feature capture layer; the fully connected layer module includes a first fully connected layer, a second fully connected layer

[0019] The output result of the first CNN module is used as the input of the second CNN module, the output result of the second CNN module is used as the input of the third CNN module, and the output result of the third CNN module is used as the input of the fourth CNN module;

[0020] The output result of the first CNN module is also used as the input of the first bottleneck module, the output result of the second CNN module is also used as the input of the second bottleneck module, and the output result of the third CNN module is also used as the input of the third bottleneck module; the output result of the first bottleneck module is used as the input of the first multi-mapper feature extraction module, the output result of the second bottleneck module is used as the input of the second multi-mapper feature extraction module, and the output result of the third bottleneck module is used as the input of the third multi-mapper feature extraction module; the output of the first multi-mapper feature extraction module is used as the input of the first feature standardization module, the output of the second multi-mapper feature extraction module is used as the input of the third feature standardization module, the output of the third multi-mapper feature extraction module is used as the input of the third feature standardization module, and the output result of the fourth CNN module is also used as the input of the fourth feature standardization module;

[0021] The output of the fourth feature normalization module is used as the label of the normalization layer, and the loss is calculated with the outputs of the first feature normalization module, the second feature normalization module, and the third feature normalization module respectively; and the loss of the first feature normalization module, the second feature normalization module, and the third feature normalization module are summed to obtain the loss of the normalization layer;

[0022] The output of the first feature standardization module is processed by the first fully connected layer as the input of the first multi-scale feature capture module, the output of the second feature standardization module is processed by the first fully connected layer as the input of the second multi-scale feature capture module, the output of the third feature standardization module is processed by the third fully connected layer as the input of the third multi-scale feature capture module, and the output of the fourth feature standardization module is processed by the fourth fully connected layer as the input of the fourth multi-scale feature capture module;

[0023] The output of the fourth multi-scale feature capture module is used as the label of the multi-scale feature capture layer, and the loss is calculated with the outputs of the first multi-scale feature capture module, the second multi-scale feature capture module, and the third multi-scale feature capture module respectively, and the losses of the first multi-scale feature capture module, the second multi-scale feature capture module, and the third multi-scale feature capture module are summed to obtain the loss of the multi-scale feature capture layer;

[0024] The outputs of the first multi-scale feature capture module, the second multi-scale feature capture module, the third multi-scale feature capture module, and the fourth multi-scale feature capture module are respectively processed by scale mean and then the corresponding prediction results are output;

[0025] The image label is calculated with the four prediction results respectively, and the loss of the image label is obtained by summing the losses of the four prediction results;

[0026] S3, using the training set in step S1 to train the self-distillation model constructed in step S2;

[0027] S4. Input the test set into the self-distillation model trained in step S3 to obtain the prediction result.

[0028] Beneficial effects of the present invention: The method of the present invention takes into account the feature similarity of adjacent acne. In previous acne grading models, acne was only treated as a natural image, ignoring its neighboring similar characteristics, resulting in unsatisfactory results of previous models. The present invention uses CNN as a framework, which can extract the features of acne images at a deeper level; the present invention also introduces a self-distillation structure based on CNN, which can enable the model to better learn image features; the method of the present invention uses a collaborative multi-mapping module to enable the model to make correct decisions when predicting stickiness to adjacent levels, so that the model learns correct knowledge; the method of the present invention divides the image features into different scales and trains the model at different scales, ensuring that the model can fully learn the image from the whole to the part; the method of the present invention has the following advantages:

[0029] 1. By jointly using multiple mappers to map acne features multiple times and summarizing all mapping results, the phenomenon of misjudgment of neighboring classes that is easy to occur in the case of a single mapper can be effectively prevented. Analysis shows that in the case of multiple mappers, it can always have a positive effect on the prediction deviation, promote the model to learn acne images with similar features, and enable them to better distinguish.

[0030] 2. The method of the present invention takes into account the uneven distribution of acne characteristics. Since the location and size of facial acne vary, when there are several areas with acne, the model tends to learn areas with more obvious acne characteristics, thereby ignoring the characteristic information of other areas and reducing the model's learning of samples. The method of the present invention splits the sample features according to different scales, so that the model can focus on the learning of each small area and can use each area with obvious characteristics at the same time, which strengthens the learning of the model and makes the model more sensitive to acne characteristics that can better affect the final grading results.

[0031] 3. Compared with previous distillation methods, this invention can significantly reduce training overhead. Traditional distillation methods need to be based on multiple models, and multiple models need to be trained simultaneously during training; this article only uses a single model, which greatly reduces the training speed and memory usage. At the same time, through the plug-and-play components proposed in this article, the performance of the model can be improved while ensuring the model training overhead. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 Examples of facial acne images with different severity levels;

[0033] Among them, (a) is a schematic diagram of level 1, (b) is a schematic diagram of level 2, (c) is a schematic diagram of level 3, and (d) is a schematic diagram of level 4.

[0034] Figure 2 The figure is a flow chart of the method of the present invention.

[0035] Figure 3 This is a model structure diagram of the present invention.

[0036] Figure 4 To visualize the results. DETAILED DESCRIPTION

[0037] To facilitate those skilled in the art to understand the technical content of the present invention, the present invention is further explained below with reference to the accompanying drawings.

[0038] The computer-aided acne grading method of the present invention comprises four parts: 1. data preparation, 2. model design, 3. model training, and 4. model testing. The overall structural relationship diagram is shown in FIG. Figure 2 As shown:

[0039] The four modules together constitute the overall structure of the method of the present invention, wherein each module also includes certain submodules, and the general functions of each large module are as follows:

[0040] 1. Data preparation

[0041] This part is mainly responsible for collecting acne images required for training the model, and pre-processing the data before starting model training to make the data meet the requirements of model training. After that, the collected acne images will be divided into training sets and test sets according to a certain ratio. The training set is used to train the model, and the test set is used to test the final performance of the model after training.

[0042] 11. Get data and labels

[0043] The facial acne datasets of the method of the present invention include two: one is the public dataset ACNE04, which includes 1457 facial acne images and their corresponding severity level labels; the other is the dataset ACNEHX collected by the cooperative hospital, which includes 1189 facial acne images and their corresponding severity level labels. The difference is that ANCE04 divides facial acne into 4 acne severity levels, and ACNEHX makes a more detailed division based on the 4 categories, dividing each level of ACNE04 into two levels, and dividing facial acne into 8 acne severity levels in total.

[0044] 12. Data preprocessing

[0045] Data preprocessing mainly involves preprocessing the image data in the dataset before training the model so that it can meet the requirements of model training. In the method of the present invention, all data in ACNE04 and ACNEHX are in 3D format (number of channels × image length × image width), and are adjusted to 3×224×224 before being put into the model.

[0046] 13. Dataset Division

[0047] The purpose of data set partitioning is to divide the data set into a training set and a test set according to a certain ratio, which are used for training models and testing models respectively. In the method of the present invention, ACNE04 and ACNEHX are both divided into training sets and test sets according to a ratio of 4:1.

[0048] 2. Model design:

[0049] This part is the key protected part of the present invention, including the self-distillation architecture design, the inter-class similarity discrimination module for dealing with the similarity problem between acne classes, and the dispersed feature recognition module for dealing with the dispersed problem of acne features. The actual training process includes how to encode acne images through the network, how to enhance the learning of the similarity between acne classes through multiple mappers to assist the model, assist the model to make accurate judgments, and how to capture and fuse small features of different scales to enable the model to learn dispersed features.

[0050] 21. Framework design

[0051] The framework of the entire model adopts a self-distillation learning framework. Compared with the traditional knowledge distillation framework, this framework only contains a single network, and the training process only relies on the data and the network itself without introducing additional external networks. Specifically, the last layer of the network will become the teacher of other layers to guide the learning of other layers. The guidance of the last layer to other layers is divided into guidance in the feature part and guidance in the prediction part. In the guidance of the feature part, the feature map of each layer except the last layer is expected to be aligned with the feature map of the last layer; in the guidance of the prediction part, each layer except the last layer is expected to learn the relationship between the predictions of the last layer rather than the specific values. In order to ensure the convergence of the training, the labels corresponding to the acne images will also be used to guide each layer at the same time, constituting the third part of the training objective of the network.

[0052] For dataset D, suppose there are N data, X represents the overall sample set, Y represents its corresponding label set, (x, y) represents the sample label batch, and the batch number for each training is represented as B. (x i ,y i) represents the i-th sample label pair, where i∈[1,B], y i ∈[0,V], V represents the total number of levels. The model design diagram of the entire method is as follows Figure 3 Shown

[0053] in, represents the kth mapper of the lth layer, Standardization represents feature standardization, patchK represents the prediction graph at different scales, and θ represents the network parameter weight. Then the optimization goal of the entire model is:

[0054] L model (X,θ,Y)=αL Pro +βL Pat +λL Cro

[0055] Where L Pro The distance between feature maps representing the combined results of multiple mappers, L Pat Indicates the prediction distance under the combination of multiple scales, L Cro Represents the distance between each layer's prediction and the actual label. α, β, and λ are hyperparameter weights introduced to balance the three losses.

[0056] 22. Multi-mapper collaborative discrimination

[0057] In the following, for simplicity of description, the present invention describes the output features of the last layer of the network as T, and the output features of the remaining layers as S l , l∈[1,L-1], where L is the total number of network layers. For each S l , the present invention will pass through an additional module, namely through Figure 3 The bottleneck layer shown in FIG. 1 extracts its features again. The present invention extracts S l The output feature after feature extraction is expressed as B(S l ), B(S l ) will pass through multiple mappers to perform multi-mapping of features, and take the average of the multi-mapping results as the final mapping result G l , that is

[0058]

[0059] Where K represents the number of mappers at each layer. The purpose of using multiple mappers is that, since there are similarities between adjacent levels of acne images, when the feature distribution mapped by a single mapper deviates from the correct expected feature distribution, the deviation often manifests itself as a gap between adjacent levels, and the deviation on adjacent levels will not be too large. Therefore, when multiple mappers are used, the deviation between the mapped feature distribution and the expected feature distribution can be smoothed through their collaborative mapping. When most mappers map correctly, the overall feature distribution can be mapped correctly; when most mappers map incorrectly, the overall feature distribution can be more inclined to the correct distribution than the feature distribution obtained by a single mapper.

[0060] Afterwards, G l Standardize the sample dimension to get the standardized feature distribution F l , G l The component in the i-th channel is recorded as Then F l It can be calculated as:

[0061]

[0062] Among them, ε is the disturbance parameter, which is a small positive value and is used to deal with the case where the standard deviation is 0, and C is the number of channels. T It represents the result after T is standardized. At this time, the loss used to measure the distance between each layer of the network feature map and the last layer of the feature map is:

[0063]

[0064] 23. Multi-scale feature capture

[0065] After standardizing the feature maps of each layer, the feature maps are first split according to different scales. l , its features at scale m can be calculated as:

[0066] D m =AAP m (F l )

[0067] where m∈2 [0,M] , M is the total number of split scales. AAP m Represents the adaptive pooling operation, which is responsible for aggregating neighboring features of the feature map and outputting a feature map of size m×m. Intuitively, AAP m The input image is split into m×m regions. The full-scale feature D of the lth layer is represented as a linked combination of features of different scales, that is:

[0068] D=Cat(Dm )

[0069] where m∈2 [0,M] , D∈R B×C×∑m After initially obtaining the combination of features of different scales, it is also necessary to obtain the grade scores of each grade according to these combined features. l , that is:

[0070] Pats l =Reshape(FC l (D),(B,V,∑ m ))

[0071] FC l represents the linear mapping layer of the lth layer, and Reshape represents the reshaping operation. l (m) represents the grade score corresponding to scale m, pats l (m) can be expressed as:

[0072]

[0073] Pats l (m)∈R B×V . Among them, pats l [:,:,:] means in pats l The standardized feature FT of the last layer can be obtained after the above processing. T , the prediction loss at scale m can be expressed as pats at all scales T with pats l The distance, that is:

[0074]

[0075] where l∈[1,L-1], m∈2 [0,M] , KL stands for KL divergence, which is used to measure the difference between two distributions. By dividing the feature map into different scales, the model can learn more finely by region. Compared with the traditional method of simply processing the entire input, it can effectively enable the model to perceive acne lesions in different locations and avoid focusing on one area and ignoring other areas.

[0076] 24. Multi-scale hierarchical aggregation

[0077] After getting the classification score pats of the lth layer l After that, the final prediction of the lth layer is the mean of all scale predictions, that is, the prediction calculation of the lth layer is:

[0078]

[0079] Where l∈[1,L], then pats T with pats L The meanings are the same, both are the classification scores of the last layer of the model, and I represents the accumulation operation on the corresponding prediction level. l For the level prediction of each layer for the sample, the present invention uses cross entropy loss to measure the distance between the prediction of each layer and the true label at this time, that is:

[0080]

[0081] This part of the loss is used to make the output of each layer as close to the true label as possible, while ensuring the stability of model training convergence.

[0082] (3) Model testing: This part is mainly responsible for testing the performance of the model, including testing the model on the divided test set and counting various performance indicators. In the present invention, the prediction probability of the Lth layer is used as the prediction probability of the model, and the prediction result of the model is obtained on this basis.

[0083] 31. Number of mappers

[0084] As mentioned above, the scenario in which multiple mappers are used is to reverse the prediction deviation of a single mapper by superimposing the mapping of multiple mappers to obtain the correct result. However, the number of mappers also needs to be carefully selected. When there are too few mappers, the model's tolerance for deviations will be weakened. When there are mapper mapping errors, the remaining insufficient mappers with correct mapping cannot effectively reverse the deviations. When there are too many mappers, the probability of mapping errors will increase, affecting the mapping results after superposition. Therefore, the number of mappers should be selected as an appropriate value to balance the above disadvantages of too many or too few mappers. In actual use, it is recommended to set the number of mappers to between [3,5].

[0085] 32. Number of split scales

[0086] The number of split scales can be regarded as the degree of refinement of the model's learning of the sample. When the split scale is 2, the original sample is split into 2×2, that is, 4 small parts, and then each of these small parts will be learned separately. However, it should be noted that the more split scales, the better. When the number of scales is too small, many fine features in small areas will be missing; at the same time, when the number of scales is too large, the areas will lose contact, causing the model to be unable to combine the features of adjacent areas, affecting the training of the model. Therefore, it is recommended to select different split scales according to the actual situation of the sample when using it. For acne images, it is recommended that the split scale should not exceed 8.

[0087] 33. Model hyperparameter settings

[0088] The method of the present invention uses three parts of loss during the training process, and uses hyperparameters α, β, and λ as weights to control the impact of each part of the loss. In the implementation of the method of the present invention, a combination of α=0.1, β=0.1, and λ=0.8 is used.

[0089] 34. Error Back Propagation

[0090] The back propagation error process describes how the model is trained. When given an input, the model generates a predicted output. By calculating the difference between this predicted output and the actual value, the present invention can obtain the prediction error. This error is then back-propagated through the network so that the weights in the entire network can be adjusted. Next, a learning rate is applied to the weights of each point in the network to fine-tune these weights. In this article, the learning rate is set to 0.01 and decays at a rate of 0.5 every 20 rounds of training. The entire training process is trained using the SGD optimizer, and its weight decay weight is set to 0.0001. The batch size during training is set to 64, including 64 acne images and their corresponding labels.

[0091] 35. Algorithm pseudocode

[0092] The pseudo code for model training is as follows:

[0093]

[0094] 4. Model testing

[0095] The model testing phase mainly uses the trained model for testing to verify the model's predictive performance. This test includes the public ACNE04 dataset and the ACNEHX dataset collected by the hospital. The test results on the ACNE04 dataset are shown in Table 1:

[0096] Table 1 Test results of the method of the present invention on the ACNE04 dataset

[0097] Model Name Accuracy (%) Accuracy (%) Sensitivity (%) Specificity (%) Youden Index (%) Baseline 79.38 76.43 78.54 93.45 71.99 KD 83.29 80.56 79.54 93.52 73.06 LSKD 83.51 79.43 81.23 93.54 74.77 PSKD 82.24 78.53 78.11 92.54 70.65 BYOT 82.49 79.21 80.79 93.26 74.05 MixSKD 83.04 81.26 81.32 93.89 75.21 Method of the present invention 84.75 83.10 82.64 94.01 76.65

[0098] The meanings of the five indicators are as follows:

[0099] 1) Accuracy: measures the model’s classification prediction accuracy, the higher the better.

[0100] 2) Precision: The ratio of the number of truly positive examples among the detected positive examples. The higher the better.

[0101] 3) Sensitivity: The ratio of detected positive examples to all true positive examples. The higher the better.

[0102] 4) Specificity: The ratio of detected negative examples to all true negative examples. The higher the better.

[0103] 5) Youden Index: Measures the overall capability of the model, the higher the better.

[0104] It can be seen from Table 1 that the method proposed in the present invention has achieved the best performance in all indicators compared with the existing methods. And compared with other self-distillation methods, such as BYOT and MixSKD, the performance of the model is greatly improved by adding multi-mapper and multi-scale feature analysis. The present invention also conducted relevant experiments on the hospital dataset ACNEHX. The experimental results are shown in Table 2:

[0105] Table 2 Test results of the method of the present invention on the dataset ACNEHX

[0106] Model Name Accuracy (%) Accuracy (%) Sensitivity (%) Specificity (%) Youden Index (%) Baseline 57.91 54.55 40.97 92.35 33.33 KD 57.50 46.59 40.91 92.16 33.07 LSKD 57.91 54.46 41.78 92.06 33.84 PSKD 57.74 54.12 40.98 92.79 33.77 BYOT 58.14 52.29 42.12 92.57 34.69 MixSKD 58.48 52.40 42.27 92.62 34.89 Method of the present invention 60.88 59.51 45.06 93.64 38.70

[0107] Similarly, it can be seen that the method of the present invention still achieves the most advanced performance on the hospital data, which illustrates the effectiveness and versatility of the method of the present invention.

[0108] In addition, the present invention also conducts a visual test on the method of the present invention. By drawing the category attention map of the model, the attention area of ​​the model can be seen. The results are as follows: Figure 4 shown in Figure 4 In the figure, each row represents the test result of a case, and each column represents the test result of a model on different test cases. The red area represents the model's focus on the input features. It can be seen that the model proposed by the method of the present invention can perceive acne more accurately, not only can it locate acne features more accurately, but also can detect those feature areas that may be ignored by other methods, showing superior performance.

[0109] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the scope of protection of the present invention is not limited to such specific statements and embodiments. For those skilled in the art, the present invention may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A facial acne grading method based on collaborative mapping multi-scale self-distillation, characterized in that: include: S1, preprocessing the acne image data, and dividing the preprocessed acne image data into a training set and a test set; S2. Construct a self-distillation model; the self-distillation model includes: a CNN module, a bottleneck module, a multi-mapper feature extraction module, a normalization module, a multi-scale feature capture module and a fully connected layer module; the CNN module includes multiple CNN layers, the multi-mapper feature extraction module includes multiple multi-mapper feature extraction layers, the normalization module includes multiple feature normalization layers, the multi-scale feature capture module includes multiple multi-scale feature capture layers, and the fully connected layer module includes multiple fully connected layers; Among them, the number of layers of the CNN module, the normalization module, the multi-scale feature capture module and the fully connected layer module is the same; the number of layers of the bottleneck module and the multi-mapper feature extraction module is the same, and the number of layers is one less than that of the CNN module, the normalization module and the multi-scale feature capture module; the output of the previous CNN layer in the CNN module is used as the input of the next CNN layer, and except for the last CNN layer, each CNN layer corresponds to a bottleneck layer, and the outputs of these CNN layers are used as the input of the corresponding bottleneck layer; each bottleneck layer corresponds to a multi-mapper feature extraction layer, and the output of each bottleneck layer is used as the input of the corresponding multi-mapper feature extraction layer; Except for the last standardization layer, each standardization module corresponds to a multi-mapper feature extraction layer. The input of each of these standardization layers is the output of the corresponding multi-mapper feature extraction layer. The input of the last standardization layer of the standardization module is the output of the last CNN layer of the CNN module. The output of the last layer of the standardization module is used as the label of the standardization module. The label of the standardization module is calculated with the output of the previous standardization layers respectively. The loss of the previous standardization layers is summed to obtain the loss of the standardization module. Each normalization layer in the normalization module corresponds to a fully connected layer; the output of each normalization layer is processed by the corresponding fully connected layer as the input of the corresponding multi-scale feature capture layer; The output of the last multi-scale feature capture layer of the multi-scale feature capture module is used as the label of the multi-scale feature capture module. The label of the multi-scale feature capture module is respectively calculated with the output of each previous multi-scale feature capture layer to calculate the loss. The loss of the multi-scale feature capture module is obtained by summing the losses of the previous multi-scale feature capture layers. The output of each multi-scale feature capture layer is processed by scale mean and then the corresponding prediction results are output; the image label is respectively calculated with each prediction result for loss, and the loss of the image label is obtained by summing them up; S3, using the training set in step S1 to train the self-distillation model constructed in step S2; S4. Input the test set into the self-distillation model trained in step S3 to obtain the prediction result.

2. A facial acne grading method based on collaborative mapping multi-scale self-distillation according to claim 1, characterized in that: The loss function used in step S3 during training is: L model (X,θ,Y)=αL Pro +βL Pat +λL Cro Among them, X represents the training set, Y represents the label set corresponding to X, and L Pro represents the loss of the normalization layer, L Pat represents the loss of the multi-scale feature capture layer, L Cro represents the loss of image labels, and α, β, and λ are hyperparameter weights introduced to balance the three losses.

3. A facial acne grading method based on collaborative mapping multi-scale self-distillation according to claim 2, characterized in that: L Pro The calculation formula is: Among them, L1 represents the number of standardization layers in the standardization module, represents the output of the l1th normalization layer, F T represents the output of the last normalization layer, C is the number of channels, It means to find the square of L2 norm.

4. A facial acne grading method based on collaborative mapping multi-scale self-distillation according to claim 3, characterized in that: L Pat The calculation formula is: in, represents the grade score corresponding to scale m in the l2th multi-scale feature capture layer except the last multi-scale feature capture layer, pats T (m) represents the grade score corresponding to scale m in the last multi-scale feature capture layer.

5. The facial acne grading method based on collaborative mapping multi-scale self-distillation according to claim 4, characterized in that: L Cro The calculation formula is: Among them, L CE represents the cross entropy loss, represents the l3th scale mean output result, and y represents the image label.

6. The facial acne grading method based on collaborative mapping multi-scale self-distillation according to claim 5, characterized in that: The number of mappers in each multi-mapper feature extraction layer is set between [3,5].

7. The facial acne grading method based on collaborative mapping multi-scale self-distillation according to claim 6, characterized in that: The splitting scale of each multi-scale feature capture layer is greater than or equal to 2 and less than or equal to 8.

Images