A network pruning method for skin disease imbalance data penalty score

Abstract

The application discloses a network pruning method for skin disease imbalance data penalty score, comprising the following steps: S1, acquiring a skin disease imbalance sample data set and determining tail classes according to the number of samples of each class, and marking a sample weight vector; S2, constructing a convolutional neural network; S3, calculating an imbalance data penalty score of each convolution kernel in the convolutional neural network; S4, pruning the convolution kernel of each convolution layer in the convolutional neural network according to a preset pruning rate; S5, training the pruned convolutional neural network by using the skin disease imbalance sample data set until convergence, and fine-tuning the converged convolutional neural network by using a balanced data set to obtain a final network model. The method reduces the network parameter quantity and the calculation amount, improves the classification accuracy, is convenient to deploy, and has strong generalization ability.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, specifically relating to a network pruning method for penalizing imbalanced data in skin diseases. Background Technology

[0002] The skin is the largest organ in the human body. Skin diseases are a general term for diseases occurring on the skin and its appendages, and they are diverse, with many internal organ diseases also manifesting on the skin. For example, melanoma (also known as malignant melanoma) is the most common and deadliest skin cancer, seriously threatening human health. It is a highly malignant solid skin tumor originating from melanocytes of the neural crest, generally caused by the malignant transformation of melanocytes distributed in the skin or other tissues. Ultraviolet radiation from sunlight is a significant risk factor for the occurrence and development of skin cancer, making sun exposure a major cause of skin cancer. Early lesions of melanoma present as melaninous lesions, often characterized by uneven pigmentation, irregular edges, asymmetry, and a diameter ≥6mm. Morphologically, it is extremely similar to other skin diseases such as pigmented nevi, actinic keratosis, and basal cell carcinoma, making them difficult to distinguish and easily leading to misdiagnosis and delaying optimal treatment. Therefore, using computer technology to improve the classification accuracy of skin diseases (such as melanoma) and assist doctors in diagnosis has significant research value and meaning.

[0003] However, in skin disease classification tasks, a major problem is the extreme imbalance of training data classes (i.e., the long-tail problem). In deep learning, the long-tail effect refers to the situation where one or more classes constitute the vast majority of the data, while other classes comprise a small portion of the dataset. The most direct consequence of the long-tail effect is overfitting in deep learning models, specifically manifested in poor performance when classifying or predicting minor class samples. ISIC2019 is a typical long-tail dataset. When training directly with imbalanced datasets, ideal results are rarely achieved. Models trained on imbalanced datasets have poor generalization ability and are prone to overfitting, tending to classify minor class samples as majority class samples. In reality, the most important classes are often those with a small sample size (such as tumor images), which are scarce but crucial, and the risk of misclassification is very high. Failure to correctly select these key samples from all samples significantly diminishes the value of the model.

[0004] To address the overfitting problem caused by imbalanced data, existing solutions typically include undersampling, loss function enhancement, and data augmentation. However, as the number of layers and kernels in convolutional neural networks (CNNs) increases, the number of parameters and computational cost rises, limiting their usability on mobile devices. Network pruning, on the other hand, can effectively reduce the number of parameters and computational cost of CNNs while maintaining accuracy, making them easy to deploy on mobile devices. However, existing network pruning methods often ignore the specificity of network structure. Different network layers have different sensitivities to parameters, and the network learning process is hierarchical, during which parameter sensitivity changes. Ignoring parameter sensitivity can significantly impact the results of imbalanced data pruning, leading to unstable discrimination results, difficulty in accurately uncovering the relationship between convolutional kernels and network outputs in low-sample data, and interference from numerous irrelevant factors, compromising the stability and reliability of the pruning results and resulting in poor generalization ability. Therefore, in view of the above problems, this invention proposes a network pruning method for penalized scoring of imbalanced data in dermatology. Summary of the Invention

[0005] The purpose of this invention is to address the above-mentioned problems by proposing a network pruning method for imbalanced data in dermatology, which improves classification accuracy while reducing the number of network parameters and computational load, and is easy to deploy.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] This invention proposes a network pruning method for penalizing imbalanced data in dermatology, comprising the following steps:

[0008] S1. Obtain the imbalanced sample dataset for skin diseases and determine the tail class based on the number of samples in each category, and label the sample weight vectors;

[0009] S2. Construct a convolutional neural network;

[0010] S3. Calculate the imbalanced data penalty score for each convolutional kernel in the convolutional neural network, as follows:

[0011] S31. After each mini-batch training is completed, the true and predicted labels of each sample are calculated and represented in One-Hot form, that is, the true and predicted labels of the corresponding samples are XORed to obtain a b*C dimensional matrix D:

[0012]

[0013] In the formula, matrix D consists of 0s and 1s, where 1 represents the true label y of the sample located in the i-th row and j-th column.i*j and prediction labels Inconsistent, 0 represents the true label y of the sample located in row i and column j. i*j and prediction labels Consistent, i = 1, 2, ..., b, j = 1, 2, ..., C, where b represents the size of the mini-batch and C represents the number of sample categories;

[0014] S32. Calculate the penalty coefficient for each mini-batch, using the following formula:

[0015]

[0016] In the formula, p t Represents the penalty coefficient obtained from training the t-th mini-batch, (w 1*C D t T ) i Representative sample weight vector w 1*C The transpose of matrix D obtained from training the t-th mini-batch t T The value of the i-th column after multiplication;

[0017] S33. Calculate the imbalanced data penalty score for all kernels of each convolutional layer in the convolutional neural network:

[0018]

[0019] In the formula, S l s represents the set of imbalanced data penalty scores for all convolutional kernels in the l-th convolutional layer of a convolutional neural network. l,q This represents the imbalance penalty score for the q-th convolutional kernel in the l-th convolutional layer. This represents the gradient tensor of the q-th convolutional kernel during the training of the t-th mini-batch. The L2 norm, mini_batch_num represents the number of mini-batches corresponding to one training epoch of the imbalanced dermatology dataset, c is the number of channels in the convolutional kernel, w is the width of the convolutional kernel, h is the height of the convolutional kernel, L represents the number of convolutional layers in the convolutional neural network, and I l This represents the number of convolutional kernels in the l-th convolutional layer;

[0020] S4. Prune the convolution kernels of each convolutional layer in the convolutional neural network according to the preset pruning rate;

[0021] S5. Train the pruned convolutional neural network using the imbalanced dermatology dataset until it converges. Then, fine-tune the converged convolutional neural network for 1-2 epochs using the balanced dataset to obtain the final network model. The balanced dataset is obtained by balancing sampling from the imbalanced dermatology dataset. An epoch represents the completion of one training cycle for all samples in the balanced dataset.

[0022] Preferably, determining the tail class based on the number of samples in each category specifically means considering the category whose sample count in the imbalanced dermatology dataset is below a preset percentage as the tail class.

[0023] Preferably, the imbalanced sample dataset for skin diseases is the ISIC2019 dataset, which includes nine categories: {Melanoma, Melanocytic nevus, Basal cell carcinoma, Actinic keratosis, Benign keratosis, Dermatofibroma, Vascular lesion, Squamous cell carcinoma, None of the others}. Dermatofibroma and Vascular lesion are designated as tail classes, and the sample weight vector is labeled as [0.0,0.0,0.0,0.0,0.0,0.5,0.5,0.0,0.0,0.0].

[0024] Preferably, the convolutional kernels of each convolutional layer in the convolutional neural network are pruned according to a preset pruning rate, as follows:

[0025] S41. Sort the imbalance penalty scores of all convolution kernels in each convolutional layer in descending order;

[0026] S42. Delete the first r% of convolutional kernels in the corresponding convolutional layer according to the preset pruning rate r, and at the same time delete the channels generated by the convolutional kernels deleted in this layer in the next layer.

[0027] Compared with existing technologies, the beneficial effects of this invention are as follows: This method considers the structural specificity of each convolutional layer in the convolutional neural network. Based on the parameter sensitivity of each layer structure, it obtains the corresponding imbalanced data penalty score by additionally considering the gradient information generated by each convolutional kernel for the classification error of imbalanced data, thereby guiding the network to perform pruning. For example, for datasets with long-tail effects, it can accurately mine the relationship between the convolutional kernels of few samples and the network output, eliminate the interference of a large number of irrelevant factors in the imbalanced data pruning process, ensure the stability and reliability of the pruning results, solve the problem of low classification accuracy of imbalanced datasets, and improve classification accuracy while reducing the number of network parameters and computational load. It is also easy to deploy and has strong generalization ability. Attached Figure Description

[0028] Figure 1 This is a flowchart of the network pruning method for penalizing imbalanced data in dermatology according to the present invention;

[0029] Figure 2 This is a data distribution diagram of the ISIC2019 dataset of this invention. Detailed Implementation

[0030] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0031] It should be noted that, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application.

[0032] like Figure 1-2 As shown, a network pruning method for penalizing scores in imbalanced data related to dermatology includes the following steps:

[0033] S1. Obtain the imbalanced sample dataset for skin diseases and determine the tail class based on the number of samples in each category, and label the sample weight vectors.

[0034] In one embodiment, determining the tail class based on the number of samples in each category specifically involves considering the category whose sample count in the imbalanced dermatology sample dataset is below a preset percentage as the tail class.

[0035] In one embodiment, the imbalanced sample dataset for skin diseases is the ISIC2019 dataset, which includes nine categories: {Melanoma, Melanocytic nevus, Basal cell carcinoma, Actinic keratosis, Benign keratosis, Dermatofibroma, Vascular lesion, Squamous cell carcinoma, None of the others}. Dermatofibroma and Vascular lesion are designated as tail classes, and the sample weight vector is labeled as [0.0,0.0,0.0,0.0,0.0,0.5,0.5,0.0,0.0,0.0].

[0036] The ISIC2019 dataset is a clinical dataset of skin lesions provided by the International Skin Imaging Consortium (ISIC) for automated algorithmic analysis of dermoscopic images. The dataset exhibits a severe imbalance in the distribution of lesion types, including nine categories: Melanoma, Melanocytic Nevus, Basal Cell Carcinoma, Actinic Keratosis, Benign Keratosis, Dermatofibroma, Vascular Lesion, Squamous Cell Carcinoma, and None of the Others. By defining the sample weight vector, the smaller the sample size, the larger the penalty weight. Analyzing the proportion of sample classes in the ISIC2019 dataset, the tail classes of the imbalanced dermatology dataset were determined to be Dermatofibroma and Vascular lesion, and the sample weight vector was set to [0.0,0.0,0.0,0.0,0.0,0.5,0.5,0.0,0.0]. The sample weight vector settings can be adaptively adjusted using empirical methods.

[0037] S2. Construct a convolutional neural network.

[0038] S3. Calculate the imbalanced data penalty score for each convolutional kernel in the convolutional neural network, as follows:

[0039] S31. After each mini-batch training is completed, the true label and predicted label of each sample are calculated and represented in One-Hot form, that is, the true label and predicted label of the corresponding sample are XORed to obtain a b*C dimensional matrix D:

[0040]

[0041] In the formula, matrix D consists of 0s and 1s, where 1 represents the true label y of the sample located in the i-th row and j-th column. i*j and prediction labels Inconsistent, 0 represents the true label y of the sample located in row i and column j. i*j and prediction labels Consistent, i = 1, 2, ..., b, j = 1, 2, ..., C, where b represents the size of the mini-batch and C represents the number of sample categories;

[0042] S32. Calculate the penalty coefficient for each mini-batch, using the following formula:

[0043]

[0044] In the formula, p t Represents the penalty coefficient obtained from training the t-th mini-batch, (w 1*C D t T ) i Representative sample weight vector w 1*C The transpose of matrix D obtained from training the t-th mini-batch t T The value of the i-th column after multiplication;

[0045] S33. Calculate the imbalanced data penalty score for all kernels of each convolutional layer in the convolutional neural network:

[0046]

[0047] In the formula, S l s represents the set of imbalanced data penalty scores for all convolutional kernels in the l-th convolutional layer of a convolutional neural network. l,q This represents the imbalance penalty score for the q-th convolutional kernel in the l-th convolutional layer. This represents the gradient tensor of the q-th convolutional kernel during the training of the t-th mini-batch. The L2 norm, mini_batch_num represents the number of mini-batches corresponding to one training epoch of the imbalanced dermatology dataset, c is the number of channels of the convolutional kernel, w is the width of the convolutional kernel, h is the height of the convolutional kernel, L represents the number of convolutional layers in the convolutional neural network, and I l This represents the number of convolutional kernels in the l-th convolutional layer.

[0048] like Figure 1 As shown, a convolutional neural network may include n layers of a basic network (n convolutional layers), such as any convolutional neural network structure in the prior art. This represents the gradient tensor of the q-th convolutional kernel during the training of the t-th mini-batch. The L2 norm, which is the gradient of the corresponding convolutional kernel, is represented in one-hot form. The misclassification matrix D is obtained by XORing the true and predicted labels of the corresponding samples. During training, the gradient information of the convolutional neural network is obtained, the L2 norm is calculated, and the misclassification of each sample is recorded. After each mini-batch is trained, the penalty coefficient of the current mini-batch is obtained. Then, the "responsibility" that the convolutional kernels in different convolutional layers of the convolutional neural network should bear for misclassification is further evaluated, and the imbalanced data penalty score of each convolutional kernel is calculated.

[0049] S4. Prune the convolution kernels of each convolutional layer in the convolutional neural network according to the preset pruning rate.

[0050] In one embodiment, the convolutional kernels of each convolutional layer in the convolutional neural network are pruned according to a preset pruning rate, as follows:

[0051] S41. Sort the imbalance penalty scores of all convolution kernels in each convolutional layer in descending order;

[0052] S42. Delete the first r% of convolutional kernels in the corresponding convolutional layer according to the preset pruning rate r, and at the same time delete the channels generated by the convolutional kernels deleted in this layer in the next layer.

[0053] like Figure 1 As shown, the index of convolutional kernels to be deleted displays a descending list of imbalanced data penalty scores for all convolutional kernels in each convolutional layer. Pruning of the convolutional neural network involves trimming convolutional kernels from highest to lowest score until a preset pruning rate is met. The preset pruning rate is equal for each convolutional layer, or it can be adjusted according to actual needs.

[0054] S5. Train the pruned convolutional neural network using an imbalanced dermatology dataset until convergence. Then, fine-tune the converged convolutional neural network for 1-2 epochs using a balanced dataset to obtain the final network model. The balanced dataset is obtained by balancing sampling from the imbalanced dermatology dataset. An epoch represents one training cycle for all samples in the balanced dataset. In practical applications, inputting the dermatology samples to be tested into the final network model will quickly yield accurate dermatology classification results.

[0055] Table 1 shows the comparison results between the method of the present invention and the prior art method on different models.

[0056] Table 1

[0057]

[0058]

[0059] As shown in Table 1, the convolutional neural networks employed include three existing models: VGG-16, ResNet-56, and DenseNet-40. The method of this invention was compared with existing methods under each model. Existing methods employed Augment(0.2,0.2,0.2,0.2,0.2), Augment(0.2,0.1,0.3,0.3,0.1), Augment(0.3,0.3,0.1,0.1,0.2), and Focal... Models trained using the Loss, GHM, DCGAN, Mixup, Cutout, and Cutmix methods are used. Augment(0.2,0.2,0.2,0.2,0.2,0.2) represents the proportion of samples augmented using data augmentation methods (random erasure, image scaling, histogram equalization, random warping, and random flipping, in that order) to form an augmented dataset. Augment(0.2,0.1,0.3,0.3,0.1) and Augment(0.3,0.3,0.1,0.1,0.2) work similarly. Focal Loss and GHM differ in that Focal Loss attenuates the loss based on confidence levels, while GHM attenuates the loss based on the number of samples with a certain confidence level. The DCGAN method, first proposed in the 2016 paper "Unsupervised Draft Representation Learning with Deep Convolutional Generative Adversarial Networks" by Alec et al., introduced the application of convolutional neural networks to the GAN (Generative Adversarial Network) model, replacing fully connected layers. Mixup, Cutout, and Cutmix are common image enhancement methods in existing technologies. For example, Mixup is an algorithm for mixed-class image enhancement, which can blend images from different classes using linear interpolation to construct new training samples and expand the training dataset. Test Top-1(%) refers to Top-1 accuracy, and Param / PR refers to the number of parameters (Param) and the compression ratio (PR). As can be seen from the table, the method of this invention can significantly reduce the number of parameters for different models. For example, it can reduce the number of parameters by 53.97% for the VGG-16 model, 56.47% for the ResNet-56 model, and 48.11% for the DenseNet-40 model, thereby improving the computation speed.

[0060] This method considers the structural specificity of each convolutional layer in a convolutional neural network. Based on the parameter sensitivity of each layer, it obtains the corresponding imbalanced data penalty score by additionally considering the gradient information generated by the classification error of each convolutional kernel on imbalanced data, thereby guiding the network to perform pruning. For example, for datasets with long-tail effects, it can accurately discover the relationship between convolutional kernels of few samples and network output, eliminate the interference of a large number of irrelevant factors in the imbalanced data pruning process, ensure the stability and reliability of the pruning results, solve the problem of low classification accuracy of imbalanced datasets, and improve classification accuracy while reducing the number of network parameters and computational cost. It is easy to deploy and has strong generalization ability.

[0061] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0062] The embodiments described above are merely specific and detailed examples of the embodiments described in this application, and should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the scope of protection of this application. Therefore, the scope of protection of this patent application should be determined by the appended claims.

Claims

1. A network pruning method for penalizing imbalanced data in dermatology, characterized in that: The network pruning method for penalizing imbalanced data in dermatology includes the following steps: S1. Obtain the imbalanced sample dataset of skin diseases and determine the tail class according to the number of samples in each category, and label the sample weight vector. Each sample in the imbalanced sample dataset of skin diseases is a skin disease image. S2. Construct a convolutional neural network; S3. Calculate the imbalanced data penalty score for each convolutional kernel in the convolutional neural network, as follows: S31. After each mini-batch training is completed, the true label and predicted label of each sample are calculated and represented in One-Hot form, that is, the true label and predicted label of the corresponding sample are XORed to obtain... dimensional matrix : In the formula, the matrix It consists of 0s and 1s, where 1 represents the true label of the sample located in the i-th row and j-th column. and prediction labels Inconsistent, 0 represents the true label of the sample located in row i and column j. and prediction labels Consistent, i=1,2,……,b, j=1,2,……,C, where b represents the size of the mini-batch and C represents the number of sample categories; S32. Calculate the penalty coefficient for each mini-batch, using the following formula: In the formula, This represents the penalty coefficient obtained from training the t-th mini-batch. Representative sample weight vector w 1*C The transpose of matrix D obtained from training the t-th mini-batch The value of the i-th column after multiplication; S33. Calculate the imbalanced data penalty score for all kernels of each convolutional layer in the convolutional neural network: In the formula, Represents the first in a convolutional neural network The set of imbalanced data penalty scores for all kernels of each convolutional layer, s l,q Representing the Imbalanced data penalty score for the q-th convolutional kernel in a convolutional layer This represents the gradient tensor of the q-th convolutional kernel during the training of the t-th mini-batch. The L2 norm, where mini_batch_num represents the number of mini-batches corresponding to one training epoch of the imbalanced dermatology dataset. is the number of channels in the convolution kernel, w is the width of the convolution kernel, and h is the height of the convolution kernel. This represents the number of convolutional layers in a convolutional neural network. Representing the The number of convolutional kernels contained in each convolutional layer; S4. Prune the convolution kernels of each convolutional layer in the convolutional neural network according to the preset pruning rate; S5. Train the pruned convolutional neural network using an imbalanced dermatology dataset until it converges. Then, fine-tune the converged convolutional neural network for 1-2 epochs using a balanced dataset to obtain the final network model. The balanced dataset is obtained by balancing sampling from the imbalanced dermatology dataset. An epoch represents the completion of one training cycle for all samples in the balanced dataset.

2. The network pruning method for penalizing imbalanced data in dermatology as described in claim 1, characterized in that: The method of determining the tail class based on the number of samples in each category specifically involves considering the category whose sample count in the imbalanced skin disease sample dataset is lower than a preset percentage as the tail class.

3. The network pruning method for penalizing imbalanced data in dermatology as described in claim 2, characterized in that: The imbalanced sample dataset for skin diseases is the ISIC2019 dataset, which includes nine categories: {Melanoma, Melanocytic nevus, Basal cell carcinoma, Actinic keratosis, Benign keratosis, Dermatofibroma, Vascular lesion, Squamous cell carcinoma, None of the others}. Dermatofibroma and Vascular lesion are designated as tail classes, and the sample weight vector is labeled as [0.0, 0.0, 0.0, 0.0, 0.0, 0.5, 0.5, 0.0, 0.0, 0.0].

4. The network pruning method for penalizing imbalanced data in dermatology as described in claim 1, characterized in that: The process of pruning the convolutional kernels of each convolutional layer in the convolutional neural network according to a preset pruning rate is as follows: S41. Sort the imbalance penalty scores of all convolution kernels in each convolutional layer in descending order; S42. According to the preset pruning rate Delete the first part of the corresponding convolutional layer The convolution kernel is % of the total kernel size, and channels generated by the convolution kernels deleted in this layer are removed in the next layer.

Images