Skin disease classification model training method and device, terminal and storage medium

Abstract

The embodiment of the application discloses a kind of dermopathy classification model training method and device, terminal and storage medium;The method comprises: the first training data set is input into original classification model to train, to obtain first classification model;Second training data set is input into original classification model to train, to obtain second classification model;Based on the first convolution kernel and classification output of first classification model and the second convolution kernel and classification output of second classification model, determine the fusion convolution kernel and classification output of fusion classification model;Based on the second loss function of second classification model and the difference between the parameters of fusion classification model and the parameters of first classification model, determine the total loss function of fusion classification model;Based on total loss function training fusion classification model, to obtain third classification model;Thus the embodiment of the application makes that new and old data can reach threshold accuracy under the premise, so that new and old data do not use, improve the reliability of each hospital data.

Description

Technical Field

[0001] This invention relates to, but is not limited to, the fields of medical application technology or artificial intelligence technology, and particularly to a training method, device, terminal and storage medium for a skin disease classification model. Background Technology

[0002] With the recent development of deep learning, such as its continuous updates in image processing techniques for segmentation and / or detection, more and more medical applications are incorporating deep learning technologies, enabling the integration of artificial intelligence (AI) into a wider range of medical applications. In computer vision, convolutional neural networks (CNNs) are indispensable for AI vision tasks such as object detection, semantic segmentation, and classification. CNNs are a type of feedforward neural network (FNN) with a deep structure that includes convolutional computations, and are one of the representative algorithms of deep learning. CNNs possess representation learning capabilities, enabling shift-invariant classification of input information according to its hierarchical structure; therefore, they are also known as shift-invariant artificial neural networks (SIANNs). Thus, this CNN technology can be applied to the classification of facial skin lesions.

[0003] However, the application of AI in medical scenarios has its own unique characteristics. Most importantly, medical data usually involves patient privacy, and hospitals have strict requirements for user privacy. They usually require medical data to be anonymized before discharge or to be used only within the hospital. For example, use after discharge may lead to privacy leaks, and the security of medical data is not high. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a training method, apparatus, terminal and storage medium for a skin disease classification model, so as to at least solve some of the above-mentioned technical problems.

[0005] The technical solution of this invention is implemented as follows:

[0006] In a first aspect, embodiments of the present invention provide a method for training a skin disease classification model, the method comprising:

[0007] The first training dataset is input into the original classification model for training to obtain the first classification model; wherein, the first training dataset includes a first number of training samples of skin diseases in different categories;

[0008] The second training dataset is input into the original classification model for training to obtain the second classification model; wherein, the second training dataset includes a second number of training samples of skin diseases.

[0009] The first convolutional kernel of the first classification model and the second convolutional kernel of the second classification model are fused to determine the fusion convolutional kernel of the fused classification model;

[0010] Based on the first number of classification outputs of the first classification model and the second number of classification outputs of the second classification model, the fusion classification output of the fusion classification model is determined;

[0011] Based on the second loss function of the second classification model and the difference between the parameters of the fusion classification model and the parameters of the first classification model, the total loss function of the fusion classification model is determined;

[0012] The second training dataset is input into the fusion classification model for training until the total loss function converges, thereby obtaining the trained third classification model.

[0013] In some embodiments, fusing the first convolutional kernel of the first classification model and the second convolutional kernel of the second classification model to determine the fused convolutional kernel of the fused classification model includes:

[0014] The first weight is determined based on the parameters of the i-th first convolutional kernel in the L-th layer and the sum of the parameters of the 1st to i-th first convolutional kernels in the L-th layer; where L is an integer greater than 0 and i is an integer greater than 0.

[0015] The second weights are determined based on the parameters of the i-th second convolutional kernel in the L-th layer and the sum of the parameters of the 1st to i-th second convolutional kernels in the L-th layer.

[0016] Based on the first weight and the first convolution kernel, and the second weight and the second convolution kernel, the i-th fusion convolution kernel of the L-th layer is determined.

[0017] In some embodiments, determining the fusion classification output of the fusion classification model based on the first number of classification outputs of the first classification model and the second number of classification outputs of the second classification model includes:

[0018] The fused classification output is determined based on the sum of the first number of classification outputs and the second number of classification outputs.

[0019] In some embodiments, determining the total loss function of the fusion classification model based on the second loss function of the second classification model and the difference between the parameters of the fusion classification model and the parameters of the first classification model includes:

[0020] Based on the i-th parameter of the L-th layer of the first classification model and the sum of the 1st to nth parameters of the L-th layer, the i-th first proportion of the L-th layer is determined; wherein, L is an integer greater than 0, i is an integer greater than 0, and n is greater than or equal to i;

[0021] Based on the difference between the parameters of the i-th fusion classification model in layer L and the parameters of the i-th first classification model in layer L, the i-th first difference in layer L is determined.

[0022] The second difference is determined by the sum of the products of the first proportions of the first to the i-th layers of the first to the L-th layers and the i-th first difference of the L-th layer.

[0023] The total loss function is determined based on the second loss function of the second classification model and the second difference.

[0024] In some embodiments, determining the total loss function based on the second loss function of the second classification model and the second difference includes:

[0025] The total loss function is determined based on the product of the third weight and the second loss function, and the product of the fourth weight and the second difference; wherein the sum of the third weight and the fourth weight is a predetermined value.

[0026] Secondly, embodiments of the present invention provide a method for classifying and treating skin diseases, the method comprising:

[0027] Obtain a first dataset, wherein the first dataset includes at least one image of a skin disease;

[0028] The first dataset is input into the third classification model to obtain the first classification recognition result; wherein, the total loss function of the third classification model is determined based on the second loss function of the second classification model and the difference between the parameters of the fusion classification model and the parameters of the first classification model; the fusion classification model is determined based on the fusion of the first classification model and the second classification model.

[0029] In some embodiments, the method includes:

[0030] Obtain a second dataset, wherein the second dataset includes at least one image of a skin disease;

[0031] The dataset is input into the third classification model to obtain the second classification recognition result; wherein, the second dataset may be a dataset obtained from a different database than the first dataset.

[0032] Thirdly, embodiments of the present invention provide a training device for a skin disease classification and treatment apparatus, comprising:

[0033] The first training module is used to input the first training dataset into the original classification model for training to obtain the first classification model; wherein, the first training dataset includes a first number of training samples of skin diseases.

[0034] The first training module is used to input the second training dataset into the original classification model for training to obtain a second classification model; wherein, the second training dataset includes a second number of training samples of skin diseases.

[0035] The first processing module is used to fuse the first convolutional kernel of the first classification model and the second convolutional kernel of the second classification model to determine the fusion convolutional kernel of the fused classification model;

[0036] The first processing module is used to determine the fusion classification output of the fusion classification model based on the first number of classification outputs of the first classification model and the second number of classification outputs of the second classification model;

[0037] The first processing module is used to determine the total loss function of the fusion classification model based on the second loss function of the second classification model and the difference between the parameters of the fusion classification model and the parameters of the first classification model;

[0038] The second training module is used to input the second training dataset into the fusion classification model for training until the total loss function converges, so as to obtain the trained third classification model.

[0039] Fourthly, embodiments of the present invention provide a terminal, the terminal including a processor and a memory for storing a computer program capable of running on the processor; wherein, when the processor runs the computer program, it implements the training method of the skin disease classification model or the skin disease classification processing method of any embodiment of the present invention.

[0040] Fifthly, embodiments of the present invention also provide a computer storage medium containing computer-executable instructions, which are executed by a processor to implement the training method for the skin disease classification model or the skin disease classification processing method described in any embodiment of the present invention.

[0041] In this embodiment of the invention, when the second training dataset is input into the original classification model for training, it is not necessary to train the entire dataset (i.e., it is not necessary to retrain the first training dataset from scratch). Only the newly added second training dataset needs to be trained, thereby saving a significant amount of training time and reducing computational load. Furthermore, since the datasets of various hospitals involve sensitive data, they cannot be used after being trained in the hospital. However, this embodiment of the invention allows for the training of newly added datasets (e.g., the second training dataset) without the datasets remaining in the hospital (e.g., the first training dataset), thereby improving the reliability of datasets within each hospital.

[0042] Furthermore, in this embodiment of the invention, the fusion classification model can retain the knowledge of the first classification model, thus achieving the accuracy of classification of the new dataset (e.g., the second training dataset) without reducing or only slightly reducing the accuracy of the old dataset (e.g., the first training dataset). It can also continuously iterate the fusion classification model to obtain a third classification model that can be used for other new and old datasets. This third classification model can ensure the accuracy of the new dataset while ensuring that the accuracy of the old dataset does not decrease or decreases relatively little. Attached Figure Description

[0043] Figure 1 This is a flowchart illustrating a training method for a skin disease classification model provided in an embodiment of the present invention.

[0044] Figure 2 This is a flowchart illustrating another method for training a skin disease classification model provided in an embodiment of the present invention.

[0045] Figure 3 This is a flowchart illustrating a training method for another skin disease classification model provided in an embodiment of the present invention.

[0046] Figure 4 This is a flowchart illustrating a method for classifying and treating skin diseases, as provided in an embodiment of the present invention.

[0047] Figure 5 This is a flowchart illustrating another method for training a skin disease classification model provided in an embodiment of the present invention.

[0048] Figure 6 This is a schematic diagram of the structure of a training device for a skin disease classification model provided in an embodiment of the present invention.

[0049] Figure 7 This is a schematic diagram of the structure of a skin disease classification and treatment device provided in an embodiment of the present invention;

[0050] Figure 8 This is a schematic diagram of the hardware structure of a terminal provided in an embodiment of the present invention. Detailed Implementation

[0051] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0052] In the following description, the use of suffixes such as "module," "part," or "unit" to denote elements is solely for the purpose of illustrative purposes and has no specific meaning in itself. Therefore, "module," "part," or "unit" may be used interchangeably. Furthermore, in the following description, the use of prefixes such as "first" or "second" to identify information is solely for the purpose of illustrative purposes and has no specific meaning in itself. Additionally, in the following description, "a plurality of" refers to two or more; "multiple types" refers to two or more kinds.

[0053] To better understand the technical solutions described in any embodiment of the present invention, some aspects of the related technology will be described first:

[0054] The application of AI in healthcare has its unique challenges. Most importantly, medical data often involves patient privacy, and hospitals have strict privacy requirements, typically requiring data to be anonymized before discharge or used only within the hospital. The situation becomes even more complex when data from multiple hospitals is not discharged, or when models are incrementally added based on disease type. For example, after training task T1, the training data D1 is not available; only the corresponding model M is obtained, achieving an accuracy of A1. Subsequent training needs to be based on model M, training data D2 on a new task T2. However, D1 cannot be used in this process. This task requires achieving an accuracy of A2, and the accuracy on task T1 must remain at least equal to or minimally lower than A1. The greater the difference between the data in T2 and T1, such as differences in type and / or style, the greater the impact on the accuracy of T1 after T2 achieves its predetermined accuracy. The situation for skin disease classification tasks is similar to the above description. That is, after using CNNs to accurately classify skin diseases (lesions) (e.g., into three categories: acne, papules, and cysts), the classification of skin diseases can be expanded (e.g., adding lesions such as exfoliation and / or blackheads). This scenario has an objective basis because the differences in facial skin disease datasets are relatively small. Although the types of lesions are expanding, they all belong to facial skin images, and at least the non-lesion skin and background areas have similar types, styles, and shooting techniques. Regardless of the method used, in principle, small class differences have a small impact on T1 descent. Therefore, expanding the categories of facial skin diseases makes it easier to minimize A1 descent when training the T2 task after the T1 task.

[0055] (1) Task-independent training method. This type of method treats each task as completely independent and trains multiple models independently for each task; that is, multiple hospitals collect and / or train data for different diseases. When performing inference after training multiple models, the results of multiple models are combined for evaluation. That is, the input sample is inferred through each model, and then the results of each model are combined, and a voting method or averaging method is used to obtain the final result.

[0056] (2) Methods based on Generative Adversarial Networks (GANs). These methods do not store samples of the old categories, but instead use GANs to learn and generate samples of the old categories. Therefore, when scaling up the model, it is only necessary to use GANs to generate images from some old datasets, without needing to store a large number of samples. Thus, models that only use discharged patients with GANs do not need to use images from outside the hospital.

[0057] (3) Transfer Learning-based methods. These methods first train model M on the D1 dataset for task T1, and then output the model after training. One approach is to keep the backbone network parameters constant during training, increase the number of fully connected layers to expand and add new disease categories; then, using the D2 dataset as positive examples and other non-D2 datasets as negative examples, train model M to identify diseases in the D2 dataset. Another approach is to not fix the backbone network parameters during training, with other details remaining the same as described above.

[0058] (4) Federated Learning-based methods. This type of method is a distributed machine learning approach. Its core idea is to train a model in a distributed manner across multiple data sources that have local data. Without exchanging local individual or sample data, a global model based on virtual fused data is constructed only by exchanging model parameters or intermediate results. This achieves a balance between data privacy protection and data sharing computation, namely, a new application paradigm of "data is available but not visible" and "the model moves while the data does not move".

[0059] However, all of the above methods have a series of drawbacks:

[0060] The disadvantage of the method described in (1) is that it requires multiple models, which increases the model size exponentially. Uncompressed models are typically 50MB to 200MB in size, and models of similar size can reach more than 1GB after 5 to 10 expansions. Furthermore, the deployment of models at the edge and end consumes a significant amount of resources. From the perspective of inference time, serial model inference also increases the time consumption exponentially, while parallel model inference causes a surge in graphics processing unit (GPU) and / or video memory resources.

[0061] Regarding the method described in (2) above, the disadvantages of this type of method are as follows: First, the upper limit is limited by the generation capability of GAN. The similarity between the images generated by GAN and the real images is still more or less different. Second, one advantage claimed by GAN-based methods is that they do not require the storage of historical data. However, in general, the GAN model itself also occupies a certain amount of storage space (usually in the range of 10MB to 100MB). If this space is used to directly store representative historical data, based on 100kB per image, hundreds or even thousands of images can be stored. Third, GAN-based methods may not be applicable in some fields. For example, in the serious medical field, the acceptance rate will decrease because it is currently impossible to verify whether the models trained by GAN and the actual medical lesion samples are consistent in clinical evaluation.

[0062] The drawback of the method described in (3) above is that it typically causes a sharp drop in the accuracy A1 of model M on task T1. This is because there is no strategy in the model structure or training method to retain the knowledge learned on task T1. The goal of this transfer learning is to improve the accuracy A2 on task T2, but there is no strategy to control the decline in A1.

[0063] The disadvantage of the method described in (4) above is that federated learning requires the exchange of model parameters or intermediate results, which necessitates network transmission; therefore, there is a communication efficiency problem, and the training time is usually several times longer than that of training data locally. Due to real-world factors such as inconsistent computing capabilities of distributed participating nodes, unstable network connection status, and non-independent data communication, the communication efficiency of federated learning can easily become one of the bottlenecks in its application.

[0064] The training method for a skin disease classification model provided in this invention aims to solve at least some of the problems in the aforementioned related technologies; for example, it aims to solve the technical problem that training datasets (e.g., skin disease images) cannot be used after discharge from hospitals. Furthermore, this invention can generate training samples of similar samples without using a GAN network. After obtaining the trained classification model, as much knowledge as possible is preserved in the classification model, and it is transferred to new tasks (e.g., new training data) for adaptation to the new training dataset. This invention can be improved for convolutional neural networks.

[0065] like Figure 1 As shown in the figure, this embodiment of the invention provides a training method for a skin disease classification model, including the following steps:

[0066] Step S11: Input the first training dataset into the original classification model for training to obtain the first classification model; wherein, the first training dataset includes a first number of training samples of skin diseases.

[0067] Step S12: Input the second training dataset into the original classification model for training to obtain a second classification model; wherein, the second training dataset includes a second number of skin disease training samples.

[0068] Step S13: Fuse the first convolution kernel of the first classification model and the second convolution kernel of the second classification model to determine the fusion convolution kernel of the fused classification model;

[0069] Step S14: Based on the first number of classification outputs of the first classification model and the second number of classification outputs of the second classification model, determine the fusion classification output of the fusion classification model;

[0070] Step S15: Based on the second loss function of the second classification model and the difference between the parameters of the fusion classification model and the parameters of the first classification model, determine the total loss function of the fusion classification model;

[0071] Step S16: Input the second training dataset into the fusion classification model for training until the total loss function converges, so as to obtain the trained third classification model.

[0072] The training method for the skin disease classification model provided in this embodiment of the invention can be executed by a terminal; the terminal can be any type of mobile terminal or fixed terminal. For example, the terminal can be, but is not limited to, at least one of the following: mobile phone, computer, server, medical device, industrial equipment, and / or wearable device, etc.

[0073] In one embodiment, the first training dataset includes a third number of skin disease training samples; the third number of skin disease training samples includes a first number of skin disease training samples of different categories. Both the first number and the third number can be integers greater than 0. For example, the terminal determines the first training dataset; the first training dataset includes N1 skin disease training samples of K1 categories; N1 and K1 are both integers greater than 0. Here, one skin disease training sample means one skin disease training sample; at least one means one or more; multiple means two or more.

[0074] In one embodiment, the second training dataset includes a fourth number of dermatology training samples; the fourth number of dermatology training samples includes a second number of dermatology training samples of different categories. Both the second and fourth numbers can be integers greater than 0. For example, the terminal determines the second training dataset; the second training dataset includes N2 images of K2 dermatology training samples; N2 and K2 are both integers greater than 0.

[0075] In this embodiment of the invention, the skin disease training samples and the skin disease images mentioned below can be any image, picture, or animation describing a skin disease or a skin disease lesion; wherein, the skin disease training sample refers to a skin disease image used as a sample. The skin disease training samples and skin disease images can be, but are not limited to, facial skin disease images, hand skin disease images, foot skin disease images, back or abdominal skin disease images, or leg skin disease images, etc. The skin diseases or skin disease lesions in the skin disease training samples and skin disease images can be, but are not limited to, acne, papules, cysts, epidermal peeling, and / or blackheads, etc.

[0076] In one embodiment, the types of skin diseases or skin lesions included in the first training dataset may be completely different from the types of skin diseases or skin lesions included in the second training dataset.

[0077] In another embodiment, the types of skin diseases or skin lesions included in the first training dataset may be at least partially the same as the types of skin diseases or skin lesions included in the second training dataset.

[0078] In one embodiment, the first classification model, the second classification model, the third classification model, the original classification model, and the fusion classification model can all be, but are not limited to, any type of convolutional neural network.

[0079] In one embodiment, step S11, inputting the first training dataset into the original classification model for training to obtain the first classification model, can be: inputting the first training dataset into the original classification model for training until the first loss function converges to obtain the first training recognition result. Here, if the first training recognition result is greater than or equal to a first predetermined value; the convergence of the first loss function can be that the first loss function is less than or equal to the first predetermined value; at this time, the parameters of the original classification model of the first training recognition result are obtained, i.e., the parameters of the first classification model. Here, the parameters can be, but are not limited to, the parameters of each convolutional kernel in each layer of the first classification model and / or the parameters of the weights of each connection layer, etc. For example, the first recognition result is an accuracy A1, such as an accuracy A1 of 90%, 95%, or 98%, etc.

[0080] In one embodiment, step S12, inputting the second training dataset into the original classification model for training to obtain the second classification model, can be: inputting the second training dataset into the original classification model for training until the second loss function converges to obtain the second training recognition result. Here, if the second training recognition result is greater than or equal to a second predetermined value; the convergence of the second loss function can be that the second loss function is less than or equal to the second predetermined value; at this time, the parameters of the original classification model for obtaining the second training recognition result are obtained, i.e., the parameters of the first classification model. Here, the parameters can be, but are not limited to, the parameters of each convolutional kernel in each layer of the first classification model and / or the parameters of the weights of each connection layer, etc. For example, the first recognition result is an accuracy A1, such as an accuracy A1 of 90%, 95%, or 98%, etc.

[0081] In one embodiment, before step S11, the method further includes: labeling the categories of each skin disease training sample included in the first training dataset.

[0082] In one embodiment, prior to step S12, the method further includes: labeling the dermatology images included in the second training dataset with categories.

[0083] In one embodiment, the fusion classification model is determined based on the first classification model and the second classification model; each convolutional kernel in each layer of the fusion classification model is determined based on the convolutional kernel at the same position in the first classification model and the convolutional kernel at the same position in the second classification model; the classification output of each connection layer of the fusion classification model is determined based on the classification output of the first classification model and the classification output of the second classification model.

[0084] In one embodiment, the convergence of the total loss function can be achieved when the total loss function is less than or equal to a third predetermined value. For example, the third predetermined value is within a predetermined range from the second predetermined value; or, the third predetermined value is within a predetermined range from the first predetermined value; or, the third predetermined value is less than or equal to the second predetermined value.

[0085] In this embodiment of the invention, when the second training dataset is input into the original classification model for training, it is not necessary to train the entire dataset (i.e., it is not necessary to retrain the first training dataset from scratch). Only the newly added second training dataset needs to be trained, thereby saving a significant amount of training time and reducing computational load. Furthermore, since the datasets of various hospitals involve sensitive data, they cannot be used after being trained in the hospital. However, this embodiment of the invention allows for the training of newly added datasets (e.g., the second training dataset) without the datasets remaining in the hospital (e.g., the first training dataset), thereby improving the reliability of datasets within each hospital.

[0086] Furthermore, in this embodiment of the invention, the fusion classification model can retain the knowledge of the first classification model, thus achieving the accuracy of classification of the new dataset (e.g., the second training dataset) without reducing or only slightly reducing the accuracy of the old dataset (e.g., the first training dataset). It can also continuously iterate the fusion classification model to obtain a third classification model that can be used for other new and old datasets. This third classification model can ensure the accuracy of the new dataset while ensuring that the accuracy of the old dataset does not decrease or decreases relatively little.

[0087] like Figure 2 As shown, step S13 includes:

[0088] Step S131: Determine the first weight based on the parameters of the i-th first convolutional kernel in the L-th layer and the sum of the parameters of the 1st to i-th first convolutional kernels in the L-th layer; wherein, L is an integer greater than 0 and i is an integer greater than 0.

[0089] Step S132: Determine the second weights based on the parameters of the i-th second convolutional kernel in the L-th layer and the sum of the parameters of the 1st to i-th second convolutional kernels in the L-th layer;

[0090] Step S133: Based on the first weight and the first convolution kernel, and the second weight and the second convolution kernel, determine the i-th fusion convolution kernel of the L-th layer.

[0091] In this embodiment of the invention, the first classification model, the second classification model, and the fusion classification model may each include L convolutional layers; any convolutional layer may include i convolutional kernels.

[0092] Here, the first weight in step S131 refers to the first weight of the i-th first convolutional kernel in layer L; of course, the first weights of each first convolutional kernel in other layers or other first convolutional kernels in layer L can be obtained in the same way as in step S131. The second weight in step S132 refers to the second weight of the i-th second convolutional kernel in layer L; of course, the second weights of each second convolutional kernel in other layers or other second convolutional kernels in layer L can be obtained in the same way as in step S132.

[0093] In one embodiment, step S131 includes:

[0094] The first sum of squares is determined based on the sum of squares of all parameters of the first convolution kernel in the Lth layer;

[0095] Determine the second sum of squares based on the sum of the first to i first sums of squares;

[0096] The first weight of the first convolution kernel of the Lth layer is determined based on the ratio of the first sum of squares to the second sum of squares.

[0097] Here, obtaining the first weights of each first convolutional kernel in layers 1 to L-1 and obtaining the first weights of the first convolutional kernels in layer L from layer 1 to layer i-1 is similar to obtaining the first weights of the i-th first convolutional kernel in layer L.

[0098] For example, a convolutional kernel K of size C1×k×k is split into two convolutional kernels K1 and K2 of size C1×k×k; the parameter values ​​of K1 and K2 are the parameter values ​​of the convolutional kernels of the first classification model and the second classification model, respectively. The terminal determines the second weight of the i-th second convolutional kernel in the L-th layer as... in, It is the sum of squares of all parameters of the i-th first convolutional kernel in the L-th layer.

[0099] In one embodiment, step S132 includes:

[0100] The third sum of squares is determined based on the sum of squares of all parameters of the i-th second convolution kernel in the L-th layer;

[0101] Determine the fourth sum of squares based on the sum of the first to the i-th third sums of squares;

[0102] The second weight of the i-th second convolution kernel in layer L is determined based on the ratio of the i-th third sum of squares to the fourth sum of squares.

[0103] Here, obtaining the second weights of each second convolutional kernel in layers 1 to L-1 and obtaining the second weights of the first to i-1 second convolutional kernels in layer L is similar to obtaining the second weights of the i-th second convolutional kernel in layer L.

[0104] For example, the terminal determines the second weight of the i-th second convolutional kernel in the L-th layer as... in, It is the sum of squares of all parameters of the i-th second convolutional kernel in the L-th layer.

[0105] In one embodiment, step S133 includes:

[0106] The first value is determined based on the product of the i-th first weight of the L-th layer and the i-th first convolution kernel of the L-th layer;

[0107] The second value is determined based on the product of the i-th second weight in the L-th layer and the i-th second convolution kernel in the L-th layer;

[0108] Based on the sum of the first and second values, the i-th fusion convolution kernel of the L-th layer is determined.

[0109] Here, obtaining each fusion convolution kernel from layer 1 to layer L-1 (i.e., the first to the ith fusion convolution kernels from layer 1 to layer L-1) and obtaining the first to the ith fusion convolution kernels from layer L are similar to obtaining the ith fusion convolution kernel from layer L.

[0110] For example, the terminal determines the i-th fusion convolutional kernel in the L-th layer as... or in, or As the first weight, or K1 is the second weight; K2 is the first convolution kernel and K1 is the second convolution kernel.

[0111] In one embodiment, the method includes: convolving the feature map based on each fused convolutional kernel of each layer to obtain a convolutional feature map. Here, the convolutional feature map output from the previous layer can be used as the feature map input to the next layer.

[0112] For example, the convolutional relationship of a classification network model can be represented as follows: This is the convolution operator; f1 is the input feature map; f2 is the output convolutional feature map. The terminal uses a fused convolution kernel to convolve the feature maps in the following convolution relationship: Where K1 is the first convolutional kernel of the first classification model, K2 is the second convolutional kernel of the second classification model, f1 is the feature map of the input of the fusion classification model, and f2 is the convolutional feature map of the output of the fusion classification model. As the first weight, The second weighting. Here, the weights of each convolutional kernel in each layer of the first and second classification models can be evaluated, and the relative importance of convolutional kernels at the same location (e.g., the same convolutional kernel in the same layer) can be assessed; thus guiding the convolutional neural network to focus on important parameters.

[0113] In this embodiment of the invention, the fused convolutional kernel can be a weighted sum of convolutional kernels, but the number of parameters of the convolutional kernels remains unchanged. The parameter values ​​of each convolutional kernel (e.g., the first sum of squares, the second sum of squares) are generated in a weighted summation manner, so the model size does not increase. Thus, the convolution based on the fused convolutional kernel in this embodiment of the invention can provide initial parameter values ​​for the next step (such as the connection layer of a fused classification model), and these initial parameter values ​​combine the model parameters of the first and second classification models.

[0114] In some embodiments, step S14 includes: determining the fused classification output based on the sum of the first number of classification outputs and the second number of classification outputs.

[0115] In one embodiment, the classification output can be the number of nodes in the last layer of the fully connected layer of the first classification model or the second classification model; the fusion classification output can be the number of nodes in the last layer of the fully connected layer of the fusion classification model. For example, the first number of classification outputs is K1, the second number of classification outputs is K2, and the fusion classification output is K1+K2.

[0116] In another embodiment, the classification output can be the parameters of the last layer of the fully connected layer of the first classification model or the second classification model; the fusion classification output can be the number of nodes in the last layer of the fully connected layer of the fusion classification model.

[0117] For example, in a typical convolutional neural network classification model, the number of nodes in the last layer is the same as the category input to the classification model (e.g., the category of skin disease or skin lesions). The number of nodes in the last layer of the fully connected layers in the first and second classification models are K1 and K2, respectively; let S be the number of nodes in the penultimate layer of the fully connected layers in the classification model, then the parameters of the last layer of the fully connected layers in the first and second classification models are S×K1 and S×K2, respectively. The number of nodes in the last layer of the fully connected layers in the fusion classification model can be extended to K1+K2; the parameters of the last layer of the fully connected layers in the fusion classification model can be extended to S×K1+S×K2. Here, the fusion classification model can directly add parameters and nodes to the fully connected layers.

[0118] Thus, in this embodiment of the invention, the sum of the classification outputs of the first classification model and the second classification model can be used to directly determine the fusion classification output of the fusion classification model; this enables the fusion of parameters of the fully connected layer, which is beneficial for constructing the fusion classification model.

[0119] like Figure 3 As shown, step S15 includes:

[0120] Step S151: Based on the i-th parameter of the L-th layer of the first classification model and the sum of the 1st to nth parameters of the L-th layer, determine the i-th first proportion of the L-th layer; wherein, L is an integer greater than 0, i is an integer greater than 0, and n is greater than or equal to i;

[0121] Step S152: Based on the difference between the parameters of the i-th fusion classification model in the L-th layer and the parameters of the i-th first classification model in the L-th layer, determine the i-th first difference in the L-th layer;

[0122] Step S153: Determine the second difference based on the sum of the products of the first proportions of the first to the i-th values ​​of the first to the L-th layers and the i-th first difference of the L-th layer;

[0123] Step S154: Determine the total loss function based on the second loss function of the second classification model and the second difference.

[0124] In one embodiment, n is the total number of fused convolutional kernels in layer L.

[0125] In one embodiment, step S151: Based on the ratio of the absolute value of the i-th parameter of the L-th layer of the first classification model to the sum of the absolute values ​​of the 1st to nth parameters of the L-th layer, determine the i-th first proportion of the L-th layer.

[0126] For example, the terminal determines that the first proportion of the i-th element in the L-th layer is... Among them, P 1,L,i Let ∑ be the i-th parameter of the L-th layer of the first classification model; n |P 1,L,n | represents the sum of the absolute values ​​of the 1st to nth parameters in layer L. Here, the first proportion, i.e., the proportion of the absolute values ​​of the parameters in the same layer of the network, can characterize the importance of the parameters in that layer of the network. Here, the first proportion can be normalized.

[0127] For example, the terminal determines the i-th first difference in the L-th layer as: P L,i -P 1,L,i .

[0128] In one embodiment, step S153: determine the second difference based on the sum of the products of the first proportions of the first to the i-th layers of the first to the L-th layers and the squares of the i-th first difference of the L-th layer.

[0129] For example, the terminal determines the second difference as: ∑ L,i λ L,i ×(P L,i -P 1,L,i ); where P L,i Let P be the i-th parameter of the L-th layer of the fusion classification model. 1,L,i λ is the i-th parameter of the L-th layer of the first classification model; L,i For the i-th first proportion of the L-th layer; ∑ L,i This is the formula for summation.

[0130] In one embodiment, step S154 includes: determining the total loss function based on the product of the third weight and the second loss function, and the product of the fourth weight and the second difference; wherein the sum of the third weight and the fourth weight is a predetermined value.

[0131] The predetermined value in step S154 is the fourth predetermined value. In one embodiment, the fourth predetermined value can be 1 or 100%, etc. Both the third weight and the fourth weight can be values ​​greater than or equal to 0 and less than or equal to 1.

[0132] In one embodiment, determining the total loss function based on the product of the third weight and the second loss function, and the product of the fourth weight and the second difference, includes: determining a first loss portion based on the product of the third weight and the second loss function; determining a second loss portion based on the product of the fourth weight and the second difference; and determining the total loss function based on the sum of the first loss portion and the second loss portion.

[0133] Here, both the second loss function and the total loss function can be classification loss functions, such as the softmax function. Of course, these can also be other classification loss functions; there are no restrictions here.

[0134] For example, the terminal determines the total loss function as: Loss = (1-β)Loss² + β∑ L,i λ L,i ×(P L,i -P 1,L,i ); where Loss2 is the second loss function; 1-β is the third weight, and β is the fourth weight.

[0135] In this embodiment of the invention, the second loss function in the first loss component of the total loss function can characterize the loss of the second training dataset (i.e., the newly added dataset). Reducing this second loss function is beneficial for learning the model on the newly added dataset. The second difference in the second loss component of the total loss function can characterize the difference between the parameters of the first classification model and the parameters of the fusion classification model. The smaller this difference, the more knowledge of the first class model is retained. The third and fourth weights can adjust the importance of the two loss components (the second loss function and the second difference).

[0136] In this embodiment of the invention, the total loss function can be trained using gradient descent to reduce it to a third predetermined value (i.e., the total loss function converges). If the accuracy of the newly added dataset (i.e., the second training dataset) is higher than a predetermined threshold (e.g., A2 or the second predetermined value in the above embodiment), training can be stopped. This yields a fusion classification model that can be used to classify new and old datasets. For example, it can achieve a second predetermined value for the second training dataset while ensuring that the first training dataset achieves a first predetermined value. Alternatively, it can be used to classify datasets from at least two other databases or at least two datasets from two hospitals, ensuring that both datasets reach a predetermined threshold.

[0137] In some embodiments, the pooling layer of the fusion classification model remains unchanged and is not expanded.

[0138] In some embodiments, the method further includes: acquiring a first dataset, wherein the first dataset includes at least one image of a skin disease; and inputting the first dataset into a third classification model to obtain a first classification recognition result.

[0139] And / or, obtain a second dataset, wherein the second dataset includes at least one dermatology image; input the dataset into the third classification model to obtain a second classification recognition result.

[0140] like Figure 4 As shown in the figure, an embodiment of the present invention provides a method for classifying and treating skin diseases, the method comprising:

[0141] Step S21: Obtain a first dataset, wherein the first dataset includes at least one image of a skin disease;

[0142] Step S22: Input the first dataset into the third classification model to obtain the first classification recognition result;

[0143] The total loss function of the third classification model is determined based on the second loss function of the second classification model and the difference between the parameters of the fusion classification model and the parameters of the first classification model; the fusion classification model is determined based on the fusion of the first classification model and the second classification model.

[0144] Here, the first classification model, the second classification model, the third classification model, and the fusion classification model are respectively the first classification model, the second classification model, the third classification model, and the fusion classification model in the above embodiments.

[0145] In one embodiment, the dermatology images included in the first dataset can be of at least one category.

[0146] In one embodiment, the first classification identification result is used to describe at least one category of skin diseases and / or skin lesions in the first dataset.

[0147] In this embodiment of the invention, a trained third classification model can be used to classify datasets from various hospitals, thereby obtaining a first classification recognition result with an accuracy exceeding a threshold.

[0148] In some embodiments, the method includes:

[0149] Obtain a second dataset, wherein the second dataset includes at least one image of a skin disease;

[0150] The dataset is input into the third classification model to obtain the second classification recognition result; wherein, the second dataset may be a dataset obtained from a different database than the first dataset.

[0151] In one embodiment, the dermatology images included in the second dataset can be of at least one category.

[0152] In one embodiment, the second classification identification result is used to describe at least one category of skin diseases and / or skin lesions in the second dataset.

[0153] In one embodiment, the first dataset and the second dataset come from different hospitals.

[0154] In this embodiment of the invention, datasets from different hospitals, etc., can be classified without requiring the datasets to be discharged from the hospital or processed after anonymization, thereby improving the privacy of medical data; and it can also ensure that after the model is discharged from the hospital, it can continue to iterate based on a new dataset even without the previous dataset.

[0155] To further explain any embodiment of the present invention, a specific embodiment is provided below.

[0156] like Figure 5 As shown, this embodiment of the invention provides a training method for a skin disease classification model, executed by a terminal, including the following steps:

[0157] Step S31: The terminal acquires the first training dataset;

[0158] For example, the first training dataset includes N1 training samples of skin diseases of category K1; the first training dataset contains the label of the category to which each skin disease training sample belongs.

[0159] Step S32: The terminal inputs the first training dataset into the original classification model to obtain the first classification model;

[0160] For example, the terminal inputs the first training dataset into the original classification model until the first loss function converges, and then stops training to obtain the first classification model; at this time, the expected accuracy achieved by the first classification model in training the first training dataset is A1.

[0161] Step S33: The terminal obtains the second training dataset;

[0162] For example, the terminal obtains a newly added second training dataset, which includes N2 training samples of skin diseases in K2 categories; the second training dataset includes the label of the category to which each skin disease training sample belongs.

[0163] Step S34: The terminal inputs the second training dataset into the original classification model to obtain the second classification model;

[0164] For example, the terminal inputs the second training dataset into the original classification model until the second loss function converges, and then stops training to obtain the second classification model; at this time, the expected accuracy achieved by the second classification model in training the second training dataset is A2.

[0165] Step S35: The terminal determines the fusion convolution kernel of the fusion classification model;

[0166] Here, step S35 can be an extension of the convolutional layer.

[0167] Here, for neural network (e.g., convolutional neural network) models, there are usually many convolutional layers to extract feature maps. For example, for a feature map of size f1 = C1 × W1 × H1 in the previous layer, after convolution by C2 convolutional kernels of size C1 × k × k, a feature map of size f2 = C2 × W2 × H2 in the next layer is obtained; where C2 × C1 × k × k parameters (ignoring bias) represent the knowledge induction learned by the neural network model in this layer, i.e., feature extraction parameters.

[0168] For example, the terminal fuses the convolutional kernels of the same layer of the first classification model and the second classification model. Taking the C2 convolutional kernels of size C1×k×k mentioned above as an example, the fusion method of convolutional kernels is explained. For example, a convolutional kernel K of size C1×k×k is split into two convolutional kernels K1 and K2 of size C1×k×k; the parameter values ​​of K1 and K2 are the parameter values ​​of the convolutional kernels of the first classification model and the second classification model, respectively. For the C2 convolutional kernels of this layer, each convolutional kernel has C1×k×k parameters; the sum of squares of all parameters of the i-th convolutional kernel (the first convolutional kernel and the second convolutional kernel) of the first classification model and the second classification model are respectively... and The first weight corresponding to the first convolutional kernel is And the second weight corresponding to the second convolution kernel is The convolution relationship where the terminal uses fused convolution kernels to convolve the feature maps is as follows: Where K1 is the first convolution kernel of the first classification model, K2 is the second convolution kernel of the second classification model, f1 is the feature map of the input of the fusion classification model, and f2 is the convolution feature map of the output of the fusion classification model.

[0169] Here, step S35 is logically a weighted summation of the convolutional kernels (the first convolutional kernel and the second convolutional kernel), but in practice, the number of parameters in the convolutional kernel (the fusion convolutional kernel and either the first or second convolutional kernel) remains unchanged. The parameter values ​​are generated according to the weighted summation method, so the size of the fusion classification model does not increase. Step S35 provides initial parameter values ​​for the next step, knowledge expansion, which integrates the parameters of the first and second classification models corresponding to the old and new datasets (the first training dataset and the second training dataset).

[0170] Step S36: The terminal determines the classification output of the fusion classification model;

[0171] Here, step S36 can be an extension of the fully connected layer. The classification output can be the number of nodes in the last layer.

[0172] For example, the number of nodes in the last layer of the first and second classification models are K1 and K2, respectively. Let the number of nodes in the penultimate layer be S; the parameters of the last layer of the first and second classification models are S×K1 and S×K2, respectively. When expanding the fusion classification model, the last layer is expanded to have K1+K2 nodes, and the parameters of the last layer are S×K1+S×K2.

[0173] Here, the fully connected layer of the fusion classification model directly adds parameters and the number of nodes, without the need for weighted summation; that is, the last layer of the fully connected layer of the fusion classification model is modified to have K1+K2 classification outputs.

[0174] Step S37: The terminal determines the total loss function of the fusion classification model;

[0175] For example, the proportion of the i-th first element in the L-th layer is determined as follows: The terminal determines the second difference as The terminal determines the total loss function as Loss = (1-β)Loss² + β∑ L,i λ L,i ×(P L,i -P 1,L,i ) ； Among them, P 1,L,i Let ∑ be the i-th parameter of the L-th layer of the first classification model; n |P 1,L,n | represents the sum of the absolute values ​​of the 1st to nth parameters in the Lth layer; Loss2 is the second loss function; 1-β is the third weight, and β is the fourth weight.

[0176] The pooling layer of the fusion classification model remains unchanged and is not expanded.

[0177] Step S38: The terminal is retrained based on the total loss function to determine the third classification model.

[0178] For example, the terminal inputs the second training dataset into the fusion classification model for training until the total loss function converges, thus obtaining the trained third classification model. This third classification model can classify and identify datasets from different hospitals.

[0179] In this embodiment of the invention, it is not necessary to train the entire dataset from scratch; instead, the incremental training dataset (e.g., the second training dataset) is specified each time. Therefore, a significant amount of training time and computational cost can be saved.

[0180] Furthermore, in sensitive scenarios such as multiple hospitals where patients cannot be discharged, the fusion classification model can be continuously iterated based on a new dataset (e.g., a second training dataset) even without an old dataset (e.g., the first training dataset). This allows the fusion classification model to be used to train datasets from other hospitals even when the datasets within a hospital are not discharged, thereby improving the reliability of datasets within each hospital.

[0181] Furthermore, in this embodiment of the invention, the knowledge of the first classification model can be retained, so the classification accuracy of the new dataset (e.g., the second training dataset) can be achieved without reducing or only slightly reducing the classification accuracy of the old dataset (e.g., the first training dataset).

[0182] In one application scenario, taking the classification task of facial skin diseases as an example, we first prepare an initial training dataset of N1 images with K1 categories, where K1 is the number of lesion categories. For example, if the initial training dataset only contains lesions of comedones, papules, and pustules, then K1 = 3. After training the original classification network using these N1 images to a certain accuracy, we stop training the network, resulting in the first classification model (Net1). Net1 can classify facial lesions of comedones, papules, and pustules with a certain accuracy. Suppose that N2 images of lesions with K2 categories are added later, such as K2 representing blackheads and scars, then K2 = 2. We then train a network with the same structure as Net1 (i.e., the original classification model) from scratch, resulting in the second classification model (Net2). By expanding the convolutional and fully connected layers of the convolutional neural network and constructing a total loss function for fusion training, we obtain the third classification model (Net3). This ensures that when Net3 adds the ability to identify new lesions, it has as little impact on the identification of the original lesion categories as possible.

[0183] The table below compares the accuracy of Net1 and the fused Net3 in a facial skin disease scenario. 1000 images were used for each lesion category, tested using a ResNetXt classification model (e.g., Net1 or Net3). It can be seen that while the accuracy after fusion is somewhat lower, it retains some of the original knowledge. When the number of categories in the new dataset (e.g., the second training dataset) is the same, the higher the accuracy of the original categories (e.g., the categories in the first training dataset), the higher the accuracy of the original categories after fusion. Furthermore, the accuracy of the original categories after fusion is higher than the overall accuracy after fusion (e.g., the average of the accuracy of the first and second training datasets).

[0184]

[0185] Table 1

[0186] It is understood that each element in Table 1 above exists independently. These elements are listed in the same table as an example, but this does not mean that all elements in the table must exist simultaneously as shown in the table. The value of each element is independent of the values ​​of any other elements in Table 1. Therefore, those skilled in the art will understand that the value of each element in Table 1 is an independent embodiment.

[0187] It should be noted that the description of the training device for the dermatology classification model below is similar to the description of the training method for the dermatology classification model described above, and the beneficial effects of the same method will not be repeated. For technical details not disclosed in the embodiments of the training device for the dermatology classification model of the present invention, please refer to the description of the embodiments of the training method for the dermatology classification model of the present invention.

[0188] like Figure 6 As shown, this embodiment of the invention provides a training device for a skin disease classification model, the device comprising:

[0189] The first training module 41 is used to input the first training dataset into the original classification model for training to obtain the first classification model; wherein, the first training dataset includes a first number of training samples of skin diseases.

[0190] The first training model 41 is used to input the second training dataset into the original classification model for training to obtain a second classification model; wherein, the second training dataset includes a second number of skin disease training samples.

[0191] The first processing module 42 is used to fuse the first convolutional kernel of the first classification model and the second convolutional kernel of the second classification model to determine the fusion convolutional kernel of the fused classification model.

[0192] The first processing module 42 is used to determine the fusion classification output of the fusion classification model based on the first number of classification outputs of the first classification model and the second number of classification outputs of the second classification model;

[0193] The first processing module 42 is used to determine the total loss function of the fusion classification model based on the second loss function of the second classification model and the difference between the parameters of the fusion classification model and the parameters of the first classification model;

[0194] The second training module 43 is used to input the second training dataset into the fusion classification model for training until the total loss function converges, so as to obtain the trained third classification model.

[0195] The training device for the skin disease classification model provided in this embodiment of the invention can be a terminal.

[0196] In some embodiments, the first processing module 42 is configured to perform the following steps:

[0197] The first weight is determined based on the parameters of the i-th first convolutional kernel in the L-th layer and the sum of the parameters of the 1st to i-th first convolutional kernels in the L-th layer; where L is an integer greater than 0 and i is an integer greater than 0.

[0198] The second weights are determined based on the parameters of the i-th second convolutional kernel in the L-th layer and the sum of the parameters of the 1st to i-th second convolutional kernels in the L-th layer.

[0199] Based on the first weight and the first convolution kernel, and the second weight and the second convolution kernel, the i-th fusion convolution kernel of the L-th layer is determined.

[0200] In some embodiments, the first processing module 42 is configured to determine the fused classification output based on the sum of the first number of classification outputs and the second number of classification outputs.

[0201] In some embodiments, the first processing module 42 is configured to perform the following steps:

[0202] Based on the i-th parameter of the L-th layer of the first classification model and the sum of the 1st to nth parameters of the L-th layer, the i-th first proportion of the L-th layer is determined; wherein, L is an integer greater than 0, i is an integer greater than 0, and n is greater than or equal to i;

[0203] Based on the difference between the parameters of the i-th fusion classification model in layer L and the parameters of the i-th first classification model in layer L, the i-th first difference in layer L is determined.

[0204] The second difference is determined by the sum of the products of the first proportions of the first to the i-th layers of the first to the L-th layers and the i-th first difference of the L-th layer.

[0205] The total loss function is determined based on the second loss function of the second classification model and the second difference.

[0206] The first processing module 42 is used to determine the total loss function based on the product of the third weight and the second loss function, and the product of the fourth weight and the second difference; wherein the sum of the third weight and the fourth weight is a predetermined value.

[0207] It should be noted that the description of the following section on the skin disease classification and treatment device is similar to the description of the above section on the skin disease classification and treatment method, and the beneficial effects of the same method will not be repeated. For technical details not disclosed in the embodiments of the skin disease classification and treatment device of the present invention, please refer to the description of the embodiments of the skin disease classification and treatment method of the present invention.

[0208] like Figure 7 As shown, an embodiment of the present invention provides a skin disease classification and treatment device, the device comprising:

[0209] Acquisition module 51 is used to acquire a first dataset, wherein the first dataset includes at least one image of a skin disease;

[0210] The third processing module 52 is used to input the first dataset into the third classification model to obtain the first classification recognition result; wherein, the total loss function of the third classification model is determined based on the second loss function of the second classification model and the difference between the parameters of the fusion classification model and the parameters of the first classification model; the fusion classification model is determined based on the fusion of the first classification model and the second classification model.

[0211] The skin disease classification and treatment device provided in this embodiment of the invention can be a terminal.

[0212] In some embodiments of the present invention, the first classification model, the second classification model, the fusion classification model, and the third classification model can be the first classification model, the second classification model, the fusion classification model, and the third classification model in the above embodiments, respectively; the second loss function and the total loss function are the second loss function and the total loss function in the above embodiments, respectively.

[0213] In some embodiments, the acquisition module 51 is used to acquire a second dataset, wherein the second dataset includes at least one skin disease image;

[0214] The third processing module is used to input the dataset into the third classification model to obtain a second classification recognition result; wherein, the second dataset may be a dataset obtained from a different database than the first dataset.

[0215] like Figure 8 As shown, this embodiment of the invention also provides a terminal, which includes a processor 61 and a memory 62 for storing computer programs that can run on the processor 61; wherein, when the processor 61 runs the computer program, it implements the training method or the skin disease classification processing method of any embodiment of the invention.

[0216] In some embodiments of the present invention, the memory may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory may be random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate Synchronous DRAM (DDRSDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0217] The processor may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the processor's hardware or by software instructions. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this invention can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0218] In some embodiments, the embodiments described herein can be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in the invention, or combinations thereof.

[0219] For software implementation, the techniques described herein can be achieved through modules (e.g., procedures, functions, etc.) that perform the functions described herein. The software code can be stored in memory and executed by a processor. The memory can be implemented within the processor or externally.

[0220] Another embodiment of the present invention provides a computer storage medium storing an executable program. When the executable program is executed by a processor, it can implement the steps of the training method or the skin disease classification processing method of any embodiment of the present invention.

[0221] In some embodiments, the computer storage medium may include various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0222] It should be noted that the technical solutions described in the embodiments of the present invention can be combined arbitrarily without conflict.

[0223] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A training method for a skin disease classification model, characterized in that, The method includes: The first training dataset is input into the original classification model for training to obtain the first classification model; wherein, the first training dataset includes a first number of training samples of skin diseases in different categories; The second training dataset is input into the original classification model for training to obtain the second classification model; wherein, the second training dataset includes a second number of training samples of skin diseases. The first convolutional kernel of the first classification model and the second convolutional kernel of the second classification model are fused to determine the fusion convolutional kernel of the fused classification model; Based on the first number of classification outputs of the first classification model and the second number of classification outputs of the second classification model, the fusion classification output of the fusion classification model is determined; Based on the second loss function of the second classification model and the difference between the parameters of the fusion classification model and the parameters of the first classification model, the total loss function of the fusion classification model is determined; wherein, determining the total loss function of the fusion classification model based on the second loss function of the second classification model and the difference between the parameters of the fusion classification model and the parameters of the first classification model includes: determining the first proportion of the i-th parameter in the L-th layer based on the i-th parameter of the first classification model and the sum of the 1st to nth parameters in the L-th layer; wherein L is greater than 0. Integer, where i is an integer greater than 0; n is greater than or equal to i; the first difference in the Lth layer is determined based on the difference between the parameters of the i-th fusion classification model in the Lth layer and the parameters of the i-th first classification model in the Lth layer; the second difference is determined based on the sum of the products of the first proportions of the 1st to ith layers in the Lth layer and the i-th first difference in the Lth layer; the total loss function is determined based on the product of the third weight and the second loss function, and the product of the fourth weight and the second difference; wherein the sum of the third weight and the fourth weight is a predetermined value; The second training dataset is input into the fusion classification model for training until the total loss function converges, thereby obtaining the trained third classification model.

2. The method according to claim 1, characterized in that, The step of fusing the first convolutional kernel of the first classification model and the second convolutional kernel of the second classification model to determine the fusion convolutional kernel of the fused classification model includes: The first weight is determined based on the parameters of the i-th first convolutional kernel in the L-th layer and the sum of the parameters of the 1st to i-th first convolutional kernels in the L-th layer; where L is an integer greater than 0 and i is an integer greater than 0. The second weights are determined based on the parameters of the i-th second convolutional kernel in the L-th layer and the sum of the parameters of the 1st to i-th second convolutional kernels in the L-th layer. Based on the first weight and the first convolution kernel, and the second weight and the second convolution kernel, the i-th fusion convolution kernel of the L-th layer is determined.

3. The method according to claim 1 or 2, characterized in that, Determining the fusion classification output of the fusion classification model based on the first number of classification outputs of the first classification model and the second number of classification outputs of the second classification model includes: The fused classification output is determined based on the sum of the first number of classification outputs and the second number of classification outputs.

4. A method for classifying and treating skin diseases, characterized in that, The method includes: Obtain a first dataset, wherein the first dataset includes at least one image of a skin disease; Input the first dataset into the third classification model to obtain the first classification recognition result; The third classification model is a classification model trained by the method according to any one of claims 1 to 3.

5. The method according to claim 4, characterized in that, The method includes: Obtain a second dataset, wherein the second dataset includes at least one image of a skin disease; The second dataset is input into the third classification model to obtain the second classification recognition result; wherein, the second dataset is a dataset obtained from a different database than the first dataset.

6. A training device for a skin disease classification and treatment system, characterized in that, include: The first training module is used to input the first training dataset into the original classification model for training to obtain the first classification model; wherein, the first training dataset includes a first number of training samples of skin diseases in different categories. The first training module is used to input the second training dataset into the original classification model for training to obtain a second classification model; wherein, the second training dataset includes a second number of skin disease training samples; The first processing module is used to fuse the first convolutional kernel of the first classification model and the second convolutional kernel of the second classification model to determine the fusion convolutional kernel of the fused classification model; The first processing module is configured to determine the fusion classification output of the fusion classification model based on the first number of classification outputs of the first classification model and the second number of classification outputs of the second classification model; The first processing module is used to determine the total loss function of the fusion classification model based on the second loss function of the second classification model and the difference between the parameters of the fusion classification model and the parameters of the first classification model; Specifically, the first processing module is configured to: determine the i-th first proportion in the L-th layer based on the i-th parameter of the L-th layer of the first classification model and the sum of the 1st to nth parameters in the L-th layer; wherein L is an integer greater than 0, i is an integer greater than 0, and n is greater than or equal to i; determine the i-th first difference in the L-th layer based on the difference between the parameter of the i-th fusion classification model in the L-th layer and the parameter of the i-th first classification model in the L-th layer; determine the second difference based on the sum of the products of the first proportions of the 1st to i-th layers of the L-th layer and the i-th first difference in the L-th layer; and determine the total loss function based on the product of the third weight and the second loss function and the product of the fourth weight and the second difference; wherein the sum of the third weight and the fourth weight is a predetermined value. The second training module is used to input the second training dataset into the fusion classification model for training until the total loss function converges, so as to obtain the trained third classification model.

7. A terminal, characterized in that, The terminal includes a processor and a memory for storing a computer program that can run on the processor; wherein, when the processor runs the computer program, it implements the training method of the skin disease classification model according to any one of claims 1 to 3, or the skin disease classification processing method according to any one of claims 4 to 5.

8. A computer storage medium, characterized in that, The computer storage medium contains computer-executable instructions, characterized in that the computer-executable instructions are executed by a processor to implement the training method of the skin disease classification model according to any one of claims 1 to 3, or the skin disease classification processing method according to any one of claims 4 to 5.

Images