A method, apparatus and device for identifying and classifying regional skin conditions

Abstract

The application discloses a regional skin disease identification and classification method, device and equipment. Based on a trained semantic segmentation model, a diseased area in a skin disease image is segmented and labeled. The training weight of a non-diseased area is reduced, the interference of a surrounding part is reduced, and the influence of a related edge part of an onset area is reduced. Then, based on a trained skin disease type identification model, the skin disease type in the segmented and labeled diseased area is identified, and the identification accuracy of the regional skin disease can be effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image processing and computer vision, and particularly relates to a regional skin disease recognition and classification method, device and equipment. BACKGROUND

[0002] Most skin diseases are exposed on the body surface, and thus are usually observed by naked eyes or with the help of magnifying glasses and other auxiliary equipment, and are visually diagnosed and classified. Since the types of skin diseases are generally identified and diagnosed by dermatologists by observing the conditions of the affected parts, the diagnostic accuracy of skin diseases is highly dependent on the personal clinical experience of the doctors. Since there are numerous types of skin diseases, misjudgment of the types of skin diseases often occurs.

[0003] With the development of computer vision technology, computer-aided diagnosis has been increasingly applied to the related analysis and processing of medical images, especially in the field of medical imaging. In the medical diagnosis of skin diseases, computer-aided diagnosis can also improve the diagnostic accuracy, improve the work efficiency and reduce the pressure on doctors. In related technologies, the skin diseases are classified and recognized by computer recognition technology according to the photographed skin disease pictures. When the skin diseases are diagnosed by computer vision technology, the types of skin diseases are mainly recognized according to the shape, color, depth and distribution of the skin disease pictures. However, this recognition method has many limitations. Because the photographed skin disease pictures also contain other parts that are not affected, the features generated by different parts are all sent to the deep learning model for training, and thus different skin diseases in the same region are often classified into one category. The classification and recognition effect of the same part with different types of skin diseases is poor when the photographed skin disease pictures are directly used.

[0004] Therefore, how to more quickly and correctly classify and recognize skin diseases is a technical problem to be solved. SUMMARY

[0005] The main purpose of the present application is to provide a regional skin disease recognition and classification method, device and equipment, which aims to solve the technical problem of poor recognition effect of different types of skin diseases in the same part when the types of skin diseases are recognized in related technologies.

[0006] In a first aspect, the present application provides a regional skin disease recognition and classification method, which comprises the following steps:

[0007] segmenting and labeling the affected area in the skin disease image based on the trained semantic segmentation model;

[0008] recognizing the type of skin disease in the segmented and labeled affected area based on the trained skin disease type recognition model.

[0009] In some embodiments, the training of the semantic segmentation model to obtain the trained semantic segmentation model comprises:

[0010] classifying the M skin disease images according to skin disease types, and performing mask annotation on the diseased areas in the M skin disease images through an image annotation tool labelme to obtain a diseased area source data set;

[0011] dividing the diseased area source data set into a segmentation training set and a segmentation test set;

[0012] training the semantic segmentation model based on the segmentation training set, so that the semantic segmentation model learns the features and attributes of the diseased areas;

[0013] based on the segmentation test set, determining whether the loss of the trained semantic segmentation model is less than a preset first loss threshold;

[0014] if yes, determining that the trained semantic segmentation model is the trained semantic segmentation model;

[0015] otherwise, adjusting the trained semantic segmentation model, and increasing the number of training times epochs to continue training the trained semantic segmentation model until the loss of the trained semantic segmentation model is less than the first loss threshold.

[0016] In some embodiments, the method further comprises:

[0017] when training the semantic segmentation model based on the segmentation training set, calculating the difference between the segmentation annotation result output by the semantic segmentation model and the corresponding labelme mask annotation result through a preset first cross-entropy loss function:

[0018]

[0019] wherein C represents the number of skin disease categories, p i is the labelme mask annotation result, q i is the segmentation annotation result output by the semantic segmentation model;

[0020] performing a random gradient descent operation according to the gradient information of the first cross-entropy loss function to update the semantic segmentation model:

[0021]

[0022] wherein t is the number of iterations, θ t is the semantic segmentation model parameter of the tthiteration, η is the learning rate of the semantic segmentation model, and is the random gradient.

[0023] In some embodiments, after the M skin disease images are classified according to the skin disease type, before the diseased area in the M skin disease images is mask-labeled by the image labeling tool labelme, the method further comprises:

[0024] performing data enhancement and data filtering on the M skin disease images.

[0025] In some embodiments, the training of the skin disease type identification model to obtain the trained skin disease type identification model comprises:

[0026] inputting the N skin disease images other than the diseased area source data set into the trained semantic segmentation model, and performing segmentation labeling on the diseased area of the input N skin disease images by the trained semantic segmentation model;

[0027] dividing the diseased area source data set and the N skin disease images after segmentation labeling into an identification training set and an identification test set again;

[0028] training the skin disease type identification model based on the identification test set, and updating the skin disease type identification model by a preset second cross-entropy loss function and a stochastic gradient descent operation during the training;

[0029] evaluating whether the loss of the trained skin disease type identification model is less than a preset second loss threshold based on the identification test set;

[0030] if yes, determining that the trained skin disease type identification model is the trained skin disease type identification model;

[0031] otherwise, adjusting the trained skin disease type identification model, and increasing the number of training times epochs to continue training the trained skin disease type identification model until the loss of the trained skin disease type identification model is less than the second loss threshold.

[0032] In some embodiments, the semantic segmentation model is an ENet model, and the skin disease type identification model is an InceptionV4_overall model.

[0033] The skin disease type identification model InceptionV4_overall is obtained by adding a global attention mechanism GAM to a deep learning original model InceptionV4.

[0034] In some embodiments, the skin disease type identification model is obtained by adding a global attention mechanism GAM to a deep learning original model, comprising:

[0035] performing average pooling on the output of the last layer of the input layer of the deep learning original model to obtain a global feature vector of the whole skin disease image.

[0036] inputting the global feature vector into a full connection layer of the deep learning original model, and calculating a weight of each pixel point:

[0037]

[0038] wherein w i,j is the weight of the i-th row and j-th column pixel point of the skin disease image, F i,j is the corresponding value of the i-th row and j-th column pixel point in the global feature vector, and a is an adjustable hyperparameter;

[0039] applying the weight of the pixel point to the output of the convolution layer to obtain a weighted feature vector;

[0040] inputting the weighted feature vector into the deep learning original model to obtain a skin disease type recognition model.

[0041] In some embodiments, the method further comprises:

[0042] composing the trained ENet model and the trained InceptionV4_overall model into a trained deep learning system ENet-InceptionV4_overall;

[0043] recognizing the skin disease type in the skin disease image through the trained ENet-InceptionV4_overall system.

[0044] In a second aspect, the present application also provides a device for recognizing and classifying regional skin diseases, the device comprising:

[0045] a segmentation module configured to segment and label a diseased area in a skin disease image based on a trained semantic segmentation model;

[0046] a recognition module configured to recognize a skin disease type in the segmented and labeled diseased area based on a trained skin disease type recognition model.

[0047] In a third aspect, the present application also provides a computer device comprising a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein the computer program, when executed by the processor, implements the steps of the method for recognizing and classifying regional skin diseases as described above.

[0048] The application provides a regional skin disease identification and classification method, device and equipment. Based on a trained semantic segmentation model, a diseased area in a skin disease image is segmented and labeled. The training weight of a non-diseased area is reduced, the interference of a surrounding part is reduced, and the influence of a related edge part of the diseased area is reduced. Then, based on a trained skin disease type identification model, the skin disease type in the segmented and labeled diseased area is identified, so that the identification accuracy of the regional skin disease can be effectively improved. BRIEF DESCRIPTION OF DRAWINGS

[0049] In order to more clearly illustrate the technical solutions of the embodiments of the application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0050] Figure 1 A flowchart of a regional skin disease identification and classification method provided by the embodiments of the application is shown.

[0051] Figure 2 A specific flowchart of the regional skin disease identification and classification method is shown.

[0052] Figure 3 A diseased area segmentation and labeling diagram is shown.

[0053] Figure 4 A network structure diagram of an InceptionV4_overall model is shown.

[0054] Figure 5 A network structure diagram of a stem layer of the InceptionV4_overall model is shown.

[0055] Figure 6 A network structure diagram of an Inception-A layer of the InceptionV4_overall model is shown.

[0056] Figure 7 A network structure diagram of an Inception-B layer of the InceptionV4_overall model is shown.

[0057] Figure 8 A network structure diagram of an Inception-C layer of the InceptionV4_overall model is shown.

[0058] Figure 9 A network structure diagram of a Reduction-A layer of the InceptionV4_overall model is shown.

[0059] Figure 10 A network structure schematic diagram of the Reduction-B layer of the InceptionV4_overall model;

[0060] Figure 11 A schematic block diagram of a regional skin disease recognition and classification device provided by an embodiment of the present application;

[0061] Figure 12 A structural schematic block diagram of a computer device related to an embodiment of the present application.

[0062] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0063] The technical solutions in the embodiments of the present application will be clearly and completely described with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0064] The flowchart shown in the accompanying drawings is only an example and does not necessarily include all the contents and operations / steps, nor does it necessarily be executed in the described order. For example, some operations / steps can be decomposed, combined or partially merged, so the actual execution order can be changed according to the actual situation.

[0065] The embodiments of the present application provide a regional skin disease recognition and classification method, device and equipment. The regional skin disease recognition and classification method can be applied to a computer device, which can be a notebook computer, a desktop computer or other electronic equipment.

[0066] Some embodiments of the present application will be described in detail below with reference to the accompanying drawings. In the case of no conflict, the embodiments described below and the features in the embodiments can be combined with each other.

[0067] Please refer to Figure 1 , Figure 1 A flowchart of a regional skin disease recognition and classification method provided by an embodiment of the present application.

[0068] Step S1, based on the trained semantic segmentation model, the diseased area in the skin disease image is segmented and labeled;

[0069] Step S2, based on the trained skin disease type recognition model, the skin disease type in the segmented and labeled diseased area is recognized.

[0070] Specifically, training the semantic segmentation model to obtain the trained semantic segmentation model comprises the following steps:

[0071] In step S101, M skin disease images are classified according to skin disease types, and the diseased areas in the M skin disease images are mask-labeled by using an image labeling tool labelme to obtain a diseased area source data set.

[0072] In step S102, the diseased area source data set is divided into a segmentation training set and a segmentation test set.

[0073] In step S103, the semantic segmentation model is trained based on the segmentation training set, so that the semantic segmentation model learns the features and attributes of the diseased area.

[0074] In step S104, whether the loss of the trained semantic segmentation model is less than a preset first loss threshold is evaluated based on the segmentation test set.

[0075] In step S105, if yes, the trained semantic segmentation model is determined as the trained semantic segmentation model.

[0076] In step S106, otherwise, the trained semantic segmentation model is adjusted, and the number of training times epochs is increased to continue training the trained semantic segmentation model until the loss of the trained semantic segmentation model is less than the first loss threshold.

[0077] It is worth noting that the semantic segmentation model in the embodiment of the present application is an ENet model, and the skin disease type recognition model is an InceptionV4_overall model, wherein the skin disease type recognition model InceptionV4_overall is obtained by adding a global attention mechanism GAM to a deep learning original model InceptionV4.

[0078] As shown in Figure 2 M original images of common skin diseases collected are classified and stored according to skin disease types. In the embodiment, the skin diseases corresponding to the collected skin disease images include 23 types, including atopic dermatitis, bullous disease, lupus erythematosus and other systemic diseases, psoriasis, seborrheic keratosis and other benign tumors, etc. The data set is classified according to categories. Due to shooting problems, there are interference information such as high light exposure, dark corner hair, etc. Therefore, after classifying the skin disease images, the original skin disease image data needs to be preprocessed such as data enhancement and data filtering.

[0079] Further, in the photographed skin disease images, there are not only diseased areas, but also surrounding healthy parts, and the shooting scale is different. The diseased areas in the skin disease images are mask-labeled by using labelme as Figure 3As shown, the mask data set is obtained as the source data set of the diseased area. The ENet semantic segmentation model is selected, and the mask labeled by labelme and the original skin disease image are used Figure 1 For training, the mask folder labels and the original image folder images correspond to the same image name due to the annotation relationship. The source data set of the diseased area is divided into a segmentation training set and a segmentation test set, which are used to train the ENet semantic segmentation model to obtain the trained ENet model.

[0080] Preferably, the ENet semantic segmentation model is trained using the segmentation training set to learn the characteristics and attributes of the key disease area of the skin disease. In the training process, a preset first cross-entropy loss function is used to calculate the difference between the segmentation annotation result output by the semantic segmentation model and the corresponding labelme mask annotation result, and the formula is:

[0081]

[0082] where C represents the number of skin disease categories, p i is the labelme mask annotation result, and q i is the segmentation annotation result output by the semantic segmentation model.

[0083] An optimizer is used to update the model according to the gradient information of the first cross-entropy loss function, so that the model can better fit the training data. Here, the stochastic gradient descent (SGD) is selected, and the formula is:

[0084]

[0085] where t is the iteration number, θ t is the semantic segmentation model parameter of the t-th iteration, η is the learning rate of the semantic segmentation model, is the stochastic gradient.

[0086] Further, in order to evaluate the performance of the semantic segmentation model, an independent segmentation test set is needed to evaluate it. The segmentation test set should have similar feature distribution as the segmentation training set, but cannot overlap with the training set. Various indicators can be used to evaluate the performance of the semantic segmentation model, such as accuracy, recall, F1 score, etc. If the loss of the semantic segmentation model is greater than or equal to the preset first loss threshold, it indicates that the current semantic segmentation model performance is poor, and the loss function is iterated to the set first loss threshold by increasing epochs, adjusting parameters, etc. to obtain a better segmentation model as a trained semantic segmentation model.

[0087] In some embodiments, the skin disease type recognition model is trained to obtain a trained skin disease type recognition model, comprising:

[0088] Step S201, inputting N skin disease images other than the diseased area source data set into the trained semantic segmentation model, and segmenting and labeling the diseased area of the inputted N skin disease images by the trained semantic segmentation model.

[0089] Step S202, dividing the diseased area source data set and the segmented and labeled N skin disease images into an identification training set and an identification test set again.

[0090] Step S203, training the skin disease type identification model based on the identification test set, and updating the skin disease type identification model by a preset second cross-entropy loss function and a random gradient descent operation during the training.

[0091] Step S204, evaluating whether the loss of the trained skin disease type identification model is less than a preset second loss threshold based on the identification test set.

[0092] Step S205, if yes, determining that the trained skin disease type identification model is the trained skin disease type identification model.

[0093] Step S206, otherwise, adjusting the trained skin disease type identification model, and increasing the training times epochs to continue training the trained skin disease type identification model until the loss of the trained skin disease type identification model is less than the second loss threshold.

[0094] Exemplarily, N skin disease images are obtained based on the diseased area source data set, and the trained semantic segmentation model ENet is used to label the key diseased areas of the N skin disease images. More skin disease image data is obtained based on the diseased area source data set, in order to further improve the generalization ability of the model. Then, the trained ENet segmentation model is used to label the key diseased areas of the N skin disease images, and the identification training set and the identification test set are re-divided. In this way, the number of training data can be increased, thereby improving the generalization ability of the model.

[0095] It is worth noting that in this embodiment, a global attention mechanism (GAM) is added to the deep learning original model InceptionV4 to enhance the skin disease classification performance of the model, so that the weight of the labeled region pixel points is higher.

[0096] Specifically, the embodiment uses the deep learning original model InceptionV4 as the skin disease category recognition basic model, and obtains a global feature vector of the whole image by performing average pooling on the output of the last layer of the input layer.

[0097]

[0098] wherein w i,j is the weight of the i-th row and j-th column pixel point of the skin disease image, F i,j is the corresponding value of the i-th row and j-th column pixel point in the global feature vector, and a is an adjustable hyperparameter.

[0099] The weight is applied to the output of the convolutional layer to obtain a weighted feature vector. Then, the weighted feature vector is input into the subsequent layers of InceptionV4 for processing to obtain an InceptionV4 model with added global attention mechanism, named InceptionV4_overall, wherein the network structure of the skin disease type recognition model InceptionV4_overall is as shown in Figure 4 to Figure 10 Specifically, the weighted feature vector is added to the original feature vector to obtain a final feature vector. Then, the final feature vector is input into a fully connected layer for classification operation. In this way, the model can focus on the regions with labeled information, improving the performance of the classifier.

[0100] The InceptionV4_overall model is trained using the newly divided recognition training set, and the weights and other parameters of the InceptionV4_overall model are re-optimized using the second cross-entropy loss function and stochastic gradient descent. Through continuous iteration, the loss function of the model is gradually reduced until it converges to the set second loss threshold. During the training process, some techniques can be used to prevent the model from overfitting, such as early stopping method, regularization, etc. Finally, we will get a model file with high recognition accuracy, which can be used to classify and recognize new skin disease images with region annotation information.

[0101] Preferably, the trained ENet model and the trained InceptionV4_overall model are combined to form a trained deep learning system ENet-InceptionV4_overall. When identifying skin diseases, the skin disease image is input into the trained ENet-InceptionV4_overall system to identify the skin disease category in the skin disease image, so as to classify and diagnose different types of skin diseases.

[0102] The embodiment of the present application provides a regional skin disease identification and classification method, the skin disease area determined in the collected skin disease image is used to train a lightweight semantic segmentation model ENet, a data set with skin disease area annotation information is obtained, the ENet semantic segmentation model is used to segment the skin disease area, the training weight of a non-disease area is reduced, the interference of a surrounding part is reduced, and the influence of a disease area associated edge part is reduced. The deep learning model InceptionV4_overall for identifying and classifying the regional skin disease is retrained by adding a global attention mechanism, key area features of the annotated data set are extracted, and skin disease classification identification and prediction are simultaneously performed, network training is more reasonable and sufficient, and the classification and identification accuracy of the model is effectively improved.

[0103] Please refer to Figure 11 , Figure 11 The embodiment of the present application provides a schematic block diagram of a regional skin disease identification and classification device.

[0104] As Figure 11 shown, the device comprises:

[0105] A segmentation module is configured to perform segmentation annotation on the disease area in the skin disease image based on the trained semantic segmentation model.

[0106] An identification module is configured to identify the skin disease type in the segmented and annotated disease area based on the trained skin disease type identification model.

[0107] The device is further configured to:

[0108] classify M skin disease images according to skin disease types, and perform mask annotation on the disease area in the M skin disease images through an image annotation tool labelme to obtain a disease area source data set;

[0109] divide the disease area source data set into a segmentation training set and a segmentation test set;

[0110] train the semantic segmentation model based on the segmentation training set, so that the semantic segmentation model learns the features and attributes of the disease area;

[0111] evaluate whether the loss of the trained semantic segmentation model is less than a preset first loss threshold based on the segmentation test set;

[0112] if yes, the trained semantic segmentation model is determined as the trained semantic segmentation model;

[0113] Otherwise, the trained semantic segmentation model is adjusted, and the number of training times epochs is increased to continue training the trained semantic segmentation model until the loss of the trained semantic segmentation model is less than the first loss threshold.

[0114] The device is further configured to:

[0115] When training the semantic segmentation model based on the segmentation training set, the difference between the segmentation annotation result output by the semantic segmentation model and the corresponding labelme mask annotation result is calculated by using a preset first cross-entropy loss function.

[0116]

[0117] wherein C represents the number of skin disease categories, p i is the labelme mask annotation result, and q i is the segmentation annotation result output by the semantic segmentation model.

[0118] The semantic segmentation model is updated according to the gradient information of the first cross-entropy loss function by using a random gradient descent operation.

[0119]

[0120] wherein t is the number of iterations, θ t is the semantic segmentation model parameter in the tthiteration, and η is the learning rate of the semantic segmentation model, is the random gradient.

[0121] The device is further configured to:

[0122] After the M skin disease images are classified according to the skin disease types, the M skin disease images are subjected to data enhancement and data filtering before the diseased areas in the M skin disease images are subjected to mask annotation by using the image annotation tool labelme.

[0123] The device is further configured to:

[0124] The N skin disease images other than the diseased area source data set are input into the trained semantic segmentation model, and the trained semantic segmentation model is used to perform segmentation annotation on the diseased areas of the input N skin disease images.

[0125] The diseased area source data set and the N skin disease images subjected to segmentation annotation are divided again into an identification training set and an identification test set.

[0126] The skin disease type identification model is trained based on the identification test set, and the skin disease type identification model is updated by using a preset second cross-entropy loss function and a random gradient descent operation during the training.

[0127] evaluating whether a loss of the trained skin disease type recognition model is less than a preset second loss threshold based on the identified test set;

[0128] If yes, the trained skin disease type recognition model is determined as a trained skin disease type recognition model.

[0129] Otherwise, the trained skin disease type recognition model is adjusted, and the number of training times epochs is increased to continue training the trained skin disease type recognition model until the loss of the trained skin disease type recognition model is less than the second loss threshold.

[0130] The semantic segmentation model is an ENet model, and the skin disease type recognition model is an InceptionV4_overall model.

[0131] The skin disease type recognition model InceptionV4_overall is obtained by adding a global attention mechanism GAM to a deep learning original model InceptionV4.

[0132] The apparatus is further configured to:

[0133] performing average pooling on outputs of a last layer of a cascaded input layer of the deep learning original model to obtain a global feature vector of the whole skin disease image;

[0134] inputting the global feature vector into a fully connected layer of the deep learning original model, and calculating a weight of each pixel point:

[0135]

[0136] wherein w i,j is a weight of an i-th row and j-th column pixel point of the skin disease image, F i,j is a corresponding value of the i-th row and j-th column pixel point in the global feature vector, and a is an adjustable hyperparameter.

[0137] applying the weight of the pixel point to an output of a convolutional layer to obtain a weighted feature vector;

[0138] inputting the weighted feature vector into the deep learning original model to obtain the skin disease type recognition model.

[0139] The apparatus is further configured to:

[0140] composing the trained ENet model and the trained InceptionV4_overall model into a trained deep learning system ENet-InceptionV4_overall.

[0141] The skin disease categories in the skin disease image are recognized by the trained ENet-InceptionV4_overall system.

[0142] It should be noted that the skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described device and each module and unit can refer to the corresponding process in the foregoing embodiments, and will not be described here.

[0143] The device provided in the foregoing embodiments can be implemented in the form of a computer program, which can run on a computer device such as the computer device shown in Figure 12 .

[0144] Please refer to Figure 12 , Figure 12 The computer device provided in the embodiments of the present application is a structural schematic block diagram of a computer device. The computer device can be a terminal.

[0145] As shown in Figure 12 , the computer device includes a processor, a memory and a network interface connected through a system bus, wherein the memory can include a non-volatile storage medium and an internal memory.

[0146] The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions, which when executed, can cause the processor to execute any one of the identification and classification methods of regional skin diseases.

[0147] The processor is used to provide computing and control capabilities to support the operation of the entire computer device.

[0148] The internal memory provides an environment for the running of the computer program in the non-volatile storage medium, which when executed by the processor, can cause the processor to execute any one of the identification and classification methods of regional skin diseases.

[0149] The network interface is used for network communication, such as sending assigned tasks. Those skilled in the art can understand that Figure 12 the structure shown in the foregoing embodiments, only the block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or less components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0150] It should be appreciated that a processor can be a central processing unit (CPU), a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or the like. The general purpose processor can be a microprocessor or the processor can be any conventional processor.

[0151] It should be noted that the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0152] The above-mentioned sequence numbers of the embodiments of the present application are only for description, and do not represent advantages or disadvantages of the embodiments. The above describes only specific implementation manners of the present application, but the protection scope of the present application is not limited to this. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present application, and these modifications or replacements should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for identifying and classifying regional skin diseases, characterized in that, The method comprises the following steps: segmenting and labeling a diseased area in a skin disease image based on a trained semantic segmentation model; identifying a skin disease type in the segmented and labeled diseased area based on a trained skin disease type identification model; wherein the trained semantic segmentation model is obtained by training the semantic segmentation model, comprising: classifying M skin disease images according to skin disease types, and performing mask labeling on the diseased areas in the M skin disease images through an image labeling tool labelme to obtain a diseased area source data set; dividing the diseased area source data set into a segmentation training set and a segmentation test set; training the semantic segmentation model based on the segmentation training set to enable the semantic segmentation model to learn the features and attributes of the diseased area; evaluating whether the loss of the trained semantic segmentation model is less than a preset first loss threshold based on the segmentation test set; if yes, determining that the trained semantic segmentation model is the trained semantic segmentation model; otherwise, adjusting the trained semantic segmentation model and increasing the number of training times epochs to continue training the trained semantic segmentation model until the loss of the trained semantic segmentation model is less than the first loss threshold; wherein the trained skin disease type identification model is obtained by training the skin disease type identification model, comprising: inputting N skin disease images other than the diseased area source data set into the trained semantic segmentation model to segment and label the diseased areas of the inputted N skin disease images through the trained semantic segmentation model; redividing the diseased area source data set and the segmented and labeled N skin disease images into an identification training set and an identification test set; training the skin disease type identification model based on the identification test set, and updating the skin disease type identification model through a preset second cross-entropy loss function and a stochastic gradient descent operation during the training; evaluating whether the loss of the trained skin disease type identification model is less than a preset second loss threshold based on the identification test set; if yes, determining that the trained skin disease type identification model is the trained skin disease type identification model; otherwise, adjusting the trained skin disease type identification model and increasing the number of training times epochs to continue training the trained skin disease type identification model until the loss of the trained skin disease type identification model is less than the second loss threshold.

2. The method of claim 1, wherein the step of identifying the region of skin disease is characterized by, Further comprising: when training the semantic segmentation model based on the segmentation training set, calculating the difference between the segmentation labeling result outputted by the semantic segmentation model and the corresponding labelme mask labeling result through a preset first cross-entropy loss function: Wherein, C represents the number of skin disease categories, The labelme mask annotation result is The segmentation annotation result output by the semantic segmentation model; performing a stochastic gradient descent operation according to the gradient information of the first cross-entropy loss function to update the semantic segmentation model: where t is the iteration number, is the semantic segmentation model parameter of the tth iteration is the learning rate of the semantic segmentation model, is the stochastic gradient.

3. The method of claim 1, wherein the step of identifying the region of skin disease is characterized by, after classifying the M skin disease images according to the skin disease types, before performing mask labeling on the diseased areas in the M skin disease images through the image labeling tool labelme, further comprising: performing data enhancement and data filtering on the M skin disease images.

4. The method of claim 1, wherein the step of identifying the region of skin disease is characterized by, Further comprising: The semantic segmentation model is an ENet model, and the skin disease type recognition model is an InceptionV4_overall model. The skin disease type recognition model InceptionV4_overall is obtained by adding a global attention mechanism GAM to a deep learning original model InceptionV4.

5. The method of claim 4, wherein the step of identifying the region of skin disease is characterized by, The global attention mechanism GAM is added to the deep learning original model to obtain the skin disease type recognition model, including: Performing average pooling on the output of the last layer of the input layer of the deep learning original model to obtain a global feature vector of the whole skin disease image; Inputting the global feature vector into the fully connected layer of the deep learning original model and calculating the weight of each pixel point: wherein, is a weight of the i-th row j-th column pixel point of the skin disease image, is a corresponding value of the i-th row j-th column pixel point in the global feature vector, is an adjustable hyperparameter; Applying the weight of the pixel point to the output of the convolution layer to obtain a weighted feature vector; Inputting the weighted feature vector into the deep learning original model to obtain the skin disease type recognition model.

6. The method of claim 4, wherein the step of identifying the region of skin disease is characterized by, Further comprising: The trained ENet model and the trained InceptionV4_overall model form a trained deep learning system ENet-InceptionV4_overall; The trained ENet-InceptionV4_overall system is used to recognize the skin disease type in the skin disease image.

7. A device for identifying and classifying regional skin diseases, which implements the method for identifying and classifying regional skin diseases according to any one of claims 1 to 6, characterized by Further comprising: A segmentation module for segmenting and labeling the diseased area in the skin disease image based on the trained semantic segmentation model; An identification module for identifying the skin disease type in the segmented and labeled diseased area based on the trained skin disease type recognition model; The device is further configured to: Classify the M skin disease images according to the skin disease type, and perform mask labeling on the diseased area in the M skin disease images through the image labeling tool labelme to obtain a diseased area source data set; Divide the diseased area source data set into a segmentation training set and a segmentation test set; Train the semantic segmentation model based on the segmentation training set to enable the semantic segmentation model to learn the features and attributes of the diseased area; Evaluate whether the loss of the trained semantic segmentation model is less than a preset first loss threshold based on the segmentation test set; If yes, the trained semantic segmentation model is determined to be a trained semantic segmentation model; Otherwise, adjust the trained semantic segmentation model and increase the number of training times epochs to continue training the trained semantic segmentation model until the loss of the trained semantic segmentation model is less than the first loss threshold. The device is further configured to: Input N skin disease images other than the diseased area source data set into the trained semantic segmentation model to segment and label the diseased area of the input N skin disease images through the trained semantic segmentation model; Divide the diseased area source data set and the N skin disease images segmented and labeled again into an identification training set and an identification test set; Train the skin disease type recognition model based on the identification test set, and update the skin disease type recognition model through a preset second cross-entropy loss function and a stochastic gradient descent operation during training; evaluating whether a loss of the trained skin disease type recognition model is less than a preset second loss threshold based on the identified test set; if yes, determining that the trained skin disease type recognition model is a trained skin disease type recognition model; otherwise, adjusting the trained skin disease type recognition model, and increasing a training times epochs to continue training the trained skin disease type recognition model until the loss of the trained skin disease type recognition model is less than the second loss threshold.

8. A computer device, comprising: The computer device comprises a processor, a memory, and a computer program stored on the memory and executable by the processor, wherein the computer program, when executed by the processor, implements the steps of the regional skin disease recognition classification method according to any one of claims 1 to 6.

Images