Skin disease classification and recognition system based on deep learning

Abstract

The invention belongs to the technical field of image recognition, and particularly relates to a skin disease classification recognition system based on deep learning, which comprises a skin image acquisition module, a skin image preprocessing module, a skin state classification module and a state classification optimization module, according to the system, a skin state recognition model is constructed, and a multi-head self-attention mechanism is introduced, so that skin state differences can be distinguished; local features are extracted through a space exploration CNN block, an AssemFormer block learns a global dependency relationship, the local features and the global features are fused, and a lesion mode can be understood; scale variant attention blocks are used for highlighting skin features of suspected lesion areas, Monte Carlo attention blocks are used for carrying out random sampling pooling calculation on feature representation, attention map enhancement blocks are used for generating attention maps, skin state categories are output, and skin state prediction labels are generated for each skin image, so that skin lesion details can be highlighted; quantitative reference is provided for medical staff, and skin state judgment is effectively assisted.

Description

Technical Field

[0001] The present invention belongs to the technical field of image recognition, and specifically refers to a skin disease classification and recognition system based on deep learning. Background Art

[0002] The skin disease classification and identification system is a tool that uses advanced technology to classify and identify skin diseases, and can objectively identify skin conditions;

[0003] However, the current skin disease classification and recognition system has the following shortcomings:

[0004] When dealing with mutations of similar size, the recognition is insufficient and the features of suspected lesion areas cannot be accurately highlighted. In terms of local feature extraction, the lesion edges and lesion textures cannot be highlighted, and the inter-class contrast and local homogeneous changes cannot be effectively learned, resulting in insufficient overall recognition ability and easy confusion between skin fungal infections and allergic reactions;

[0005] Due to the lack of an effective attention mechanism, it is impossible to fully capture multi-perspective information and it is difficult to accurately identify complex skin textures and structures. For some skin diseases with similar early symptoms, in the early stages of eczema and psoriasis, the ability to distinguish subtle differences between different skin diseases is limited due to inaccurate feature extraction, and the ability to adapt to changes in different skin characteristics is insufficient in practical applications. Summary of the invention

[0006] In view of the above problems, the present invention provides a skin disease classification and recognition system based on deep learning, introduces a multi-head self-attention mechanism, constructs a skin state recognition model, and helps to distinguish skin state differences; first, local features are extracted through the spatial exploration CNN block, the AssemFormer block learns global dependencies, and the local features are integrated with the global features, which helps to understand the lesion pattern and improve the accuracy of prediction; secondly, the scale variant attention block is used to highlight the skin features of the suspected lesion area, the Monte Carlo attention block randomly samples and pools to calculate the feature representation, the residual learning CNN block enriches the feature representation, and the attention map enhancement block generates the attention map; finally, the attention map is spliced ​​through the fully connected layer, the skin state category is output, and the skin state prediction label is generated for each skin image; the system can highlight the details of skin lesions, provide quantitative reference for medical staff, and effectively assist in judging the skin state.

[0007] A skin disease classification and recognition system based on deep learning, comprising a skin image acquisition module, a skin image preprocessing module, a skin state classification module and a state classification optimization module;

[0008] The skin image acquisition module is connected to the skin image preprocessing module, and the skin image preprocessing module is connected to the skin state classification module;

[0009] The skin image acquisition module uses a skin imaging device to acquire skin images of suspected lesion areas and annotate image information for the skin images;

[0010] The skin image preprocessing module preprocesses the skin image to obtain a preprocessed image;

[0011] The skin state classification module introduces a multi-head self-attention mechanism to construct a skin state recognition model, uses the multi-head self-attention mechanism to extract skin features, calculates the feature representation of the skin features, generates an attention map based on the feature representation, comprehensively splices the attention map to classify the skin state through the softmax function, and generates a skin state prediction label;

[0012] The state classification optimization module sets the parameters of the skin state recognition model, uses the cross entropy loss function to measure the difference between the skin state prediction label and the actual skin state, iterates the training to update the parameters, sets the evaluation index to evaluate the performance of the skin state recognition model, and adjusts the hyperparameters to optimize the skin state recognition model;

[0013] The skin image preprocessing module preprocesses the skin image to obtain a preprocessed image, comprising the following steps:

[0014] Step P1: Image format unification: convert the skin image into a unified format, crop the skin image, and focus on the suspected lesion area;

[0015] Step P2: Image enhancement: Randomly rotate, randomly flip and scale the skin image, adjust the brightness, contrast and color balance of the skin image, and highlight the details of the lesion;

[0016] Step P3: Image normalization: Use the zero-mean normalization method to normalize the pixel values ​​of the skin image, map the pixel values ​​to a specific interval, and obtain a preprocessed image.

[0017] The skin state recognition model constructed by the skin state classification module includes a space exploration CNN block, an AssemFormer block, a scale variant attention block, a Monte Carlo attention block, a residual learning CNN block and an attention map enhancement block.

[0018] The skin state classification module introduces a multi-head self-attention mechanism to construct a skin state recognition model, uses the multi-head self-attention mechanism to extract skin features, calculates the feature representation of the skin features, generates an attention map based on the feature representation, and comprehensively splices the attention map to classify the skin state through the softmax function to generate a skin state prediction label. The process includes the following steps:

[0019] Step S1: Analyze the preprocessed image: divide the preprocessed image into basic units, divide each basic unit into several sub-regions, use the multi-head self-attention mechanism to calculate the similarity score of the basic unit, obtain the calculation result, capture the global dependency relationship based on the calculation result, and extract the skin features;

[0020] Step S2: highlighting skin features: setting the time step, performing weighted summation on the sub-regions through the scale-variant attention block, generating a cross-scale attention map, and displaying the prominent skin features of the suspected lesion area based on the cross-scale attention map;

[0021] To generate a cross-scale attention map, the formula used is as follows:

[0022] ;

[0023] Among them, t represents the current time step, Represents the basic unit, represents the cross-scale attention map, represents the normalization function, n represents the total number of sub-regions, Indicates traversing the sub-region of the basic unit, s indicates the sequence number of the previous time step, represents the time step before the current time step, represents the j-th sub-region at the s-th time step, Represents the probability of controlling the correspondence between basic units across time steps;

[0024] Step S3: Random sampling pooling: Through the Monte Carlo attention block, randomly select the pooling tensor, and perform random sampling pooling operations on the basic units through the average pooling function to calculate the feature representation of each prominent skin feature;

[0025] Step S4: enriching feature representation: using residual learning CNN blocks, focusing on local features and global features, using an inverted bottleneck structure, increasing the number of channels to enrich the feature representation of prominent skin features, and obtaining an overall feature representation;

[0026] Step S5: Generate an attention map: Through the attention map enhancement block, calculate the contribution of the feature representation to the overall feature representation, determine the association probability based on the contribution, and generate an attention map based on the pooled tensor combined with the association probability;

[0027] To generate the attention map, the formula used is as follows:

[0028] ;

[0029] Among them, m represents the mark that distinguishes it from the cross-scale attention map formula, represents the attention map, i represents the sequence number of the pooling tensor, represents the total number of pooled tensors, represents the association probability, represents the average pooling function;

[0030] Step S6: classify skin conditions: use the fully connected layer of the attention map enhancement block to comprehensively splice the attention map, use the softmax function to classify the skin conditions, output the skin condition category, and generate a skin condition prediction label for each skin image;

[0031] The skin condition prediction label is generated for each skin image. The formula used is as follows:

[0032] ;

[0033] Where X represents the preprocessed image, represents the skin condition category output by the softmax function, i' represents the basic unit, c represents the total number of skin condition categories, represents the i'th basic unit in the preprocessed image, represents the prediction score of the i'th basic unit in the preprocessed image, Represents the index sum of all skin condition category scores.

[0034] Furthermore, step S1 includes the following steps:

[0035] Step S11: assigning position information: dividing the preprocessed image into a plurality of non-overlapping basic units, assigning position information to each basic unit, and using the position information to clarify the position of the basic unit in the preprocessed image;

[0036] Step S12: Introducing a self-attention mechanism: Introducing a multi-head self-attention mechanism, analyzing the basic units of the preprocessed image through different subspaces in the self-attention mechanism, calculating the similarity score between each basic unit of the preprocessed image in each subspace, and obtaining the calculation result;

[0037] The calculation results are obtained using the following formula:

[0038] ;

[0039] in, They represent the query vector, key vector, and value vector in the multi-head self-attention mechanism, respectively. Represents the calculation result of the multi-head self-attention mechanism, represents the dimension of the key vector K, Represents a normalization operation;

[0040] Step S13: Extract skin features: Integrate the calculation results of each subspace, capture the global dependencies between basic units, and extract skin features.

[0041] Further, step S13 includes the following steps:

[0042] Step S131: extracting local features: through the convolution layer and pooling layer of the spatial exploration CNN block, 3×3 convolution and 1×1 convolution operations are performed on each basic unit to learn the local features of the suspected lesion area, and pooling operations are performed on each basic unit to highlight the inter-class contrast and local homogeneous changes, gradually extract high-level abstract local features, and generate a local feature map containing local features;

[0043] Step S132: extracting global features: using the AssemFormer block, learning the global dependency between local features, extracting the global features of the suspected lesion area, and generating a global feature map containing global features, the global features including morphology, depth, color distribution, and position association;

[0044] Generate a global feature map containing global features. The formula used is as follows:

[0045] ;

[0046] in, represents the global feature map, represents a scalar function, represents the basic unit centered at the jth sub-region;

[0047] Step S133: Fusion of global features: Through the AssemFormer block, a convolution operation is performed on the global feature map, local features and global features are fused, and a fused feature map is output to provide skin features of the suspected lesion area;

[0048] The convolution operation is performed on the global feature map, and the formula used is as follows:

[0049] ;

[0050] in, represents the basic unit coordinates, represents the sub-region coordinates, represents the convolutional layer index of the spatial exploration CNN block, Represents the convolution kernel number index, Indicates The kth convolution kernel of layer is at position The convolution result at , c' represents the channel index, Represents the preprocessed image in the sub-region coordinates The local features of Indicates The kth convolution kernel of the layer is in the basic unit coordinates Local features of the place.

[0051] The state classification optimization module sets the parameters of the skin state recognition model, uses the cross entropy loss function to measure the difference between the skin state prediction label and the actual skin state, iterates the training to update the parameters, sets the evaluation index to evaluate the performance of the skin state recognition model, and adjusts the hyperparameters to optimize the skin state recognition model, including the following steps:

[0052] Step D1: Divide the preprocessed images: divide the preprocessed images into a training set, a validation set, and a test set;

[0053] Step D2: Define the loss function: set the parameters of the skin state recognition model, use the cross entropy loss function, calculate the cross entropy loss, and measure the difference between the skin state prediction label and the actual skin state;

[0054] To calculate the cross entropy loss, the formula used is as follows:

[0055] ;

[0056] Among them, y represents the real skin condition, represents the skin condition prediction label, represents the cross entropy loss, represents the number of samples, c represents the total number of skin condition categories, i'' represents the sample number, j'' represents the skin condition category number, Indicates the actual skin status category number, represents the predicted value of the skin condition prediction label of sample i'' in category j'';

[0057] Step D3: Update model parameters: set hyperparameters, use the Adam optimizer on the training set, train the skin state recognition model, calculate the gradient based on the cross entropy loss function, and update the parameters of the skin state recognition model;

[0058] Step D4: Evaluate model performance: Set evaluation indicators, evaluate the performance of the skin condition recognition model on the validation set, adjust hyperparameters according to the evaluation indicators, and optimize the skin condition recognition model.

[0059] The beneficial effects achieved by the present invention are as follows:

[0060] (1) The system introduces a multi-head self-attention mechanism to calculate the unit similarity scores from multiple subspaces, and the comprehensive results capture the global dependencies. The multi-head self-attention mechanism can analyze the preprocessed image from multiple perspectives, effectively identify skin texture and skin structure, enhance the expression ability of skin features, and help distinguish the subtle differences between different skin diseases; the spatial exploration CNN block extracts local features through convolution and pooling operations, focusing on local details, including lesion edges and lesion textures, highlighting inter-class contrast and local homogeneous changes, and better capturing local lesion features; the AssemFormer block learns the global dependencies between local features and fuses local and global features, grasps the skin features as a whole, understands the pattern of lesions in the entire preprocessed image, improves the recognition ability of skin diseases, and improves the accuracy of skin condition prediction labels;

[0061] (2) The system uses scale-variant attention blocks to generate cross-scale attention maps, highlight the characteristics of suspected lesion areas, and weightedly highlight lesion details at different scales, thereby improving the model's recognition of lesions of different sizes and shapes. The Monte Carlo attention block randomly samples and pools the feature representation, which can effectively enhance the robustness of the model and better adapt to the changes in skin features of different skin diseases. The residual learning CNN block uses deep separable convolution and inverted bottleneck structure to enrich the feature representation, reduce the amount of calculation while improving feature expression, better capture the complex skin features of skin diseases, further enhance the model's learning and expression capabilities for skin diseases, and significantly improve classification performance.

[0062] (3) The system calculates the association probability through the attention map enhancement block to generate an attention map. The attention map highlights key skin features and guides the model to focus on important information. The fully connected layer integrates various skin features. After splicing the attention maps, the softmax function is used for classification and generates prediction labels. This not only achieves accurate classification, but also provides quantitative reference to assist medical staff in accurately judging skin conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 A schematic diagram of the structure of the Monte Carlo attention block proposed in the present invention;

[0064] Figure 2 This is a schematic diagram of the structure of the AssemFormer block proposed in the present invention;

[0065] Figure 3 This is a schematic diagram of the structure of the residual learning CNN block proposed in the present invention;

[0066] Among them, Conv represents the convolution kernel, Transformer represents the Transformer layer, ReLU represents the nonlinear function, C represents the convolution operation, V represents the vectorization processing, and + represents the feature map addition operation. Represents an element-wise multiplication operation. DETAILED DESCRIPTION

[0067] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0068] Embodiment 1: This embodiment provides a skin disease classification and recognition system based on deep learning, including a skin image acquisition module, a skin image preprocessing module, a skin state classification module and a state classification optimization module;

[0069] The skin image acquisition module is connected to the skin image preprocessing module, and the skin image preprocessing module is connected to the skin state classification module;

[0070] The skin image acquisition module uses a skin imaging device to acquire skin images of suspected lesion areas and annotate image information for the skin images;

[0071] The skin image preprocessing module preprocesses the skin image to obtain a preprocessed image;

[0072] The skin state classification module introduces a multi-head self-attention mechanism to construct a skin state recognition model, uses the multi-head self-attention mechanism to extract skin features, calculates the feature representation of the skin features, generates an attention map based on the feature representation, comprehensively splices the attention map to classify the skin state through the softmax function, and generates a skin state prediction label;

[0073] The state classification optimization module sets the parameters of the skin state recognition model, uses the cross entropy loss function to measure the difference between the skin state prediction label and the actual skin state, iterates the training to update the parameters, sets the evaluation index to evaluate the performance of the skin state recognition model, and adjusts the hyperparameters to optimize the skin state recognition model.

[0074] Embodiment 2: This embodiment is based on the above embodiment. The skin image preprocessing module preprocesses the skin image to obtain a preprocessed image, including the following steps:

[0075] Step P1: Image format unification: convert the skin image into a unified format, crop the skin image, and focus on the suspected lesion area;

[0076] Step P2: Image enhancement: Randomly rotate, randomly flip and scale the skin image, adjust the brightness, contrast and color balance of the skin image, and highlight the details of the lesion;

[0077] Step P3: Image normalization: Use the zero-mean normalization method to normalize the pixel values ​​of the skin image, map the pixel values ​​to a specific interval, and obtain a preprocessed image.

[0078] Embodiment 3: The skin state recognition model constructed by the skin state classification module includes a spatial exploration CNN block, an AssemFormer block, a scale variant attention block, a Monte Carlo attention block, a residual learning CNN block and an attention map enhancement block.

[0079] Embodiment 4: This embodiment is based on Embodiment 1, and the skin state recognition model constructed by the skin state classification module includes a CNN block, a scale variant attention block, a Monte Carlo attention block and an attention map enhancement block.

[0080] Embodiment 5: This embodiment is based on Embodiment 3. The skin state classification module introduces a multi-head self-attention mechanism to construct a skin state recognition model, extracts skin features using the multi-head self-attention mechanism, calculates the feature representation of the skin features, generates an attention map based on the feature representation, and comprehensively splices the attention map to classify the skin state through the softmax function. The process of generating a skin state prediction label includes the following steps:

[0081] Step S1: Analyze the preprocessed image: divide the preprocessed image into basic units, and divide each basic unit into 128 sub-regions;

[0082] Step S1 includes the following steps:

[0083] Step S11: assigning position information: dividing the preprocessed image into 256 non-overlapping basic units, assigning position information to each basic unit, and using the position information to clarify the position of the basic unit in the preprocessed image;

[0084] Step S12: Introducing a self-attention mechanism: Introducing a multi-head self-attention mechanism, analyzing the basic units of the preprocessed image through different subspaces in the self-attention mechanism, calculating the similarity score between each basic unit of the preprocessed image in each subspace, and obtaining the calculation result;

[0085] The calculation results are obtained using the following formula:

[0086] ;

[0087] in, They represent the query vector, key vector, and value vector in the multi-head self-attention mechanism, respectively. Represents the calculation result of the multi-head self-attention mechanism, represents the dimension of the key vector K, Represents a normalization operation;

[0088] Step S13: extracting skin features: integrating the calculation results of each subspace, capturing the global dependency between basic units, and extracting skin features;

[0089] Step S13 includes the following steps:

[0090] Step S131: extracting local features: through the convolution layer and pooling layer of the spatial exploration CNN block, 3×3 convolution and 1×1 convolution operations are performed on each basic unit to learn the local features of the suspected lesion area, and pooling operations are performed on each basic unit to highlight the inter-class contrast and local homogeneous changes, gradually extract high-level abstract local features, and generate a local feature map containing local features;

[0091] Step S132: extracting global features: using the AssemFormer block, learning the global dependency between local features, extracting the global features of the suspected lesion area, and generating a global feature map containing global features, the global features including morphology, depth, color distribution, and position association;

[0092] Generate a global feature map containing global features. The formula used is as follows:

[0093] ;

[0094] in, represents the global feature map, represents a scalar function, represents the basic unit centered at the jth sub-region;

[0095] Step S133: Fusion of global features: Through the AssemFormer block, a convolution operation is performed on the global feature map, local features and global features are fused, and a fused feature map is output to provide skin features of the suspected lesion area;

[0096] The convolution operation is performed on the global feature map, and the formula used is as follows:

[0097] ;

[0098] in, represents the basic unit coordinates, represents the sub-region coordinates, represents the convolutional layer index of the spatial exploration CNN block, Represents the convolution kernel number index, Indicates The kth convolution kernel of layer is at position The convolution result at , c' represents the channel index, Represents the preprocessed image in the sub-region coordinates The local features of Indicates The kth convolution kernel of the layer is in the basic unit coordinates Local features of the place;

[0099] Step S2: highlighting skin features: setting the time step, performing weighted summation on the sub-regions through the scale-variant attention block, generating a cross-scale attention map, and displaying the prominent skin features of the suspected lesion area based on the cross-scale attention map;

[0100] To generate a cross-scale attention map, the formula used is as follows:

[0101] ;

[0102] Among them, t represents the current time step, Represents the basic unit, represents the cross-scale attention map, represents the normalization function, n represents the total number of sub-regions, Indicates traversing the sub-region of the basic unit, s indicates the sequence number of the previous time step, represents the time step before the current time step, represents the j-th sub-region at the s-th time step, Represents the probability of controlling the correspondence between basic units across time steps;

[0103] Step S3: Random sampling pooling: Through the Monte Carlo attention block, randomly select the pooling tensor, and perform random sampling pooling operations on the basic units through the average pooling function to calculate the feature representation of each prominent skin feature;

[0104] Step S4: enriching feature representation: using residual learning CNN blocks, focusing on local features and global features, using an inverted bottleneck structure, increasing the number of channels to enrich the feature representation of prominent skin features, and obtaining an overall feature representation;

[0105] Step S5: Generate an attention map: Through the attention map enhancement block, calculate the contribution of the feature representation to the overall feature representation, determine the association probability based on the contribution, and generate an attention map based on the pooled tensor combined with the association probability;

[0106] To generate the attention map, the formula used is as follows:

[0107] ;

[0108] Among them, m represents the mark that distinguishes it from the cross-scale attention map formula, represents the attention map, i represents the sequence number of the pooling tensor, represents the total number of pooled tensors, represents the association probability, represents the average pooling function;

[0109] Step S6: classify skin conditions: use the fully connected layer of the attention map enhancement block to comprehensively splice the attention map, use the softmax function to classify the skin conditions, output the skin condition category, and generate a skin condition prediction label for each skin image;

[0110] The skin condition prediction label is generated for each skin image. The formula used is as follows:

[0111] ;

[0112] Where X represents the preprocessed image, represents the skin condition category output by the softmax function, i' represents the basic unit, c represents the total number of skin condition categories, represents the i'th basic unit in the preprocessed image, represents the prediction score of the i'th basic unit in the preprocessed image, Represents the index sum of all skin condition category scores.

[0113] Embodiment 6: This embodiment is based on Embodiment 4, and the process of generating a skin state prediction label by the skin state classification module includes the following steps:

[0114] Step M1: Analyze the preprocessed image: divide the preprocessed image into basic units, extract local features through convolution and pooling operations, concatenate local features to obtain global features, and directly merge local features and global features to obtain skin features;

[0115] Step M2: highlight skin features: set the time step, perform weighted summation on the sub-regions through the scale variant attention block, generate a cross-scale attention map, and display the prominent skin features of the suspected lesion area based on the cross-scale attention map;

[0116] To generate a cross-scale attention map, the formula used is as follows:

[0117] ;

[0118] Among them, t represents the current time step, Represents the basic unit, represents the cross-scale attention map, represents the normalization function, n represents the total number of sub-regions, Indicates traversing the sub-region of the basic unit, s indicates the sequence number of the previous time step, represents the time step before the current time step, represents the j-th sub-region at the s-th time step, Represents the probability of controlling the correspondence between basic units across time steps;

[0119] Step M3: Random sampling pooling: Through the Monte Carlo attention block, randomly select the pooling tensor, and perform random sampling pooling operations on the basic units through the average pooling function to calculate the feature representation of each prominent skin feature, and generate an attention map based on the feature representation;

[0120] Step M4: Classify skin conditions: Use the fully connected layer of the attention map enhancement block to comprehensively concatenate the attention maps, use the softmax function to classify skin conditions, output the skin condition category, and generate a skin condition prediction label for each skin image;

[0121] The skin condition prediction label is generated for each skin image. The formula used is as follows:

[0122] ;

[0123] Where X represents the preprocessed image, represents the skin condition category output by the softmax function, i' represents the basic unit, c represents the total number of skin condition categories, represents the i'th basic unit in the preprocessed image, represents the prediction score of the i'th basic unit in the preprocessed image, Represents the index sum of all skin condition category scores.

[0124] Embodiment 7: This embodiment is based on Embodiment 5. The state classification optimization module sets the parameters of the skin state recognition model, uses the cross entropy loss function to measure the difference between the skin state prediction label and the actual skin state, iterates the training to update the parameters, sets the evaluation index to evaluate the performance of the skin state recognition model, and adjusts the hyperparameters to optimize the skin state recognition model. The process includes the following steps:

[0125] Step D1: Divide the preprocessed images: Divide the preprocessed images into training set, validation set and test set in a ratio of 8:1:1;

[0126] Step D2: Define the loss function: set the parameters of the skin state recognition model, use the cross entropy loss function, calculate the cross entropy loss, and measure the difference between the skin state prediction label and the actual skin state;

[0127] To calculate the cross entropy loss, the formula used is as follows:

[0128] ;

[0129] Among them, y represents the real skin condition, represents the skin condition prediction label, represents the cross entropy loss, represents the number of samples, c represents the total number of skin condition categories, i'' represents the sample number, j'' represents the skin condition category number, Indicates the actual skin status category number, represents the predicted value of the skin condition prediction label of sample i'' in category j'';

[0130] Step D3: Update model parameters: set hyperparameters, learning rate to 0.001, number of iterations to 100, batch size to 32, weight decay to 0.0001, use Adam optimizer on the training set to train the skin state recognition model, calculate the gradient according to the cross entropy loss function, and update the parameters of the skin state recognition model;

[0131] Step D4: Evaluate model performance: Set evaluation indicators, evaluate the performance of the skin condition recognition model on the validation set, adjust hyperparameters according to the evaluation indicators, and optimize the skin condition recognition model.

[0132] The present invention and its implementation methods are described above. Such description is not restrictive. If a person skilled in the art is inspired by it and creatively designs structures and implementation methods similar to the technical solution without departing from the purpose of the present invention, they should all fall within the protection scope of the present invention.

Claims

1. A skin disease classification and recognition system based on deep learning, comprising a skin image acquisition module, wherein the skin image acquisition module uses a skin imaging device to acquire skin images of suspected lesion areas and annotate image information for the skin images, characterized in that: The system also includes a skin image preprocessing module, a skin state classification module and a state classification optimization module; The skin image preprocessing module preprocesses the skin image to obtain a preprocessed image; The skin state classification module introduces a multi-head self-attention mechanism to construct a skin state recognition model, uses the multi-head self-attention mechanism to extract skin features, calculates the feature representation of the skin features, generates an attention map based on the feature representation, comprehensively splices the attention map to classify the skin state through the softmax function, and generates a skin state prediction label; The state classification optimization module sets the parameters of the skin state recognition model, uses the cross entropy loss function to measure the difference between the skin state prediction label and the actual skin state, iterates the training to update the parameters, sets the evaluation index to evaluate the performance of the skin state recognition model, and adjusts the hyperparameters to optimize the skin state recognition model; The skin state recognition model constructed by the skin state classification module includes a space exploration CNN block, an AssemFormer block, a scale variant attention block, a Monte Carlo attention block, a residual learning CNN block and an attention map enhancement block.

2. A skin disease classification and recognition system based on deep learning according to claim 1, characterized in that: The skin condition classification module generates a skin condition prediction label, comprising the following steps: Step S1: Analyze the preprocessed image: divide the preprocessed image into basic units, divide each basic unit into sub-regions, use the multi-head self-attention mechanism to calculate the similarity score of the basic units, obtain the calculation results, capture the global dependency relationship based on the calculation results, and extract skin features; Step S2: highlighting skin features: setting the time step, performing weighted summation on the sub-regions through the scale-variant attention block, generating a cross-scale attention map, and displaying the prominent skin features of the suspected lesion area based on the cross-scale attention map; Step S3: Random sampling pooling: Through the Monte Carlo attention block, randomly select the pooling tensor, and perform random sampling pooling operations on the basic units through the average pooling function to calculate the feature representation of each prominent skin feature; Step S4: enriching feature representation: using residual learning CNN blocks, focusing on local features and global features, using an inverted bottleneck structure, increasing the number of channels to enrich the feature representation of prominent skin features, and obtaining an overall feature representation; Step S5: Generate an attention map: Through the attention map enhancement block, calculate the contribution of the feature representation to the overall feature representation, determine the association probability based on the contribution, and generate an attention map based on the pooled tensor combined with the association probability; Step S6: Classify the skin condition: The attention map is integrated through the fully connected layer of the attention map enhancement block, the softmax function is used to classify the skin condition, the skin condition category is output, and the skin condition prediction label is generated for each skin image.

3. The skin disease classification and recognition system based on deep learning according to claim 2 is characterized by: Step S1 includes the following steps: Step S11: assigning position information: dividing the preprocessed image into non-overlapping basic units, assigning position information to each basic unit, and using the position information to clarify the position of the basic unit in the preprocessed image; Step S12: Introducing a self-attention mechanism: Introducing a multi-head self-attention mechanism, analyzing the basic units of the preprocessed image through different subspaces in the self-attention mechanism, calculating the similarity score between each basic unit of the preprocessed image in each subspace, and obtaining the calculation result; Step S13: Extract skin features: Integrate the calculation results of each subspace, capture the global dependencies between basic units, and extract skin features.

4. The skin disease classification and recognition system based on deep learning according to claim 3 is characterized by: Step S13 includes the following steps: Step S131: extracting local features: by spatially exploring the CNN block, performing convolution operations on each basic unit, learning local features of the suspected lesion area, performing pooling operations on each basic unit, gradually extracting high-level abstract local features, and generating a local feature map containing local features; Step S132: extracting global features: using the AssemFormer block, learning the global dependency between local features, extracting the global features of the suspected lesion area, and generating a global feature map containing the global features; Step S133: Fusion of global features: Through the AssemFormer block, a convolution operation is performed on the global feature map, local features and global features are fused, and a fused feature map is output to provide skin features of the suspected lesion area.

5. The skin disease classification and recognition system based on deep learning according to claim 4 is characterized by: The state classification optimization module optimizes the process of the skin state recognition model, including the following steps: Step D1: Divide the preprocessed images: divide the preprocessed images into a training set, a validation set, and a test set; Step D2: Define the loss function: set the parameters of the skin state recognition model, use the cross entropy loss function, calculate the cross entropy loss, and measure the difference between the skin state prediction label and the actual skin state; Step D3: Update model parameters: set hyperparameters, use the Adam optimizer on the training set, train the skin state recognition model, calculate the gradient based on the cross entropy loss function, and update the parameters of the skin state recognition model; Step D4: Evaluate model performance: Set evaluation indicators, evaluate the performance of the skin condition recognition model on the validation set, adjust hyperparameters according to the evaluation indicators, and optimize the skin condition recognition model.

Images