A fluorescence recognition method for skin fungi based on deep learning

Abstract

This invention discloses a deep learning-based method for identifying dermatophyte fluorescence. The method comprises obtaining an input image of dermatophyte fluorescence; encoding the input image using a Transformer model; and decoding and outputting the encoded image using a sequence pooling module to obtain a dermatophyte category prediction result. This method can efficiently and quickly identify dermatophyte species.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence and biomedical image processing technology, and in particular to a skin fungus fluorescence recognition method based on deep learning. Background Art

[0002] Dermatophytes primarily include Malassezia, Candida, Aspergillus, and Cryptococcus. Currently, a widely used method for fungal examination is fungal fluorescence microscopy. This method utilizes a special fluorescent marker to react chemically with chitin and other polysaccharides in the fungal cell wall. Combined with a fluorescence microscope, the fluorescent reaction of the reagent and the microscope's specific excitation wavelengths enable clear and accurate visualization of the fungi within the microscope's field of view. This method offers high recognition and contrast, effectively improving the accuracy and efficiency of fungal microscopy.

[0003] Transformers are one of the most popular standards in natural language processing. They are also currently being applied in computer vision. Transformers can address the issue of excessive layers in traditional convolutional neural networks, expanding the receptive field of an image and capturing more contextual information, effectively capturing global information about the image. They also possess strong modality fusion capabilities. However, Transformers often require large amounts of data, and big data also means increased computing resources and demanding hardware. However, in medicine, particularly for fluorescent identification of skin fungi, collecting large amounts of data is difficult because these data come from clinical samples that require staining with fluorescent fluid, a labor-intensive and time-consuming process. Furthermore, the number of positive cases for rare pathogenic fungi is significantly smaller, making accurate labeling even more challenging. This further hinders the development of a dataset suitable for Transformer training, hindering its effective application. Therefore, using Transformer models to improve the accuracy and efficiency of skin fungi identification is a pressing challenge facing existing technologies. Summary of the Invention

[0004] In order to solve the problems existing in the above-mentioned prior art, the present invention provides a skin fungus fluorescence recognition method based on deep learning, which can efficiently and quickly identify the types of skin fungi.

[0005] In order to achieve the above technical objectives, the present invention provides the following technical solutions: a method for fluorescence identification of skin fungi based on deep learning, comprising:

[0006] Obtain an input image of skin fungus fluorescence; encode the input image through the Transformer model; decode and output the encoded result through the sequence pooling module to obtain the skin fungus category prediction result.

[0007] Optionally, before encoding the input image, the method further includes preprocessing the input image, wherein the preprocessing includes image enhancement.

[0008] Optionally, the Transformer model includes a convolutional tokenizer module and a Transformer encoder module connected in sequence.

[0009] Optionally, the convolutional tokenizer module includes a first sublayer and a second sublayer connected in sequence; wherein the first sublayer includes a 64×3×3 convolutional layer, a zero-padding layer and a maximum pooling layer connected in sequence; the second sublayer includes a 128×3×3 convolutional layer, a zero-padding layer and a maximum pooling layer connected in sequence, and position embedding is added to the convolutional tokenizer module structure.

[0010] Optionally, the Transformer encoder module includes two third sublayers connected in sequence; wherein the third sublayer includes a LayerNormalization layer, a Multi-head self-attention layer, a regularization layer, a LayerNormalization layer, an MLP layer and a regularization layer connected in sequence.

[0011] Optionally, the sequence pooling module adopts a linear layer.

[0012] Optionally, the process of decoding and outputting the encoded result includes:

[0013] Processing the encoded result through a linear layer output to generate a one-dimensional vector, wherein the one-dimensional vector includes the predicted probabilities of different dermatophyte categories;

[0014] The skin fungus category corresponding to the maximum prediction probability in the one-dimensional vector is selected as the skin fungus category prediction result.

[0015] Optionally, before encoding the input image through the Transformer model, the step further includes: training the Transformer model and the sequence pooling module;

[0016] The training process includes obtaining a skin fungus fluorescence dataset, annotating the skin fungus fluorescence dataset, and dividing the annotation results into a training set, a validation set, and a test set; training the Transformer model and sequence pooling module through the training set and validation set, and performing skin fungus fluorescence recognition through the trained Transformer model and sequence pooling module.

[0017] Optionally, after training the Transformer model and sequence pooling module, the following steps are also performed:

[0018] The output results of the Transformer model and sequence pooling module are evaluated for model accuracy using the test set. The Transformer model and sequence pooling module are then optimized based on the model accuracy and retrained after optimization.

[0019] The tuning process includes: using a learning rate cosine annealing algorithm to adjust the learning rate or adjusting the convolution kernel size, training cycle and other hyperparameters of the convolution layer.

[0020] The present invention has the following technical effects:

[0021] Through the above technical solution, the present invention overcomes the limitation of traditional Transformers, which are only applicable to large datasets. It enables Transformers to be trained from scratch on small datasets, expanding the application scenarios of Transformers. It also makes the model structure more compact while maintaining a certain accuracy, increasing the training efficiency of Transformers. At the same time, it can also reduce the number of model parameters and reduce the dependence on hardware computing devices. This allows for efficient and rapid identification of dermatophyte species. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0023] Figure 1 A schematic flow chart of a method for fluorescent identification of skin fungi provided by an embodiment of the present invention;

[0024] Figure 2 Schematic diagram of a fluorescent sample of skin fungi provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0025] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0026] Example 1

[0027] In order to solve the problems existing in the prior art, the present invention provides the following solution: a method for fluorescence identification of skin fungi based on deep learning, comprising the following steps:

[0028] (1) Preprocess the input image;

[0029] (2) Input the image into the Transformer model for encoding;

[0030] (3) Output the encoded result.

[0031] The step (1) preprocesses the input image to make the image more suitable for network training.

[0032] The step (1) specifically includes: rescaling the image and performing data enhancement using an image enhancement library. Common image enhancement methods may be used, such as random brightness and contrast adjustment, horizontal / vertical flipping, and adding Gaussian noise. By using data enhancement methods, the sample diversity of the small-scale dataset is increased to improve the generalization ability of the final model.

[0033] The step (2) inputs the image into the Transformer model for encoding, and the Transformer model includes a convolutional tokenizer module and a Transformer encoder module.

[0034] The step (2) specifically includes: passing the image processed in step (1) through a convolutional tokenizer, which specifically includes two sublayers, each of which includes a convolutional layer, a zero-filling layer, and a maximum pooling layer. Position embedding is added to the convolutional tokenizer module.

[0035] Specifically, the convolutional tokenizer module includes two sublayers: the first sublayer consists of a 64×3×3 convolutional layer, a zero-padding layer, and a 3×3 max pooling layer, all connected in sequence; the second sublayer consists of a 128×3×3 convolutional layer, a zero-padding layer, and a 3×3 max pooling layer, all connected in sequence. Position embedding encodes the position information between image components into a vector and adds it to the output of the convolutional tokenizer module.

[0036] The output of the convolutional tokenizer is then input into the Transformer encoder for encoding. The Transformer encoder consists of two sublayers, each of which contains a Layer Normalization (LN) layer, a Multi-head self-attention layer, a regularization layer, a LayerNormalization (LN) layer, an MLP layer, and a regularization layer connected in sequence.

[0037] The step (3) outputs the image encoded in the step (2), and the output is the predicted probability of the image in each category.

[0038] The step (3) specifically includes: passing the output of step (2) through a sequence pooling module, which uses a linear layer, and outputting a one-dimensional vector after passing through the module. This vector contains the predicted probabilities of all categories, and selects the category corresponding to the maximum predicted probability as the final output of the model.

[0039] Before recognition, the overall model composed of the Transformer model and the decoding module, i.e., the sequence pooling module, needs to be trained. The training of the model includes the following steps: obtaining a skin fungus fluorescence dataset; manually annotating all images and then dividing them into a training set, a validation set, and a test set according to a certain ratio; building the overall model according to steps (2) and (3); preprocessing the dataset using the method of step (1); and using the dataset to train the overall model.

[0040] After training the model using the data set, the method further includes: using the test set to evaluate the recognition accuracy of the model, tuning the model according to the accuracy of the model, and training the model again after the tuning. The tuning refers to introducing a learning rate cosine annealing algorithm to adjust the learning rate or adjusting other hyperparameters such as the convolution kernel size and training cycle of the convolution layer.

[0041] The above is the recognition process for a single image. In practice, the input image should be a training set consisting of multiple fungal categories. The training set is trained using the above steps, then tested using the test set. The output is compared with the test set labels to determine the fungal category recognition accuracy. The model is then fine-tuned based on this accuracy and trained again until the target accuracy is achieved, thus meeting the required accuracy. Fine-tuning can involve changing the learning rate during training, for example, by introducing a learning rate cosine annealing algorithm or adjusting other hyperparameters such as the kernel size and training period of the convolutional layer.

[0042] Example 2

[0043] This invention overcomes the limitation of traditional Transformers, which are only applicable to large datasets, by enabling Transformers to be trained from scratch on smaller datasets, thus expanding their application scenarios. It also makes the model structure more compact while maintaining a certain level of accuracy, increasing Transformer training efficiency. Furthermore, it reduces the number of model parameters and reduces dependence on hardware computing devices. This allows for efficient and rapid identification of dermatophyte species.

[0044] The present invention is further described below by taking the five classifications of dermatophytes as examples.

[0045] Follow the steps below to perform fluorescence identification of five types of skin fungi. For the overall flow chart, see Figure 1 :

[0046] 1. Preprocess the input image; this example uses a dataset of five fungal fluorescence recognition types, including negative skin fungi, Candida, Malassezia, Cryptococcus, and Aspergillus. The dataset is derived from clinical samples and has been manually annotated by professional personnel. The sample images in the dataset are as follows: Figure 2 As shown, their pixel values ​​are all 1920×1200. Their numbers are: 745, 344, 1000, 133 and 1000 respectively. As can be seen from the above, the number of Candida and Cryptococcus is less than that of other categories, so data enhancement can be performed on them using methods such as rotation, translation and mirroring. In this embodiment, two times the data enhancement is performed on Candida and three times the data enhancement is performed on Cryptococcus to expand the skin fungus fluorescence recognition dataset. For the data-enhanced dataset, the dataset is divided into a ratio of 8:1:1 for training set: validation set: test set. Input into the model for model training.

[0047] 2. Input the image into the Transformer model for encoding. In this embodiment, the convolutional tokenizer module contains two sublayers. The first sublayer consists of a 64×3×3 convolutional layer, a zero-padding layer, and a 3×3 max pooling layer connected in sequence. The second sublayer consists of a 128×3×3 convolutional layer, a zero-padding layer, and a 3×3 max pooling layer connected in sequence. Position embedding is also added.

[0048] Next comes the Transformer encoding layer. In this embodiment, the Transformer encoding layer specifically includes two sublayers, each of which contains a LayerNormalization layer, a Multi-head self-attention layer, a regularization layer, a LayerNormalization layer, an MLP layer, and a regularization layer, which are connected in sequence.

[0049] 3. Output the encoded result. Pass the Transformer encoding layer through a sequence pooling module, and then output the final recognition result. In this embodiment, the output obtained in step 2 is input into the sequence pooling module. The sequence pooling module uses a linear layer and ultimately outputs a one-dimensional vector containing the predicted probabilities of all categories. The category corresponding to the maximum probability is selected as the final output of the model.

[0050] In this example, we first divided the training set into a validation set and a test set according to step 1. Then, we built a deep learning model using steps 2 and 3. The data from step 1 was then trained on the deep learning model. A training period of 50 was selected, and cosine annealing of the learning rate was added. Training was then performed on a CPU. The total training time was less than 30 minutes, and the model achieved 90% accuracy on the test set. The model had only 0.4M parameters.

[0051] This example uses Transformer and deep learning technology to design a method for fluorescence identification of skin fungi, which can effectively identify fungal types and has algorithmic innovation and practical application value.

[0052] This paper proposes a deep learning-based method for identifying dermatophyte fluorescence. This method is suitable for small datasets, such as those for dermatophyte fluorescence, allowing for the training of Transformer-based models from scratch. It can be trained using only a CPU, eliminating the need for expensive GPUs, making it less demanding than other algorithms. Furthermore, it requires fewer model parameters. Furthermore, it boasts high recognition accuracy, reaching over 90% after approximately 50 training cycles.

[0053] In summary, this method can identify the types of skin fungi more efficiently and quickly on skin fungus fluorescence datasets with less data volume with fewer parameters and lower computational requirements, providing more competitive identification results.

[0054] The above are merely preferred embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A method for fluorescence identification of skin fungi based on deep learning, characterized in that: include: Obtaining an input image of skin fungus fluorescence; Encode the input image through the Transformer model; The encoded results are decoded and output through the sequence pooling module to obtain the category prediction results of skin fungi; The method further includes preprocessing the input image before encoding the input image, wherein the preprocessing includes image enhancement; The Transformer model includes a convolutional tokenizer module and a Transformer encoder module connected in sequence; The convolutional tokenizer module includes a first sublayer and a second sublayer connected in sequence; wherein the first sublayer includes a 64×3×3 convolutional layer, a zero-padding layer, and a maximum pooling layer connected in sequence; the second sublayer includes a 128×3×3 convolutional layer, a zero-padding layer, and a maximum pooling layer connected in sequence, and position embedding is added to the convolutional tokenizer module structure; The Transformer encoder module includes two third sublayers connected in sequence; wherein the third sublayer includes a Layer Normalization layer, a Multi-head self-attention layer, a regularization layer, a LayerNormalization layer, an MLP layer, and a regularization layer connected in sequence; The sequence pooling module adopts a linear layer; The process of decoding and outputting the encoded result includes: Processing the encoded result through a linear layer output to generate a one-dimensional vector, wherein the one-dimensional vector includes the predicted probabilities of different dermatophyte categories; The skin fungus category corresponding to the maximum prediction probability in the one-dimensional vector is selected as the skin fungus category prediction result.

2. The method for fluorescent identification of dermatophytes according to claim 1, wherein: Before encoding the input image through the Transformer model, the following steps are also included: training the Transformer model and the sequence pooling module; The training process includes obtaining a skin fungus fluorescence dataset, performing data augmentation preprocessing on the skin fungus fluorescence dataset, annotating the preprocessed skin fungus fluorescence dataset, and dividing the annotation results into a training set, a validation set, and a test set; training the Transformer model and sequence pooling module through the training set and validation set, and performing skin fungus fluorescence recognition through the trained Transformer model and sequence pooling module.

3. The method for fluorescent identification of dermatophytes according to claim 2, wherein: After training the Transformer model and sequence pooling module, it also includes: The output results of the Transformer model and sequence pooling module are evaluated for model accuracy using the test set. The Transformer model and sequence pooling module are then optimized based on the model accuracy. After optimization, the Transformer model and sequence pooling module are retrained. The tuning process includes: using a learning rate cosine annealing algorithm to adjust the learning rate or adjusting the convolution kernel size, training cycle and other hyperparameters of the convolution layer.

Images