A method and system for automatically screening thyroid disease using infrared thermal images
By using a deep learning model based on Vision Transformer to preprocess and extract features from infrared thermal images, the problems of low sensitivity and high cost in early screening of thyroid diseases are solved, achieving efficient and accurate disease risk assessment and screening.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- WUHAN UNIV
- Filing Date
- 2022-10-17
- Publication Date
- 2026-07-03
AI Technical Summary
Existing methods for early screening of thyroid diseases, such as color Doppler ultrasound, have low sensitivity, are greatly affected by the doctor's professional background knowledge, require high-end and expensive testing equipment, and rely on human experience for infrared thermal image analysis, resulting in inaccurate results.
By employing a Vision Transformer-based deep learning model combined with medical infrared thermal images, and through preprocessing, feature extraction, and classification, thyroid diseases are automatically screened. Transfer learning and label smoothing loss functions are used to prevent overfitting and provide an objective disease risk assessment.
It improves the sensitivity of early thyroid disease detection, reduces reliance on specialized knowledge, lowers testing costs, and provides objective and accurate screening results, thus improving screening efficiency and accuracy.
Smart Images
Figure CN115662630B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of image processing and automated screening for thyroid diseases, specifically to a classification method for thyroid infrared thermal images and a system for screening thyroid diseases based on infrared thermal images, belonging to the application of deep learning methods in the field of medical diagnosis within artificial intelligence technology. Background Technology
[0002] As an important endocrine organ, the thyroid gland secretes thyroid hormones that play a crucial role in maintaining normal metabolism, ensuring the proper functioning of various organ systems, and maintaining the excitability of the nervous system. Survey data shows that thyroid diseases are prevalent, making early screening essential.
[0003] Currently, color Doppler ultrasound is a commonly used method for early screening of thyroid diseases. However, color Doppler ultrasound has the following problems for early screening of thyroid diseases: First, ultrasound detection can only show abnormal structural changes after the lesion has developed to a certain size, density, or deformation. Early metabolic and circulatory changes in thyroid diseases are difficult to detect through ultrasound images, resulting in low sensitivity of ultrasound in detecting early thyroid lesions. Second, ultrasound image acquisition requires physicians with specialized knowledge, and the results are affected by the physician's familiarity with thyroid diseases. Third, ultrasound detection requires sophisticated equipment and is expensive, making it unsuitable for large-scale disease screening. Unlike ultrasound detection, which identifies abnormalities based on the morphological structure of the lesion, medical infrared thermography receives infrared radiation signals from the body's cellular metabolism and processes them to create a temperature distribution map of the body surface. This allows for sensitive detection of temperature changes caused by metabolic and circulatory changes in the early stages of disease. In addition, infrared thermal imaging does not require the collector to have professional medical background knowledge, and its detection cost is lower than that of ultrasound detection, making it more suitable for large-scale disease screening.
[0004] Previous studies have mostly relied on methods such as manually building physical models, mathematical statistical analysis, and doctors interpreting images visually to analyze infrared thermal images. These methods depend on the researchers' own experience, and subjective differences can affect the accuracy of the analysis results. Summary of the Invention
[0005] This invention primarily addresses the problems mentioned above regarding ultrasound in early screening for thyroid diseases, including low sensitivity, susceptibility to subjective factors such as physician's professional background knowledge, high requirements for testing equipment, and relatively high testing costs. The first aspect of this invention provides a method for automatically screening thyroid diseases using infrared thermal images. This method combines medical infrared thermal images with the Vision Transformer deep learning model, assessing the risk of thyroid disease in the examinee based on the classification results of the infrared thermal images by the deep learning model. The method mainly includes the following steps:
[0006] S1: Create a dataset of thyroid infrared thermal images, collect infrared thermal images of the neck and above of the subjects and the results of thyroid ultrasound examination and grading of the subjects, and use the infrared thermal images of the subjects as samples and the results of thyroid ultrasound examination and grading of the subjects as sample labels.
[0007] S2: Preprocess the infrared thermal image to remove background information and obtain the neck region;
[0008] S3: Construct an image classification model based on Vision Transformer, including a Transformer-based feature extraction module for feature extraction from thyroid infrared thermal images; and an MLP-based image classification module for classifying the features extracted by Transformer.
[0009] S4: Divide the preprocessed thyroid infrared thermal image dataset in S2 into training and test sets, define the loss function and optimizer, and define the learning rate decrement method;
[0010] S5: Using a transfer learning strategy, load the pre-trained weights of Vision Transformer on ImageNet, perform multiple iterations of model training based on the backpropagation algorithm, and save the model structure and parameters with the best performance after testing.
[0011] S6: After the thyroid infrared thermal image that needs to be classified is preprocessed in step S2, it is input into the trained image classification model based on Vision Transformer, and the classification result based on the infrared thermal image is output. The risk of the examinee having thyroid disease is assessed based on the classification result.
[0012] Furthermore, the preprocessing in step S2 includes color thresholding, extracting the maximum contour region, and extracting the maximum inscribed rectangle region within the maximum contour region.
[0013] Furthermore, the preprocessing in step 2 specifically includes the following steps:
[0014] S2.1: Color thresholding based on HSV space. First, the acquired RGB infrared thermal image is converted to HSV color space. Then, the minimum and maximum thresholds are set to perform color segmentation on the thyroid infrared thermal image.
[0015] The color space conversion formula is:
[0016] R′=R / 255
[0017] G′=G / 255
[0018] B′=B / 255
[0019] C max =max(R′, G′, B′)
[0020] C min =min(R′, G′, B′)
[0021] Δ=C max -C min
[0022]
[0023]
[0024] Where R, G, and B represent the pixel values in the three color channels of the RGB image, H, S, and V are the values of each channel in the converted HSV image, and R′, G′, B′, and C′ are the pixel values in the three color channels of the RGB image. max C min Both Δ and Δ are intermediate calculation results.
[0025] S2.2: Calculate the area of all contour regions in the image obtained by color segmentation in S2.1, and obtain the contour coordinates of the region with the largest area;
[0026] S2.3: Calculate the coordinates of the largest inscribed rectangle of the region based on the contour coordinates of the largest area region obtained in S2.2;
[0027] S2.4: Cut out the neck region from the original image based on the coordinates of the largest inscribed rectangle.
[0028] Furthermore, step S3 extracts features from the thyroid infrared thermal image based on the Transformer encoder module, and classifies the extracted image features based on the MLP module, specifically including the following steps:
[0029] S3.1: Resize the image obtained after preprocessing in step 2 to a specific resolution size H*W, and divide the image into Patch image blocks of size M*M, resulting in a total of N image blocks, which together form a Patch image block;
[0030] S3.2: Perform image patch embedding operation on these N Patch block sequences, so that each Patch block is expanded into a one-dimensional vector, and a sequence of N one-dimensional vectors is obtained;
[0031] S3.3: Introduce positional encoding through positional embedding operation to add positional information from the image sequence to each Patch image block. Add the sequence of N one-dimensional vectors obtained in S3.2 to the positional embedding result to obtain a new one-dimensional vector sequence.
[0032] S3.4: Introduce the learnable category embedding vector Cls Token, which is ultimately used for thyroid infrared image classification, and concatenate it with the vector sequence obtained in S3.3 to obtain the final one-dimensional vector sequence used for feature extraction;
[0033] S3.5: Input the one-dimensional vector sequence obtained in S3.4 into the Transformer encoder module. The encoder module consists of an MSA Block composed of a multi-head attention mechanism, layer normalization and skip connections, and an MLP Block composed of a feedforward network, layer normalization and skip connections.
[0034] S3.6: Obtain the output of the first position learnable class embedding vector Cls Token of the Transformer encoder module as the learned feature vector about the thyroid infrared thermal image;
[0035] S3.7: The MLP module consists of two fully connected layers. The feature vector obtained in S3.6 is used as the input of the MLP module, and the final output is a vector with dimension (1, number of categories).
[0036] Furthermore, in step S3.3, absolute position encoding is used to directly generate position matrices (p0, p1, p2, ...) by randomly initializing the positions (0, 1, 2, ...) of different Patch image blocks. The dimension of p0 is the same as the dimension of the vector obtained by the image block embedding operation for each Patch image block in S3.2. The absolute position encoding is continuously updated as a trainable parameter during the training process.
[0037] Furthermore, in step S3.4, Cls Token is a randomly initialized embedding vector with the same dimension as the vector obtained by the image patch embedding operation for each Patch image patch. It is a learnable vector that is continuously updated during training.
[0038] Furthermore, in step S3.5, the feedforward network contains two fully connected layers, with the activation function in between using GeLU (Gaussian Error Linear Unit).
[0039] Furthermore, the loss function defined in step S4 is the label smoothing loss function, which effectively alleviates the overfitting situation during training.
[0040]
[0041]
[0042] Where z i q represents the value at the i-th position in the output vector of the MLP module. i p represents the confidence probability that the input image is predicted to be each category, k represents the number of categories, and p represents the confidence probability. i This represents the one-hot encoding corresponding to the true class label of the input image;
[0043] The purpose of the label smoothing loss function is to transform the target from a one-hot label like [1,0,0] into the following form:
[0044]
[0045] Where y refers to the class label of the input image, which is a vector with dimension equal to the number of classes. i This represents the value at the i-th position in the vector label, k represents the number of categories, and ε is a constant.
[0046] The learning rate decreasing method described in step S4 refers to the learning rate decreasing as the number of training iterations increases, which helps the model converge.
[0047] Furthermore, the training process using the transfer learning strategy in step S5 includes the following steps:
[0048] S5.1: Use the model weights trained on the ImageNet dataset as initial parameters;
[0049] S5.2: Fine-tuning all layers of the pre-trained model on the thyroid infrared thermal image dataset;
[0050] S5.3: Save the parameters of the round with the highest accuracy in the validation set during training as the model parameters loaded into the thyroid disease screening system.
[0051] On the other hand, the present invention provides a system for automatically screening thyroid diseases using infrared thermal images, including a file module, an infrared thermal image display module, an infrared thermal image processing module, an infrared thermal image classification module, a screening result display module, and a screening report generation module;
[0052] The file module is used to read thyroid infrared thermal images and classification models, as well as to save the generated screening reports;
[0053] The classification model includes a Transformer-based feature extraction module for feature extraction from thyroid infrared thermal images; and an MLP-based image classification module for classifying the features extracted by the Transformer.
[0054] The infrared thermal image display module is used to display and zoom in / out of infrared thermal images of the thyroid gland.
[0055] The infrared thermal image processing module is used for image preprocessing operations on thyroid infrared thermal images, including color thresholding, extraction of the maximum contour region, and extraction of the largest inscribed rectangle region within the maximum contour region.
[0056] The infrared thermal image classification module is used to train the image classification model and to classify thyroid infrared thermal images using the trained thyroid infrared thermal image classification model.
[0057] The screening results display module assesses the risk of thyroid disease in the examinee based on the classification results of the examinee's infrared thermal image and displays the assessment results.
[0058] The screening report generation module is used to generate a visual report of the examinee's infrared thermal image and the examinee's assessment results.
[0059] This invention has the following advantages compared to existing research:
[0060] 1. This invention utilizes medical infrared thermal imaging technology to collect infrared thermal images of the subject's neck for initial screening of thyroid diseases. Compared with the current method of initial screening of thyroid diseases based on ultrasound images, it has higher sensitivity and is more likely to detect early lesions of thyroid diseases.
[0061] 2. This invention preprocesses the acquired thyroid infrared thermal image to remove background noise information, segment the neck region, and improve the classification accuracy of the thyroid infrared thermal image.
[0062] 3. This invention uses a label smoothing loss function and a parameter transfer learning strategy during the training of the deep learning model, which effectively prevents overfitting caused by the small size of the thyroid infrared thermal image dataset.
[0063] 4. This invention classifies thyroid infrared thermal images based on the Vision Transformer deep learning model, assesses the risk of thyroid diseases in examinees based on the classification results, improves the screening efficiency of thyroid diseases, provides doctors with objective and accurate screening results, and guides doctors to select appropriate treatment plans.
[0064] 5. This invention designs a thyroid disease screening system based on infrared thermal imaging to provide an intuitive visualization of screening results and generate screening result reports. Attached Figure Description
[0065] Figure 1 This is the infrared thermal image of the thyroid gland used in this invention;
[0066] Figure 2 This is a flowchart of the screening method for thyroid diseases based on infrared thermal images in this invention;
[0067] Figure 3 This is a flowchart of the preprocessing of thyroid infrared thermal images in this invention;
[0068] Figure 4 This is a structural diagram of the deep learning model based on Vision Transformer in this invention;
[0069] Figure 5 This is a structural diagram of the Transformer encoder in this invention;
[0070] Figure 6 This is a schematic diagram of the thyroid disease screening system module in this invention. Detailed Implementation
[0071] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0072] The first aspect of this invention provides a method for automatically screening thyroid diseases using infrared thermal imaging, the workflow of which is as follows: Figure 2 As shown, it includes the following steps:
[0073] S1: Create a thyroid infrared thermal image dataset by collecting infrared thermal images of the neck and above of the subjects and the thyroid ultrasound grading results. The subjects' infrared thermal images are used as samples, and the ultrasound results are used as sample labels. In this invention, the sample labels include two types: High Risk and Low Risk. Ultrasound indicating a TI-RADs grade of 4a or higher is considered high risk, and below 4a is considered low risk.
[0074] S2: Preprocessing the infrared thermal image to remove background information and obtain the neck region. Specifically, this includes color thresholding segmentation, extraction of the maximum contour region, and extraction of the largest inscribed rectangle region within the maximum contour region. The preprocessing flowchart and the images obtained at each step are shown below. Figure 3 As shown;
[0075] S3: Construct an image classification model based on Vision Transformer, including building a Transformer-based feature extraction module for feature extraction from thyroid infrared thermal images; and building an MLP-based image classification module to classify the features extracted by the Transformer. Specifically, the overall architecture of the Vision Transformer-based thyroid infrared thermal image model is as follows: Figure 4 As shown, the structure of the feature extraction module based on Transformer is as follows: Figure 5 As shown;
[0076] S4: Divide the preprocessed thyroid infrared thermal image dataset in S2 into training and test sets, define the loss function and optimizer, and define the learning rate decrement method;
[0077] S5: Using a transfer learning strategy, load the pre-trained weights of Vision Transformer on ImageNet, perform multiple iterations of neural network training based on the backpropagation algorithm, and save the model structure and parameters with better performance as the target for system model loading after testing.
[0078] S6: After the thyroid infrared thermal image that needs to be classified is preprocessed in step S2, it is input into the trained image classification model based on Vision Transformer, and the classification result based on the infrared thermal image is output. The risk of the examinee having thyroid disease is assessed based on the classification result.
[0079] The preprocessing work described in step S2 includes:
[0080] S2.1: Based on HSV color thresholding, the acquired RGB infrared thermal image is first converted to the HSV color space. The color space conversion formula is as follows:
[0081] R′=R / 255
[0082] G′=G / 255
[0083] B′=B / 255
[0084] C max =max(R′, G′, B′)
[0085] Cmin =min(R′, G′, B′)
[0086] Δ=C max -C min
[0087]
[0088]
[0089] V = C max
[0090] Where R, G, and B represent the pixel values in the three color channels of the RGB image, and H, S, and V are the values of each channel in the converted HSV image. R′, G′, B′, C max C min Δ and Δ are intermediate calculation results.
[0091] Then, minimum and maximum thresholds are set to perform color segmentation on the thyroid infrared thermal image; specifically, in this invention, the minimum threshold is set to [0,0,0] and the maximum threshold is set to [34,255,255].
[0092] S2.2: Calculate the area of all contour regions in the image obtained by color segmentation in S2.1, and obtain the contour coordinates of the region with the largest area;
[0093] S2.3: Calculate the coordinates of the largest inscribed rectangle of the region based on the contour coordinates of the largest area region obtained in S2.2;
[0094] S2.4: Cut out the neck region from the original image based on the coordinates of the largest inscribed rectangle.
[0095] The image classification module based on Transformer and MLP in step S3 mainly includes:
[0096] S3.1: Resize the image obtained in step 2 to a specific resolution size H*W. Divide the image into M*M patch image blocks, resulting in N image blocks, which form a patch image block sequence. Specifically, in this invention, H*W = 224*224, M = 16*16, and N = 14*14;
[0097] S3.2: Perform image patch embedding on these N Patch block sequences, expanding each Patch block into a one-dimensional vector, resulting in a sequence of N one-dimensional vectors. Specifically, in this invention, N = 14 * 14;
[0098] S3.3: Positional encoding is introduced through positional embedding operations to add positional information from the image sequence to each patch image block. The sequence of N one-dimensional vectors obtained in S3.2 is added to the positional embedding result to obtain a new one-dimensional vector sequence. Specifically, this invention uses absolute positional encoding to directly generate a position matrix (p0, p1, p2...) by randomly initializing the positions (0, 1, 2...) of different patch image blocks, where the dimension of p0 is the same as the dimension of the vector obtained by the image block embedding operation for each patch image block in S3.2. The absolute positional encoding is continuously updated as a trainable parameter during the training process.
[0099] S3.4: Introduce the learnable category embedding vector ClsToken, which is ultimately used for thyroid infrared image classification. Concatenate it with the vector sequence obtained in S3.3 to obtain the final one-dimensional vector sequence used for feature extraction. Specifically, ClsToken is a randomly initialized embedding vector with the same dimension as the vector obtained by the image patch embedding operation for each patch image block. It is a learnable vector that is continuously updated during training.
[0100] S3.5: Input the one-dimensional vector sequence obtained in S3.4 into the Transformer encoder module. The encoder module consists of an MSA Block composed of a multi-head attention mechanism, layer normalization and skip connections, and an MLP Block composed of a feedforward network, layer normalization and skip connections. The feedforward network contains two fully connected layers, and the activation function in the middle uses GeLU (Gaussian Error Linear Unit).
[0101] S3.6: Obtain the output of the first position learnable class embedding vector Cls Token of the Transformer encoder module as the learned feature vector about the thyroid infrared thermal image;
[0102] S3.7: The MLP module consists of two fully connected layers. The feature vector obtained in S3.6 is used as the input of the MLP module, and the final output is a vector with dimension (1, number of categories). Specifically, in this invention, the number of categories is equal to 2, namely benign and malignant.
[0103] The definitions of some configurations during the training process in step S4 include:
[0104] S4.1: The loss function defined during training is the label smoothing loss function, which can prevent overfitting. Traditional classification loss uses Softmax, first calculating the Softmax of the output of the fully connected layer as the confidence probability of each category, and then using cross-entropy to calculate the loss.
[0105]
[0106]
[0107] Where z i q represents the value at the i-th position in the output vector of the MLP module. i p represents the confidence probability that the input image is predicted to be in each category (good or malignant in this invention), k represents the number of categories, and p represents the confidence probability that the input image is predicted to be in each category. i This represents the value at the i-th position in the one-hot encoding corresponding to the true class label of the input image, where p represents the true distribution and q represents the distribution of the network output.
[0108] In traditional classification loss calculations, the output probability of each sample in the correct class is maximized to be 1, which requires a z-value of +∞. This increases the distance between the sample and other classes. Now, suppose a multi-class task has a label of [1,0,0]. If the label itself is problematic, it can severely damage the model because forcibly learning a non-class sample with a very high probability during training affects the estimation of the posterior probability. Furthermore, classes are not always unrelated; if the output probabilities differ too much, it can lead to overfitting. Therefore, the idea behind Label Smoothing is to change the target from a one-hot label like [1,0,0] to the following form:
[0109]
[0110] Where y refers to the class label of the input image, which is a vector with dimension equal to the number of classes. i Let represent the value at the i-th position in the label vector, and k represent the number of categories. The number of categories to be classified is consistent with the length of the final label vector, so i can be interpreted as both a category and a position. ε is a small constant, which makes the probability optimization objective in the softmax loss no longer 1 and 0, and the optimal solution for z is no longer positive infinity, but a specific numerical value. This avoids overfitting to some extent and mitigates the impact of mislabeled labels.
[0111] S4.2: The optimization algorithm defined during training is stochastic gradient descent (SGD) combined with momentum learning. Specifically, in this invention, the initial learning rate is set to 0.0001, and the momentum value is set to 0.9.
[0112] S4.3: The learning rate decay method defined during training is fixed step decay. Specifically, in this invention, the learning rate decay rate is set to 0.00004, and the step size is set to 30;
[0113] The training process using the transfer learning strategy in step S5 includes the following steps:
[0114] S5.1: Use the model weights trained on the ImageNet dataset as initial parameters;
[0115] S5.2: Fine-tune all layers of the pre-trained model on the thyroid infrared thermal image dataset. Specifically, in this invention, the total number of training rounds is set to 100, and the training batch size is set to 32.
[0116] S5.3: After training and testing, the model structure and parameters with good performance are saved as the target for system model loading. Specifically, in this invention, the parameters with the highest accuracy in the validation set during training are saved as the model parameters loaded by the thyroid disease screening system.
[0117] Currently, early screening for thyroid diseases still relies on ultrasound imaging. However, ultrasound imaging has limitations in detecting early thyroid lesions, including low sensitivity, high equipment requirements, and relatively high costs. Furthermore, manual image interpretation is subject to subjective factors such as the doctor's professional background knowledge. This invention provides a thyroid disease screening method based on infrared thermal imaging. This method combines medical infrared thermal imaging with the Vision Transformer deep learning model. Based on the classification results of the infrared thermal images by the deep learning model, the risk of the examinee having thyroid disease is assessed. This method can assist doctors in thyroid disease screening, reducing their workload, improving screening efficiency, providing doctors with objective and accurate screening results, guiding them to choose appropriate treatment plans, and enabling patients with thyroid diseases to receive timely treatment in the early stages, reducing the risk of disease progression.
[0118] Based on the same inventive concept, this invention provides a system for automatically screening thyroid diseases using infrared thermal imaging, such as... Figure 6 As shown, the system includes the following 6 modules:
[0119] The file module is used to read thyroid infrared thermal images and classification models, as well as to save the generated screening reports;
[0120] The classification model includes a Transformer-based feature extraction module for feature extraction from thyroid infrared thermal images; and an MLP-based image classification module for classifying the features extracted by the Transformer.
[0121] The infrared thermal image display module is used to display, zoom, and perform other operations on the infrared thermal image of the thyroid gland;
[0122] The infrared thermal image processing module is used for image preprocessing operations such as color thresholding segmentation, extraction of the maximum contour region, and extraction of the largest inscribed rectangle region in the maximum contour region of the thyroid infrared thermal image.
[0123] The infrared thermal image classification module is used to train the image classification model and to classify thyroid infrared thermal images using the trained thyroid infrared thermal image classification model.
[0124] The screening results display module assesses the risk of thyroid disease in the examinee based on the classification results of the examinee's infrared thermal image and displays the assessment results.
[0125] The screening report generation module is used to generate a visual report of the examinee's infrared thermal image and assessment results. This system facilitates the visualization of screening results and the generation of screening result reports.
[0126] The thyroid disease screening system and method based on infrared thermal imaging provided by this invention are of great significance for popularizing thyroid disease screening.
[0127] Finally, it should be noted that the above-described embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A method for automatically screening thyroid diseases using infrared thermal imaging, characterized in that, Includes the following steps: S1: Create a dataset of thyroid infrared thermal images, collect infrared thermal images of the neck and above of the subjects and the results of thyroid ultrasound examination and grading of the subjects, and use the infrared thermal images of the subjects as samples and the results of thyroid ultrasound examination and grading of the subjects as sample labels. S2: Preprocess the infrared thermal image to remove background information and obtain the neck region; S3: Construct an image classification model based on Vision Transformer, including a Transformer-based feature extraction module for feature extraction from thyroid infrared thermal images; and an MLP-based image classification module for classifying the features extracted by Transformer. Step S3 extracts features from the thyroid infrared thermal image based on the Transformer encoder module, and classifies the extracted image features based on the MLP module. Specifically, this includes the following steps: S3.1: Resize the image obtained after preprocessing in step 2 to a specific resolution size H×W, and divide the image into patch image blocks of size M×M, resulting in a total of N image blocks, which together form a patch image block; S3.2: Perform image patch embedding operation on these N Patch block sequences, so that each Patch block is expanded into a one-dimensional vector, and a sequence of N one-dimensional vectors is obtained. S3.3: Introduce positional encoding through positional embedding operation to add positional information from the image sequence to each Patch image block. Add the sequence of N one-dimensional vectors obtained in S3.2 to the positional embedding result to obtain a new one-dimensional vector sequence. S3.4: Introduce the learnable category embedding vector Cls Token, which is ultimately used for thyroid infrared image classification, and concatenate it with the vector sequence obtained in S3.3 to obtain the final one-dimensional vector sequence used for feature extraction; S3.5: Input the one-dimensional vector sequence obtained in S3.4 into the Transformer encoder module. The encoder module consists of an MSA Block composed of a multi-head attention mechanism, layer normalization and skip connections, and an MLP Block composed of a feedforward network, layer normalization and skip connections. S3.6: Obtain the output of the first position learnable class embedding vector Cls Token of the Transformer encoder module as the learned feature vector about the thyroid infrared thermal image; S3.7: The MLP module consists of two fully connected layers. The feature vector obtained in S3.6 is used as the input of the MLP module, and the final output is a vector with dimension (1, number of categories). S4: Divide the preprocessed thyroid infrared thermal image dataset in S2 into training and test sets, define the loss function and optimizer, and define the learning rate decrement method; S5: Using a transfer learning strategy, load the pre-trained weights of Vision Transformer on ImageNet, perform multiple iterations of model training based on the backpropagation algorithm, and save the model structure and parameters with the best performance after testing. S6: After the thyroid infrared thermal image that needs to be classified is preprocessed in step S2, it is input into the trained image classification model based on Vision Transformer, and the classification result based on the infrared thermal image is output. The risk of the examinee having thyroid disease is assessed based on the classification result.
2. The method according to claim 1, characterized in that: The preprocessing in step S2 includes color thresholding, extracting the maximum contour region, and extracting the largest inscribed rectangle region within the maximum contour region.
3. The method according to claim 1 or 2, characterized in that: Step 2, the preprocessing, specifically includes the following steps: S2.1: Color thresholding based on HSV space. First, the acquired RGB infrared thermal image is converted to HSV color space. Then, the minimum and maximum thresholds are set to perform color segmentation on the thyroid infrared thermal image. The color space conversion formula is: Where R, G, and B represent the pixel values in the three color channels of the RGB image, and H, S, and V are the values of each channel in the converted HSV image. , , , , , These are all intermediate calculation results; S2.2: Calculate the area of all contour regions in the image obtained by color segmentation in S2.1, and obtain the contour coordinates of the region with the largest area; S2.3: Calculate the coordinates of the largest inscribed rectangle of the region based on the contour coordinates of the largest area region obtained in S2.2; S2.4: Cut out the neck region from the original image based on the coordinates of the largest inscribed rectangle.
4. The method according to claim 1, characterized in that: In step S3.3, absolute position encoding is used to directly generate position matrices (p0, p1, p2, ...) by randomly initializing the positions (0, 1, 2, ...) of different Patch image blocks. The dimension of p0 is the same as the dimension of the vector obtained by the image block embedding operation for each Patch image block in S3.
2. The absolute position encoding is continuously updated as a trainable parameter during the training process.
5. The method according to claim 1, characterized in that: In step S3.4, Cls Token is a randomly initialized embedding vector with the same dimension as the vector obtained by embedding each Patch image patch through the image patch operation. It is a learnable vector that is continuously updated during training.
6. The method according to claim 1, characterized in that: In step S3.5, the feedforward network contains two fully connected layers, with the activation function in between using the Gaussian error linear unit (GeLU).
7. The method according to claim 1, characterized in that: The loss function defined in step S4 is the label smoothing loss function, which effectively alleviates overfitting during training. in This represents the value at the i-th position in the output vector of the MLP module. This represents the confidence probability that the input image is predicted to belong to each category. Indicates the number of categories. This represents the one-hot encoding corresponding to the true class label of the input image; The purpose of the label smoothing loss function is to transform the target from a one-hot label like [1,0,0] into the following form: Where y refers to the class label of the input image, which is a vector with dimension equal to the number of classes. This represents the value at the i-th position in the vector label, k represents the number of categories, and ε is a constant. The learning rate reduction method described in step S4 refers to the learning rate decreasing as the number of training iterations increases, which helps the model converge.
8. The method according to claim 1, characterized in that: The training process using the transfer learning strategy in step S5 includes the following steps: S5.1: Use the model weights trained on the ImageNet dataset as initial parameters; S5.2: Fine-tuning all layers of the pre-trained model on the thyroid infrared thermal image dataset; S5.3: Save the parameters of the round with the highest accuracy in the validation set during training as the model parameters loaded into the thyroid disease screening system.
9. A system for automatically screening thyroid diseases using infrared thermal imaging, for implementing the method as described in any one of claims 1-8, characterized in that: It includes a file module, an infrared thermal image display module, an infrared thermal image processing module, an infrared thermal image classification module, a screening result display module, and a screening report generation module; The file module is used to read thyroid infrared thermal images and classification models, as well as to save the generated screening reports; The classification model includes a Transformer-based feature extraction module for feature extraction from thyroid infrared thermal images; and an MLP-based image classification module for classifying the features extracted by the Transformer. The infrared thermal image display module is used to display and zoom in / out of infrared thermal images of the thyroid gland. The infrared thermal image processing module is used for image preprocessing operations on thyroid infrared thermal images, including color thresholding, extraction of the maximum contour region, and extraction of the largest inscribed rectangle region within the maximum contour region. The infrared thermal image classification module is used to train the image classification model and to classify thyroid infrared thermal images using the trained thyroid infrared thermal image classification model. The screening results display module assesses the risk of thyroid disease in the examinee based on the classification results of the examinee's infrared thermal image and displays the assessment results. The screening report generation module is used to generate a visual report of the examinee's infrared thermal image and the examinee's assessment results.