Thyroid nodule classification method based on global lightweight attention module

Abstract

The invention relates to a thyroid nodule classification method based on a global lightweight attention module, and belongs to the field of deep learning and medical image processing. Comprising the steps of constructing a thyroid nodule data set; preprocessing and expanding a data set; constructing an attention mechanism-based improved classification model; training an attention mechanism-based improved classification model; the model is used for thyroid nodule benign and malignant classification; according to the method, the region of interest of the thyroid nodule ultrasonic image with the unified size is input, a global lightweight attention module and a high-low channel jump connection module are combined, and particularly, local features and global dependency of a feature map are adaptively combined through the global lightweight attention module, so that the local features and the global dependency of the feature map are combined, and the local features and the global dependency of the feature map are combined. And according to the channel direction, high and low frequency features are divided into a jump connection module, and feature extraction is further carried out on the thyroid nodule. More comprehensive thyroid nodule features can be obtained by using the network model, so that a more accurate and more efficient classification result can be obtained.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image processing and deep learning, and specifically relates to a thyroid nodule classification method based on a global lightweight attention module. Background Art

[0002] Among the current thyroid nodule diagnostic technologies, ultrasound detection is the most widely used diagnostic method for clinical diagnosis of thyroid diseases due to its advantages of real-time, no radiation, no damage, and good repeatability. The detection mainly relies on doctors to observe the tissue information of the internal area of ​​the thyroid gland through an ultrasound probe. However, due to the poor quality of thyroid ultrasound images and the vague tissue morphology of thyroid nodules, the diagnosis results will rely on the doctor's theoretical experience and subjective conclusions. At the same time, the amount of ultrasound image data is large, and too much repetitive detection and review of patients' thyroid ultrasound images require a lot of time and energy. With the continuous development of artificial intelligence technology, the use of deep learning methods can use the computing power of computers to automatically analyze the benign and malignant nodules in thyroid ultrasound images, and it has begun to show its advantages in assisting medical staff to improve the accuracy rate, improve work efficiency, and improve the cure rate of patients. However, the classification efficiency in the thyroid nodule ultrasound image classification model is not high enough, and the insufficient extraction of feature information of nodules is the main problem at present.

[0003] The Chinese patent publication number is "CN115641486A", and the name is "An ultrasonic thyroid nodule identification device, method, equipment and storage medium". This method automatically divides the training set after obtaining data from multiple medical centers, and uses an SSD model based on the CBAM attention mechanism module for nodule classification. This method ignores the overall and detailed features of the nodules during feature extraction, and the classification efficiency is not high enough. Summary of the invention

[0004] The existing thyroid nodule classification model based on the attention mechanism has the problem of low classification efficiency. In addition, the existing model ignores the overall characteristics and detailed features of the nodules during the feature extraction process. In order to solve the above problems, the present invention proposes a thyroid nodule classification method based on a global lightweight attention module. The present invention inputs the region of interest of the thyroid nodule ultrasound image of uniform size, combines the global lightweight attention module with the high and low channel skip connection module, and adaptively combines the local features with their global dependencies through the global lightweight attention module during the training process. It is divided into high and low frequency features according to the channel direction in the skip connection module to further extract features of the thyroid nodules. Using this network model, a more comprehensive thyroid nodule feature can be obtained, thereby obtaining a more accurate and efficient classification result.

[0005] The present invention is achieved through the following technical solutions:

[0006] A thyroid nodule classification method based on a global lightweight attention module, the specific steps are as follows:

[0007] Step 1: Construct a thyroid nodule dataset: collect ultrasound results of thyroid nodule examinations performed in hospitals and construct a thyroid nodule dataset;

[0008] Step 2: Preprocess and expand the data set: Perform various preprocessing operations on the initially constructed data set and further expand the data set;

[0009] Step 3: Construct an improved classification model based on the attention mechanism: This classification model combines the global lightweight attention module and the high-low channel skip connection module to adaptively combine the global features and detail features of the nodules, optimize the original network model, and improve the classification accuracy and diagnostic efficiency;

[0010] Step 4: Train the improved classification model based on the attention mechanism: Use the training set to train the network model. After the training times reach the preset threshold, use the test set to test the trained model. When the accuracy of the network model stabilizes within a certain range, the network model training is considered complete and the parameters of the network model are saved. After the network training is completed, the network model parameters are fixed and the network model is determined to be the final nodule prediction model. When it is necessary to predict the benign or malignant nature of thyroid nodule ultrasound images, the image can be directly input into the network model to obtain the final prediction result.

[0011] Step 5: Use the model to classify benign and malignant thyroid nodules: scale the ultrasound image of the thyroid nodules to be classified to a size of 640×640, normalize it, and input it into the trained model to output the classification result.

[0012] Furthermore, the thyroid nodule dataset in step 1 refers to ultrasound images of thyroid nodules of patients who underwent thyroid ultrasound examination in the hospital, and are divided into benign nodules and malignant nodules through puncture pathological surgery under the guidance of professional doctors to construct a thyroid nodule dataset.

[0013] Furthermore, the preprocessing in step 2 refers to removing patient information from the data set constructed in step 1, unifying the size, and performing normalization processing; expanding the data set refers to performing contrast enhancement, brightness enhancement, and mirroring operations on the preprocessed data set, expanding the number of data sets to 3-5 times the original, and dividing the expanded data set into a training set and a test set in a ratio of 7:3.

[0014] Furthermore, the improved classification model based on the attention mechanism in step 3 includes a 7×7 convolution layer, a maximum pooling layer, an average pooling layer, a fully connected layer, two global lightweight attention modules and three high and low channel jump connection modules; the global lightweight attention module includes three convolution blocks, five different feature maps, an attention map, two average pooling layers and an upsampling layer; the high and low channel jump connection module includes three high and low channel convolution blocks, three standardization operation layers and three activation function layers.

[0015] Compared with the prior art, the advantages of the present invention are as follows:

[0016] 1. The global lightweight attention module constructed by the present invention improves the shortcomings of the traditional self-attention mechanism, which has large computational complexity and slow running speed. It improves the running speed while keeping the classification accuracy of the original model unchanged, effectively saving the doctor's diagnosis time.

[0017] 2. The present invention adopts a method of extracting features from high and low channels simultaneously to more comprehensively extract the global features and detail features of the ultrasound image, effectively improving the classification accuracy.

[0018] 3. The activation function in the improved jump connection module of the present invention uses the swish activation function, which has a small amount of data calculation and a fast operation speed, effectively improving the model running time. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for the specific embodiments or the description of the prior art. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn according to the actual scale.

[0020] Figure 1 This is a flow chart of a thyroid nodule classification method based on a global lightweight attention module of the present invention.

[0021] Figure 2 This is a network structure diagram of a thyroid nodule classification method based on a global lightweight attention module of the present invention.

[0022] Figure 3 This is a structural diagram of the global lightweight attention module described in the present invention.

[0023] Figure 4 This is a structural diagram of the high and low channel jump connection described in the present invention.

[0024] Figure 5 This is a structural diagram of the high and low channel convolution block described in the present invention. DETAILED DESCRIPTION

[0025] In order to clearly and completely describe the technical solution and its specific working process of the present invention, the specific implementation methods of the present invention are as follows in conjunction with the accompanying drawings of the specification:

[0026] Example 1

[0027] like Figure 1 As shown, a thyroid nodule classification method based on a global lightweight attention module comprises the following steps:

[0028] Step 1, construct a thyroid nodule dataset: The thyroid nodule dataset refers to the ultrasound images of thyroid nodules of patients who underwent thyroid ultrasound examinations in the hospital. Under the guidance of professional doctors, the images are divided into benign nodules and malignant nodules through puncture pathological surgery to construct a thyroid nodule dataset.

[0029] Step 2, preprocessing and expanding the data set: remove patient information from the data set constructed in step 1, unify the size, and perform normalization; expanding the data set means contrast enhancement, brightness enhancement, and mirroring the preprocessed data set, expanding the data set to 3-5 times the original size, and dividing the expanded data set into a training set and a test set in a ratio of 7:3.

[0030] Step 3, the improved classification model based on the attention mechanism includes a 7×7 convolution layer, a maximum pooling layer, an average pooling layer, a fully connected layer, two global lightweight attention modules and three high-low channel jump connection modules; the global lightweight attention module includes three convolution blocks, five different feature maps, an attention map, two average pooling layers and an upsampling layer; the high-low channel jump connection module consists of two high-low channel jump connections, each high-low channel jump connection includes three high-low channel convolution blocks, three standardization operation layers and three activation function layers.

[0031] Step 4, training the improved classification model based on the attention mechanism: use the training set to train the network model, and use the test set to test the trained model after the training times reach the preset threshold. When the accuracy of the network model stabilizes within a certain range, the network model training is considered complete, and the parameters of the network model are saved. After the network training is completed, the network model parameters are fixed, and the network model is determined to be the final nodule prediction model. When it is necessary to predict the benign or malignant nature of thyroid nodule ultrasound images, the image can be directly input into the network model to obtain the final prediction result.

[0032] Step 5: Use the model to classify benign and malignant thyroid nodules: scale the ultrasound image of the thyroid nodules to be classified to a size of 640×640, normalize it, and input it into the trained model to output the classification result.

[0033] Example 2

[0034] like Figure 1 As shown, a thyroid nodule classification method based on a global lightweight attention module comprises the following steps:

[0035] Step 1: Construct a thyroid nodule dataset. Under the guidance of experienced clinicians, medical images of malignant and benign thyroid nodules were obtained under ultrasound. The original JPG format images collected had a resolution of 1440×1080 pixels, and this part of the images was constructed as the initial thyroid nodule dataset.

[0036] Step 2: Preprocess and expand the dataset. Remove patient information from the dataset constructed in step 1, unify the image size to 640×640 pixels, and perform normalization. Further perform contrast enhancement, brightness enhancement, and mirroring on the preprocessed dataset, and expand the dataset to 3-5 times the initial dataset to obtain the final dataset. Divide the expanded dataset into a training set and a test set in a ratio of 7:3.

[0037] Step 3: Build an improved classification model based on the attention mechanism. The structure of the improved classification model based on the attention mechanism is shown in the figure below: Figure 2 As shown in the figure, the image is input into the model, passing through a convolution layer with a convolution kernel size of 7×7, a maximum pooling layer, two global lightweight attention modules, three high and low channel skip connection modules, an average pooling layer and a fully connected layer;

[0038] Among them, the global lightweight attention module is as follows Figure 3 As shown in the figure, the input original feature map A is divided into three branches, each of which passes through a convolution layer with a convolution kernel size of 1×1 to obtain feature maps B, C, and D. Feature map C undergoes an average pooling operation and matrix multiplication with feature map B, and then undergoes a Softmax function operation to obtain feature map E. Feature map D undergoes an average pooling operation and matrix multiplication with feature map E, and then undergoes an upsampling operation, and is superimposed with the original feature map A in the channel dimension to obtain the final attention feature map F;

[0039] The Softmax function is defined as follows:

[0040]

[0041] Where: zi is the output value of the i-th node, and C is the number of nodes.

[0042] Among them, the high-low channel jump connection module consists of two high-low channel jump connections, and each high-low channel jump connection includes three high-low channel convolution blocks, such as Figure 4As shown, the high-low channel skip connection trunk includes two high-low channel convolution blocks with a convolution kernel size of 3×3, two standardization operation layers and two Swish activation function layers; the skip connection branch includes a high-low channel convolution block with a convolution kernel size of 1×1, a standardization operation layer and a Swish activation function layer, wherein the high-low channel skip connection module 1;

[0043] Among them, the high and low channel convolution blocks are as follows Figure 5 As shown in Figure 1, the high-frequency channel includes convolution and pooling operations, and the low-frequency channel includes average pooling operations, upsampling operations, and convolution operations, where the convolution kernel size in the convolution operation is 2×2. H and low frequency X L , and the resolution of the low-frequency image is reduced to half of the high-frequency image, and the high-frequency channel output result is Y H , the low frequency channel output is Y L , the final output can be expressed as:

[0044] Y=Y H +Y L

[0045] Among them, Y H and Y L The expression is:

[0046] Y H =f(X H , ·W H→H )+upsample[f(X L ; W L→H ), 2]

[0047] Y L =f(X L , ·W L→L )+f[pool(X H , 2); W H→ L]

[0048] Wherein: upsample represents the upsampling operation, pool represents the pooling operation, and f(X, W) represents the convolution operation with the convolution kernel W.

[0049] Where: upsample represents the upsampling operation, pool represents the pooling operation, and f(X,W) represents the convolution operation with the convolution kernel W.

[0050] The Swish activation function is defined as follows:

[0051] Swish(x)=x·sigmoid(x),

[0052]

[0053] Step 4: Train the improved classification model based on the attention mechanism. During the network training process, the number of training iterations is set to 500, the learning rate is set to 10-3, and the network training is considered complete when the prediction accuracy reaches more than 95%. After the network training is completed, the network model parameters are fixed. The network model using the network model parameters is determined as the final prediction model.

[0054] Step 5: Use the model to classify benign and malignant thyroid nodules. After scaling the ultrasound image of the thyroid nodules to be classified to a size of 640×640, normalize it, and input it into the trained model, the thyroid nodule classification result can be obtained to assist doctors in diagnosing the disease.

[0055] The classification of the improved classification model based on the attention mechanism is analyzed: the test set divided in step 2 is input into the classification model, and the accuracy, recall, precision and time are used as evaluation indicators. As shown in Table 1, the network model experimental results of the present invention show that the network model proposed by the present invention has the best effect.

[0056] Table 1 Network model experimental results

[0057] Network Model Accuracy Recall Accuracy time Resnet18 0.8927 0.896 0.892 4.63s ResNet50 0.8902 0.855 0.922 4.76s Ours 0.9687 0.988 0.949 1.95s

[0058] Among them, the implementation of convolution, activation function, pooling and upsampling are algorithms well known to those skilled in the art, and the specific processes and methods can be found in corresponding textbooks or technical literature.

[0059] The preferred embodiments of the present invention are described in detail above in conjunction with the accompanying drawings. However, the present invention is not limited to the specific details in the above embodiments. Within the technical concept of the present invention, a variety of simple modifications can be made to the technical solution of the present invention, and these simple modifications all belong to the protection scope of the present invention.

[0060] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present invention will not further describe various possible combinations.

[0061] In addition, various embodiments of the present invention may be arbitrarily combined, and as long as they do not violate the concept of the present invention, they should also be regarded as the contents disclosed by the present invention.

Claims

1. A thyroid nodule classification method based on a global lightweight attention module, characterized in that: The specific steps are as follows: Step 1: Construct a thyroid nodule dataset: collect ultrasound results of thyroid nodule examinations performed in hospitals and construct a thyroid nodule dataset; Step 2: Preprocess and expand the data set: Perform various preprocessing operations on the initially constructed data set and further expand the data set; Step 3: Construct an improved classification model based on the attention mechanism: This classification model combines the global lightweight attention module and the high-low channel skip connection module to adaptively combine the global features and detail features of the nodules, optimize the original network model, and improve the classification accuracy and diagnostic efficiency; Step 4: Train the improved classification model based on the attention mechanism: Use the training set to train the network model. After the training times reach the preset threshold, use the test set to test the trained model. When the accuracy of the network model stabilizes within a certain range, the network model training is considered complete and the parameters of the network model are saved. After the network training is completed, the network model parameters are fixed and the network model is determined to be the final nodule prediction model. When it is necessary to predict the benign or malignant nature of thyroid nodule ultrasound images, the image can be directly input into the network model to obtain the final prediction result. Step 5: Use the model to classify benign and malignant thyroid nodules: scale the ultrasound image of the thyroid nodules to be classified to a size of 640×640, normalize it, and input it into the trained model to output the classification result.

2. A multi-scale and multi-region thyroid nodule prediction method as claimed in claim 1, characterized in that: In step 1, the thyroid nodule dataset refers to ultrasound images of thyroid nodules of patients who undergo thyroid ultrasound examination in the hospital, and are divided into benign nodules and malignant nodules through puncture pathological surgery under the guidance of professional doctors to construct the thyroid nodule dataset.

3. The multi-scale and multi-region thyroid nodule prediction method according to claim 1, characterized in that: In the step 2, preprocessing refers to removing patient information from the data set constructed in claim 2 to unify the size and perform normalization processing; expanding the data set refers to contrast enhancement, brightness enhancement and mirroring operations on the preprocessed data set, expanding the number of data sets to 3-5 times the original, and dividing the expanded data set into a training set and a test set in a ratio of 7:

3.

4. The multi-scale and multi-region thyroid nodule prediction method according to claim 1, characterized in that: In the step 3, the improved classification model based on the attention mechanism includes a 7×7 convolution layer, a maximum pooling layer, an average pooling layer, a fully connected layer, two global lightweight attention modules and three high and low channel jump connection modules; the global lightweight attention module includes three convolution blocks, five different feature maps, an attention map, two average pooling layers and an upsampling layer; the high and low channel jump connection module includes three high and low channel convolution blocks, three standardization operation layers and three activation function layers.

Images