Thyroid cancer ultrasound image classification method and system based on multi-modal feature fusion and bayesian optimization

Abstract

The application discloses a thyroid cancer ultrasound image classification method and system based on multi-modal feature fusion and Bayesian optimization, relates to the technical field of medical image processing, collects a thyroid ultrasound image dataset and performs a pretreatment operation; improved local binary pattern features, Haralick texture features and VGG16 deep features are extracted to construct a mixed feature space; features in the mixed feature space are spliced into a 4119-dimensional mixed feature vector, and ExtraTrees is used for feature importance screening.The application fuses improved local binary pattern features, Haralick texture features and VGG16 deep features, constructs a mixed feature space, and uses ExtraTrees for feature importance screening and PCA dimension reduction, effectively extracts multi-level features of an image, significantly improves the accuracy of benign and malignant classification of thyroid nodules, simultaneously, a Bayesian optimization algorithm dynamically adjusts network hyperparameters, combines a cosine annealing strategy to accelerate model convergence, and enhances the generalization ability of the model, so that the model can perform well on different datasets.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, specifically to a method and system for classifying thyroid cancer ultrasound images based on multimodal feature fusion and Bayesian optimization. Background Technology

[0002] Thyroid nodules are one of the most common diseases of the endocrine system. Their benign or malignant classification is of great significance for the selection of clinical diagnosis and treatment plans. Early and accurate diagnosis of thyroid cancer can significantly improve the cure rate and survival rate of patients. With the incidence of thyroid malignant tumors rising year by year, the development of more accurate and efficient diagnostic methods has become an urgent need.

[0003] However, traditional diagnostic methods such as ultrasound imaging, CT, and MRI rely heavily on physicians' experience, are highly subjective, and have poor consistency in diagnostic results. While deep learning methods (such as CNN) improve classification performance, they also suffer from problems such as susceptibility to noise interference, low feature interpretability, reliance on single features, and low efficiency of traditional hyperparameter optimization methods (such as grid search).

[0004] Furthermore, existing studies mostly employ single-modal features (such as pure depth features or traditional texture features) or deep learning models with fixed hyperparameters, such as deep feature extraction based on VGG16 or traditional feature analysis based on LBP and Haralick textures. However, these methods suffer from incomplete feature representation or low optimization efficiency. To address these issues, we propose a classification method and system for thyroid cancer ultrasound images based on multimodal feature fusion and Bayesian optimization. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for classifying thyroid cancer ultrasound images based on multimodal feature fusion and Bayesian optimization, so as to solve the problems mentioned in the background art.

[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0007] Firstly, a classification method for thyroid cancer ultrasound images based on multimodal feature fusion and Bayesian optimization includes the following steps:

[0008] S1. Collect thyroid ultrasound image dataset and perform preprocessing operations;

[0009] S2. Extract improved Local Binary Pattern (LBP) features, Haralick texture features, and VGG16 depth features to construct a hybrid feature space;

[0010] S3. Concatenate the features in the mixed feature space into a 4119-dimensional mixed feature vector, use ExtraTrees to filter the feature importance, select the top 200 key features, and reduce the dimensionality to 12 dimensions using PCA.

[0011] S4. Construct a lightweight fully connected neural network (LightFC-DNN). The input layer receives a 12-dimensional feature vector, and the output is the classification result after passing through a double fully connected layer.

[0012] S5. Use Bayesian optimization algorithm to dynamically adjust network hyperparameters, and combine cosine annealing and linear warm-up learning rate strategy to accelerate model convergence and avoid getting trapped in local extrema.

[0013] S6. Train and validate the model on the TN-SCUI 2020 dataset, and plot the confusion matrix and ROC curve for analysis.

[0014] A further improvement to the technical solution of the present invention is that: S1 specifically includes:

[0015] Thyroid ultrasound images were obtained from the TN-SCUI 2020 dataset to ensure that the dataset covers both benign and malignant cases. Each thyroid ultrasound image was standardized and uniformly scaled to 256×256 pixels to eliminate scale inconsistencies caused by differences in equipment between images from different sources. Then, the optimal cropping parameters determined based on Monte Carlo experiments were used to crop the edges of the standardized images, so that the cropped images retain the effective area of ​​thyroid nodules with a probability of 96.7%, highlighting the nodule features.

[0016] Medium filtering was applied to the size-normalized and edge-cropped thyroid ultrasound images, and nonlinear filtering was performed using a 3×3 filter kernel.

[0017] Histogram equalization was performed on the thyroid ultrasound images after median filtering. After processing, the information entropy of the thyroid ultrasound images was increased from 5.32 to 6.18, and the contrast of microcalcifications was improved by 39.7%, further optimizing the image preprocessing effect.

[0018] A further improvement to the technical solution of the present invention is that: S2 specifically includes:

[0019] For the preprocessed thyroid ultrasound image, the improved Local Binary Pattern (LBP) algorithm is used to extract texture features. The LBP operator of uniform mode is used, with sampling radius R=2 and number of sampling points P=16. The LBP value of each pixel is calculated and a 17-dimensional histogram feature is generated to reflect the texture information of the image.

[0020] Based on the preprocessed thyroid ultrasound images, a gray-level co-occurrence matrix (GLCM) was constructed. Haralick texture features were extracted by calculating six statistics of the GLCM, including contrast, dissimilarity, homogeneity, energy, correlation, and second moment of angle.

[0021] We used a VGG16 convolutional neural network pre-trained on the ImageNet dataset to extract depth features from thyroid ultrasound images. We removed the top classifier of the VGG16 network and retained only the feature extraction part. We converted the image to RGB format, resized it to 224×224 pixels, and normalized it. Then we input the pre-processed image into the adjusted VGG16 network to extract the 4096-dimensional feature vector of the penultimate fully connected layer.

[0022] The extracted improved LBP features, Haralick texture features, and VGG16 depth features are concatenated to form a hybrid feature space, providing a more comprehensive and accurate image description and further improving classification performance.

[0023] A further improvement to the technical solution of the present invention is that: S3 specifically includes:

[0024] Based on the concatenation of the extracted improved LBP features, Haralick texture features and VGG16 depth features, an initial 4119-dimensional hybrid feature vector is formed, covering multiple feature dimensions from low-level texture information to high-level semantic information.

[0025] The ExtraTrees classifier is used to filter the importance of a 4119-dimensional mixed feature vector. ExtraTrees constructs multiple random decision trees, calculates the split contribution of each feature in the tree, calculates the average impurity reduction of each feature in all trees to measure the importance of the feature, and selects the top 200 features as key features based on the feature importance score output by the ExtraTrees algorithm.

[0026] Principal Component Analysis (PCA) algorithm is used to reduce the dimensionality of the first 200 key features. The original feature space is mapped to a new low-dimensional feature space through linear transformation, so that the new features (principal components) are independent of each other and sorted by variance. Principal components that retain 90% of the variance information are selected, and finally the feature vector is reduced to 12 dimensions.

[0027] A further improvement to the technical solution of the present invention is that: S4 specifically includes:

[0028] The architecture of a lightweight fully connected neural network (LightFC-DNN) is designed. Its input layer receives a 12-dimensional feature vector after dimensionality reduction. The network architecture includes two fully connected layers. The first fully connected layer maps the 12-dimensional input features to 256 dimensions, and the second fully connected layer further compresses the 256-dimensional features to 128 dimensions. The layer-by-layer compression design can effectively extract high-order information of features while reducing computational complexity. The activation function is LeakyReLU with α=0.2.

[0029] An output layer is constructed at the end of a lightweight fully connected neural network. The output layer contains two neurons, which correspond to the benign and malignant classifications of thyroid nodules, respectively. The Softmax function is used as the activation function to map the 128-dimensional features onto the probability distributions of the two classifications.

[0030] After completing the architecture design and output layer construction of the lightweight fully connected neural network, the network is initialized and configured. The weights and bias parameters of the network are initialized using a random initialization method to ensure that the network can effectively propagate gradients in the early stage of training. The loss function of the network is configured, and the cross-entropy loss function is used to measure the difference between the model output probability distribution and the true label. At the same time, the Adam optimizer is selected to update the network parameters during training to minimize the loss function value.

[0031] A further improvement to the technical solution of the present invention is that: S5 specifically includes:

[0032] Define the search space for the Bayesian optimization algorithm, and clarify the hyperparameters to be optimized and their value ranges, where the learning rate ranges from... arrive The range of weight decay is from arrive The Dropout rate ranges from 0.2 to 0.5, the gradient clipping threshold ranges from 0.5 to 1.5, and the Bayesian optimization algorithm is initialized, with a Gaussian process selected as the probabilistic surrogate model and the expected improvement function selected as the acquisition function.

[0033] In the early stages of training, a linear warm-up strategy is adopted, in which the learning rate increases linearly from the initial value to the set maximum learning rate. After the warm-up phase, a cosine annealing strategy is adopted to adjust the learning rate. The learning rate gradually decreases in the form of a cosine function as the number of training rounds increases, which helps the model to adjust parameters more finely in the later stages of training and avoid getting trapped in local extrema. Thus, the linear warm-up and cosine annealing strategies are combined to form a dynamically adjusted learning rate strategy.

[0034] The Bayesian optimization algorithm is used to search within a set range of hyperparameters. In each iteration, a set of hyperparameter combinations is selected based on the acquisition function. The model is trained using a learning rate strategy that combines cosine annealing and linear warm-up. After each iteration, the model performance is evaluated, the cross-entropy loss function value is recorded, and the probabilistic surrogate model of the Bayesian optimization algorithm is updated based on the evaluation results. After multiple iterations, a set of hyperparameter combinations that optimizes the model's performance on the validation set is found. This set of hyperparameters is then used for the final model training to obtain a thyroid nodule benign / malignant classification model with the best performance.

[0035] A further improvement to the technical solution of the present invention is that: S6 specifically includes:

[0036] The TN-SCUI 2020 dataset was divided into training, validation, and test sets in a ratio of 7:1.5:1.5 to ensure a balanced distribution of data across different sets and to accurately evaluate the model's performance at different stages. Images in the training, validation, and test sets were preprocessed, including normalization and resizing, to meet the model's input requirements. Labels were also encoded. The constructed lightweight fully connected neural network (LightFC-DNN) was trained using the training set. Hyperparameters were dynamically adjusted using a Bayesian optimization algorithm, and a learning rate strategy combining linear warm-up and cosine annealing was adopted. During training, the loss function values ​​and accuracy of the training and validation sets were recorded to monitor model convergence.

[0037] The model is periodically evaluated during training using a validation set. Indicators such as classification accuracy, recall, and F1 score are calculated. Based on the evaluation results, it is observed whether the model is overfitting or underfitting. Training strategies or hyperparameters are adjusted accordingly. Confusion matrices of the validation set and prediction set are plotted to show the model's classification of benign and malignant thyroid nodules. The classification accuracy and misclassification of each category are analyzed to identify the model's weaknesses. Based on the analysis results of the confusion matrix, the model structure or hyperparameters are adjusted accordingly.

[0038] The optimized model is then evaluated using a test set. Evaluation metrics such as accuracy and mean absolute error are calculated. If the classification accuracy does not reach 95%, the reasons are analyzed to determine whether it is due to factors such as differences in the dataset or limitations in the model structure. Based on the prediction results of the test set, an ROC curve is plotted, and the AUC value is calculated. Then, based on the analysis results of the ROC curve and classification accuracy, the model is further optimized by adding data augmentation, adjusting network depth, etc. The training, validation, and evaluation process is repeated until the model achieves a classification accuracy of 95% on the test set.

[0039] A further improvement to the technical solution of the present invention is that the formula for calculating the accuracy is as follows: ;

[0040] In the formula, Accuracy represents the proportion of samples correctly predicted by the model out of the total number of samples. This indicates the number of samples that the model correctly predicts as positive when they are actually positive. This indicates the number of samples that the model correctly predicted as negative when they were actually negative. This represents the number of samples that the model incorrectly predicts as positive when they are actually negative. This indicates the number of samples that the model incorrectly predicts as negative when they are actually positive.

[0041] The formula for calculating the mean absolute error is as follows: ;

[0042] In the formula, Mean absolute error (MAE) represents the average magnitude of the difference between the predicted and actual values. and These represent the height and width of the sample, respectively. This represents the total number of sample pairs. The model predicts the first Line number The probability that a sample belongs to a certain category. Indicates the first Line number The true class probability of the column sample.

[0043] A further improvement to the technical solution of the present invention lies in the calculation process of the AUC value:

[0044] The optimized model is run on the test set to obtain the predicted probability that each sample belongs to the positive class, i.e., malignant thyroid nodules. This ensures that the prediction results correspond one-to-one with the true labels in the test set. At the same time, the data is checked for missing values ​​or outliers. If any are found, they need to be processed accordingly to ensure the accuracy of subsequent ROC curve plotting and AUC value calculation.

[0045] Using the obtained predicted probabilities and the true labels of the test set, combined with the scikit-learn and matplotlib libraries in Python, the true positive rate and false positive rate at different thresholds are calculated by using the roc_curve function of scikit-learn, passing in the true labels of the test set and the positive class probability predicted by the model. The ROC curve is plotted with the false positive rate on the horizontal axis and the true positive rate on the vertical axis using the plotting function of matplotlib.

[0046] The `roc_auc_score` function from the scikit-learn library is used to calculate the area under the ROC curve (AUC), which is between 0 and 1. AUC values ​​closer to 1 indicate better classification performance. An AUC of 0.5 indicates that the model's classification ability is equivalent to random guessing. Based on the calculated AUC value and the shape of the ROC curve, the model's performance is evaluated and analyzed. If the AUC value does not meet expectations, the model can be further optimized, such as by adjusting hyperparameters or increasing training data.

[0047] Secondly, a thyroid cancer ultrasound image classification system based on multimodal feature fusion and Bayesian optimization is provided to implement the aforementioned thyroid cancer ultrasound image classification method based on multimodal feature fusion and Bayesian optimization. The system includes an ultrasound image classification platform, which is communicatively connected to the following modules:

[0048] The data preprocessing module is used to perform preprocessing operations on thyroid ultrasound images, including size standardization, edge cropping, median filtering, and histogram equalization.

[0049] The multimodal feature extraction module is used to extract improved local binary pattern features, Haralick texture features, and VGG16 depth features to construct a hybrid feature space.

[0050] The feature filtering and dimensionality reduction module is used to filter features based on their importance using ExtraTrees and reduce the dimensionality to 12 dimensions using PCA to remove redundant information and retain the most important features.

[0051] The lightweight neural network classification module is used to build a lightweight fully connected neural network to learn and classify the dimensionality-reduced feature vectors.

[0052] The Bayesian optimization and hyperparameter tuning module is used to dynamically adjust the network hyperparameters using the Bayesian optimization algorithm, and combines cosine annealing and linear warm-up learning rate strategies to accelerate model convergence.

[0053] Due to the adoption of the above technical solution, the technical progress achieved by this invention compared to the prior art is as follows:

[0054] 1. This invention provides a method and system for classifying thyroid cancer ultrasound images based on multimodal feature fusion and Bayesian optimization. By fusing improved local binary pattern features, Haralick texture features, and VGG16 depth features, a hybrid feature space is constructed. ExtraTrees are used for feature importance screening and PCA dimensionality reduction, effectively extracting multi-level features of the image and significantly improving the accuracy of benign and malignant classification of thyroid nodules. At the same time, the Bayesian optimization algorithm dynamically adjusts the network hyperparameters, and the cosine annealing strategy accelerates model convergence, enhancing the model's generalization ability and enabling it to perform well on different datasets.

[0055] 2. This invention provides a method and system for classifying thyroid cancer ultrasound images based on multimodal feature fusion and Bayesian optimization. By combining traditional texture features and depth features, it not only captures the abstract semantic information of the image, but also retains the physical interpretability of traditional features. The improved local binary pattern and Haralick texture features provide rich texture information, which helps to interpret the model decision criteria from a medical perspective, enhances the interpretability of the model, and makes it easier for doctors to understand and trust the model output results. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0057] Figure 1 This is a schematic diagram of the workflow of the present invention;

[0058] Figure 2 This is a schematic diagram of the confusion matrix between the training set and the prediction set of this invention. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0060] Example 1, such as Figure 1 , Figure 2 As shown, this invention provides a method for classifying thyroid cancer ultrasound images based on multimodal feature fusion and Bayesian optimization, including the following steps:

[0061] S1. Collect a dataset of thyroid ultrasound images and perform preprocessing operations such as size standardization, edge cropping, median filtering, and histogram equalization. Thyroid ultrasound images are obtained from the TN-SCUI 2020 dataset, ensuring that the dataset covers both benign and malignant cases. Each thyroid ultrasound image is standardized and uniformly scaled to 256×256 pixels to eliminate scale inconsistencies caused by differences in equipment between images from different sources. Then, based on the optimal cropping parameters determined by Monte Carlo experiments, edge cropping is performed on the size-standardized images, ensuring that the cropped images retain the effective region of thyroid nodules with a 96.7% probability, while significantly reducing the interference of irrelevant tissues on subsequent analysis and highlighting nodule features. Median filtering is applied to the size-standardized and edge-cropped thyroid ultrasound images, using a 3×3 filter kernel for nonlinear filtering to effectively remove image... The image is filtered to remove random noise while preserving important structural and edge information. Median filtering smooths local noise and improves the signal-to-noise ratio (SNR) from 12.4 dB to 15.7 dB, avoiding over-smoothing that could lead to loss of image details. Histogram equalization is then performed on the thyroid ultrasound image after median filtering. In this process, the brightness component of the image is equalized in the HSV color space. By redistributing the pixel values, the contrast and detail of the image are enhanced, making nodule features clearer and more identifiable. After processing, the information entropy of the thyroid ultrasound image is increased from 5.32 to 6.18, and the contrast of microcalcifications is improved by 39.7%, further optimizing the image preprocessing effect.

[0062] S2. Extract improved Local Binary Pattern (LBP) features, Haralick texture features, and VGG16 depth features to construct a hybrid feature space. For the preprocessed thyroid ultrasound image, extract texture features using the improved LBP algorithm. The LBP is represented by the LBP operator in uniform mode, with a sampling radius R=2 and a sampling point count P=16. Calculate the LBP value for each pixel and generate a 17-dimensional histogram feature to reflect the image's texture information. Based on the preprocessed thyroid ultrasound image, construct the Gray-Level Co-occurrence Matrix (GLCM). Then, calculate the 6... Several statistical measures, including contrast, dissimilarity, homogeneity, energy, correlation, and second moment of angle, were used to extract Haralick texture features, describing the texture characteristics of the image from different perspectives, reflecting the spatial distribution and interrelationships of textures in the image, and providing rich texture information for the mixed feature space. A VGG16 convolutional neural network pre-trained on the ImageNet dataset was used to extract depth features from thyroid ultrasound images. The top-level classifier of the VGG16 network was removed, retaining only the feature extraction part. The image was converted to RGB format, resized to 224×224 pixels, and normalized (mean value [0.485, ...). [0.456, 0.406], with standard deviations of [0.229, 0.224, 0.225]), and then input the preprocessed image into the adjusted VGG16 network, extracting the 4096-dimensional feature vector of the penultimate fully connected layer. VGG16 deep features can capture the high-level semantic information of the image, providing the feature representation capability of the deep learning model for the hybrid feature space, further improving classification performance. The extracted improved LBP features, Haralick texture features and VGG16 deep features are concatenated to form a hybrid feature space, providing a more comprehensive and accurate image description, further improving classification performance;

[0063] S3. Concatenate the features in the mixed feature space into a 4119-dimensional mixed feature vector. Use ExtraTrees for feature importance filtering, selecting the top 200 key features, and reduce the dimensionality to 12 dimensions using PCA. Based on the concatenation results of the extracted improved LBP features, Haralick texture features, and VGG16 deep features, an initial 4119-dimensional mixed feature vector is formed, covering multiple feature dimensions from low-level texture information to high-level semantic information. Confirm that the concatenated 4119-dimensional mixed feature vector is constructed correctly and that each feature dimension corresponds correctly. Use the ExtraTrees classifier to filter the feature importance of the 4119-dimensional mixed feature vector. ExtraTrees constructs multiple random decision trees, calculates the split contribution of each feature in the tree, and calculates the average non-split contribution of each feature in all trees. Purity reduction measures the importance of features, and based on the feature importance score output by the ExtraTrees algorithm, the top 200 most important features are selected as key features. This aims to remove redundant and irrelevant features, reduce feature dimensionality, and retain the most valuable information for the classification task, thereby improving the training efficiency and classification performance of the model. Principal Component Analysis (PCA) is used to reduce the dimensionality of the selected top 200 key features. Through linear transformation, the original feature space is mapped to a new low-dimensional feature space, making the new features (principal components) independent of each other and sorted by variance. Principal components that retain 90% of the variance information are selected, and finally the feature vector is reduced to 12 dimensions. The 12-dimensional feature vector after dimensionality reduction is evaluated to confirm that it can effectively retain the main information in the original feature space, while improving the performance and generalization ability of the classification model.

[0064] S4. Construct a lightweight fully connected neural network (LightFC-DNN). The input layer receives a 12-dimensional feature vector, which outputs a classification result after passing through two fully connected layers. The architecture of the lightweight fully connected neural network (LightFC-DNN) is designed, with its input layer receiving the 12-dimensional feature vector after dimensionality reduction. The network architecture includes two fully connected layers. The first fully connected layer maps the input 12-dimensional features to 256 dimensions, and the second fully connected layer further compresses the 256-dimensional features to 128 dimensions. This layer-by-layer compression design effectively extracts high-order information of the features while reducing computational complexity. The activation function chosen is LeakyReLU with α=0.2 to alleviate the gradient vanishing problem and allow negative inputs to retain some gradients. Compared to the traditional ReLU function, LeakyReLU can alleviate the gradient vanishing problem and allows negative inputs to retain some gradients, making it suitable for scenarios with many low-contrast regions in medical images. An output layer is constructed at the end of the lightweight fully connected neural network. The output layer contains two neurons, corresponding to the benign and malignant classifications of thyroid nodules, respectively. The Softmax function is used as the activation function, mapping the 128-dimensional features to the probability distributions of the two categories. The Softmax function normalizes the output values ​​to probability values, ensuring that the output value for each category is between 0 and 1, and the sum of the output values ​​for all categories is 1. Through the Softmax function, the model outputs the probability corresponding to each category, thereby achieving the benign and malignant classification of thyroid nodules. After completing the architecture design of the lightweight fully connected neural network and constructing the output layer, the network is initialized and configured. Random initialization methods (Xavier initialization or He initialization) are used to initialize the network weights and bias parameters to ensure that the network can effectively propagate gradients in the early stages of training. The network loss function is then configured, using the cross-entropy loss function to measure the difference between the model's output probability distribution and the true label. Simultaneously, the Adam optimizer is selected to update the network parameters during training to minimize the loss function value.

[0065] S5. Use the Bayesian optimization algorithm to dynamically adjust network hyperparameters, including learning rate, weight decay, and dropout rate. Combine this with a learning rate strategy that integrates cosine annealing and linear warm-up to accelerate model convergence and avoid getting trapped in local optima. Define the search space for the Bayesian optimization algorithm, as shown in Table 1. Clearly define the hyperparameters to be optimized and their value ranges. The learning rate ranges from... arrive The range of weight decay is from arrive The dropout rate ranges from 0.2 to 0.5, and the gradient clipping threshold ranges from 0.5 to 1.5. A Bayesian optimization algorithm is initialized, with a Gaussian process chosen as the probabilistic surrogate model and the expected improvement function as the acquisition function. A linear warm-up strategy is used in the early stages of training, where the learning rate increases linearly from the initial value to the set maximum learning rate. This helps the model converge quickly in the early stages of training and avoids instability caused by an excessively large learning rate. After the warm-up phase, a cosine annealing strategy is used to adjust the learning rate. Here, the learning rate gradually decreases according to a cosine function as the number of training epochs increases, helping the model to finely adjust parameters in the later stages of training and avoid getting trapped in local optima. Therefore, the linear warm-up and cosine annealing strategies are combined to form a dynamically adjusted learning rate strategy. In the linear warm-up phase, the learning rate increases linearly from 0 to a maximum value in the first 5 epochs of training. By gradually increasing the learning rate, the model is helped to quickly reach a stable state. During the cosine annealing phase, starting from the 6th epoch, the learning rate decays according to a cosine function, with a period length of 80 epochs, and the minimum learning rate decays to... The cosine annealing strategy improves the model's generalization ability by gradually reducing the learning rate, allowing for more precise parameter adjustments in the later stages of training. A Bayesian optimization algorithm is used to search within a set range of hyperparameters. In each iteration, a set of hyperparameter combinations is selected based on the acquisition function. The model is trained using a learning rate strategy combining cosine annealing and linear warm-up. After each iteration, the model's performance is evaluated, the cross-entropy loss function value is recorded, and the probabilistic surrogate model of the Bayesian optimization algorithm is updated based on the evaluation results. After multiple iterations, a set of hyperparameter combinations that optimizes the model's performance on the validation set is found. This set of hyperparameters is then used for final model training to obtain the best-performing benign / malignant thyroid nodule classification model.

[0066] Table 1. Search Space Definition Table

[0067]

[0068] S6. Train and validate the model on the TN-SCUI 2020 dataset, and plot the confusion matrix and ROC curve for analysis, so that the classification accuracy of the optimized model reaches 95%.

[0069] Example 2, as Figure 1 , Figure 2 As shown, based on Embodiment 1, the present invention provides a technical solution: Preferably, S6 specifically includes:

[0070] The TN-SCUI 2020 dataset was divided into training, validation, and test sets in a ratio of 7:1.5:1.5 to ensure a balanced distribution of data across different datasets and accurately evaluate the model's performance at different stages. Images in the training, validation, and test sets were preprocessed, including normalization and resizing, to meet the model's input requirements. Labels were also encoded. The constructed lightweight fully connected neural network (LightFC-DNN) was trained using the training set, with hyperparameters dynamically adjusted using a Bayesian optimization algorithm. A learning rate strategy combining linear warm-up and cosine annealing was employed. During training, the loss function values ​​and accuracy of the training and validation sets were recorded to monitor model convergence. The model was periodically evaluated using the validation set, calculating metrics such as classification accuracy, recall, and F1 score. Based on the evaluation results, overfitting or underfitting was observed, and training strategies or hyperparameters were adjusted accordingly. Confusion matrices for the validation and prediction sets were plotted to demonstrate the model's ability to classify thyroid nodules. The model is classified into benign and malignant forms. The classification accuracy and misclassification rates for each category are analyzed to identify weaknesses in the model. Based on the confusion matrix analysis, the model structure or hyperparameters are adjusted accordingly, including fine-tuning the Dropout rate and learning rate, to improve performance on the validation set. The optimized model is then evaluated using the test set, calculating accuracy and mean absolute error (AUC). If the classification accuracy does not reach 95%, the reasons are analyzed to determine if it is due to dataset differences, model structure limitations, or other factors. Based on the prediction results on the test set, an ROC curve is plotted, and the AUC value is calculated. The ROC curve visually displays the true positive rate and false positive rate of the model at different thresholds. The closer the AUC value is to 1, the better the model's classification performance. Based on the ROC curve and classification accuracy analysis, the model is further optimized by adding data augmentation, adjusting network depth, etc. The training, validation, and evaluation process is repeated until the model achieves a classification accuracy of 95% on the test set.

[0071] Furthermore, the formula for calculating accuracy is as follows: ;

[0072] In the formula, Accuracy represents the proportion of samples correctly predicted by the model out of the total number of samples. This indicates the number of samples that the model correctly predicts as positive when they are actually positive. This indicates the number of samples that the model correctly predicted as negative when they were actually negative. This represents the number of samples that the model incorrectly predicts as positive when they are actually negative. Accuracy indicates the number of samples that are actually positive but are incorrectly predicted as negative. The value of accuracy ranges from 0 to 1, where 0 means that the model did not correctly predict any samples, and 1 means that the model correctly predicted all samples. The higher the accuracy, the better the model's performance.

[0073] The formula for calculating the mean absolute error is as follows: ;

[0074] In the formula, Mean absolute error (MAE) represents the average magnitude of the difference between the predicted and actual values. and These represent the height and width of the sample, respectively. This represents the total number of sample pairs. The model predicts the first Line number The probability that a sample belongs to a certain category. Indicates the first Line number The true class probability of the column sample. The smaller the value, the smaller the mean absolute error between the model's predicted probability and the true probability; that is, the closer the model's predicted probability is to the true probability, and the better the model's performance. The larger the value, the larger the mean absolute error between the model's predicted probability and the true probability, and the worse the model's performance. For all and When they are all established, At this point, the model's predicted probability is completely consistent with the actual probability;

[0075] Furthermore, the calculation process for AUC values:

[0076] The optimized model is run on the test set to obtain the predicted probability that each sample belongs to the positive class, i.e., malignant thyroid nodules. This ensures that the prediction results correspond one-to-one with the true labels in the test set. At the same time, the data is checked for missing values ​​or outliers. If any are found, they need to be processed accordingly to ensure the accuracy of subsequent ROC curve plotting and AUC value calculation.

[0077] Using the obtained predicted probabilities and the true labels of the test set, and combining the scikit-learn and matplotlib libraries in Python, the `roc_curve` function of scikit-learn, with the true labels of the test set and the model's predicted positive class probability as input, calculates the true positive rate (TPR) and false positive rate (FPR) at different thresholds. Plotting the false positive rate on the horizontal axis and the true positive rate on the vertical axis using matplotlib's plotting capabilities, the ROC (Receiver Operating Characteristic) curve is generated. The ROC curve illustrates the model's classification performance by plotting the true positive rate (TPR) and false positive rate (FPR) at different thresholds. The true positive rate is the proportion of samples correctly predicted as positive out of all actual positive samples, while the false positive rate is the proportion of samples incorrectly predicted as positive out of all actual negative samples. The closer the ROC curve is to the upper left corner, the better the model's classification performance.

[0078] The `roc_auc_score` function from the scikit-learn library is used to calculate the area under the ROC curve (AUC), which is between 0 and 1. The closer the AUC value is to 1, the better the model's classification performance. If the AUC value is 0.5, the model's classification ability is equivalent to random guessing. Based on the calculated AUC value and the shape of the ROC curve, the model's performance is evaluated and analyzed. If the AUC value does not meet expectations, the model is further optimized, such as by adjusting the model's hyperparameters or increasing the training data.

[0079] The formula for calculating the AUC value is as follows: ;

[0080] In the formula, The AUC value. The total number of samples in the test set. and They are the first The and the first The predicted probability that a sample belongs to the positive class. and They are the first The and the first The true label (0 or 1) of each sample. The variance of the predicted probabilities is used to adjust for the impact of the variability in predicted probabilities on AUC. It is a constant factor used to standardize the output of the arctangent function to match the range of AUC values. The function (arctangent function) is used to convert probability differences into angles, representing the relative ranking between sample pairs.

[0081] Example 3, as Figure 1 , Figure 2 As shown, based on Examples 1-2, this invention also provides a thyroid cancer ultrasound image classification system based on multimodal feature fusion and Bayesian optimization, used to implement the above-mentioned thyroid cancer ultrasound image classification method based on multimodal feature fusion and Bayesian optimization. The system includes an ultrasound image classification platform, which is communicatively connected to the following modules:

[0082] The data preprocessing module is used to perform preprocessing operations on thyroid ultrasound images, including size standardization, edge cropping, median filtering, and histogram equalization, to improve image quality, reduce noise interference, optimize image feature expression, and improve classification accuracy.

[0083] The multimodal feature extraction module is used to extract improved Local Binary Pattern (LBP) features, Haralick texture features, and VGG16 depth features, construct a hybrid feature space, and combine traditional texture features with deep learning features to provide comprehensive and accurate image descriptions and enhance classification performance.

[0084] The feature selection and dimensionality reduction module is used to select features based on their importance using ExtraTrees and reduce the dimensionality to 12 dimensions using PCA. This removes redundant information, retains the most important features, reduces model complexity, and improves classification efficiency.

[0085] The lightweight neural network classification module is used to build a lightweight fully connected neural network (LightFC-DNN) to learn and classify the dimensionality-reduced feature vectors. It adopts a lightweight design to reduce the computational threshold while maintaining high-performance classification capabilities, making it suitable for clinical deployment.

[0086] The Bayesian optimization and hyperparameter tuning module is used to dynamically adjust the network hyperparameters using the Bayesian optimization algorithm. It combines cosine annealing and linear warm-up learning rate strategies to accelerate model convergence, improve model generalization ability and training stability, avoid getting trapped in local optima, and achieve optimal classification performance.

[0087] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A classification method for thyroid cancer ultrasound images based on multimodal feature fusion and Bayesian optimization, characterized in that, Includes the following steps: S1. Collect thyroid ultrasound image dataset and perform preprocessing operations; S2. Extract improved local binary pattern features, Haralick texture features, and VGG16 depth features to construct a hybrid feature space, specifically including: For the preprocessed thyroid ultrasound images, the improved Local Binary Pattern (LBP) algorithm is used to extract texture features. The LBP algorithm is a uniform LBP operator with a sampling radius of R=2 and a sampling point count of P=16. The LBP value of each pixel is calculated and a 17-dimensional histogram feature is generated. Based on the preprocessed thyroid ultrasound images, a gray-level co-occurrence matrix was constructed. Haralick texture features were extracted by calculating six statistics of the gray-level co-occurrence matrix, including contrast, dissimilarity, homogeneity, energy, correlation, and second moment of angle. We used a VGG16 convolutional neural network pre-trained on the ImageNet dataset to extract depth features from thyroid ultrasound images. We removed the top classifier of the VGG16 network and retained only the feature extraction part. We converted the image to RGB format, resized it to 224×224 pixels, and normalized it. Then we input the pre-processed image into the adjusted VGG16 network to extract the 4096-dimensional feature vector of the penultimate fully connected layer. The extracted improved LBP features, Haralick texture features, and VGG16 depth features are concatenated to form a hybrid feature space. S3. Concatenate the features in the mixed feature space into a 4119-dimensional mixed feature vector. Use ExtraTrees to filter feature importance, selecting the top 200 key features, and then reduce the dimensionality to 12 dimensions using PCA. Specifically, this includes: Based on the concatenation of the extracted improved LBP features, Haralick texture features and VGG16 depth features, an initial 4119-dimensional hybrid feature vector is formed. The ExtraTrees classifier is used to filter the feature importance of a 4119-dimensional mixed feature vector. ExtraTrees constructs multiple random decision trees, calculates the split contribution of each feature in the tree, and selects the top 200 features based on the feature importance score output by the ExtraTrees algorithm as key features. Principal component analysis algorithm is used to reduce the dimensionality of the first 200 key features. The original feature space is mapped to a new low-dimensional feature space through linear transformation, so that the new features are independent of each other and sorted by variance. Principal components that retain variance information are selected, and finally the feature vector is reduced to 12 dimensions. S4. Construct a lightweight fully connected neural network. The input layer receives a 12-dimensional feature vector, and the output is the classification result after passing through two fully connected layers. Specifically, this includes: Design a lightweight fully connected neural network architecture. Its input layer receives a 12-dimensional feature vector after dimensionality reduction. The network architecture includes two fully connected layers. The first fully connected layer maps the 12-dimensional input features to 256 dimensions, and the second fully connected layer further compresses the 256-dimensional features to 128 dimensions. The activation function is LeakyReLU with α=0.

2. An output layer is constructed at the end of a lightweight fully connected neural network. The output layer contains two neurons, which correspond to the benign and malignant classifications of thyroid nodules, respectively. The Softmax function is used as the activation function to map the 128-dimensional features onto the probability distributions of the two classifications. After completing the architecture design and output layer construction of the lightweight fully connected neural network, the network is initialized and configured. The weights and bias parameters of the network are initialized using a random initialization method, and the loss function of the network is configured. The cross-entropy loss function is used to measure the difference between the model output probability distribution and the true label. At the same time, the Adam optimizer is selected to update the network parameters during training to minimize the loss function value. S5. Use Bayesian optimization algorithm to dynamically adjust network hyperparameters, and combine cosine annealing and linear warm-up learning rate strategy to accelerate model convergence; S6. Train and validate the model on the TN-SCUI 2020 dataset, and plot the confusion matrix and ROC curve for analysis.

2. The thyroid cancer ultrasound image classification method based on multimodal feature fusion and Bayesian optimization according to claim 1, characterized in that: S1 specifically includes: Thyroid ultrasound images were obtained from the TN-SCUI 2020 dataset, and each thyroid ultrasound image was standardized to a uniform size of 256×256 pixels. Then, the images were cropped at the edges using the optimal cropping parameters determined by Monte Carlo experiments. This ensured that the cropped images retained the effective area of ​​thyroid nodules with a probability of 96.7%, highlighting the nodule features. Medium filtering was applied to the size-normalized and edge-cropped thyroid ultrasound images, and nonlinear filtering was performed using a 3×3 filter kernel. Histogram equalization was performed on the thyroid ultrasound images after median filtering. After processing, the information entropy of the thyroid ultrasound images was increased from 5.32 to 6.18, and the contrast of microcalcifications was improved by 39.7%, further optimizing the image preprocessing effect.

3. The thyroid cancer ultrasound image classification method based on multimodal feature fusion and Bayesian optimization according to claim 1, characterized in that: S5 specifically includes: Define the search space for the Bayesian optimization algorithm, and clarify the hyperparameters to be optimized and their value ranges, where the learning rate ranges from... arrive The range of weight decay is from arrive The Dropout rate ranges from 0.2 to 0.5, the gradient clipping threshold ranges from 0.5 to 1.5, and the Bayesian optimization algorithm is initialized, with a Gaussian process selected as the probabilistic surrogate model and the expected improvement function selected as the acquisition function. In the early stages of training, a linear warm-up strategy is adopted, in which the learning rate increases linearly from the initial value to the set maximum learning rate. After the warm-up phase, a cosine annealing strategy is adopted to adjust the learning rate. The learning rate gradually decreases in the form of a cosine function as the number of training rounds increases. Thus, the linear warm-up and cosine annealing strategies are combined to form a dynamically adjusted learning rate strategy. The Bayesian optimization algorithm is used to search within a set range of hyperparameters. In each iteration, a set of hyperparameter combinations is selected based on the acquisition function. The model is trained using a learning rate strategy that combines cosine annealing and linear warm-up. After each iteration, the model performance is evaluated, the cross-entropy loss function value is recorded, and the probabilistic surrogate model of the Bayesian optimization algorithm is updated based on the evaluation results. After multiple iterations, a set of hyperparameter combinations that optimizes the model's performance on the validation set is found. This set of hyperparameters is then used for the final model training to obtain a thyroid nodule benign / malignant classification model with the best performance.

4. The thyroid cancer ultrasound image classification method based on multimodal feature fusion and Bayesian optimization according to claim 1, characterized in that: S6 specifically includes: The TN-SCUI 2020 dataset was divided into training, validation, and test sets. Images in the training, validation, and test sets were preprocessed, and labels were encoded. The lightweight fully connected neural network was trained using the training set. Hyperparameters were dynamically adjusted using a Bayesian optimization algorithm, and a learning rate strategy combining linear warm-up and cosine annealing was adopted. During training, the loss function values ​​and accuracy of the training and validation sets were recorded. The model is evaluated periodically during training using a validation set. Based on the evaluation results, it is observed whether the model is overfitting or underfitting. The training strategy or hyperparameters are adjusted accordingly. The confusion matrix of the validation set and the test set is plotted to show the model's classification of benign and malignant thyroid nodules. The classification accuracy and misclassification of each category are analyzed to identify the weaknesses of the model. Based on the analysis results of the confusion matrix, the model structure or hyperparameters are adjusted accordingly. The optimized model is evaluated using a test set. The evaluation metrics of accuracy and mean absolute error are calculated. If the classification accuracy does not reach 95%, the reasons are analyzed. Based on the prediction results of the test set, the ROC curve is plotted and the AUC value is calculated. Then, based on the analysis results of the ROC curve and classification accuracy, the model is further optimized. The training, validation and evaluation process is repeated until the model achieves a classification accuracy of 95% on the test set.

5. The thyroid cancer ultrasound image classification method based on multimodal feature fusion and Bayesian optimization according to claim 4, characterized in that: The formula for calculating the accuracy rate is as follows: ； In the formula, Accuracy represents the proportion of samples correctly predicted by the model out of the total number of samples. This indicates the number of samples that the model correctly predicts as positive when they are actually positive. This indicates the number of samples that the model correctly predicted as negative when they were actually negative. This represents the number of samples that the model incorrectly predicts as positive when they are actually negative. This indicates the number of samples that the model incorrectly predicts as negative when they are actually positive. The formula for calculating the mean absolute error is as follows: ； In the formula, Mean absolute error (MAE) represents the average magnitude of the difference between the predicted and actual values. and These represent the height and width of the sample, respectively. This represents the total number of sample pairs. The model predicts the first Line number The probability that a sample belongs to a certain category. Indicates the first Line number The true class probability of the column sample.

6. The thyroid cancer ultrasound image classification method based on multimodal feature fusion and Bayesian optimization according to claim 4, characterized in that: The calculation process of the AUC value: The optimized model is run on the test set to obtain the predicted probability that each sample belongs to the positive class, i.e., malignant thyroid nodules. Using the obtained predicted probabilities and the true labels of the test set, combined with the scikit-learn and matplotlib libraries in Python, the true positive rate and false positive rate at different thresholds are calculated by using the roc_curve function of scikit-learn, passing in the true labels of the test set and the positive class probability predicted by the model. The ROC curve is plotted with the false positive rate on the horizontal axis and the true positive rate on the vertical axis using the plotting function of matplotlib. The `roc_auc_score` function from the scikit-learn library is used to calculate the area under the ROC curve (AUC), which is between 0 and 1. AUC values ​​closer to 1 indicate better classification performance. An AUC of 0.5 indicates that the model's classification ability is equivalent to random guessing. Based on the calculated AUC value and the shape of the ROC curve, the model's performance is evaluated and analyzed. If the AUC value does not meet expectations, the model is further optimized.

7. A thyroid cancer ultrasound image classification system based on multimodal feature fusion and Bayesian optimization, used to implement the thyroid cancer ultrasound image classification method based on multimodal feature fusion and Bayesian optimization as described in any one of claims 1-6, comprising an ultrasound image classification platform, characterized in that, The ultrasound image classification platform has the following communication modules, wherein: The data preprocessing module is used to preprocess thyroid ultrasound images; The multimodal feature extraction module is used to extract improved local binary pattern features, Haralick texture features, and VGG16 depth features to construct a hybrid feature space. The feature selection and dimensionality reduction module is used to select features based on their importance using ExtraTrees and reduce the dimensionality to 12 dimensions using PCA. The lightweight neural network classification module is used to build a lightweight fully connected neural network to learn and classify the dimensionality-reduced feature vectors. The Bayesian optimization and hyperparameter tuning module is used to dynamically adjust the network hyperparameters using the Bayesian optimization algorithm, and combines cosine annealing and linear warm-up learning rate strategies to accelerate model convergence.

Images