A lightweight multi-feature fusion method for thyroid nodule classification

Abstract

A lightweight multi-feature fusion method for thyroid nodule classification is disclosed. This invention relates to the field of medical image processing technology, specifically a lightweight multi-feature fusion method for thyroid nodule classification. By improving the Hifuse model, this invention can fully mine global and local features of images, achieving multi-view and multi-directional comprehensive discrimination of benign and malignant thyroid nodules while reducing the number of parameters and computational complexity. The method includes the following steps: acquiring and preprocessing a dataset of ultrasound diagnostic report images of thyroid nodules; constructing and improving the Hifuse model; introducing a radiomics feature learning module into the Hifuse model, replacing the local attention module with an improved local attention module, and replacing the global attention module with an EfficientViM global extraction module; and using the trained improved Hifuse model to complete the thyroid nodule classification.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, specifically to a lightweight multi-feature fusion method for classifying thyroid nodules. Background Technology

[0002] Thyroid nodules are generally classified as benign or malignant. Malignant nodules are called thyroid cancer. In recent years, with changes in people's environment and lifestyles, factors such as uneven iodine intake in the diet, obesity, and high stress have led to a year-on-year increase in the incidence of thyroid cancer. Ultrasound, unlike CT or radionuclide scans, does not involve ionizing radiation and is highly efficient, safe, and suitable for all populations, making it the preferred method for screening and evaluating thyroid nodules. To standardize the ultrasound evaluation of nodules, the Thyroid Group of the Radiology Branch of the Chinese Medical Association published the "Chinese Thyroid Nodule Ultrasound Risk Grading System" (C-TIRADS) in 2020. This system uses a weighted scoring based on five features of ultrasound images to initially determine the benign or malignant nature of thyroid nodules. However, its evaluation process still relies on the subjective judgment of physicians, resulting in some experience variability, and the workload is significant when interpreting massive amounts of images. Deep learning-based medical image analysis methods have shown promising application prospects in the auxiliary diagnosis of benign and malignant thyroid nodules. However, existing deep learning models typically rely on large-scale training data, while the high cost and limited sample size of medical image data hinder their widespread application in clinical practice. Furthermore, malignant features are often not isolated; they frequently exhibit boundary expansion and spatial relationships with surrounding tissues. These are crucial indicators for assessing invasiveness and malignant potential. However, most methods focus only on the texture and edge information of the nodule region, making it difficult to model the morphological structure, spatial dependence, and combination features of nodules in the global image. This limits the improvement of classification model performance.

[0003] Medical image classification tasks often face challenges such as limited data scale and insufficient sample size, leading to model overfitting and affecting the final diagnostic accuracy. To overcome these challenges, existing research typically enhances the model's ability to represent subtle lesions and overall lesion regions by simultaneously extracting global and local features.

[0004] The HiFuse model, proposed in the paper "HiFuse: Hierarchical multi-scale feature fusion network for medical image classification" (Xiangzuo Huo) published in the journal Biomedical Signal Processing and Control, has demonstrated its ability to fuse multi-scale features in medical image classification tasks.

[0005] The paper "Efficient Vision Mamba with Hidden State Mixer based State Space Duality" (Sanghyeok Lee) presented at CVPR 2025 proposes the EfficientViM module, a lightweight visual backbone network that can efficiently model global features with fewer parameters and lower computational overhead.

[0006] However, while some existing hierarchical multi-scale feature fusion methods (such as the Hifuse model) can effectively extract global and local features and improve diagnostic accuracy, the Transformer structure they typically use has a large parameter scale and computational complexity, which makes it difficult to meet the requirements of lightweight and efficient models in real medical applications. Summary of the Invention

[0007] To address the limitations of existing medical classification tasks, such as limited data scale and insufficient sample size, which can easily lead to overfitting, as well as the large parameter scale and computational complexity of existing classification techniques, this invention proposes a lightweight multi-feature fusion method for thyroid nodule classification. By improving the Hifuse model, it can fully explore the global and local features of the image, achieving multi-view and multi-directional comprehensive discrimination of benign and malignant thyroid nodules while reducing the number of parameters and computational complexity.

[0008] The method includes the following steps:

[0009] S1. Obtain and preprocess the dataset of ultrasound diagnostic report images of thyroid nodules, and divide it into training set and test set in a 7:3 ratio;

[0010] S2. Construct and improve the Hifuse model to obtain the improved Hifuse model:

[0011] S21. Introduce a radiomics feature learning module into the Hifuse model;

[0012] S22. In the local feature extraction branch of the Hifuse model, replace the local attention module with the improved local attention module;

[0013] The improved local attention module is specifically: an improved dynamic adaptive convolution module is added before the depthwise separable convolution in the local attention module.

[0014] S23. In the global feature extraction branch of the Hifuse model, replace the global attention module with the EfficientViM global extraction module;

[0015] S3. Train the improved Hifuse model using the training set and the test set to obtain the trained improved Hifuse model;

[0016] S4. Use the trained Hifuse improved model to classify thyroid nodules.

[0017] Furthermore, the preprocessing specifically involves performing size normalization, grayscale standardization, and noise suppression sequentially on the thyroid nodule ultrasound diagnostic report image dataset.

[0018] Furthermore, the function of the radiomics feature learning module is to extract radiomics features and guide the deep feature attention distribution based on the radiomics features.

[0019] Furthermore, the radiomics feature learning module passes through the radiomics feature extraction module, the first fully connected layer, and the second fully connected layer sequentially from input to output;

[0020] The radiomics feature extraction module uses the Pyradiomics method to extract radiomics features;

[0021] The second fully connected layer uses the Sigmoid activation function.

[0022] Furthermore, the improved dynamic adaptive convolution module is specifically described as follows: in the dynamic adaptive convolution module, the module for calculating the average width and height is replaced by a module for calculating texture complexity;

[0023] The texture complexity calculation module uses the following formula: Calculating texture complexity ,in, This indicates the number of pixels in the image to be extracted. Indicates the first image to be extracted 1 pixel, Indicates the first The horizontal gradient magnitude of each pixel. Indicates the first The vertical gradient magnitude of each pixel.

[0024] Furthermore, the processor implements the steps of the above method when executing the computer program.

[0025] Furthermore, when the computer instructions are executed by the processor, they implement the steps of the above method.

[0026] The beneficial effects of the method described in this invention are as follows:

[0027] (1) This invention introduces a radiomics feature learning module into the Hifuse model and fuses the fusion features extracted based on deep learning methods with the radiomics features extracted by the radiomics feature learning module. This effectively integrates features at different information levels, takes into account the structure, texture and statistical features of the image, significantly improves the accuracy and robustness of classifying benign and malignant thyroid nodules, realizes multi-source feature fusion, and improves the model's discrimination ability.

[0028] (2) The improved dynamic adaptive convolution introduced in the local feature extraction branch of the Hifuse model in this invention can dynamically adjust the receptive field and sampling strategy of the convolution according to the texture complexity of the local region in the image, thereby improving the model's ability to recognize blurred boundaries, small lesions and complex structural regions, and enhancing the perception ability of different images.

[0029] (3) In the global feature extraction branch of the Hifuse model, this invention replaces the global attention module with the EfficientViM global extraction module, enabling the method described in this invention to effectively model long-distance spatial dependencies in images while maintaining the model's lightweight nature, overcoming the problems of high computational overhead and slow convergence of the traditional Transformer structure. The EfficientViM global extraction module optimizes the extraction method of global features by simplifying the computation process, while maintaining strong modeling capabilities, significantly improving computational efficiency and convergence speed.

[0030] (4) Integrating radiomics-guided attention mechanism to enhance model interpretability and medical credibility: This invention maps key radiomics features selected by the LASSO method into channel attention weights to guide the importance distribution of deep feature channels, realizing an attention modulation mechanism based on external prior information, improving the model's ability to focus on key areas, and enhancing the interpretability of classification decisions. Attached Figure Description

[0031] Figure 1 This is a flowchart of the method described in an embodiment of the present invention;

[0032] Figure 2 This is a schematic diagram of the improved Hifuse model structure in an embodiment of the present invention;

[0033] Figure 3 This is a schematic diagram of the local attention module structure of the Hifuse model in an embodiment of the present invention;

[0034] Figure 4 This is a schematic diagram of the improved local attention module structure of the Hifuse improved model in an embodiment of the present invention;

[0035] Figure 5This is a schematic diagram of the improved dynamic adaptive convolution module in an embodiment of the present invention;

[0036] Figure 6 This is a schematic diagram of the global attention module structure of the Hifuse model in an embodiment of the present invention;

[0037] Figure 7 This is a schematic diagram of the EfficientViM global extraction module structure of the Hifuse improved model in this embodiment of the invention;

[0038] Figure 8 This is a schematic diagram of the attention mapping and deep feature fusion mechanism guided by radiomics features in the implementation of this invention. Detailed Implementation

[0039] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0040] This embodiment provides a lightweight multi-feature fusion method for thyroid nodule classification, the flowchart of which is shown below. Figure 1 The method includes the following steps:

[0041] S1. Obtain and preprocess the dataset of ultrasound diagnostic report images of thyroid nodules, and divide it into training set and test set in a 7:3 ratio.

[0042] The thyroid nodule ultrasound diagnostic report image dataset used in this embodiment comes from ultrasound examination reports issued by hospitals. To protect patient privacy, images of thyroid nodules are extracted and labeled as benign or malignant.

[0043] Preprocessing operations were performed on the ultrasound diagnostic report image data of thyroid nodules. These operations included: cropping the region of interest to remove invalid background information, standardizing image size, grayscale normalization, noise suppression, and ultrasound image contrast enhancement. Finally, the preprocessed image data was divided into training and testing sets in a 7:3 ratio for model training and performance evaluation.

[0044] S2. Construct and improve the Hifuse model to obtain the improved Hifuse model (thyroid nodule classification model):

[0045] like Figure 2 As shown, in the Hifuse model, a radiomics feature learning module is introduced in parallel. The radiomics feature learning module processes the input data of the Hifuse model in parallel with the local feature extraction branch and the global feature extraction branch.

[0046] The radiomics feature learning module passes through the radiomics feature extraction module, the first fully connected layer, and the second fully connected layer sequentially from input to output.

[0047] The image omics feature extraction module uses the Pyradiomics method to perform quantitative feature engineering on the input data, extracting image omics features such as first-order statistical features, texture features, shape features, and filtering features.

[0048] The radiomics feature extraction module extracts one-dimensional data corresponding to each image for correlation analysis. L1 regularized LASSO regression is used to select features from the extracted radiomics features, and the optimal regularization intensity parameter is automatically determined through cross-validation. This effectively removes redundant information while retaining core features that significantly distinguish between benign and malignant nodules, thus achieving compression and optimization of high-dimensional image features.

[0049] The role of the radiomics feature learning module is to extract radiomics features and guide the distribution of deep feature attention based on the radiomics features.

[0050] S22. In the local feature extraction branch of the Hifuse model, replace the local attention module with the improved local attention module;

[0051] The local attention module of the Hifuse model, such as Figure 3 As shown.

[0052] like Figure 4 As shown, in the local attention module Before depthwise separable convolution, an improved dynamic adaptive convolution module is added to obtain an improved local attention module;

[0053] The improved local attention module adopts Depthwise separable convolution reduces computational overhead, and combined with improved dynamic adaptive convolution to handle images of varying complexity, the improved local attention module of this invention can fully combine texture complexity perception mechanism and attention enhancement strategy, enabling the model to dynamically adjust the convolution receptive field and sampling strategy according to the texture distribution of different regions in the image, thereby effectively enhancing the model's ability to perceive complex structures, blurred boundaries and lesions of different sizes in medical images.

[0054] The improved dynamic adaptive convolution module is specifically described as follows: In the dynamic adaptive convolution module, the module for calculating the average width and height is replaced with a module for calculating texture complexity;

[0055] The texture complexity calculation module is based on gradient magnitude, using the formula: Calculating texture complexity ,in, This indicates the number of pixels in the image to be extracted. Indicates the first image to be extracted 1 pixel, Indicates the first The horizontal gradient magnitude of each pixel. Indicates the first The vertical gradient magnitude of each pixel.

[0056] The improved dynamic adaptive convolution module also includes a receptive field size learning module and a convolution kernel size dynamic adjustment module.

[0057] Receptive Field Size Learning Module: This module predicts the height (h) and width (w) of the convolutional kernel at each spatial location on the input feature map. This process is dynamically adjusted based on the features of the local image, dynamically selecting different convolutional kernel sizes.

[0058] Kernel Size Dynamic Adjustment Module: The kernel size dynamic adjustment module (Kernel SamplingController) calculates the sampling kernel size at each position based on the predicted height (h) and width (w), and adjusts the number of sampling points in combination with the image texture complexity (such as gradient magnitude). If the texture complexity is large, the sampling density is increased (the stride is reduced). If the texture complexity is small, the sampling density is appropriately reduced.

[0059] like Figure 5 As shown, the receptive field size learning module predicts the height (h) and width (w) of the convolution kernel based on the input feature map X (B represents the batch number, C represents the number of channels, H represents the height, and W represents the width). The convolution kernel size dynamic adjustment module combines the predicted convolution height (h) and width (w) with the texture complexity calculated by the texture complexity calculation module to dynamically determine the texture complexity and adjust the number of sampling points to select an appropriate convolution operation.

[0060] S23. In the global feature extraction branch of the Hifuse model, replace the global attention module with the EfficientViM global extraction module (efficient visual mamba module).

[0061] The global attention module of the Hifuse model is as follows: Figure 6 As shown, the Global FeatureBlock is a key module in the HiFuse model used to extract global semantic information from images. This module adopts the window multi-head self-attention mechanism (W-MSA) introduced by SwinTransformer, which divides the feature map into 7×7 non-overlapping local windows and uses 3 attention heads to compute in parallel within each window.

[0062] EfficientViM global extraction module, such as Figure 7 As shown, the HSM-SSD (Hidden State Mixer-based SSD) module represents the state space dual (SSD) module based on the hidden state mixer.

[0063] S3. Train the improved Hifuse model using the training set and the test set to obtain the trained improved Hifuse model;

[0064] S4. Use the trained Hifuse improved model to classify thyroid nodules.

[0065] like Figure 2 As shown, the input image simultaneously passes through a local feature extraction branch, a global feature extraction branch, and an image omics feature learning module. The local feature extraction branch and the global feature extraction branch extract local and global features respectively, and they run independently without interfering with each other. Between the local feature extraction branch and the global feature extraction branch, a feature fusion module is used to fuse the local and global features to obtain the fused deep learning features.

[0066] The radiomics feature learning module extracts radiomics features and guides the distribution of deep feature attention based on these features.

[0067] like Figure 2 and 8 As shown, in the radiomics feature learning module, the radiomics feature learning module extracts radiomics features (one-dimensional radiomics information); the first fully connected layer compresses redundant information; the second fully connected layer uses the Sigmoid activation function to map the compressed information dimension to the number of channels (channel attention weights), thus obtaining the output of the radiomics feature learning module.

[0068] The output of the radiomics feature learning module is used as "conditional information" to guide the deep feature attention distribution of the fused deep learning features, and finally completes the classification. Specifically, the output of the radiomics feature learning module and the fused deep learning features are weighted and fused, and then input into the classifier module for classification to obtain the output of the Hifuse improved model (classification result: benign / malignant).

Claims

1. A lightweight multi-feature fusion thyroid nodule classification method, characterized in that, The method comprises the following steps: S1, acquiring and preprocessing the thyroid nodule ultrasound diagnosis report image dataset, and dividing it into a training set and a test set in a 7:3 ratio; S2, constructing and improving the Hifuse model to obtain an improved Hifuse model: S21, introducing an imageomics feature learning module into the Hifuse model, which processes the input data of the Hifuse model in parallel with the local feature extraction branch and the global feature extraction branch; The imageomics feature learning module sequentially passes through an imageomics feature extraction module, a first full connection layer and a second full connection layer from input to output; The imageomics feature learning module extracts imageomics features; the first full connection layer compresses redundant information; and the second full connection layer uses a Sigmoid activation function to map the dimension of the compressed information to the number of channels to obtain the output of the imageomics feature learning module; The imageomics feature extraction module uses the Pyradiomics method to extract imageomics features, specifically: the imageomics feature extraction module uses the Pyradiomics method to perform quantitative feature engineering processing on the input data of the Hifuse model to extract first-order statistical features, texture features, shape features, and filter features, etc. The second full connection layer uses a Sigmoid activation function; The imageomics feature extraction module performs correlation analysis on the one-dimensional data corresponding to each image in the input data of the Hifuse model, uses LASSO regression based on L1 regularization to select features from the extracted imageomics features, automatically determines the optimal regularization strength parameter through cross-validation, and removes redundant information; The imageomics feature learning module extracts imageomics features and guides the attention distribution of deep features according to the imageomics features; S22, in the local feature extraction branch of the Hifuse model, replace the local attention module with an improved local attention module; The improved local attention module specifically adds an improved dynamic adaptive convolution module before the depth separable convolution in the local attention module. The improved dynamic adaptive convolution module specifically replaces the average value calculation module with a texture complexity calculation module in the dynamic adaptive convolution module. The texture complexity calculation module uses the following formula: Calculating texture complexity ,in, This indicates the number of pixels in the image to be extracted. Indicates the first image to be extracted 1 pixel, Indicates the first The horizontal gradient magnitude of each pixel. Indicates the first The vertical gradient magnitude of each pixel; S23, in the global feature extraction branch of the Hifuse model, replace the global attention module with an EfficientViM global extraction module; S3, training the improved Hifuse model using the training set and the test set to obtain a trained improved Hifuse model; S4, using the trained improved Hifuse model to complete thyroid nodule classification.

2. The method according to claim 1, wherein, The preprocessing specifically includes size normalization, grayscale standardization and noise suppression on the thyroid nodule ultrasound diagnosis report image dataset. 3.An electronic device comprising a memory and a processor, the memory storing a computer program, wherein, The processor executes the computer program to implement the steps of the method of any one of claims 1-2.

4. A computer readable storage medium storing computer instructions, characterized in that, The computer instructions are executed by the processor to implement the steps of the method of any one of claims 1-2.

Images