Thyroid nodule detection and evaluation method and system based on deep learning

Abstract

The application discloses a thyroid nodule detection and evaluation method and system based on deep learning, relates to the technical field of medical image processing and deep learning, and comprises the following steps: constructing a training data set, constructing a nodule detection model and training, inputting a preprocessed ultrasonic image sequence into the nodule detection model frame by frame, obtaining the detection position and confidence of a thyroid nodule in each frame image, processing the detected nodule region to generate an attention heat map of the nodule region, performing time sequence fusion and consistency optimization on the same nodule region, and outputting the final classification probability and ACR TI-RADS grading result of the nodule after time sequence fusion as the final nodule detection and evaluation result. The application has real-time performance in the detection and evaluation of thyroid nodules, realizes automatic multi-feature risk evaluation, enhances the explainability of the classification result, eliminates frame-to-frame detection jump, can assist doctors in improving the diagnosis efficiency, and reduces the risk of misdiagnosis and missed diagnosis.

Description

Technical Field

[0001] This invention relates to the fields of medical image processing and deep learning technology, and in particular to a method and system for detecting and evaluating thyroid nodules based on deep learning. Background Technology

[0002] Thyroid nodules are a common clinical problem, and accurate assessment of their benign or malignant nature is crucial for the early diagnosis and treatment of thyroid cancer. Currently, ultrasound imaging, due to its advantages of being non-invasive, real-time, radiation-free, and low-cost, is widely used in the diagnosis of thyroid nodules and is the preferred imaging method for clinical examination. In actual clinical practice, physicians typically use an ultrasound probe to dynamically scan the thyroid region, acquiring continuous scan videos and relying on experience to determine the location, morphology, and benign or malignant characteristics of the nodules in real time during the scan. However, due to speckle noise, low contrast, and the complex characteristics of nodules in ultrasound images, physicians face a certain degree of subjectivity and risk of misdiagnosis. Accurately determining the benign or malignant nature of nodules is particularly challenging for inexperienced physicians or medical institutions with limited resources.

[0003] To standardize the ultrasound diagnostic criteria for thyroid nodules, the American College of Radiology (ACR) proposed the Thyroid Imaging Reporting and Data System (ACR TI-RADS). This method quantifies and grades the risk of nodules based on five categories of ultrasound features. Each feature is assigned a different score according to its malignancy risk, and the scores of the five categories are summed to obtain a total score. The total score is used to determine the TI-RADS classification (TR1~TR5) to guide clinical decision-making. However, in actual clinical practice, physicians need to manually analyze and score a large number of nodule features one by one. This is not only time-consuming and labor-intensive, but may also lead to inconsistent diagnostic results due to the subjectivity of feature interpretation.

[0004] With the development of artificial intelligence technology, deep learning is increasingly widely used in the field of medical image-assisted diagnosis, and computer-aided diagnosis technology for thyroid nodules has gradually matured. Yu Z et al. (Yu Z, Liu S, Liu P, et al. Automatic detection and diagnosis of thyroid ultrasound images based on attention mechanism[J]. Comput Biol Med, 2023, 155:106468.) proposed an automatic detection and diagnosis method for thyroid ultrasound images based on an attention mechanism. They used an improved CSPDarkNet as the backbone network to extract features, embedded a coordinate attention (CA) module in the feature aggregation network, and combined spatial pyramid pooling and path aggregation networks for multi-scale feature fusion. This enabled automatic localization and benign / malignant classification of thyroid nodules. A detection speed of 0.89 mAP and 7ms per image was achieved on 1921 ultrasound images, validating the effectiveness of the lightweight attention mechanism in real-time detection and classification of thyroid ultrasound images. Wang X et al. (Wang X, Niu Y, Liu H, et al. ThyroNet-X4Genesis: an advanced deep learning model for auxiliary diagnosis of thyroid nodules' malignancy[J]. Sci Rep, 2025, 15(1): 4214.) proposed the ThyroNet-X4 Genesis model based on an improvement of ResNet. By introducing expansion coefficients, 5×5 convolution kernels, and grouped convolutions to optimize feature extraction capabilities, and combining GradCAM heatmaps to provide interpretability analysis of classification results, the model achieved classification accuracies of 71.70% and 67.02% on the internal validation set and the single-center external validation set, respectively, validating the application potential of deep learning in the classification of benign and malignant thyroid nodules.Zou W et al. (Zou W, Zhou Y, Yao J, et al. Age-stratified deeplearning model for thyroid tumor classification: a multicenter diagnosticstudy[J]. Eur Radiol, 2025, 35:4599–4609.) proposed an age-stratified multimodal classification network ASMCNet based on DenseNet121. This network incorporates patient age as a stratification factor into the model training and combines it with GradCAM heatmaps to assist physicians in understanding the model's decision-making basis. On three-center datasets, it achieved a classification performance of AUC 0.906. Furthermore, physician studies demonstrated that heatmap visualization significantly improves the diagnostic accuracy of radiologists with different experience levels, proving the clinical value of deep learning-assisted diagnosis and interpretability analysis in thyroid nodule classification.

[0005] However, current research in the detection and evaluation of thyroid nodules has the following main shortcomings: existing methods analyze single static ultrasound images, ignoring the temporal continuity information of dynamic scanning videos in clinical practice; the inherent speckle noise and low contrast of ultrasound images have not been effectively addressed, and there is a lack of adaptive enhancement mechanisms for the importance of feature channels; existing classification methods are mostly binary classifications of benign and malignant nodules, lacking automated methods that combine ACR TI-RADS criteria to score nodules feature by feature; furthermore, the classification results of existing models are not interpretable enough, making it difficult for doctors to fully understand the basis for decision-making.

[0006] To address the aforementioned issues, this invention proposes a deep learning-based method and system for thyroid nodule detection and evaluation. The system is systematically designed from five aspects: image preprocessing, lightweight attention-enhanced detection, ACR TI-RADS standardized classification scoring, interpretable heatmap generation, and video temporal fusion. The aim is to achieve accurate detection, standardized scoring, and stable evaluation of nodules in thyroid ultrasound scan videos. Summary of the Invention

[0007] The purpose of this invention is to provide a method and system for detecting and evaluating thyroid nodules based on deep learning, to solve the problems in the prior art, and to achieve accurate detection, standardized scoring and stable evaluation of thyroid nodules in ultrasound scan videos.

[0008] The technical solution adopted by this invention to solve its technical problem is as follows:

[0009] A deep learning-based method for the detection and evaluation of thyroid nodules includes the following steps:

[0010] S101. Construct a training dataset, obtain thyroid ultrasound scan video data, extract frames from the thyroid ultrasound scan video to obtain ultrasound image sequences, perform CLAHE preprocessing on the ultrasound image sequences, and label the nodule regions of the ultrasound image sequences to construct a training dataset.

[0011] S102. Construct a nodule detection model. The nodule detection model is based on the YOLOv11 target detection framework. An efficient channel attention mechanism (ECA) is embedded in the C3k2 module of the neck network to form a C3k2-ECA module. The model is trained using the training dataset to obtain a trained nodule detection model.

[0012] S103. Input the preprocessed ultrasound image sequence from step S101 frame by frame into the nodule detection model in step S102 to obtain the detection location and confidence level of thyroid nodules in each frame.

[0013] S104. The nodule regions detected in each frame are cropped and input into the nodule classification network for feature extraction. The five ultrasound features of the nodule—composition, echo, morphology, edge, and hyperechoic foci—are classified and scored using the ACR TI-RADS scoring system to obtain the classification evaluation results of the nodule region in each frame. At the same time, based on the feature map of the MobileViT classification network, the GradCAM gradient weighted class activation mapping method is used to generate an attention heatmap of the nodule region to help doctors understand the classification basis.

[0014] S105. Track and match the detection results of the same nodule in consecutive frames of the ultrasound image sequence. Construct a temporal feature sequence from the classification feature vectors belonging to the same nodule. Input the temporal feature sequence into a bidirectional long short-term memory network (BiLSTM) for temporal fusion and consistency optimization. Output the final classification probability and ACR TI-RADS classification result of the nodule after temporal fusion as the final nodule detection and evaluation result.

[0015] Preferably, the CLAHE preprocessing converts the original ultrasound image to the LAB color space, and modifies the luminance channel. Applying the CLAHE algorithm to enhance contrast, contrast limiting parameters. The range of values ​​is Grid size parameter The range of values ​​is Preferred =2.0, =8.

[0016] Preferably, the ECA attention mechanism is embedded at the output of the C3k2 module, and cross-channel interaction is achieved through one-dimensional convolution with adaptive kernel size. No dimensionality reduction operation is required, and the kernel size is adaptively determined according to the number of channels, thereby enhancing the feature discrimination ability of the neck network with minimal parameter quantity and computational overhead.

[0017] Preferably, the MobileViT classification network outputs prediction results from five parallel classification heads, which correspond to five ultrasound features in the ACRTI-RADS scoring system: composition, echo, morphology, margin, and hyperechoic lesions. The prediction and grading results of the five ultrasound features are assigned corresponding scores and summed to obtain a total score. The TI-RADS grading of the nodule is determined based on the total score.

[0018] Preferably, the GradCAM heatmap uses the ACR TI-RADS total score as the backpropagation target, calculates the gradient weight of the total score on each channel of the MobileViT feature map, generates a weighted heatmap, and visualizes the region of interest of the model for key nodule discrimination features.

[0019] Preferably, the BiLSTM performs temporal modeling on the temporal feature sequence along both the forward and reverse directions, and after concatenating the hidden states in the forward and reverse directions, it outputs the final classification probability after temporal fusion through a fully connected layer and a Softmax function.

[0020] This invention also provides a deep learning-based thyroid nodule detection and evaluation system, comprising: a video preprocessing module, which acquires thyroid ultrasound scan video data, extracts frames from the video to obtain an ultrasound image sequence, performs CLAHE adaptive histogram equalization preprocessing on the ultrasound image sequence, and labels the nodule regions to construct a training dataset;

[0021] The nodule detection module performs frame-by-frame nodule detection on the preprocessed ultrasound image sequence based on the YOLOv11 model. The YOLOv11 model embeds the ECA efficient channel attention mechanism in the C3k2 module of the neck network to form the C3k2-ECA module, which outputs the detection location and confidence of thyroid nodules in each frame of the image.

[0022] The nodule classification module uses the MobileViT lightweight visual Transformer model to extract features from the detected nodule regions, combines the ACR TI-RADS scoring system to assess the benign or malignant risk of nodules, and uses the GradCAM gradient weighted class activation mapping method to generate attention heatmaps of nodule regions.

[0023] The temporal fusion module, based on the temporal information between video frames, uses a bidirectional long short-term memory network (BiLSTM) to perform temporal modeling and fusion of the classification features of the same nodule in consecutive frames, and outputs the final nodule detection and evaluation results after temporal consistency optimization.

[0024] The present invention has the following beneficial effects:

[0025] (1) This invention enhances the local contrast of ultrasound images in the luminance channel of the LAB color space through CLAHE preprocessing, making the nodule boundary and internal echo features clearer, while preserving color information without distortion, effectively improving the inherent low contrast and speckle noise problems of ultrasound images, and providing higher quality input images for subsequent deep learning models.

[0026] (2) The present invention embeds an efficient channel attention mechanism (ECA) at the output of the C3k2 module of the YOLOv11 neck network. It achieves lightweight cross-channel interaction through one-dimensional convolution with adaptive kernel size. It can adaptively enhance the feature channels that make key contributions to nodule detection without dimensionality reduction operation, suppress irrelevant channel information, improve the feature discrimination ability of the model with minimal parameter quantity and computational overhead, and balance detection accuracy and real-time performance.

[0027] (3) This invention uses the MobileViT lightweight visual Transformer model, combined with the ACR TI-RADS scoring system, to classify and score the five types of ultrasound features of nodules, namely composition, echo, morphology, edge and strong echo foci, and to classify the risk level. It realizes standardized automated multi-feature scoring, which makes up for the shortcomings of existing methods that only perform benign and malignant classification. The diagnostic results can be directly connected to the clinical ACR TI-RADS reporting system.

[0028] (4) Based on the MobileViT feature map and ACR TI-RADS total score, this invention uses GradCAM to generate an attention heat map, and uses the score results as the backpropagation target to visualize the specific discrimination area and discrimination basis of the model for various ultrasound features of nodules. It provides feature-by-feature interpretability analysis based on a standardized scoring system, which enhances clinicians' understanding and trust in the model output results.

[0029] (5) This invention is aimed at the video scene of thyroid ultrasound scan. It uses BiLSTM to perform bidirectional temporal modeling and fusion of the classification features of the same nodule in consecutive frames. It makes full use of the temporal continuity information between video frames, eliminates the frame jump of frame-by-frame detection and classification results, improves the stability of nodule localization and classification, and improves the accuracy of judgment by using multi-angle scan information. Attached Figure Description

[0030] Figure 1This is a flowchart illustrating the overall method of an embodiment of the present invention;

[0031] Figure 2 This is a schematic diagram of the overall model architecture of an embodiment of the present invention;

[0032] Figure 3 This is a schematic diagram comparing the CLAHE preprocessing before and after in an embodiment of the present invention;

[0033] Figure 4 This is a schematic diagram of the nodule detection model structure according to an embodiment of the present invention;

[0034] Figure 5 This is a schematic diagram of the GradCAM heat map according to an embodiment of the present invention;

[0035] Figure 6 The ACR TI-RADS five-category ultrasound feature scoring criteria and TI-RADS classification comparison table;

[0036] Figure 7 This is a schematic diagram of the BiLSTM timing fusion architecture according to an embodiment of the present invention;

[0037] Figure 8 This is a schematic diagram of the final output result of an embodiment of the present invention.

[0038] Figure 9 This is a system structure diagram of an embodiment of the present invention. Detailed Implementation

[0039] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments, but the present invention is not limited to the embodiments.

[0040] The deep learning-based method for thyroid nodule detection and evaluation according to embodiments of the present invention, such as... Figure 1 As shown, it includes the following steps:

[0041] S101, Construct a training dataset, acquire thyroid ultrasound scan video data, extract frames from thyroid ultrasound scan videos to obtain ultrasound image sequences, perform CLAHE (Adaptive Histogram Equalization) preprocessing on ultrasound image frames, and label nodule regions to construct a training dataset.

[0042] S102, Construct a nodule detection model. Construct a nodule detection model based on the YOLOv11 target detection framework. Embed an efficient channel attention mechanism (ECA) into the C3k2 module of the neck network to form a C3k2-ECA module. Train the model using the training dataset to obtain a trained nodule detection model.

[0043] S103, input the preprocessed ultrasound image sequence frame by frame into the nodule detection model in step S102 to obtain the detection location and confidence level of thyroid nodules in each frame of the image;

[0044] S104: The nodule regions detected in each frame are cropped and input into the MobileViT classification network for feature extraction. The ACR TI-RADS scoring system is used to classify and score the five types of ultrasound features of the nodules: composition, echo, morphology, edge, and strong echo foci, to obtain the classification evaluation results of the nodule regions in each frame. At the same time, based on the feature map of the MobileViT classification network, the GradCAM gradient weighted class activation mapping method is used to generate the attention heatmap of the nodule regions.

[0045] S105 tracks and matches the detection results of the same nodule in consecutive frames of the ultrasound image sequence, constructs a temporal feature sequence from the classification feature vectors belonging to the same nodule, inputs the temporal feature sequence into the bidirectional long short-term memory network BiLSTM for temporal fusion and consistency optimization, and outputs the final classification probability and ACR TI-RADS classification result of the nodule after temporal fusion as the final nodule detection and evaluation result.

[0046] Specifically, such as Figure 2 As shown, the system of this embodiment includes four modules: a video preprocessing module, a nodule detection module, a nodule classification module, and a temporal fusion module. The data flow between the modules is as follows: The video preprocessing module receives the input thyroid ultrasound scan video, completes frame extraction and CLAHE preprocessing, and outputs the preprocessed ultrasound image sequence to the nodule detection module; the nodule detection module detects nodules frame by frame based on the YOLOv11 model, and outputs the detected nodule location and confidence level to the nodule classification module; the nodule classification module uses MobileViT to perform ACR TI-RADS scoring on the cropped nodule region and generates a GradCAM heatmap, outputting the frame-by-frame classification feature vector to the temporal fusion module; the temporal fusion module performs BiLSTM temporal modeling on the classification features of the same nodule in consecutive frames, and outputs the final nodule detection and evaluation results. The specific implementation methods of each step are described in detail below.

[0047] The specific process of performing CLAHE preprocessing on ultrasound image frames in step S101 is as follows: First, the thyroid ultrasound scan video is extracted at equal intervals. The extraction frequency is set according to the video frame rate and scan speed. In this embodiment, the extraction frequency is 3 frames per second. Invalid frames containing probe movement transitions, blurred images, etc., are removed, and valid frames containing clear thyroid tissue areas are retained to form an ultrasound image sequence. ,in This represents the total number of frames extracted.

[0048] Then, single-frame ultrasound images from the ultrasound image sequence are... Convert from BGR color space to LAB color space to separate the luminance channel. and color channels , For the brightness channel Applying the CLAHE algorithm, the luminance channel is divided into... Each grid region is independently histogram-equalized, with contrast-limited parameters. Limiting the contrast enhancement of the histogram is expressed as:

[0049]

[0050] in, The range of values ​​is , The range of values ​​is In this embodiment, it is preferred that... , . It effectively suppresses the amplification of speckle noise while enhancing the contrast of the nodule region; Make each grid cover pixels (input image size is) (At that time), the particle size was moderate, the statistics were sufficient, and it matched the typical size range of thyroid nodules.

[0051] Finally, the enhanced brightness channel... With the original color channel , Merge the images, convert them back to the BGR color space, and obtain the preprocessed ultrasound images. For a detailed comparison of the before and after CLAHE preprocessing, please refer to the diagram below. Figure 3 As shown, Figure 3 This demonstrates the enhancement effect of CLAHE processing on the nodule boundary and internal echo characteristics of ultrasound images.

[0052] Regarding the construction of the nodule detection model in step S102, as follows: Figure 4 As shown, this invention is based on the YOLOv11 object detection model. YOLOv11 adopts a single-stage detection architecture, which includes three parts: a backbone network, a neck network, and a detection head. The backbone network is responsible for extracting multi-scale intermediate layer features from the input image. The neck network performs multi-scale intermediate layer feature fusion through a Feature Pyramid Network (FPN) and a Path Aggregation Network (PAN). The detection head outputs bounding box coordinates, object confidence, and class probability on feature maps at three different scales.

[0053] This invention embeds an efficient channel attention mechanism (ECA) into the C3k2 module of the YOLOv11 neck network, forming a C3k2-ECA module. The C3k2 module is the core feature fusion component of the neck network, achieving efficient multi-scale intermediate layer feature aggregation through two gradient flow paths and the Bottleneck structure. This invention embeds the ECA attention mechanism at the output of each C3k2 module, rather than at the input or intermediate layer. Its technical advantages are:

[0054] The C3k2 module first completes multi-scale intermediate layer feature fusion, fully aggregating information from different receptive fields. ECA then performs channel importance weighting on the fused features, enabling adaptive evaluation of the contribution of each channel after fusion based on global information, thereby avoiding information loss caused by channel weighting before feature fusion.

[0055] The calculation process of the C3k2-ECA module is as follows: The input multi-scale intermediate layer features are first fused by the C3k2 module to obtain the output feature map. , dimension ; An ECA attention mechanism is embedded at the output of the C3k2 module to... Perform channel importance weighting.

[0056] right Adaptive global average pooling is performed to compress the two-dimensional spatial features of each channel into a single scalar, resulting in a channel-level description vector. Channel-level description vector Represented as:

[0057]

[0058] in, This indicates that the spatial dimensions are adaptively compressed to... , Dimensions .

[0059] right Perform a dimensionality transformation, removing the spatial dimension and swapping the channel and sequence dimensions to adapt it to the input format of a one-dimensional convolution, represented as:

[0060]

[0061] in, This indicates the removal of spatial dimensions. This indicates that the channel dimension is transposed to the sequence dimension, after which... Dimensions That is, a one-dimensional channel sequence.

[0062] Determine the kernel size of a one-dimensional convolution. According to the number of channels Adaptive computation, expressed as:

[0063]

[0064] in, This indicates taking the nearest odd number.

[0065] Will With kernel size Fill as One-dimensional convolutions are used to aggregate cross-channel information, and channel attention weights are generated after dimension reduction and Sigmoid activation. Channel attention weight Represented as:

[0066]

[0067] in, This indicates that the number of input channels is 1, the number of output channels is 1, and the core size is [value missing]. Fill as One-dimensional convolution operation without bias terms; This indicates that the output dimension will be changed from... Restore to ; This represents the Sigmoid activation function; Dimensions The range of values ​​for each element is: .

[0068] Channel attention weights Broadcast extended to Same dimensions Attention-enhancing features are obtained through element-wise multiplication. Attention enhancement features Represented as:

[0069]

[0070] in, This represents element-wise multiplication based on a broadcast mechanism, i.e. The weight scalar of each channel and The entire corresponding channel Feature map multiplication enhances key feature channels and suppresses irrelevant channels. This is the final output of the C3k2-ECA module.

[0071] The ECA module has only one-dimensional convolution kernel parameters. With a small number of weight parameters and minimal computational complexity, the feature discrimination capability of the neck network is enhanced, meeting the real-time requirements of ultrasound scan video processing.

[0072] The model was trained using the AdamW optimizer with an initial learning rate of 0.001, a cosine annealing learning rate scheduling strategy, 300 training epochs, a batch size of 16, 5 warm-up epochs, a label smoothing coefficient of 0.02, and an early stopping patience value of 50 epochs.

[0073] Regarding step S103, the ultrasound image sequence preprocessed by CLAHE...

[0074]

[0075] The trained nodule detection model is input frame by frame, and the model outputs a set of detection results for each frame. ,in For the first The center coordinates and width and height of each detection box To obtain the confidence score, redundant boxes are removed after non-maximum suppression post-processing to obtain the final detection result for each frame. The location and confidence score of the detected nodules in each frame are recorded.

[0076] Regarding step S104, the detected nodule regions are classified, scored, and analyzed for interpretability. First, the nodule region image is cropped from the detection results based on the bounding box coordinates. The cropped nodule region image is then scaled to a uniform size. The pixels are then input into the MobileViT classification network. MobileViT is a lightweight visual Transformer model that combines the local feature extraction capability of convolution with the global context modeling capability of Transformer. It is suitable for controlling model complexity while maintaining high classification accuracy, meeting the real-time processing requirements of video scenarios.

[0077] The core building block of MobileViT is the MobileViT Block. Each Block consists of two concatenated parts: convolutional local feature extraction and Transformer global feature modeling, represented as follows:

[0078]

[0079]

[0080]

[0081]

[0082] in, This is a cropped and scaled image of the nodule region. This represents pointwise convolution. This represents depthwise separable convolution. This means expanding local features into a sequence to fit the Transformer input. This indicates a multi-head self-attention encoder. This indicates concatenation along the channel dimension. This refers to the output characteristics of MobileViTBlock.

[0083] The MobileViT classification network performs global average pooling on the features output from the final layer, then connects to five parallel fully connected classification heads to predict the classification results of the five ultrasound features in ACR TI-RADS:

[0084]

[0085] in, For the prediction and classification of component characteristics, For the prediction and grading of echo features, For the prediction and grading of morphological features, Hierarchical prediction of edge features Predictive grading of strong echogenic foci features. The output features of the last layer of MobileViT This represents five parallel fully connected classification heads, each of which outputs the classification probability of the corresponding feature class via the Softmax function.

[0086] According to the ACR TI-RADS scoring system, the predicted grading results of the five features are assigned corresponding scores and summed to obtain the total score:

[0087]

[0088] in, For the overall score, , , , , These are the ACR TI-RADS score mapping functions corresponding to the predicted grades for each feature category.

[0089] The classification and corresponding scores for each feature category, as well as the ACR TI-RADS classification corresponding to the total score, are as follows: Figure 6 As shown, based on the total score The risk level of nodules is determined, enabling automated assessment from ultrasound image features to quantitative scoring.

[0090] Feature map based on the last convolutional layer of the MobileViT classification network and overall score The GradCAM gradient-weighted class activation mapping method is used to generate attention heatmaps of nodule regions, such as... Figure 5 As shown, Figure 5 The GradCAM heatmap shows the overlay effect of the attention heatmap generated by GradCAM on the nodule region and the original ultrasound image, intuitively presenting the model's attention to the key discrimination region of the nodule.

[0091] This invention uses the ACR TI-RADS total score Instead of a single binary classification probability, the heatmap is used as the backpropagation target, allowing it to reflect the model's attention to the comprehensive discrimination region of various ultrasonic features. The specific process is as follows:

[0092] Calculate the total score For feature maps The gradient of each channel is used to obtain the importance weights of each channel through global average pooling:

[0093]

[0094]

[0095] in, For the first Each channel contributes to the overall score. Importance weights and The spatial dimensions of the feature map. For the total score, the feature map of the first... Channel coordinates The gradient at that point.

[0096] The GradCAM heatmap is obtained by weighting the importance weights of each channel with the corresponding feature maps and then applying ReLU activation.

[0097]

[0098] in, The number of channels in the feature map. Dimensions ReLU activates the filtering of negative value regions, retaining regions that contribute positively to the score.

[0099] Will Bilinear interpolation upsampling to the original nodule region image size, normalized to The interval is then superimposed onto the original nodule region image to generate a visual heatmap. The red and yellow areas in the heatmap represent the areas that the model focuses on classifying, helping doctors understand the classification criteria of the model.

[0100] The training loss of the classification model is the sum of the cross-entropy losses of the five classifiers. The AdamW optimizer is used with an initial learning rate of 0.001, cosine annealing decay, and 200 training epochs with a batch size of 16.

[0101] Regarding step S105, as follows Figure 7 As shown, node tracking and matching are performed on the detection results of each frame in the video sequence, and detection results belonging to the same node in consecutive frames are grouped into the same node sequence. The matching strategy adopts a dual criterion based on the intersection-union ratio (IU) and the center point distance. When the IU of the detection boxes between two frames is greater than 0.3 and the normalized center point distance is less than 0.5, the current detection box is assigned to an existing trajectory; otherwise, it is regarded as the starting frame of a new node. The Hungarian algorithm is used for global optimal allocation during matching.

[0102] For each frame belonging to the same nodule after tracking and matching, the classification feature vector of that nodule region before the fully connected layer of the MobileViT classification network is extracted to form the temporal feature sequence of that nodule.

[0103]

[0104] in For the first The classification feature vector of the nodule in the frame. For feature dimension, This represents the total number of frames in the nodule sequence.

[0105] The temporal feature sequence is input into a bidirectional long short-term memory network (BiLSTM), and temporal modeling is performed on the sequence along both the forward and reverse directions:

[0106]

[0107]

[0108]

[0109] in, and These are the hidden state and memory unit of a forward LSTM, respectively. and These are the hidden state and memory unit of the inverse LSTM, respectively. This represents vector concatenation. For the first The two-way fusion feature of the step.

[0110] Forward LSTM captures the temporal dependencies from the start of the scan to the current time step, while backward LSTM captures the inverse temporal dependencies from the end of the scan to the current time step. The fusion of bidirectional information enables the model to make comprehensive use of the contextual information of previous and next frames, thereby improving the accuracy of judgment in single-frame blurred or occluded scenes.

[0111] The temporal feature vector output by BiLSTM is then subjected to global average pooling and input into a fully connected layer and a softmax function, outputting the final classification probability after temporal fusion:

[0112]

[0113] in, and Here, T represents the weights and biases of the fully connected layer, and T represents the total number of frames in the nodal sequence.

[0114] Based on the final classification probability after temporal fusion, the final five-class feature scores, total score, and TI-RADS rating are calculated using the ACR TI-RADS scoring system, serving as the final evaluation result for this node. Compared to the results of frame-by-frame independent classification, the results after temporal fusion eliminate inter-frame jumps, exhibiting higher stability and accuracy.

[0115] In the final output results of the embodiments of the present invention, such as Figure 8 As shown, each detected thyroid nodule is labeled with a bounding box, and the results include item-by-item scores for the five ACR TI-RADS features, the total score, the TI-RADS grading determination, and GradCAM heatmap visualization. The final assessment report includes the nodule's location, morphological description, detailed scores for various ultrasound features, risk level, and recommended clinical management, assisting physicians in rapid screening and diagnostic decision-making.

[0116] This is a system structure diagram of an embodiment of the present invention, such as... Figure 9 As shown, it includes:

[0117] The video preprocessing module 901 acquires thyroid ultrasound scan video data, extracts frames from the video to obtain ultrasound image sequences, performs CLAHE adaptive histogram equalization preprocessing on the ultrasound image frames, and labels the nodule regions to construct a training dataset.

[0118] The nodule detection module 902 performs frame-by-frame nodule detection on the preprocessed ultrasound image sequence based on the improved YOLOv11 model, and outputs the detection location and confidence level of thyroid nodules in each frame.

[0119] The nodule classification module 903 uses the MobileViT classification model to extract features from the detected nodule regions. It combines the ACR TI-RADS scoring system to classify and score five types of ultrasound features of nodules: composition, echo, morphology, edge, and strong echo foci. It also uses the GradCAM gradient weighted class activation mapping method to generate an attention heatmap of the nodule regions.

[0120] The temporal fusion module 904 tracks and matches the detection results of the same nodule in consecutive frames based on the temporal information between video frames. It uses a bidirectional long short-term memory network (BiLSTM) to perform temporal modeling and fusion of the classification features of the same nodule, and outputs the final nodule detection and evaluation results after temporal consistency optimization.

[0121] The workflow of each module in this system is the same as the deep learning-based method for thyroid nodule detection and evaluation, and will not be described again here.

[0122] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A deep learning-based method for the detection and evaluation of thyroid nodules, characterized in that, Includes the following steps: S101. Construct a training dataset, obtain thyroid ultrasound scan video data, extract frames from the thyroid ultrasound scan video to obtain ultrasound image sequences, perform CLAHE preprocessing on the ultrasound image sequences, and label the nodule regions of the ultrasound image sequences to construct a training dataset. S102. Construct a nodule detection model. The nodule detection model is based on the YOLOv11 target detection framework. An efficient channel attention mechanism (ECA) is embedded in the C3k2 module of the neck network to form a C3k2-ECA module. The model is trained using the training dataset to obtain a trained nodule detection model. S103. Input the preprocessed ultrasound image sequence from step S101 frame by frame into the nodule detection model in step S102 to obtain the detection location and confidence level of thyroid nodules in each frame. S104. The nodule regions detected in each frame are cropped and input into the nodule classification network for feature extraction. The ACRTI-RADS scoring system is used to classify and score the five types of ultrasound features of the nodule: composition, echo, morphology, edge, and strong echo foci, to obtain the classification evaluation results of the nodule region in each frame. At the same time, based on the feature map of the MobileViT classification network, the GradCAM gradient weighted class activation mapping method is used to generate an attention heatmap of the nodule region to help doctors understand the classification basis. S105. Track and match the detection results of the same nodule in consecutive frames of the ultrasound image sequence. Construct a temporal feature sequence from the classification feature vectors belonging to the same nodule. Input the temporal feature sequence into a bidirectional long short-term memory network (BiLSTM) for temporal fusion and consistency optimization. Output the final classification probability and ACR TI-RADS classification result of the nodule after temporal fusion as the final nodule detection and evaluation result.

2. The deep learning-based method for detecting and evaluating thyroid nodules according to claim 1, characterized in that, Step S101 involves performing CLAHE preprocessing on the ultrasound image sequence, including the following steps: The single-frame ultrasound images in the ultrasound image sequence are converted from the BGR color space to the LAB color space, separating the luminance channel L and color channels A and B; the CLAHE algorithm is applied to the luminance channel L to divide the luminance channel L into... Each grid region is independently histogram-equalized, and parameters are limited by contrast. By limiting the contrast enhancement of the histogram, the enhanced luminance channel is obtained. Brightness channel Represented as: in, The range of values ​​is , The range of values ​​is ; Finally, the enhanced brightness channel... The images are then merged with the separated color channels A and B, converted back to the BGR color space, and the preprocessed ultrasound image is obtained.

3. The deep learning-based method for detecting and evaluating thyroid nodules according to claim 1, characterized in that, Step S102, which involves embedding an efficient channel attention mechanism (ECA) into the C3k2 module of the neck network to form a C3k2-ECA module, includes the following steps: The C3k2 module in the neck network receives multi-scale intermediate layer features extracted from the ultrasound image sequence, and performs multi-scale feature fusion through cross-stage partial connections and the Bottleneck structure to obtain the output feature map. , dimension ; An ECA attention mechanism is embedded at the output of the C3k2 module. The channel importance weighting is performed, and the specific calculation process is as follows: right Adaptive global average pooling is performed to compress the two-dimensional spatial features of each channel into a single scalar, resulting in a channel-level description vector. Channel-level description vector Represented as: in, This indicates that the spatial dimensions are adaptively compressed to... , Dimensions ; right Perform a dimensionality transformation, removing the spatial dimension and swapping the channel and sequence dimensions to adapt it to the input format of a one-dimensional convolution, represented as: in, This indicates the removal of spatial dimensions. This indicates that the channel dimension is transposed to the sequence dimension, after which... Dimensions That is, a one-dimensional channel sequence; Determine the kernel size of a one-dimensional convolution. According to the number of channels Adaptive computation, expressed as: in, This indicates taking the nearest odd number; With kernel size Fill as One-dimensional convolutions are used to aggregate cross-channel information, and channel attention weights are generated after dimension reduction and Sigmoid activation. Channel attention weight Represented as: in, This indicates that the number of input channels is 1, the number of output channels is 1, the core size is k, and the padding is... One-dimensional convolution operation without bias terms; This indicates that the output dimension will be changed from... Restore to ; This represents the Sigmoid activation function; Dimensions Channel attention weights Broadcast extended to Same dimensions Attention-enhancing features are obtained through element-wise multiplication. Attention enhancement features Represented as: in, This represents element-wise multiplication based on a broadcast mechanism, i.e. The weight scalar of each channel and The entire corresponding channel Feature map multiplication enhances key feature channels and suppresses irrelevant channels. This is the final output of the C3k2-ECA module.

4. The deep learning-based method for detecting and evaluating thyroid nodules according to claim 1, characterized in that, The specific steps of step S104 are as follows: From the detection results output by the nodule detection model, the nodule region image is cropped according to the bounding box coordinates, scaled to a uniform size, and then input into the MobileViT classification network; The MobileViT classification network employs a lightweight visual Transformer structure that combines convolutional local feature extraction with global feature modeling. It extracts features from nodule region images and outputs prediction results from five parallel classification heads, which correspond to five ultrasound features in the ACR TI-RADS scoring system: composition, echo, morphology, edge, and strong echo foci. According to the ACR TI-RADS scoring system, the predicted grading results of the five ultrasound features are assigned corresponding scores and summed to obtain the total score. The TI-RADS grade of the nodule is determined based on the total score to achieve benign or malignant risk assessment. Based on the feature map and total score of the MobileViT classification network, the GradCAM gradient weighted class activation mapping method is used to generate an attention heatmap of the nodule region, which visualizes the degree of attention the model pays to the key discrimination region of the nodule and helps doctors understand the classification basis.

5. The deep learning-based method for detecting and evaluating thyroid nodules according to claim 1, characterized in that, The specific steps of step S105 are as follows: The detection results of each frame in the video sequence are subjected to nodal tracking and matching. The detection results of the same nodal in consecutive frames are grouped into the same nodal sequence. The classification feature vector of the nodal region in each frame is extracted to form a temporal feature sequence. The temporal feature sequence of the same nodule is input into the Bidirectional Long Short-Term Memory (BiLSTM) network. The BiLSTM network performs temporal modeling on the sequence along the forward and backward directions, respectively, and extracts the temporal context information of the nodule under different scanning angles and scanning times. The hidden states of the forward and backward directions are concatenated to obtain the fused temporal feature vector. The fused temporal feature vector is passed through a fully connected layer and a softmax function to output the final classification probability of the nodule after temporal fusion. The final total score and TI-RADS classification are calculated in conjunction with the ACR TI-RADS scoring system as the final evaluation result of the nodule. The temporal fusion process of the Bidirectional Long Short-Term Memory (BiLSTM) network is represented as follows: in, For the first The classification feature vector of the nodule in the frame. and These are the hidden state and memory unit of a forward LSTM, respectively. and These are the hidden state and memory unit of the inverse LSTM, respectively. This represents vector concatenation. For the first The bidirectional fusion feature of the step, This represents the total number of frames in the nodule sequence. and For the weights and biases of the fully connected layer, This represents the final classification probability after temporal fusion.

6. A deep learning-based system for detecting and evaluating thyroid nodules, characterized in that, include: The video preprocessing module acquires thyroid ultrasound scan video data, extracts frames from the video to obtain ultrasound image sequences, performs CLAHE adaptive histogram equalization preprocessing on the ultrasound image sequences, and labels the nodule regions to construct a training dataset. The nodule detection module performs frame-by-frame nodule detection on the preprocessed ultrasound image sequence based on the YOLOv11 model. The YOLOv11 model embeds the ECA efficient channel attention mechanism in the C3k2 module of the neck network to form the C3k2-ECA module, which outputs the detection location and confidence of thyroid nodules in each frame of the image. The nodule classification module uses the MobileViT lightweight visual Transformer model to extract features from the detected nodule regions, combines the ACR TI-RADS scoring system to assess the benign or malignant risk of nodules, and uses the GradCAM gradient weighted class activation mapping method to generate attention heatmaps of nodule regions. The temporal fusion module, based on the temporal information between video frames, uses a bidirectional long short-term memory network (BiLSTM) to perform temporal modeling and fusion of the classification features of the same nodule in consecutive frames, and outputs the final nodule detection and evaluation results after temporal consistency optimization.

Images