Construction and application of mask-guided benign and malignant thyroid nodule classification model

Abstract

The application provides a kind of construction and application of thyroid nodule benign and malignant classification model based on mask guide, the present application simultaneously inputs thyroid ultrasound original map, lesion mask and patient clinical structured information, designs anatomical boundary enhancement network, generates lesion anatomical boundary graph by mask inflation, erosion operation, adopts double-branch deep convolution to strengthen internal and boundary features of nodule respectively;Propose mask conditional attention mechanism, fuse three types of channel descriptors to weaken irrelevant background interference, and build BRM bidirectional mutual feedback modulation module, realize image and structured feature adaptive mutual calibration fusion through bidirectional gate weight, can accurately extract nodule boundary key diagnostic features, improve the defects of low contrast of ultrasound image, multi-modal information fusion imbalance, improve the accuracy of nodule benign and malignant identification.

Description

Technical Field

[0001] This invention relates to the fields of medical image processing, deep learning, and intelligent auxiliary diagnosis of thyroid nodules. In particular, it relates to the construction and application of a mask-guided classification model for benign and malignant thyroid nodules. This model is applicable to complex scenarios in thyroid ultrasound images, such as blurred boundaries, low contrast, speckle noise interference, and uneven expression of information in different modalities. It can assist doctors in determining the benign or malignant nature of thyroid nodules. Background Technology

[0002] Thyroid nodules are a common thyroid disease in clinical practice, and their benign or malignant differentiation is crucial for subsequent treatment selection. Ultrasound imaging, due to its non-invasive, economical, convenient, and real-time characteristics, has become an important imaging tool for thyroid nodule screening and diagnosis. However, in actual clinical settings, thyroid ultrasound images are often affected by factors such as imaging equipment, operator experience, patient size, tissue scattering, and speckle noise. These factors can easily lead to problems such as low image contrast, blurred lesion boundaries, irregular nodule morphology, and significant background tissue interference. These factors increase the difficulty for physicians in determining the benign or malignant nature of nodules and also affect the extraction of key discriminative features by deep learning models.

[0003] In recent years, deep learning-based automatic classification methods for thyroid nodules have made some progress. Traditional methods often use the entire ultrasound image as input, automatically learning image features through convolutional neural networks and outputting benign / malignant predictions. However, these methods typically lack explicit constraints on the anatomical structure of the lesion, and the model is easily affected by background areas, irrelevant tissues, and noise artifacts, leading to unstable classification criteria. Especially when the nodule boundaries are blurred, the internal echoes are heterogeneous, or the surrounding tissue structure is complex, the model may struggle to accurately capture the boundary morphology, lesion contour, and local texture information related to benignity / malignancy.

[0004] On the other hand, in the clinical diagnosis of thyroid nodules, doctors typically rely not only on ultrasound images but also on structured information such as patient age, gender, ultrasound findings, nodule size, shape, borders, echogenicity, and calcification for a comprehensive assessment. Therefore, a single image modality is insufficient to fully reflect all diagnostic information about the nodule. Existing multimodal methods usually fuse image features and structured features directly, but this approach lacks an intermodal information exchange mechanism and cannot dynamically adjust the importance of image and structured modalities based on the feature differences of different samples. This can easily lead to an over-dominance of one modality or insufficient utilization of effective information.

[0005] Furthermore, existing mask-guided classification methods typically only use the lesion segmentation mask as spatial attention to guide the model to focus on the lesion region, but rarely further distinguish between the internal region of the lesion and the anatomical boundary region. For the identification of benign and malignant thyroid nodules, features such as whether the boundary is clear, whether it is spiculated, whether it is irregular, and whether there is invasion of surrounding tissue have important diagnostic value. If only the lesion region mask is used without explicitly modeling the boundary region, the model's ability to perceive boundary-related features remains limited. Summary of the Invention

[0006] The purpose of this invention is to provide the construction and application of a mask-guided classification model for benign and malignant thyroid nodules. This model constructs an anatomical boundary map using a lesion segmentation mask, enhances the features of the internal and boundary regions of the lesion in image branches, and further utilizes a mask-conditional attention mechanism to achieve boundary-aware channel recalibration. Simultaneously, a BRM bidirectional feedback modulation module enables mutual calibration and adaptive fusion between image features and structured features, thereby improving the accuracy and robustness of benign and malignant thyroid nodule classification.

[0007] To achieve the above objectives, this technical solution provides a method for constructing a mask-guided classification model for benign and malignant thyroid nodules, comprising the following steps: S1: Obtain multiple sets of ultrasound scan data labeled with the results of benign or malignant thyroid nodules as training datasets. Each set of ultrasound scan data includes the original ultrasound image, a mask image of the nodule region delineated by a professional doctor, and the corresponding clinical structured information of the patient. S2: The training dataset is input into the thyroid nodule benign / malignant classification framework to train the thyroid nodule benign / malignant classification model. The thyroid nodule benign / malignant classification framework includes an input layer, a feature extraction and fusion network, and a classification network. The feature extraction network further includes an anatomical boundary enhancement network, a text multilayer perception network, and a bidirectional feedback modulation module. The anatomical boundary enhancement network includes multiple cascaded anatomical boundary enhancement convolutional blocks. Each anatomical boundary enhancement convolutional block introduces an anatomical boundary enhancement module and a mask conditional attention mechanism. The original ultrasound image and the mask image are input from the input layer into the anatomical boundary enhancement network for processing to obtain image features. Clinical structured information is input from the input layer into the text multilayer perception network for processing to obtain structured features. The image features and structured features are input into the bidirectional feedback modulation module to output fused features. The fused features are input into the classification network to output the prediction result. The loss function of the thyroid nodule benign / malignant classification framework is a weighted combination of classification loss, attention mask loss, and gating weight loss.

[0008] Compared with existing technologies, this technical solution has the following characteristics and beneficial effects: 1. To address the issues of small grayscale differences, abundant speckle noise and artifacts, and blurred lesion boundaries in thyroid ultrasound, traditional CNN global feature extraction easily incorporates irrelevant information such as background, scale, and normal tissue, weakening lesion discrimination features. This solution designs an anatomical boundary enhancement module, which generates lesion anatomical boundary maps using mask dilation and erosion operations. Dual-depth convolution branches enhance the differential features within the nodules and at their boundaries, respectively. Furthermore, learnable parameters are introduced to adaptively fuse the original image, internal enhancement features, and boundary enhancement features, accurately capturing boundary pathological information such as spiculations and irregular contours, effectively suppressing irrelevant background interference, and strengthening the expression of lesion region features.

[0009] 2. Traditional channel attention methods, such as SE, rely solely on global average pooling to generate channel weights, which is susceptible to interference from image background, noise, and irrelevant tissue, failing to effectively focus on lesions and boundary diagnostic features. This solution proposes a mask-conditional attention mechanism that simultaneously constructs and adaptively fuses three types of channel descriptors: global, lesion, and boundary. Channel weight calibration is constrained by the lesion's anatomical structure, avoiding weight bias caused by non-lesion regions and improving the network's response strength to key diagnostic features of nodules.

[0010] 3. Traditional multimodal fusion methods, which employ direct stitching and fixed weighting, suffer from drawbacks such as high-dimensional image features suppressing clinical structured information, the inability to dynamically adjust weights based on case data, and the lack of deep interactive calibration of modalities. This solution combines image and structured features with a bidirectional feedback modulation module to achieve bimodal bidirectional gating interaction. It utilizes mutually generated dynamic gating weights to complete the mutual calibration and fusion of image and clinical features, achieving adaptive complementarity of modal information. This fully leverages the joint diagnostic value of image and clinical data, improving the utilization rate of multimodal information and classification performance. Attached Figure Description

[0011] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a schematic diagram of a classification framework for benign and malignant thyroid nodules according to an embodiment of this application. Figure 2 This is a schematic diagram of the framework of the anatomical boundary enhancement network; Figure 3 This is a schematic diagram of the framework of the anatomical boundary enhancement module; Figure 4 This is a schematic diagram of the framework of mask-conditional attention mechanism; Figure 5 This is a schematic diagram of the bidirectional mutual feedback modulation module. Figure 6 This is a schematic diagram of the experimental results; Figure 7This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0012] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with one or more embodiments of this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0013] It should be noted that the steps of the corresponding methods are not necessarily performed in the order shown and described in this specification in other embodiments. In some other embodiments, the methods may include more or fewer steps than described in this specification. Furthermore, a single step described in this specification may be broken down into multiple steps in other embodiments; and multiple steps described in this specification may be combined into a single step in other embodiments.

[0014] Example 1 This approach provides a method for constructing a mask-guided classification model for benign and malignant thyroid nodules, comprising the following steps: S1: Obtain multiple sets of ultrasound scan data labeled with the results of benign or malignant thyroid nodules as training datasets. Each set of ultrasound scan data includes the original ultrasound image, a mask image of the nodule region delineated by a professional doctor, and the corresponding clinical structured information of the patient. S2: The training dataset is input into the thyroid nodule benign / malignant classification framework to train the thyroid nodule benign / malignant classification model. The thyroid nodule benign / malignant classification framework includes an input layer, a feature extraction and fusion network, and a classification network. The feature extraction network further includes an anatomical boundary enhancement network, a text multilayer perception network, and a bidirectional feedback modulation module. The anatomical boundary enhancement network includes multiple cascaded anatomical boundary enhancement convolutional blocks. Each anatomical boundary enhancement convolutional block introduces an anatomical boundary enhancement module and a mask conditional attention mechanism. The original ultrasound image and the mask image are input from the input layer into the anatomical boundary enhancement network for processing to obtain image features. Clinical structured information is input from the input layer into the text multilayer perception network for processing to obtain structured features. The image features and structured features are input into the bidirectional feedback modulation module to output fused features. The fused features are input into the classification network to output the prediction result. The loss function of the thyroid nodule benign / malignant classification framework is a weighted combination of classification loss, attention mask loss, and gating weight loss.

[0015] like Figure 1 As shown, Figure 1This is a framework diagram of the benign and malignant classification framework for thyroid nodules in this scheme. As can be seen, the same set of ultrasound scan data is input into the feature extraction and fusion network from the input layer. The original ultrasound image and the mask image are input into the anatomical boundary enhancement network for processing to obtain image features. The clinical structured information is input into the text multilayer perception network for processing to obtain structured features. The image features and structured features are then fused by the bidirectional feedback modulation module to obtain fused features. The fused features are input into the classification network to output the prediction results of benign and malignant classification of thyroid nodules.

[0016] In some embodiments, the text multilayer perceptron includes a linear layer, a normalization layer, a ReLU activation layer, a dropout layer, a normalization layer, and a ReLU activation layer cascaded in sequence. Clinical structured information is input into the text multilayer perceptron and outputs structured features.

[0017] Regarding the clinical structured information of this protocol, the clinical structured information of this protocol mainly includes information related to the five major attributes of the nodules: shape, margin, composition, echo, and calcification.

[0018] like Figure 2 As shown, the anatomical boundary enhancement network comprises cascaded anatomical boundary enhancement convolutional blocks of different scales, wherein each anatomical boundary enhancement convolutional block comprises sequentially connected 1 1 convolution, k k-depth convolution, anatomical boundary enhancement module, mask conditional attention mechanism and 1 1 convolution, the features input to the anatomical boundary enhancement convolution block and the final 1 The output features of the convolution are summed element by element to obtain the output features of the current anatomical boundary enhancement convolution block.

[0019] In some embodiments, the anatomical boundary enhancement network includes cascaded connections of 3 3 convolutions, 1 scale of 3 3-dimensional anatomical boundary enhancement convolution block, 2 scales of 3 3 anatomical boundary enhanced convolution blocks, 3 scales of 5 5-dimensional anatomical boundary enhancement convolution block, 3 scales. Anatomical boundary enhancement convolution blocks of 3, 4 scales of 5 5-dimensional anatomical boundary enhanced convolution block, 2 scales of 5 5 anatomical boundary enhancement convolution blocks and 1 1. Convolutional and pooling layers.

[0020] It should be noted that in thyroid ultrasound images, the grayscale difference between the nodule region and the surrounding thyroid tissue is usually small, and there is significant speckle noise and artifact interference in the images, making it difficult to accurately perceive lesion boundaries and irregularly shaped areas. Traditional convolutional neural networks typically perform uniform feature extraction on the entire image, which can easily encode irrelevant background tissue, scales, text labels, or features of surrounding normal tissue, weakening the discriminative representation of the lesion region. Therefore, this solution specifically designs an anatomical boundary enhancement module as the core image feature enhancement module of the anatomical boundary enhancement network.

[0021] Regarding the structure of the anatomical boundary enhancement module, as follows: Figure 3 As shown, the anatomical boundary enhancement module includes a first depthwise convolution branch and a second depthwise convolution branch connected in parallel, wherein each of the first depthwise convolution branch and the second depthwise convolution branch includes k The system employs a k-depth convolution, with a mask processing module between the first and second branches. The mask image is input into the mask processing module and processed to obtain the lesion anatomical boundary map. The original ultrasound image is input into the first and second depth convolution branches, respectively, to output the first and second features. The mask image and the first feature are multiplied element-wise to obtain the lesion internal enhancement feature. The lesion anatomical boundary map and the second feature are multiplied element-wise to obtain the lesion boundary enhancement feature. The lesion internal enhancement feature, the lesion boundary enhancement feature, and the original ultrasound image are added element-wise to obtain the output feature of the current anatomical boundary enhancement module.

[0022] The anatomical boundary enhancement module of this scheme uses the mask image as the structural prior. First, it generates the anatomical boundary map of the lesion based on the lesion mask in the mask image. Then, it performs feature enhancement on the internal region of the lesion and the boundary region of the lesion respectively. Unlike traditional spatial attention, this anatomical boundary enhancement module does not simply multiply the mask with the feature map, but explicitly constructs the boundary region of the lesion and designs two differentiated enhancement paths for the internal region of the lesion and the boundary region, thereby enhancing the model's ability to perceive nodule morphology, boundary continuity and local texture anomalies.

[0023] It should be noted that the original ultrasound image input to the current anatomical boundary enhancement module is fed into the first depth convolution branch and the second depth convolution branch respectively to output the first feature and the second feature.

[0024] In the step of "inputting the mask image into the mask processing module to obtain the lesion anatomical boundary map," the mask processing module performs a dilation operation on the mask image to obtain a dilated map, performs an erosion operation on the mask image to obtain an eroded map, and calculates the difference between the dilated map and the eroded map to obtain the lesion anatomical boundary map. The lesion anatomical boundary map of this scheme can highlight the nodule contour region, enabling the model to explicitly focus on the lesion edge during feature extraction.

[0025] In the step of “multiplying the mask image and the first feature element-wise to obtain the enhancement feature inside the lesion, and multiplying the lesion anatomical boundary map and the second feature element-wise to obtain the enhancement feature of the lesion boundary”, the lesion area in the mask image is set to 1, the non-lesion area is set to 0, the boundary area in the lesion anatomical boundary map is set to 1, and the non-boundary area is set to 0.

[0026] In the step of "adding the enhancement features inside the lesion, the enhancement features at the lesion boundary, and the features input to the current anatomical boundary enhancement module element by element to obtain the output features of the current anatomical boundary enhancement module", learnable parameters are introduced for the enhancement features inside the lesion and the enhancement features at the lesion boundary. Based on the learnable parameters, the enhancement features inside the lesion, the enhancement features at the lesion boundary, and the features input to the current anatomical boundary enhancement module are added element by element to obtain the output features of the current anatomical boundary enhancement module. The advantage of this is that, while preserving the original image features, it adaptively enhances the discriminative information of the lesion interior and the anatomical boundary region to finally output the output features of the current anatomical boundary enhancement module.

[0027] For example, the features input into the anatomical boundary enhancement module are represented as follows: ; in Features are input into the anatomical boundary enhancement module. Here, H is the number of channels, H is the height of the input feature map, and W is the width of the input feature map. R is the input feature map, and R is the set of real numbers.

[0028] The mask image input into the anatomical boundary enhancement module is represented as follows: ; Where M is the mask image. Where H is the number of channels, H is the height of the input mask image, and W is the width of the input mask image. It is the set of real numbers.

[0029] The anatomical boundary diagram of the lesion is shown as follows: ; Where Dilate represents the mask dilation operation, Erode represents the mask erosion operation, M represents the mask image, and B represents the lesion anatomical boundary map.

[0030] The features input to the anatomical boundary enhancement module are fed into two depthwise convolutional branches, which output the first feature respectively. Second feature : ; in Indicates k Depth-k convolution, Indicates the first feature, Indicates the second characteristic, H is the number of channels, H is the height of the corresponding feature map, W is the width of the corresponding feature map, and R is the set of real numbers.

[0031] The enhancement features inside the lesion are obtained by element-wise multiplication of the mask image and the first feature, and the enhancement features at the lesion boundary are obtained by element-wise multiplication of the lesion anatomical boundary map and the second feature, as shown below: ; in For element-wise multiplication, The lesion exhibits internal enhancement characteristics. The image shows the enhanced features of the lesion boundary. M represents the mask image, and B represents the anatomical boundary map of the lesion. Indicates the first feature, Indicates the second characteristic, H is the number of channels, H is the height of the corresponding feature map, W is the width of the corresponding feature map, and R is the set of real numbers.

[0032] The enhancement features inside the lesion, the enhancement features at the lesion boundary, and the features input to the current anatomical boundary enhancement module are added element-wise to obtain the output features of the current anatomical boundary enhancement module, as shown below: ; in The lesion exhibits internal enhancement characteristics. The lesion exhibits enhanced border characteristics. The features input into the anatomical boundary enhancement module are α and β, which are learnable parameters that control the enhancement features inside the lesion and the enhancement features at the lesion boundary, respectively.

[0033] Furthermore, existing channel attention mechanisms (such as the SE module) typically utilize global average pooling to generate channel weights and recalibrate the importance of different channels based on global contextual information. However, in thyroid ultrasound images, truly diagnostically valuable features are often concentrated in the lesion region and its boundary regions, while background tissue, irrelevant structures, and noise artifacts in the entire image may affect the global pooling results, causing channel attention weights to be interfered with by non-lesion regions. Therefore, a channel attention mechanism that can simultaneously utilize global context, lesion region, and boundary region information is needed, so that the channel weight generation process is constrained by the lesion's anatomical structure, thereby improving the image branch's responsiveness to key diagnostic features. Therefore, this proposal suggests a masked conditional attention mechanism.

[0034] The structure of the mask-conditioned attention mechanism is as follows: Figure 4As shown, this mask-conditional attention mechanism introduces constraints from the mask image and the lesion anatomical boundary map on the basis of traditional SE channel attention. It no longer relies solely on the global average pooling descriptor, but simultaneously constructs three types of channel descriptors: global descriptor, lesion region descriptor, and boundary region descriptor. Subsequently, the three types of descriptors are adaptively fused through learnable weights, and channel recalibration coefficients are generated, so that mask-conditional attention can simultaneously perceive the overall context, the main lesion region, and the anatomical boundary region.

[0035] Specifically, the anatomical boundary enhancement module outputs features, mask images, and lesion anatomical boundaries. Figure 1 The data is then input into the mask-conditional attention mechanism. Global average pooling is performed on the features output by the anatomical boundary enhancement module to obtain the global channel descriptor. Mask pooling is performed on the features output by the anatomical boundary enhancement module using a mask image to obtain the lesion region channel descriptor. Mask pooling is performed on the features output by the anatomical boundary enhancement module using the lesion anatomical boundary map to obtain the boundary region channel descriptor. The global channel descriptor, the lesion region channel descriptor, and the boundary region channel descriptor are weighted to obtain the fused descriptor. The fused descriptor is input into the channel mapping structure to obtain the channel recalibration weights. The channel recalibration weights are applied to the features output by the anatomical boundary enhancement module to obtain the output features of the current mask-conditional attention mechanism.

[0036] In some embodiments, the channel mapping structure includes 1 connected in sequence. 1. Convolution, Nonlinear Activation Function, 1 1. Convolution and Sigmoid activation function.

[0037] Specifically, the lesion region channel descriptor is obtained by mask pooling the features output by the anatomical boundary enhancement module using a masked image, as shown in the following formula: ; in For the lesion area channel descriptor, The features output by the anatomical boundary enhancement module are represented by i and j, which represent spatial location indices. To prevent constant terms with a denominator of 0, M is a mask image. This represents the number of channels.

[0038] Similarly, the features output by the anatomical boundary enhancement module are masked and pooled using the lesion anatomical boundary map to obtain the boundary region channel descriptor, as shown in the following formula: ; in For boundary region channel descriptors, The features output by the anatomical boundary enhancement module are represented by i and j, which represent spatial location indices. To prevent constant terms with a denominator of 0, M is a mask image. This represents the number of channels.

[0039] In the process of "obtaining a fusion descriptor from the weighted global channel descriptor, lesion region channel descriptor, and boundary region channel descriptor", learnable weights are introduced into the lesion region channel descriptor and the boundary region channel descriptor. Based on the learnable weights, the global channel descriptor, lesion region channel descriptor, and boundary region channel descriptor are added element-wise to obtain the fusion descriptor.

[0040] Furthermore, in the classification of benign and malignant thyroid nodules, ultrasound images and structured clinical data are two key complementary modalities. Traditional multimodal fusion methods typically employ simple splicing or weighted averaging strategies, which have significant limitations in practical applications: First, since the dimensionality of image features is much higher than that of structured features, direct splicing easily leads to the dominance of high-dimensional modalities, suppressing the effective information expression of low-dimensional structured data; second, fixed fusion weights cannot adapt to the differences in modal importance in different cases, lacking dynamic adjustment capabilities; third, this method lacks a deep interaction mechanism between modalities, making it difficult to achieve calibration and complementarity of information features. Therefore, there is an urgent need for a fusion method that can achieve intermodal modulation and dynamic optimization of feature expression to improve the utilization efficiency and classification performance of multimodal information. To address this issue, this solution proposes a bidirectional mutual feedback modulation module to achieve bidirectional gating interaction between image features and structured features.

[0041] The structure of the bidirectional mutual feedback modulation module is as follows: Figure 5 As shown, Image features are input into the first gating module of the bidirectional feedback modulation module to generate structured mode gating weights. Structured features are input into the second gating module of the bidirectional feedback modulation module to generate image mode gating weights. Image features are modulated based on image mode gating weights to obtain image modulation features. Structured features are modulated based on structured mode gating weights to obtain structured modulation features. Image modulation features and structured modulation features are input into image modulation feature projection layer and structured feature projection layer for linear mapping and normalization. The processed features are then stitched together to obtain fused features.

[0042] The bidirectional feedback modulation module of this scheme first uses structured features to generate image modal gating weights and modulates the image features. At the same time, it uses image features to generate structured modal gating weights and modulates the structured features. Through this bidirectional feedback mechanism, the two types of modalities can be mutually calibrated before fusion, thereby achieving a more complete complementary expression.

[0043] Specifically, let the image features be: ; Where v represents the image feature. R represents the image feature dimension, and R denotes the set of real numbers. Let the structured features be ; Where t represents the structured feature. For structured feature dimensions, R represents the set of real numbers; The formula for generating image modality gating weights using structured features is as follows:

[0044] in A gating module for generating image gating weights, This represents the Sigmoid activation function. Represents the image modality gating weights; The formula for obtaining image modulation features by modulating image features using image modality gating weights is as follows: ; in Image modulation features, This indicates element-wise multiplication, where v represents the image feature. This represents the image modality gating weights.

[0045] Similarly, the formula for generating structured modal gating weights using image features is as follows: ; in To generate the gating module for structured modal gating weights, This represents the Sigmoid activation function. Represents the structured modal gating weights; The formula for obtaining structured modulated features by modulating structured features using structured modal gating weights is as follows: ; in It is a structured modulation feature. This indicates element-wise multiplication, where t is a structured feature. This represents the structured modal gating weights.

[0046] After linear mapping and normalization of the image modulation features and structured modulation features into the image modulation feature projection layer and structured feature projection layer, the processed features are concatenated to obtain the fused features, as follows: ; ; ; in and These correspond to the image modulation feature projection layer and the structured feature projection layer, respectively.

[0047] Regarding the classification network of this scheme: The classification network of this scheme consists of a linear layer, a ReLU activation layer, a dropout layer, and another linear layer connected in sequence. The fused feature input is fed into the classification network to output the prediction result, which is either a benign or malignant result for the thyroid nodule.

[0048] The loss function for the training phase of this thyroid nodule benign / malignant classification framework is as follows: L_total=L_cls+λ_attn L_attn+λ_gate L_gate; Where L_cls is the classification loss, L_attn is the attention mask loss, L_gate is the gating weight loss, and λ_attn and λ_gate are the weights of the corresponding losses.

[0049] In some embodiments, λ_attn=0.2, λ_gate=0.01.

[0050] Regarding the classification of losses in this plan: The classification loss in this scheme uses cross-entropy loss, as shown in the following formula: L_cls= CrossEntropyLoss(logits, y); Where Logits is the predicted value, y is the true value, and CrossEntropyLoss() is the cross-entropy loss.

[0051] The formula for the attention masking loss in this scheme is as follows: L_attn= L_BCE(A, M) + L_Dice(A, M) L_Dice=1-(2Σ(A·M)+1) / (ΣA+ΣM+1); Where A is the attention map generated by the model, M is the manually labeled nodule mask, L_BCE() is the constraint for pixel-level consistency, and L_Dice() is the constraint for the degree of overlap of the overall region.

[0052] The formula for the gating weight loss in this scheme is as follows: L_gate=mean((mean(g_image)-0.5)^2) +mean((mean(g_text)-0.5)^2); Where g_image represents the image modality gating weights, and g_text represents the text or clinical table modality gating weights.

[0053] Example 2 As shown in Example 1, this scheme constructs a classification model for benign and malignant thyroid nodules that can be used to predict the outcome of benign and malignant thyroid nodules. Correspondingly, this scheme provides an application method for the classification model for benign and malignant thyroid nodules, including the following steps: Acquire ultrasound scan data, which includes the original ultrasound image, a masked image of the nodule region delineated by a professional physician, and the patient's corresponding structured clinical information; The ultrasound scan data is input into the thyroid nodule benign or malignant classification model obtained by Example 1 and the prediction result is output, where the prediction result is either benign or malignant.

[0054] To verify the effectiveness of this approach, this study used actual clinical thyroid nodule ultrasound image data provided by a tertiary hospital in Zhejiang Province. This dataset, selected, labeled, and organized by professional physicians, contains 3896 thyroid nodule ultrasound images from 1616 patients. Each patient's data includes several sets of ultrasound scans, each set consisting of three parts: the original ultrasound image, a masked image of the nodule region delineated by a professional physician, and the patient's corresponding clinical structured information. The clinical structured information mainly includes five attributes of the nodule: shape, margin, composition, echogenicity, and calcification, providing a reliable data foundation for model training and performance validation.

[0055] In terms of data partitioning, this study strictly divided the dataset into training, validation, and test sets in a 7:2:1 ratio. To comprehensively evaluate model performance, this study selected F1 score, accuracy, area under the receiver operating characteristic curve (AUC), recall, sensitivity, specificity, and parameters as the main evaluation metrics. Specifically, the F1 score comprehensively measures the model's classification performance, accuracy reflects the overall prediction accuracy, AUC evaluates the model's ability to distinguish between classes, recall measures the model's ability to detect target samples, and params are used to evaluate the model's complexity and lightweight nature. During the experiment, "MAGE-Net" was used to refer to the thyroid nodule benign / malignant classification model of this approach.

[0056] All experiments were conducted on the same workstation equipped with an NVIDIA RTX 4080 GPU, using uniform training settings. Specifically, the optimizer employed stochastic gradient descent (SGD) with momentum, an initial learning rate of 0.0001, a momentum coefficient of 0.8, a batch size of 4, a maximum training epoch of 200, and an early stopping mechanism (Patience=50) to reduce overfitting and improve training efficiency.

[0057] The experimental test results are shown in Table 1 below; Table 1. Test results of MAGE-Net and SOTA detectors on the dataset. As shown in Table 1, MAGE-Net achieved the best overall performance in the thyroid nodule ultrasound image classification task. The fusion module in the table refers to the fusion method of image features and structured clinical information, i.e., jointly modeling ultrasound image features as well as attributes such as nodule shape, edge, composition, echo, and calcification. Under the condition of using the BRM fusion module, MAGE-Net's F1, ACC, AUC, Sensitivity, and Specificity reached 0.9260, 0.9410, 0.9801, 0.9290, and 0.9489, respectively, all outperforming comparative models such as ResNet-18, ViT-b16, ConvNext, DenseNet-121, Swin-T, MedMamba, and Efficient-b3, indicating that it has stronger classification and discrimination capabilities and stability.

[0058] Compared to the superior Efficient-b3, MAGE-Net improved F1, ACC, Sensitivity, and Specificity by 3.16, 3.99, 3.21, and 3.77 percentage points, respectively, indicating that it can simultaneously improve the detection capability of positive samples and reduce the risk of false positives for negative samples. Meanwhile, MAGE-Net has only 12.1201M parameters, significantly lower than models such as ViT-b16, ConvNext, and Swin-T, demonstrating a good balance between accuracy and lightweight design, and possessing certain potential for clinical application.

[0059] To verify the effectiveness and independent contribution of each module in MAGE-Net, this study conducted systematic ablation experiments. Using the Efficient-b3 network as the baseline model, we tested the impact of different modules on model performance on the experimental dataset. In the experiments, the MBConv module in the baseline model was replaced with the proposed Anatomical Boundary Enhancement module (MAGE-Block), the original SE attention mechanism was replaced with the Mask Conditional Attention mechanism (MCAM module), and the traditional Concat method in multimodal information fusion was replaced with the Bidirectional Feedback Modulation module (BRM module). By progressively introducing and combining different modules, we examined the contribution of each module to the model's classification performance. The experimental results are shown in Table 2.

[0060] Table 2 Ablation Experiment Results

[0061] Ablation experiments demonstrate that the proposed MAGE-Block, MCAM, and BRM modules effectively improve model performance. Compared to the EfficientNet baseline model, the gradual introduction of each module resulted in improvements in F1, ACC, AUC, and Recall metrics. The complete model achieved the best results, with F1 increasing from 0.8503 to 0.9260, ACC from 0.8744 to 0.9410, AUC from 0.9574 to 0.9801, and Recall from 0.8710 to 0.9290. This indicates that the three modules effectively complement each other in feature extraction, attention enhancement, and multimodal information fusion, significantly improving the model's classification performance and robustness.

[0062] In addition, to further demonstrate the discriminative ability and feature focus area of ​​the proposed model in image classification tasks, this study selected the top five models in terms of ACC index in Table 2 for visualization comparison, including MAGE-Net, ViT-b16, ConvNeXt, Efficient-b3, and Swin-T. Figure 6 The results show the feature response of different models on benign and malignant samples. The first column is the ground truth (GT) region, with green outlines indicating the location of lesions. The remaining columns are the heatmap visualization results generated by different models. The corresponding category and prediction confidence are labeled below the image.

[0063] As shown in Figure 6, MAGE-Net can accurately focus on the main lesion region in both benign and malignant samples, with high consistency between the thermal response and the ground truth labeled region, and overall high prediction confidence. In contrast, ViT-b16, ConvNeXt, Efficient-b3, and Swin-T exhibit issues such as scattered regions of interest, false activation of background regions, or insufficient response at lesion boundaries in some samples, indicating that their ability to locate key lesion features and their discrimination stability still have certain limitations. Overall, MAGE-Net can more effectively capture key regions related to classification and reduce irrelevant background interference, further validating the effectiveness of the proposed module in feature extraction, attention enhancement, and multimodal information fusion.

[0064] Example 3 This embodiment also provides an electronic device, see reference. Figure 7 It includes a memory 402 and a processor 401, the memory 402 storing a computer program and the processor 401 being configured to run the computer program to perform the steps in any of the embodiments of the mask-guided method for constructing a benign and malignant classification model of thyroid nodules described above.

[0065] Specifically, the processor 401 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0066] The memory 402 may include a large-capacity memory 402 for data or instructions. The memory 402 can be used to store or cache various data files that need to be processed and / or communicated, as well as possible computer program instructions executed by the processor 401.

[0067] The processor 401 reads and executes computer program instructions stored in the memory 402 to implement any of the methods for constructing a cross-condition health index prediction model for wind turbine gearbox components or a cross-condition health index prediction method for wind turbine gearbox components in the above embodiments.

[0068] Optionally, the electronic device may further include a transmission device 403 and an input / output device 404, wherein the transmission device 403 is connected to the processor 401 and the input / output device 404 is connected to the processor 401.

[0069] Transmission device 403 can be used to receive or send data via a network. Specific examples of the network described above may include wired or wireless networks provided by the communication provider of the electronic device. In one example, the transmission device includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In one example, transmission device 404 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0070] The input / output device 404 is used to input or output information. In this embodiment, the input information may be ultrasound scan data, etc., and the output information may be prediction results, etc.

[0071] Optionally, in this embodiment, the processor 401 can be configured to perform the following steps via a computer program: S1: Obtain multiple sets of ultrasound scan data labeled with the results of benign or malignant thyroid nodules as training datasets. Each set of ultrasound scan data includes the original ultrasound image, a mask image of the nodule region delineated by a professional doctor, and the corresponding clinical structured information of the patient. S2: The training dataset is input into the thyroid nodule benign / malignant classification framework to train a thyroid nodule benign / malignant classification model. This framework includes an input layer, a feature extraction and fusion network, and a classification network. The feature extraction network further includes an anatomical boundary enhancement network, a text multilayer perception network, and a bidirectional feedback modulation module. The anatomical boundary enhancement network comprises multiple cascaded anatomical boundary enhancement convolutional blocks. Each block incorporates an anatomical boundary enhancement module and a mask conditional attention mechanism. The original ultrasound image and the mask image are input from the input layer into the anatomical boundary enhancement network for image feature processing. Clinical structured information is input from the input layer into the text multilayer perception network for structured features. Image features and structured features are input into the bidirectional feedback modulation module to output fused features. The fused features are then input into the classification network to output the prediction result. The loss function of the thyroid nodule benign / malignant classification framework is a weighted combination of classification loss, attention mask loss, and gating weight loss. It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0072] Generally, various embodiments can be implemented in hardware or dedicated circuitry, software, logic, or any combination thereof. Some aspects of the invention can be implemented in hardware, while others can be implemented by firmware or software executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flowcharts, or using some other graphical representation, it should be understood that, by way of non-limiting example, these blocks, apparatuses, systems, techniques, or methods described herein can be implemented in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0073] Embodiments of the present invention can be implemented by computer software, which may be executable by a data processor of a mobile device, such as a processor entity, or by hardware, or by a combination of software and hardware. Computer software or programs (also referred to as program products), including software routines, applets, and / or macros, can be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. A computer program product may include one or more computer-executable components configured to perform embodiments when the program is run. One or more computer-executable components may be at least one piece of software code or a portion thereof. Additionally, it should be noted that any block in the logical flow of the figures may represent a program step, or interconnected logical circuitry, blocks and functions, or a combination of program steps and logical circuitry, blocks and functions. The software may be stored on physical media such as memory chips or blocks of storage implemented within a processor, magnetic media such as hard disks or floppy disks, and optical media such as, for example, DVDs and their data variants, CDs, etc. The physical medium is a non-transient medium.

[0074] Those skilled in the art should understand that the technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0075] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for constructing a mask-guided classification model for benign and malignant thyroid nodules, characterized in that, Includes the following steps: S1: Obtain multiple sets of ultrasound scan data labeled with the results of benign or malignant thyroid nodules as training datasets. Each set of ultrasound scan data includes the original ultrasound image, a mask image of the nodule region delineated by a professional doctor, and the corresponding clinical structured information of the patient. S2: The training dataset is input into the thyroid nodule benign / malignant classification framework to train the thyroid nodule benign / malignant classification model. The thyroid nodule benign / malignant classification framework includes an input layer, a feature extraction and fusion network, and a classification network. The feature extraction network further includes an anatomical boundary enhancement network, a text multilayer perception network, and a bidirectional feedback modulation module. The anatomical boundary enhancement network includes multiple cascaded anatomical boundary enhancement convolutional blocks. Each anatomical boundary enhancement convolutional block introduces an anatomical boundary enhancement module and a mask conditional attention mechanism. The original ultrasound image and the mask image are input from the input layer into the anatomical boundary enhancement network for processing to obtain image features. Clinical structured information is input from the input layer into the text multilayer perception network for processing to obtain structured features. The image features and structured features are input into the bidirectional feedback modulation module to output fused features. The fused features are input into the classification network to output the prediction result. The loss function of the thyroid nodule benign / malignant classification framework is a weighted combination of classification loss, attention mask loss, and gating weight loss.

2. The method for constructing a mask-guided classification model for benign and malignant thyroid nodules according to claim 1, characterized in that, The anatomical boundary enhancement network comprises cascaded anatomical boundary enhancement convolutional blocks of different scales, wherein each anatomical boundary enhancement convolutional block comprises sequentially connected 1 1 convolution, k k-depth convolution, anatomical boundary enhancement module, mask conditional attention mechanism and 1 1 convolution, the features input to the anatomical boundary enhancement convolution block and the last 1 The output features of the convolution are summed element by element to obtain the output features of the current anatomical boundary enhancement convolution block.

3. The method for constructing a mask-guided classification model for benign and malignant thyroid nodules according to claim 1, characterized in that, The anatomical boundary enhancement module includes a first depthwise convolution branch and a second depthwise convolution branch connected in parallel, wherein each of the first and second depthwise convolution branches includes k The system employs a k-depth convolution, with a mask processing module between the first and second branches. The mask image is input into the mask processing module and processed to obtain the lesion anatomical boundary map. The original ultrasound image is input into the first and second depth convolution branches, respectively, to output the first and second features. The mask image and the first feature are multiplied element-wise to obtain the lesion internal enhancement feature. The lesion anatomical boundary map and the second feature are multiplied element-wise to obtain the lesion boundary enhancement feature. The lesion internal enhancement feature, the lesion boundary enhancement feature, and the original ultrasound image are added element-wise to obtain the output feature of the current anatomical boundary enhancement module.

4. The method for constructing a mask-guided classification model for benign and malignant thyroid nodules according to claim 3, characterized in that, The mask processing module performs a dilation operation on the mask image to obtain a dilation map, performs an erosion operation on the mask image to obtain an erosion map, and calculates the difference between the dilation map and the erosion map to obtain the lesion anatomical boundary map.

5. The method for constructing a mask-guided classification model for benign and malignant thyroid nodules according to claim 3, characterized in that, In the mask image, the value of the lesion area is set to 1, and the value of the non-lesion area is set to 0. In the lesion anatomical boundary map, the value of the boundary area is set to 1, and the value of the non-boundary area is set to 0.

6. The method for constructing a mask-guided classification model for benign and malignant thyroid nodules according to claim 3, characterized in that, The features output by the anatomical boundary enhancement module, the mask image, and the lesion anatomical boundary map are input into the mask conditional attention mechanism. Global average pooling is performed on the features output by the anatomical boundary enhancement module to obtain the global channel descriptor. Mask pooling is performed on the features output by the anatomical boundary enhancement module using the mask image to obtain the lesion region channel descriptor. Mask pooling is performed on the features output by the anatomical boundary enhancement module using the lesion anatomical boundary map to obtain the boundary region channel descriptor. The global channel descriptor, the lesion region channel descriptor, and the boundary region channel descriptor are weighted to obtain the fused descriptor. The fused descriptor is input into the channel mapping structure to obtain the channel recalibration weights. The channel recalibration weights are applied to the features output by the anatomical boundary enhancement module to obtain the output features of the current mask conditional attention mechanism.

7. The method for constructing a mask-guided classification model for benign and malignant thyroid nodules according to claim 1, characterized in that, Image features are input into the first gating module of the bidirectional feedback modulation module to generate structured mode gating weights. Structured features are input into the second gating module of the bidirectional feedback modulation module to generate image mode gating weights. Image features are modulated based on image mode gating weights to obtain image modulation features. Structured features are modulated based on structured mode gating weights to obtain structured modulation features. Image modulation features and structured modulation features are input into image modulation feature projection layer and structured feature projection layer for linear mapping and normalization. The processed features are then stitched together to obtain fused features.

8. The method for constructing a mask-guided classification model for benign and malignant thyroid nodules according to claim 1, characterized in that, The classification network consists of a linear layer, a ReLU activation layer, a dropout layer, and another linear layer connected in sequence. The fused feature input is fed into the classification network to output the prediction result, which is either a benign or malignant result for the thyroid nodule.

9. A method for applying a classification model for benign and malignant thyroid nodules, characterized in that, Includes the following steps: Acquire ultrasound scan data, which includes the original ultrasound image, a masked image of the nodule region delineated by a professional physician, and the patient's corresponding structured clinical information; The ultrasound scan data is input into the thyroid nodule benign and malignant classification model to output the prediction result, wherein the prediction result is either a benign thyroid nodule result or a malignant thyroid nodule result, and the thyroid nodule benign and malignant classification model is constructed according to the construction method of the thyroid nodule benign and malignant classification model according to any one of claims 1 to 8.

10. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the method for constructing a benign or malignant classification model for thyroid nodules according to any one of claims 1 to 8.

Images