Multi-mode prompt fine tuning method based on thyroid ultrasound image combined with target detection
By combining target detection and soft cueing fine-tuning techniques with multimodal fusion, the problems of noise interference and insufficient utilization of semantic information in ultrasound images for thyroid nodule analysis are solved, achieving efficient feature extraction and risk assessment, and improving the robustness and accuracy of ultrasound images in thyroid nodule analysis.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- YANGZHOU FIRST PEOPLES HOSPITAL
- Filing Date
- 2025-12-30
- Publication Date
- 2026-04-21
AI Technical Summary
Existing ultrasound images for thyroid nodule analysis suffer from low contrast, significant speckle noise interference, and marked differences in lesion morphology. Furthermore, they lack effective utilization of semantic information from clinical descriptive texts, resulting in limited feature expression capabilities and insufficient robustness of the model under conditions with few samples.
By introducing a target detection module to locate thyroid lesion areas, combining pre-trained image feature extraction technology to obtain high-level visual features, and using soft cueing fine-tuning enhanced text processing technology to perform semantic mining on clinical report text, a multi-modal fusion mechanism is constructed using a multi-head attention network to integrate visual features and semantic embedding vectors, and finally a risk score for nodules is generated through a risk assessment module.
It improves the robustness and discriminativeness of ultrasound image features, enhances feature learning ability under limited sample conditions, generates more reliable thyroid nodule risk assessment results, and reduces noise interference and background effects.
Smart Images
Figure CN121905511A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of multimodal medical image processing technology, and in particular to a multimodal cue fine-tuning method based on thyroid ultrasound images combined with target detection. Background Technology
[0002] With ultrasound examination becoming the preferred non-invasive, real-time, and low-cost imaging method in clinical practice, ultrasound imaging has also been widely used in thyroid nodule-related analysis. However, compared with other imaging methods such as CT, ultrasound images suffer from low contrast, significant speckle noise interference, and significant differences in lesion morphology, posing considerable challenges to image interpretation and computer-aided analysis. Especially in scenarios with few samples, models often struggle to accurately capture the key structural features of lesions. Existing methods largely rely on purely visual features for analysis, ignoring the rich semantic information contained in clinical descriptive texts, resulting in limited feature representation capabilities and insufficient robustness to noise interference and device domain bias.
[0003] Traditional methods are easily affected by background interference when processing ultrasound images and lack effective utilization of semantic information from clinical texts, resulting in deficiencies in feature representation and robustness, especially with a significant performance decline under conditions of few samples. Summary of the Invention
[0004] To address the shortcomings of existing technologies, this invention provides a multimodal cue fine-tuning method based on thyroid ultrasound images combined with target detection. By introducing a target detection module to locate and extract thyroid lesion regions, background interference is reduced and the expressive power of visual features is enhanced. Subsequently, the lesion image and symptom description text are jointly input into the multimodal cue fine-tuning model. Soft cue vectors are adapted to a pre-trained language model to achieve more discriminative multimodal feature representation under limited sample conditions. This approach integrates information from both image and text modalities, improving the robustness of feature learning and providing a more reliable data foundation for subsequent risk assessment.
[0005] The objective of this invention is achieved as follows: a multimodal cue fine-tuning method based on thyroid ultrasound images combined with target detection, comprising the following steps:
[0006] 1) Accurate localization of thyroid nodules is achieved by introducing a target detection module, and then high-level visual features are obtained by combining pre-trained image feature extraction technology;
[0007] 2) Employing text processing technology with soft prompts and fine-tuning enhancement, semantic mining is performed on clinical report text input to achieve efficient domain adaptation under limited ultrasound image and text data;
[0008] 3) A multimodal fusion mechanism is constructed through a multi-head attention network to integrate visual features and semantic embedding vectors, and a risk score and abnormality probability index for thyroid nodules are generated through a risk assessment module.
[0009] Furthermore, step 1) specifically includes:
[0010] Step 1.1) By preprocessing the ultrasound images using a target detection tool, the model is guided to focus on the lesion-related area and suppress irrelevant background noise, thereby achieving automatic detection and localization of the lesion area; for the predicted lesion boundary of the i-th input image... Represented as:
[0011]
[0012] in, The predicted lesion boundary of the i-th input image is represented by a polygon consisting of k vertices; each point The x and y coordinates of the j-th vertex on the lesion outline are represented, where j = 1, 2, 3, ..., k;
[0013] Step 1.2) Based on the predicted polygon By applying polygon masks, lesion regions are extracted from the original image, obtaining image data containing only lesion information; lesion region image Represented as:
[0014]
[0015] in, This represents the extracted lesion area image. For the original image, This represents element-wise multiplication. For predicting polygons Constructed indicator mask function;
[0016] Step 1.3) Obtain the cropped lesion area image The input is fed into the Beit encoder to extract high-level features, forming an expanded input representation, which in turn forms a visual feature representation for subsequent risk assessment; the Beit encoder extracts visual features from the input image. Represented as:
[0017] .
[0018] Furthermore, step 2) specifically includes:
[0019] Step 2.1) In order to capture the text information corresponding to the i-th ultrasound image, a soft cue vector is introduced. , represented as:
[0020]
[0021] Where m represents the cue vector length and d represents the embedding dimension;
[0022] and soft suggestion vectors Sample symptom description text Combined with the input to form the model, Represented as:
[0023] in, This represents the j-th word in the text, and n is the length of the text.
[0024] Step 2.2) Take the text sample The embeddings are converted through the embedding layer of the pre-trained language model and represented as follows:
[0025]
[0026] in, This represents the i-th text sample. Let n represent a space of real matrices with n rows and d columns;
[0027] Next, with soft cue vectors Dimensional concatenation is performed to form the model's input embedding matrix. , represented as:
[0028]
[0029] in, The final input embedding matrix retains the original semantic information of the symptom description text while incorporating task guidance information from soft cues, providing an input foundation for subsequent Transformer encoding.
[0030] Step 2.3) To extract the contextual dependencies contained in the input embedding matrix, the concatenated matrix is... The input is fed into the PLM Transformer pre-trained language model; the PLM Transformer pre-trained language model incorporates a cueing fine-tuning mechanism on the basis of the traditional Transformer structure, and achieves deep encoding of text semantics through iterative calculation of multi-layer multi-head self-attention and feedforward neural networks, and finally outputs a continuous high-dimensional continuous text semantic feature representation. Represented as:
[0031]
[0032] in, This represents the high-dimensional continuous text semantic feature representation extracted for the i-th input sample; This represents the parameterization function of the PLM Transformer pre-trained language model with parameter θ; θ represents the set of all trainable parameters of the PLM Transformer pre-trained language model.
[0033] Step 2.4) Employ a joint optimization strategy to ensure the medical semantic accuracy of text features: During model training, simultaneously optimize the core parameter θ of the PLM Transformer and the soft cue vector. The system is updated to minimize the text semantic encoding loss, enabling it to encode key medical information within the text description. To quantify the encoding quality of text features, a text feature semantic consistency index is introduced. Used to measure The degree of matching with the original semantics of the text is expressed as:
[0034]
[0035] MeanPool is the mean pooling operation. The closer the value is to 1, the more textual features it represents. The better the semantic encoding effect.
[0036] Furthermore, step 3) specifically includes:
[0037] Step 3.1) After extracting the multimodal features individually, perform deep fusion processing on the two types of features;
[0038] First, the semantic feature representation of high-dimensional continuous text. With visual features Perform dimensional concatenation to obtain the initial joint feature vector. , represented as:
[0039]
[0040] in, Represented as the dimension identifier of a real number vector space;
[0041] Then, the initial joint feature vector The input is fed into a multi-head attention network, which calculates attention weights between features from two different modalities to achieve cross-modal information interaction. It then combines fully connected layers with non-linear activation functions to perform dimensionality mapping and non-linear transformations on the interacted features, ultimately obtaining a unified-dimensional multi-modal fusion feature. , represented as:
[0042]
[0043] in, This is the final multimodal fusion feature vector; MHTtn is a multi-head attention operation used to calculate cross-modal attention weights and realize feature interaction; , Learnable parameters of fully connected layers; It is a non-linear activation function used to enhance the model's ability to fit complex non-linear feature relationships;
[0044] Step 3.2) Introduce fusion feature enhancement factor Optimize the discriminative power of fused features, fused feature enhancement factor Represented as:
[0045]
[0046] in, Image feature quality index;
[0047] If image features Quality indicators The semantic consistency index is greater than the corresponding text features. If visual features are increased, then the weight of visual features in the fusion process is increased; if text features are increased, then the weight of visual features in the fusion process is increased. semantic consistency Quality index greater than the corresponding image feature This increases the weight of text features, and the enhanced multimodal fusion features are used... Represented as:
[0048]
[0049] Multimodal fusion features By integrating the original fusion features with the single-modal features after dynamic weight adjustment, quantitative characterization of nodule-related structural information is provided.
[0050] Step 3.3) Enhance the fusion features The input is fed into the risk assessment module, which outputs continuous quantized values through the mapping of the multilayer perceptron (MLP), represented as:
[0051]
[0052] Among them, the output This is a risk score used to reflect the characteristics associated with nodules.
[0053] Compared with the prior art, the beneficial effects of the present invention are as follows: 1) The present invention achieves precise localization of thyroid nodules by introducing a target detection module, reducing interference caused by speckle noise, blurred boundaries and artifacts in ultrasound images, and then combines pre-trained image feature extraction technology to obtain high-level visual features to improve the extraction quality of nodule-related features;
[0054] 2) This invention employs a text processing technique that enhances text processing with soft prompts and fine-tuning to perform semantic mining on text inputs such as clinical descriptions and reports. It achieves efficient domain adaptation with limited ultrasound image and text data and can enhance semantic context extraction capabilities without extensive retraining.
[0055] 3) This invention constructs a multimodal fusion mechanism through a multi-head attention network, integrates visual features and semantic embedding vectors, and generates risk scores and abnormality probability indices for thyroid nodules through a risk assessment module to assist in clinical analysis. Attached Figure Description
[0056] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0057] Figure 1 Overall framework diagram of the present invention. Detailed Implementation
[0058] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0059] like Figure 1 The multimodal cueing fine-tuning method based on thyroid ultrasound images combined with target detection, as shown, includes the following steps:
[0060] 1) By introducing a target detection module, the precise localization of thyroid nodules is achieved, reducing interference from speckle noise, blurred boundaries and artifacts in ultrasound images. Then, combined with pre-trained image feature extraction technology, high-level visual features are obtained to improve the extraction quality of nodule-related features.
[0061] Step 1.1) By preprocessing the ultrasound images using a target detection tool, the model is guided to focus on the lesion-related area and suppress irrelevant background noise, thereby achieving automatic detection and localization of the lesion area; for the predicted lesion boundary of the i-th input image... Represented as:
[0062]
[0063] in, The predicted lesion boundary of the i-th input image is represented by a polygon consisting of k vertices; each point The x and y coordinates of the j-th vertex on the lesion outline are represented, where j = 1, 2, 3, ..., k;
[0064] Step 1.2) Based on the predicted polygon By applying polygon masks, lesion regions are extracted from the original image, obtaining image data containing only lesion information; lesion region image Represented as:
[0065]
[0066] in, This represents the extracted lesion area image. For the original image, This represents element-wise multiplication. For predicting polygons Constructed indicator mask function;
[0067] Step 1.3) Obtain the cropped lesion area image The input is fed into the Beit encoder to extract high-level features, forming an expanded input representation, which in turn forms a visual feature representation for subsequent risk assessment; the Beit encoder extracts visual features from the input image. Represented as:
[0068]
[0069] This module can flexibly adapt to ultrasound images and effectively capture discriminative visual features, thereby addressing issues such as speckle noise, blurred boundaries, and various artifacts.
[0070] 2) Employing soft-cue fine-tuning enhanced text processing technology, semantic mining is performed on clinical report text input to achieve efficient domain adaptation under limited ultrasound image and text data, and the semantic context extraction capability can be enhanced without extensive retraining;
[0071] Step 2.1) In order to capture the text information corresponding to the i-th ultrasound image, a soft cue vector is introduced. , represented as:
[0072]
[0073] Where m represents the cue vector length and d represents the embedding dimension;
[0074] and soft suggestion vectors Symptom description text Combined with the input to form the model, Represented as:
[0075]
[0076] in, This represents the j-th word in the text, and n is the length of the text.
[0077] Step 2.2) Take the text sample The embeddings are converted through the embedding layer of the pre-trained language model and represented as follows:
[0078]
[0079] in, This represents the i-th text sample. Let n represent a space of real matrices with n rows and d columns;
[0080] Next, with soft cue vectors Dimensional concatenation is performed to form the model's input embedding matrix. , represented as:
[0081]
[0082] in, The final input embedding matrix retains the original semantic information of the symptom description text while incorporating task guidance information from soft cues, providing an input foundation for subsequent Transformer encoding.
[0083] Step 2.3) To extract the contextual dependencies contained in the input embedding matrix, the concatenated matrix is... The input is fed into the PLM Transformer pre-trained language model; the PLM Transformer pre-trained language model incorporates a cueing fine-tuning mechanism on the basis of the traditional Transformer structure, and achieves deep encoding of text semantics through iterative calculation of multi-layer multi-head self-attention and feedforward neural networks, and finally outputs a high-dimensional continuous text semantic feature representation. Represented as:
[0084]
[0085] in, This represents the high-dimensional continuous text semantic feature representation extracted for the i-th input sample; This represents the parameterization function of the PLM Transformer pre-trained language model with parameter θ; θ represents the set of all trainable parameters of the PLM Transformer pre-trained language model.
[0086] Step 2.4) Employ a joint optimization strategy to ensure the medical semantic accuracy of text features: During model training, simultaneously optimize the core parameter θ of the PLM Transformer and the soft cue vector. The system is updated to minimize the text semantic encoding loss, enabling it to encode key medical information within the text description. To quantify the encoding quality of text features, a text feature semantic consistency index is introduced. Used to measure The degree of matching with the original semantics of the text is expressed as:
[0087]
[0088] MeanPool is the mean pooling operation. The closer the value is to 1, the more textual features it represents. The better the semantic encoding effect.
[0089] 3) A multimodal fusion mechanism is constructed through a multi-head attention network to integrate visual features and semantic embedding vectors, and a risk score and abnormality probability index for thyroid nodules are generated through a risk assessment module.
[0090] Step 3.1) After extracting the multimodal features individually, perform deep fusion processing on the two types of features;
[0091] First, the semantic feature representation of high-dimensional continuous text. With visual features Perform dimensional concatenation to obtain the initial joint feature vector. , represented as:
[0092]
[0093] in, Represented as the dimension identifier of a real number vector space;
[0094] Then, the initial joint feature vector The input is fed into a multi-head attention network, which calculates attention weights between features from two different modalities to achieve cross-modal information interaction. It then combines fully connected layers with non-linear activation functions to perform dimensionality mapping and non-linear transformations on the interacted features, ultimately obtaining a unified-dimensional multi-modal fusion feature. , represented as:
[0095]
[0096] in, This is the final multimodal fusion feature vector; MHTtn is a multi-head attention operation used to calculate cross-modal attention weights and realize feature interaction; , Learnable parameters of fully connected layers; It is a non-linear activation function used to enhance the model's ability to fit complex non-linear feature relationships;
[0097] Step 3.2) Introduce fusion feature enhancement factor Optimize the discriminative power of fused features, fused feature enhancement factor Represented as:
[0098]
[0099] in, For image feature quality indicators;
[0100] If image features Quality indicators The semantic consistency index is greater than the corresponding text features. If visual features are increased, then the weight of visual features in the fusion process is increased; if text features are increased, then the weight of visual features in the fusion process is increased. semantic consistency Quality index greater than the corresponding image feature This increases the weight of text features, and the enhanced multimodal fusion features are used... Represented as:
[0101]
[0102] Multimodal fusion features By integrating the original fusion features with the single-modal features after dynamic weight adjustment, quantitative characterization of nodule-related structural information is provided.
[0103] Step 3.3) Enhance the fusion features The input is fed into the risk assessment module, which outputs continuous quantized values through the mapping of the multilayer perceptron (MLP), represented as:
[0104]
[0105] Among them, the output This is a risk score used to reflect the characteristics associated with nodules.
[0106] To test the effectiveness of this invention, experiments were conducted on two thyroid datasets, DDTI and Thyroid-YZU. To ensure a true measure of the evaluation results, four representative evaluation metrics were selected: Accuracy, Precision, Recall, and F1Score, defined as follows:
[0107]
[0108]
[0109]
[0110]
[0111] To verify the effectiveness of the proposed model, we systematically compared it with various baseline methods and representative methods for medical images, including image processing methods: AlexNet, ResNet, and InceptionV3; multimodal large language model methods (M-LLMs): BaitRadar, Paligemma, and OTE; and medical multimodal large language model method: RexNet_150. These methods cover a variety of technical approaches, from traditional convolutional neural networks to cross-modal feature fusion and deep semantic reasoning, enabling a comprehensive evaluation of the proposed method's performance in visual feature extraction, multimodal feature interaction, and adaptability to medical scenarios. The prediction evaluation results are shown in Table 1.
[0112] Table 1. Experimental results on the two datasets
[0113]
[0114] Compared to traditional unimodal methods (such as AlexNet, ResNet, and InceptionV3), this invention outperforms traditional methods in feature extraction and task adaptation, maintaining high feature learning quality even with limited sample sizes. While InceptionV3 performs well in medical image tasks, it struggles to fully utilize textual semantic information for joint modeling. Compared to multimodal large language models (such as BaitRadar, Paliemma, and OTE), this invention is more robust to ultrasound image noise, blurred boundaries, and artifact interference, exhibiting higher stability of fused features. Compared to medical multimodal models (such as RexNet_150), this invention improves the accuracy of lesion region information extraction by introducing lesion region target detection and an improved image-text feature fusion method, resulting in a more reliable risk score.
[0115] Experimental results show that the proposed method exhibits consistent performance advantages across different training sample sizes. Even with limited samples, it maintains near-saturation classification performance, demonstrating low dependence on labeled data and strong generalization ability. Standard deviation analysis across multiple experiments reveals that the proposed method exhibits smaller performance fluctuations across various datasets and sample settings, indicating better stability and reproducibility in ultrasound risk assessment tasks.
[0116] Experimental results show that the proposed method exhibits consistent performance advantages across different training sample sizes. Even with limited samples, it maintains near-saturated feature modeling capabilities, demonstrating low dependence on labeled data and strong generalization ability. Standard deviation analysis across multiple experiments reveals that the proposed method exhibits minimal performance fluctuations across various datasets and sample settings, indicating better stability and reproducibility in ultrasound image correlation analysis tasks.
[0117] The above description of the embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. It should be noted that those skilled in the art can make several improvements and modifications to the present invention without departing from the principles of the present invention, and these improvements and modifications also fall within the protection scope of the claims of the present invention.
Claims
1. A multimodal cueing fine-tuning method based on thyroid ultrasound images combined with target detection, characterized in that, Includes the following steps: 1) Accurate localization of thyroid nodules is achieved by introducing a target detection module, and then high-level visual features are obtained by combining pre-trained image feature extraction technology; 2) Employing text processing technology with soft prompts and fine-tuning enhancement, semantic mining is performed on clinical report text input to achieve efficient domain adaptation under limited ultrasound image and text data; 3) A multimodal fusion mechanism is constructed through a multi-head attention network to integrate visual features and semantic embedding vectors, and a risk score and abnormality probability index for thyroid nodules are generated through a risk assessment module.
2. The multimodal cueing fine-tuning method based on thyroid ultrasound images combined with target detection according to claim 1, characterized in that, Step 1) specifically includes: Step 1.1) By preprocessing the ultrasound images using a target detection tool, the model is guided to focus on the lesion-related area and suppress irrelevant background noise, thereby achieving automatic detection and localization of the lesion area; for the predicted lesion boundary of the i-th input image... Represented as: ； in, The predicted lesion boundary of the i-th input image is represented by a polygon consisting of k vertices; each point The x and y coordinates of the j-th vertex on the lesion outline are represented, where j = 1, 2, 3, ..., k; Step 1.2) Based on the predicted polygon By applying polygon masks, lesion regions are extracted from the original image, obtaining image data containing only lesion information; lesion region image Represented as: ； in, This represents the extracted lesion area image. For the original image, This represents element-wise multiplication. For predicting polygons Constructed indicator mask function; Step 1.3) Obtain the cropped lesion area image The input is fed into the Beit encoder to extract high-level features, forming an expanded input representation, which in turn forms a visual feature representation for subsequent risk assessment; the Beit encoder extracts visual features from the input image. Represented as: 。 3. The multimodal cueing fine-tuning method based on thyroid ultrasound images combined with target detection according to claim 1, characterized in that, Step 2) specifically includes: Step 2.1) In order to capture the text information corresponding to the i-th ultrasound image, a soft cue vector is introduced. , represented as: ； Where m represents the cue vector length and d represents the embedding dimension; and soft suggestion vectors Sample symptom description text Combined with the input to form the model, Represented as: ； in, This represents the j-th word in the text, and n is the length of the text. Step 2.2) Take the text sample The embeddings are converted through the embedding layer of the pre-trained language model and represented as follows: ； in, This represents the i-th text sample. Let n represent a space of real matrices with n rows and d columns; Next, with soft cue vectors Dimensional concatenation is performed to form the model's input embedding matrix. , represented as: ； in, The final input embedding matrix retains the original semantic information of the symptom description text while incorporating task guidance information from soft cues, providing an input foundation for subsequent Transformer encoding. Step 2.3) involves extracting the contextual dependencies inherent in the input embedding matrix, and then concatenating the... The input is fed into the PLMTransformer pre-trained language model; the PLMTransformer pre-trained language model incorporates a cueing fine-tuning mechanism on the basis of the traditional Transformer structure, and achieves deep encoding of text semantics through iterative calculation of multi-layer multi-head self-attention and feedforward neural networks, and finally outputs a high-dimensional continuous text semantic feature representation. Represented as: ； in, This represents the high-dimensional continuous text semantic feature representation extracted for the i-th input sample; This represents the parameterization function of the PLM Transformer pre-trained language model with parameter θ; θ represents the set of all trainable parameters of the PLM Transformer pre-trained language model. Step 2.4) Employ a joint optimization strategy to ensure the medical semantic accuracy of text features: During model training, simultaneously optimize the core parameter θ of the PLM Transformer and the soft cue vector. The system is updated to minimize the text semantic encoding loss, enabling it to encode key medical information within the text description. To quantify the encoding quality of text features, a text feature semantic consistency index is introduced. Used to measure The degree of matching with the original semantics of the text is expressed as: ； MeanPool is the mean pooling operation. The closer the value is to 1, the more textual features it represents. The better the semantic encoding effect.
4. The multimodal cueing fine-tuning method based on thyroid ultrasound images combined with target detection according to claim 1, characterized in that, Step 3) specifically includes: Step 3.1) After extracting the multimodal features individually, perform deep fusion processing on the two types of features; First, the semantic feature representation of high-dimensional continuous text. With visual features Perform dimensional concatenation to obtain the initial joint feature vector. , represented as: ； in, Represented as the dimension identifier of a real number vector space; Then, the initial joint feature vector The input is fed into a multi-head attention network, which calculates attention weights between features from two different modalities to achieve cross-modal information interaction. Combined with a fully connected layer with a non-linear activation function, the network performs dimensionality mapping and non-linear transformation on the interacted features, ultimately yielding a unified-dimensional multi-modal fusion feature. , represented as: ； in, This is the final multimodal fusion feature vector; MHTtn is a multi-head attention operation used to calculate cross-modal attention weights and realize feature interaction; , Learnable parameters of fully connected layers; It is a non-linear activation function used to enhance the model's ability to fit complex non-linear feature relationships; Step 3.2) Introduce fusion feature enhancement factor Optimize the discriminative power of fused features, fused feature enhancement factor Represented as: ； in, Image feature quality index; If image features Quality indicators The semantic consistency index is greater than the corresponding text features. If visual features are increased, then the weight of visual features in the fusion process is increased; if text features are increased, then the weight of visual features in the fusion process is increased. semantic consistency Quality index greater than the corresponding image feature This increases the weight of text features, and the enhanced multimodal fusion features are used... Represented as: ； Multimodal fusion features By integrating the original fusion features with the single-modal features after dynamic weight adjustment, quantitative characterization of nodule-related structural information is provided. Step 3.3) Enhance the fusion features The input is fed into the risk assessment module, which outputs continuous quantized values through the mapping of the multilayer perceptron (MLP), represented as: ； Among them, the output This is a risk score used to reflect the characteristics associated with nodules.