Intelligent identification method and system for thyroid nodules based on deep learning

Abstract

The present application relates to the technical field of medical image processing, and more particularly to a thyroid nodule intelligent identification method and system based on deep learning, which acquires thyroid ultrasound images, performs region segmentation to locate nodule target regions, constructs multi-dimensional image feature expression, generates joint feature representation in a unified semantic space through a cross-modal feature alignment framework, inputs the joint feature representation into a classification network to obtain preliminary diagnosis results and confidence distribution, establishes a dynamic uncertainty quantification mechanism to perform feature enhancement and dynamically reshape the classification network decision boundary on difficult samples with classification uncertainty through adversarial sample generation to obtain a revised diagnosis result, and outputs the final diagnosis result and decision basis visualization information, and the present application improves the accuracy and reliability of thyroid nodule identification through cross-modal feature fusion and uncertainty-driven decision boundary optimization.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and in particular to a method and system for intelligent identification of thyroid nodules based on deep learning. Background Technology

[0002] Thyroid ultrasound is an important tool for clinical screening of benign and malignant thyroid nodules. With the development of deep learning technology, intelligent identification methods based on models such as convolutional neural networks are gradually being applied to assist in diagnosis. Current conventional practices typically begin by preprocessing ultrasound images and locating the target nodule region using image segmentation algorithms. Then, a pre-trained deep neural network is used to extract multidimensional image features of the nodules, such as texture, edges, and grayscale distribution. Building on this, some methods incorporate patient clinical history information, including structured indicators such as age, gender, number of nodules, and blood flow signals. These image features are combined with clinical features through feature concatenation or weighted fusion, and finally input into a classifier (such as a fully connected layer or support vector machine) to output the benign / malignant classification result. While these frameworks have achieved good average accuracy on publicly available datasets, they still have significant limitations in practical applications.

[0003] One of the main drawbacks of existing conventional methods is the lack of an effective mechanism for handling classification uncertainty. When the imaging appearance of a nodule is blurry, the boundaries are unclear, or the contrast with surrounding tissue is low, the probability distribution output by the classifier often shows little difference between benign and malignant categories. In such cases, directly using the category with the highest probability as the final result can easily lead to misjudgment. In clinical diagnosis, these difficult samples are precisely the ones that doctors need to focus on, but traditional methods cannot automatically identify and adjust the decision-making logic, resulting in a decrease in the reliability of the system in high-risk scenarios.

[0004] Another significant drawback is the overly simplistic cross-modal feature fusion approach, which fails to fully explore the intrinsic connections between imaging features and clinical semantic features. Direct stitching or post-fusion is insufficient to bridge the semantic gap between modalities, and the potential representations contained in clinical history (such as thyroid function indicators, patient past medical history, and other abstract information) are difficult to effectively correspond to the spatial structural features of the images. This shallow fusion not only introduces noise but also limits the model's ability to discriminate complex cases, especially in cases with small nodules or atypical features, where performance improvement is extremely limited. Summary of the Invention

[0005] This invention provides a method and system for intelligent identification of thyroid nodules based on deep learning, which can solve the problems in the prior art.

[0006] A first aspect of this invention provides a deep learning-based intelligent identification method for thyroid nodules, comprising: Acquire ultrasound image data of the thyroid gland to be detected, perform regional segmentation on the ultrasound image data to locate the target region of the nodule, and construct a multidimensional image feature representation of the target region of the nodule. A cross-modal feature alignment framework is constructed. By establishing a mapping relationship between the image domain feature space and the clinical semantic feature space, the multidimensional image feature expression of the nodule target region and the patient's clinical history features are semantically coordinated and reconstructed in a unified semantic space to generate a joint feature representation containing information related to image phenotype and clinical phenotype. The joint feature representation is input into a classification network, a preliminary diagnosis result is generated according to the benign / malignant discrimination rule, and the confidence distribution corresponding to the preliminary diagnosis result is output. A dynamic uncertainty quantification mechanism is established. By evaluating the boundary ambiguity of the probability differences between multiple candidate categories in the confidence distribution, difficult samples with uncertain classification decisions are located. For the difficult samples, a feature enhancement strategy based on adversarial sample generation is triggered to discriminately enhance the joint feature representation. The decision boundary of the classification network is dynamically reshaped using the discriminatively enhanced joint feature representation to generate a corrected diagnostic result. The corrected diagnostic result is output as the final identification result, and the decision basis visualization information corresponding to the final identification result is generated.

[0007] The process of performing region segmentation on the thyroid ultrasound image data to locate the target nodule region and constructing a multidimensional image feature representation of the target nodule region includes: An adaptive boundary-aware region segmentation module is constructed. By establishing a constraint relationship between pixel-level gradient response and local texture consistency in thyroid ultrasound image data, preliminary segmentation results of nodule candidate regions and their corresponding boundary confidence distributions are generated. Based on the boundary confidence distribution, the blurred boundary regions in the preliminary segmentation results are refined and corrected. By introducing morphological prior constraints, the boundary contours are topologically optimized to generate the accurate segmentation boundary of the nodule target region. A multi-level feature decoupling and recombination framework is established. The precise segmentation boundary of the nodule target region is used as the spatial positioning benchmark. The morphological feature dimension, echo feature dimension and blood flow distribution feature dimension are decoupled and separated within the nodule target region. The decoupled features of each dimension are then recombined in a structured manner through cross-dimensional semantic association to generate a multi-dimensional image feature expression.

[0008] A cross-modal feature alignment framework is constructed. By establishing a mapping relationship between the image domain feature space and the clinical semantic feature space, the multidimensional image feature expression of the nodule target region and the patient's clinical history features are semantically coordinated and reconstructed in a unified semantic space, generating a joint feature representation containing information on the association between image phenotype and clinical phenotype, including: The multidimensional image feature representation of the target region of the nodule is projected onto a unified semantic space through semantic embedding transformation to generate a semantic vector representation of the image phenotype. At the same time, the patient's clinical history features are semantically encoded, and clinical diagnostic information, laboratory test indicators and medical history records are mapped to the unified semantic space to generate a semantic vector representation of the clinical phenotype. In the unified semantic space, the cross-modal semantic similarity distribution between the semantic vector representation of the image phenotype and the semantic vector representation of the clinical phenotype is calculated. Based on the cross-modal semantic similarity distribution, the semantic correspondence between the image feature dimension and the clinical feature dimension is identified, and the association weight of the semantic correspondence is quantified. Based on the association weights of the semantic correspondence, the semantic vector representations of the image phenotype and the clinical phenotype are complementaryly integrated. Through vector recombination driven by association weights, deep coupling of cross-modal features is achieved, generating a joint feature representation.

[0009] Based on the association weights of the semantic correspondence, the semantic vector representations of the image phenotype and the clinical phenotype are complementaryly integrated. Deep coupling of cross-modal features is achieved through vector recombination driven by association weights, generating a joint feature representation including: A cross-modal complementarity metric function is constructed based on the association weights of semantic correspondences. The information gain relationship between each feature dimension in the semantic vector representation of the image phenotype and each feature dimension in the semantic vector representation of the clinical phenotype is calculated through the cross-modal complementarity metric function, and a feature dimension-level complementary weight distribution is generated. Based on the complementary weight distribution, a weighted vector concatenation is performed between the semantic vector representation of the image phenotype and the semantic vector representation of the clinical phenotype. By using the complementary weight distribution as a concatenation coefficient to adjust the expression intensity of each feature dimension in the concatenated vector, a preliminary integrated vector is generated. The initial integrated vector is subjected to a nonlinear transformation of the semantic space. By introducing the complementary weight distribution as a regularization constraint during the transformation process, the semantic correspondence structure between cross-modal features is maintained, thereby generating a joint feature representation.

[0010] A dynamic uncertainty quantification mechanism is established. By evaluating the boundary ambiguity of the probability differences among multiple candidate categories in the confidence distribution, the problematic samples with uncertain classification decisions are identified, including: A dynamic uncertainty quantification mechanism based on probability difference evolution tracking is established. By continuously measuring the probability differences between multiple candidate categories in the confidence distribution at multiple decision times, a temporal evolution trajectory reflecting the probability differences as the decision progresses is constructed, and a dynamic evaluation matrix of boundary ambiguity is generated based on the fluctuation pattern of the temporal evolution trajectory. By using the adaptive weight allocation strategy in the dynamic uncertainty quantification mechanism, differentiated weights are assigned to the boundary ambiguity evaluation values ​​at different times based on the ambiguity change rate at different times in the dynamic evaluation matrix of boundary ambiguity, thereby generating a weighted and fused comprehensive uncertainty quantification index. Based on the combined uncertainty quantification index and the fluctuation amplitude of the time series evolution trajectory, a joint judgment is made. By identifying samples where the combined uncertainty quantification index exceeds a preset uncertainty threshold and the fluctuation amplitude of the time series evolution trajectory exceeds a preset fluctuation threshold, difficult samples are located.

[0011] A dynamic uncertainty quantification mechanism based on probability difference evolution tracking is established. By continuously measuring the probability differences between multiple candidate categories in the confidence distribution at multiple decision times, a temporal evolution trajectory reflecting the probability differences as the decision progresses is constructed, including: Multiple decision-making sampling points are set at different stages of classification decision-making. By extracting the probability values ​​of multiple candidate categories from the confidence distribution at each decision-making sampling point, the probability interval between any two candidate categories is calculated, and the probability difference measurement value corresponding to each decision-making sampling point is generated. A time-series correlation analysis is performed on the measured probability difference values. By calculating the amount and rate of change of the measured probability difference values ​​between sampling points at adjacent decision times, time-series feature parameters reflecting the evolution of the probability competition relationship of candidate categories are extracted, and the evolution rate and trend of probability difference are quantified based on the time-series feature parameters. By using the probability difference measurement values ​​of sampling points at each decision moment as trajectory nodes, the evolution rate as the connection strength between trajectory nodes, and the evolution trend as the directional constraint of the trajectory, a temporal evolution trajectory reflecting the probability difference as the decision progresses is generated.

[0012] For the challenging samples, a feature enhancement strategy based on adversarial sample generation is triggered to discriminately enhance the joint feature representation. The decision boundary of the classification network is dynamically reshaped using the discriminatively enhanced joint feature representation to generate a corrected diagnostic result, including: For the joint feature representation of difficult samples, an adversarial perturbation field is generated in the feature space. The gradient distribution of the adversarial perturbation field is constructed by calculating the perturbation direction that causes the maximum change in the decision confidence of the classification network, and an adversarial enhanced feature representation containing decision-sensitive features is generated based on the gradient distribution. A comparative analysis of the feature space of the adversarial enhanced feature representation and the original joint feature representation is performed. By extracting the feature dimensions in the adversarial enhanced feature representation that cause the decision boundary to shift, a discriminative enhancement vector for the difficult sample is constructed. The discriminative enhancement vector is then fused into the joint feature representation to generate a discriminatively enhanced joint feature representation. The discriminatively enhanced joint feature representation is used as the constraint input for reshaping the decision boundary of the classification network. The distribution characteristics of the discriminatively enhanced joint feature representation in the feature space are used to adjust the decision boundary parameters of the classification network, thereby completing the dynamic reshaping of the decision boundary. Based on the reshaped decision boundary, the difficult samples are reclassified to generate a corrected diagnostic result.

[0013] A second aspect of this invention provides a deep learning-based intelligent identification system for thyroid nodules, comprising: The image segmentation unit is used to acquire ultrasound image data of the thyroid gland to be detected, perform regional segmentation on the ultrasound image data of the thyroid gland to locate the target region of the nodule, and construct a multidimensional image feature representation of the target region of the nodule. The feature alignment unit is used to construct a cross-modal feature alignment framework. By establishing a mapping relationship between the image domain feature space and the clinical semantic feature space, the multidimensional image feature expression of the nodule target region and the patient's clinical history features are semantically coordinated and reconstructed in a unified semantic space to generate a joint feature representation containing information related to image phenotype and clinical phenotype. The classification and discrimination unit is used to input the joint feature representation into the classification network, generate a preliminary diagnosis result according to the benign and malignant discrimination rules, and output the confidence distribution corresponding to the preliminary diagnosis result; The dynamic correction unit is used to establish a dynamic uncertainty quantification mechanism. By evaluating the boundary ambiguity of the probability differences between multiple candidate categories in the confidence distribution, it locates the difficult samples with uncertainty in classification decision. For the difficult samples, it triggers a feature enhancement strategy based on adversarial sample generation to discriminately enhance the joint feature representation. The decision boundary of the classification network is dynamically reshaped using the discriminatively enhanced joint feature representation to generate a corrected diagnostic result. The result output unit is used to output the corrected diagnostic result as the final identification result and generate the decision basis visualization information corresponding to the final identification result.

[0014] A third aspect of the present invention provides an electronic device, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0015] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0016] A deep learning-based intelligent identification method for thyroid nodules significantly improves the accuracy and robustness of nodule benign / malignant differentiation. It performs fine-grained region segmentation on thyroid ultrasound images, accurately locates the target nodule region, and constructs multi-dimensional image feature representations, fully extracting key imaging information such as morphology, boundaries, internal echoes, and calcifications. A cross-modal feature alignment framework establishes a mapping relationship between the image domain feature space and the clinical semantic feature space, semantically coordinating and reconstructing the representation of nodule imaging features and patient clinical history features in a unified semantic space. The generated joint feature representation deeply integrates the correlation information between imaging phenotype and clinical phenotype, greatly enhancing the ability to distinguish complex and atypical lesions.

[0017] The classification network takes the joint feature representation as input and generates preliminary diagnostic results and confidence distributions based on benign / malignant discrimination rules. A dynamic uncertainty quantification mechanism accurately identifies problematic samples with uncertain classification decisions by assessing the boundary ambiguity of probability differences between multiple candidate categories in the confidence distribution. For these samples, a feature enhancement strategy based on adversarial sample generation discriminatively enhances the joint feature representation, effectively strengthening the feature's sensitivity to high-risk lesions. By dynamically reshaping the decision boundary of the classification network, the corrected diagnostic results significantly reduce false negative and false positive rates, thus decreasing the risk of missed and misdiagnosed lesions in clinical practice. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating the intelligent identification method for thyroid nodules based on deep learning, as described in an embodiment of the present invention. Figure 2 This is a flowchart of dynamic uncertainty quantification based on probability difference evolution tracking in an embodiment of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0021] Figure 1 This is a flowchart illustrating the intelligent identification method for thyroid nodules based on deep learning, as described in an embodiment of the present invention.

[0022] Deep learning-based intelligent identification methods for thyroid nodules include: Acquire ultrasound image data of the thyroid gland to be detected, perform regional segmentation on the ultrasound image data to locate the target region of the nodule, and construct a multidimensional image feature representation of the target region of the nodule. A cross-modal feature alignment framework is constructed. By establishing a mapping relationship between the image domain feature space and the clinical semantic feature space, the multidimensional image feature expression of the nodule target region and the patient's clinical history features are semantically coordinated and reconstructed in a unified semantic space to generate a joint feature representation containing information related to image phenotype and clinical phenotype. The joint feature representation is input into a classification network, a preliminary diagnosis result is generated according to the benign / malignant discrimination rule, and the confidence distribution corresponding to the preliminary diagnosis result is output. A dynamic uncertainty quantification mechanism is established. By evaluating the boundary ambiguity of the probability differences between multiple candidate categories in the confidence distribution, difficult samples with uncertain classification decisions are located. For the difficult samples, a feature enhancement strategy based on adversarial sample generation is triggered to discriminately enhance the joint feature representation. The decision boundary of the classification network is dynamically reshaped using the discriminatively enhanced joint feature representation to generate a corrected diagnostic result. The corrected diagnostic result is output as the final identification result, and the decision basis visualization information corresponding to the final identification result is generated.

[0023] In one optional implementation, the thyroid ultrasound image data is segmented to locate the target nodule region, and a multidimensional image feature representation of the target nodule region is constructed, including: An adaptive boundary-aware region segmentation module is constructed. By establishing a constraint relationship between pixel-level gradient response and local texture consistency in thyroid ultrasound image data, preliminary segmentation results of nodule candidate regions and their corresponding boundary confidence distributions are generated. Based on the boundary confidence distribution, the blurred boundary regions in the preliminary segmentation results are refined and corrected. By introducing morphological prior constraints, the boundary contours are topologically optimized to generate the accurate segmentation boundary of the nodule target region. A multi-level feature decoupling and recombination framework is established. The precise segmentation boundary of the nodule target region is used as the spatial positioning benchmark. The morphological feature dimension, echo feature dimension and blood flow distribution feature dimension are decoupled and separated within the nodule target region. The decoupled features of each dimension are then recombined in a structured manner through cross-dimensional semantic association to generate a multi-dimensional image feature expression.

[0024] After acquiring the ultrasound image data of the thyroid gland to be examined, it is necessary to accurately locate and extract features from the nodule region in the image. Thyroid ultrasound images are usually presented as grayscale images, in which the nodule region differs from the surrounding normal thyroid tissue in terms of echo intensity, texture pattern, and boundary clarity. However, due to problems such as speckle noise, acoustic artifacts, and blurred transition areas between the nodule boundary and the surrounding tissue during ultrasound imaging, it is difficult to obtain accurate nodule contours by directly using traditional segmentation methods.

[0025] To address the aforementioned issues, an adaptive boundary-aware region segmentation module is constructed. The core of this module lies in establishing a constraint relationship between pixel-level gradient response and local texture consistency. Specifically, multi-scale gradient calculations are performed on the input thyroid ultrasound image data. The Sobel or Scharr operator is applied in the horizontal and vertical directions, respectively, to obtain the gradient magnitude and direction at each pixel location. Regions with larger gradient magnitudes typically correspond to nodule boundaries, but relying solely on gradient information is susceptible to noise interference. Therefore, a local texture consistency index is calculated simultaneously. The gray-level co-occurrence matrix is ​​used to extract texture features such as contrast, correlation, energy, and homogeneity within the local window. Texture continuity is evaluated by analyzing the gray-level distribution relationship between adjacent pixels.

[0026] When establishing constraints, gradient response is used as a preliminary indicator of boundary candidates, and texture consistency is used as a verification criterion for the homogeneity within a region. For pixels with gradient magnitudes exceeding an adaptive threshold, the texture consistency within their neighborhood is checked. If there are significant differences in texture features on both sides of the pixel, it is marked as a reliable boundary point; if the texture features are similar, the gradient response is considered to be caused by noise, and its boundary confidence is reduced. Through this dual constraint mechanism, preliminary segmentation results of nodule candidate regions are generated. This preliminary segmentation result is represented in the form of a binary mask, where pixels with a value of 1 belong to the nodule region, and pixels with a value of 0 belong to the background region. Simultaneously, a boundary confidence score is calculated for each boundary pixel. This confidence score comprehensively considers gradient intensity, texture difference, and local boundary continuity, and its value ranges from 0 to 1. The closer the value is to 1, the higher the confidence that the location is a true boundary.

[0027] Preliminary segmentation results often exhibit ambiguity in certain areas of the nodule boundary, particularly in regions where the echo difference between the nodule and surrounding tissue is small, or in areas with complex internal structures such as calcification or cystic changes. Based on the boundary confidence distribution, regions with ambiguous boundaries whose confidence is below a preset threshold are identified. These regions undergo refined correction using a region growing strategy. High-confidence boundary points are used as seed points, and the boundary position is gradually adjusted by expanding outward or contracting inward based on the gray-level similarity and texture continuity of adjacent pixels. During the adjustment process, morphological prior constraints are introduced, utilizing the medical knowledge that thyroid nodules typically present as elliptical or near-circular shapes to perform topological optimization of the segmentation boundaries.

[0028] The implementation of morphological prior constraints includes the following operations: First, morphological closing operations are performed on the preliminary segmentation results to fill small holes and fractures inside the nodules and eliminate pseudo-voids caused by noise or internal uneven echoes; second, morphological opening operations are performed to remove small protrusions and burrs on the outer side of the boundary and smooth the boundary contour; then, the convex hull of the segmented region is calculated, and for boundary depressions with a depth exceeding a reasonable range, appropriate filling is performed according to the shape of the convex hull to make the boundary more consistent with the natural shape of the nodule. During the topology optimization process, it is also necessary to detect and process existing multi-connected regions. If multiple independent connected components appear in the segmentation results, the main nodule region is retained and isolated noise regions are removed based on the area, positional relationship, and degree of consistency with clinical priors of each connected component. After the above fine-tuning and topology optimization, an accurate segmentation boundary of the nodule target region is generated. This boundary is represented in the form of a pixel coordinate sequence or polygonal contour, which can accurately describe the spatial range and geometric shape of the nodule.

[0029] After obtaining precise segmentation boundaries, multidimensional imaging features need to be extracted from the target nodule region. Thyroid nodules exhibit multifaceted imaging characteristics, and single-dimensional features often fail to fully reflect the pathological features of the nodule. A multi-level feature decoupling and reconstructing framework is established, using precise segmentation boundaries as a spatial positioning benchmark to ensure that subsequent feature extraction is performed only within the nodule region, avoiding interference from background tissue.

[0030] Morphological features reflect the geometric shape and boundary characteristics of nodules. The area, perimeter, major axis length, and minor axis length of the nodule region are calculated, yielding shape descriptors such as roundness, aspect ratio, and compactness. Roundness is defined as the ratio of the region's area to the area of ​​its circumcircle; a value close to 1 indicates the nodule is nearly circular. Compactness is defined as the ratio of the square of the perimeter to the area, reflecting the complexity of the boundary. Furthermore, the smoothness of the boundary is analyzed, quantifying the irregularity by calculating the standard deviation of the boundary curvature variation. Malignant nodules often have irregular boundaries with drastic curvature changes. The directional features of the nodule are also extracted; principal component analysis is used to calculate the principal axis direction of the nodule region, describing the spatial orientation of the nodule.

[0031] Echo feature dimension reflects the gray-level distribution and texture pattern within the nodule. The mean, standard deviation, skewness, and kurtosis of pixel gray-level values ​​within the nodule region are statistically analyzed to describe the overall level and distribution characteristics of echo intensity. Hypoechoic, isoechoic, hyperechoic, and mixed echogenicity are important distinguishing features of thyroid nodules. Further texture features are extracted, and gray-level co-occurrence matrix is ​​used to calculate statistics such as contrast, correlation, energy, and homogeneity. Gray-level run-length matrix is ​​used to calculate indices such as short-run dominance, long-run dominance, and gray-level non-uniformity. Local binary mode is used to statistically analyze the microstructure of the texture. These texture features can quantify the uniformity, roughness, and directionality of echoes within the nodule, providing important evidence for the differentiation between benign and malignant nodules.

[0032] Blood flow distribution characteristics reflect the blood supply to the nodule. In color Doppler ultrasound images, blood flow signals are color-coded and superimposed on a grayscale image. Color pixels within the nodule region are extracted, and the distribution area of ​​the blood flow signal, the proportion of the blood flow signal to the nodule area, and the spatial distribution pattern of the blood flow signal are statistically analyzed. Based on the location of the blood flow signal, the distribution is classified into peripheral, central, and mixed types. Peripheral blood flow is more common in benign nodules, while central blood flow and abundant blood flow suggest an increased likelihood of malignancy. The density distribution of the blood flow signal is calculated, and the richness of blood flow is quantified by analyzing the degree of aggregation and dispersion of blood flow pixels. For images with spectral Doppler data, hemodynamic parameters such as blood flow velocity and drag index can also be extracted.

[0033] The three dimensions of features mentioned above are extracted independently, forming a decoupled feature set. However, there are inherent semantic relationships between these feature dimensions. For example, nodules with irregular boundaries are often accompanied by heterogeneous internal echoes and abnormal blood flow distribution. By establishing cross-dimensional semantic relationships, association patterns between different feature dimensions are established. The correlation between morphological features and echo features is calculated to analyze the correspondence between boundary characteristics and internal texture; the synergy between echo features and blood flow features is calculated to analyze the spatial overlap between hypoechoic regions and blood flow signals. Based on these relationships, the decoupled features of each dimension are structurally reorganized to construct a multi-dimensional image feature representation in the form of feature vectors or feature matrices. This feature representation not only includes independent features of each dimension but also integrates cross-dimensional association information, enabling a more comprehensive description of the nodule's image phenotype and providing rich input information for subsequent cross-modal feature alignment and classification diagnosis.

[0034] In one optional implementation, a cross-modal feature alignment framework is constructed. By establishing a mapping relationship between the image domain feature space and the clinical semantic feature space, the multidimensional image feature expression of the nodule target region and the patient's clinical history features are semantically coordinated and reconstructed in a unified semantic space, generating a joint feature representation containing information on the association between image phenotype and clinical phenotype, including: The multidimensional image feature representation of the target region of the nodule is projected onto a unified semantic space through semantic embedding transformation to generate a semantic vector representation of the image phenotype. At the same time, the patient's clinical history features are semantically encoded, and clinical diagnostic information, laboratory test indicators and medical history records are mapped to the unified semantic space to generate a semantic vector representation of the clinical phenotype. In the unified semantic space, the cross-modal semantic similarity distribution between the semantic vector representation of the image phenotype and the semantic vector representation of the clinical phenotype is calculated. Based on the cross-modal semantic similarity distribution, the semantic correspondence between the image feature dimension and the clinical feature dimension is identified, and the association weight of the semantic correspondence is quantified. Based on the association weights of the semantic correspondence, the semantic vector representations of the image phenotype and the clinical phenotype are complementaryly integrated. Through vector recombination driven by association weights, deep coupling of cross-modal features is achieved, generating a joint feature representation.

[0035] After obtaining the multidimensional imaging features of the target nodule region, it is necessary to deeply integrate the imaging information with the patient's clinical history to fully utilize multi-source heterogeneous data and improve diagnostic accuracy. Since imaging features and clinical features belong to different data modalities, their original representation spaces exhibit significant distributional differences and semantic gaps, making it difficult to capture the intrinsic relationship between the two through direct feature stitching or simple fusion. Therefore, a cross-modal feature alignment framework is constructed to achieve collaborative representation of imaging phenotypes and clinical phenotypes within a unified semantic space.

[0036] To address the multidimensional image feature representation of the nodule target region, a semantic embedding transformation is used to project it onto a unified semantic space. Multidimensional image feature representations typically include texture features, morphological features, edge features, and high-level semantic features extracted by deep convolutional networks, with dimensions ranging from hundreds to thousands of dimensions. The semantic embedding transformation employs a feature mapping module driven by fully connected neural network layers or an attention mechanism to map high-dimensional image features into fixed-dimensional semantic vectors. Specifically, the unified semantic space is set to 512 or 1024 dimensions. Through a combination of learnable linear transformation matrices and nonlinear activation functions, the original image features are progressively compressed, and their semantic essence is extracted. During the mapping process, key information related to nodule benignity / malignancy is retained, such as calcification patterns, boundary clarity, and internal echo homogeneity, generating a semantic vector representation of the image phenotype. This semantic vector not only contains the visual features of the image but also implicitly contains the pathological significance of the nodule.

[0037] Simultaneously, the patient's clinical history features are semantically encoded. These features encompass multiple dimensions, including the patient's age, gender, family history, history of thyroid disease, thyroid function test results such as thyroid-stimulating hormone (TSH) levels, free triiodothyronine (TTI) levels, and free thyroxine (FTT) levels, as well as laboratory test indicators such as thyroglobulin antibodies and thyroid peroxidase antibodies. This clinical information exists in the form of structured data or text descriptions and requires unified numerical encoding. For continuous numerical indicators such as age and hormone levels, standardization is performed before direct use as feature input. For categorical indicators such as gender and family history, one-hot encoding or embedding vector representation is used. For textual medical history records, natural language processing techniques are used to extract key clinical terms, and pre-trained medical text encoding models such as BioBERT or ClinicalBERT convert the text into semantic vectors. These multidimensional clinical features are integrated into a unified clinical feature vector through a feature fusion layer. This vector is then passed through a semantic mapping network of the same dimension as the imaging features, mapping clinical diagnostic information, laboratory test indicators, and medical history records to a unified semantic space, generating a semantic vector representation of the clinical phenotype. This semantic vector captures the patient's clinical risk factors and physiological state characteristics.

[0038] In a unified semantic space, the semantic vectors of imaging phenotypes and clinical phenotypes are comparable. The cross-modal semantic similarity distribution between the two is calculated, and the semantic closeness between the vectors is assessed using a cosine similarity metric or a bilinear similarity function. Specifically, the similarity between the imaging semantic vectors and clinical semantic vectors is calculated dimension-by-dimensionally to generate a similarity matrix. Each element of this matrix reflects the strength of the association between a certain semantic dimension of an imaging feature and a certain semantic dimension of a clinical feature. By analyzing the similarity matrix, the semantic correspondence between the dimensions of imaging features and the dimensions of clinical features is identified. For example, the dimension representing irregular nodule boundaries in imaging features has a high semantic similarity to the dimension representing thyroid dysfunction in clinical features, suggesting a correlation in their pathological mechanisms.

[0039] Based on cross-modal semantic similarity distribution, the association weights of semantic correspondences are quantified, reflecting the importance of different feature dimension pairs. An attention mechanism is used to calculate the association weights, and the similarity matrix is ​​normalized using a softmax function, ensuring that the sum of the weights of all clinical feature dimensions corresponding to each image feature dimension is 1, and vice versa. The calculation of association weights considers the global dependencies between features, adaptively highlighting feature correspondences that contribute significantly to the diagnostic task. Larger association weights are assigned to feature dimension pairs with high similarity; for dimension pairs with low similarity or semantic irrelevance, the association weights are close to 0, thus achieving selective feature fusion.

[0040] Based on the association weights of semantic correspondences, the semantic vector representations of image phenotypes and clinical phenotypes are complementaryly integrated. The core of this complementary integration lies in utilizing a vector recombination mechanism driven by association weights to achieve deep coupling of cross-modal features. Specifically, each dimension of the image semantic vector is weighted and summed with all dimensions of the clinical semantic vector; the weights are the corresponding association weights, generating a fused image enhancement vector. Similarly, each dimension of the clinical semantic vector is weighted and summed with all dimensions of the image semantic vector, generating a fused clinical enhancement vector. The image enhancement vector not only retains the semantic information of the original image features but also incorporates related clinical feature information, thereby enhancing the discriminative power of image features. The clinical enhancement vector supplements the original clinical features with supporting information from the image phenotype. The image enhancement vector and the clinical enhancement vector are concatenated or added element-wise to form a joint feature representation. This joint feature representation contains the association information between the image phenotype and the clinical phenotype, achieving deep fusion of multimodal information.

[0041] In practical applications, the cross-modal feature alignment framework is optimized through end-to-end training. During training, a contrastive learning loss function or a triplet loss function is used to ensure that the image semantic vector and clinical semantic vector of the same patient are closer in a unified semantic space, while samples from different patients or different diagnostic categories are farther apart in the semantic space. In this way, the semantic embedding transformation network learns a mapping relationship that can effectively align image features and clinical features. Simultaneously, the module for calculating association weights automatically learns the optimal weight allocation strategy through backpropagation, eliminating the need for manually setting feature fusion rules. The entire framework has strong generalization capabilities and can adapt to the feature fusion needs of different patient groups and clinical scenarios.

[0042] The aforementioned cross-modal feature alignment framework integrates diagnostic information that was originally scattered across different data modalities into a unified joint feature representation, providing richer and more discriminative input features for subsequent classification networks, thereby improving the accuracy and reliability of differentiating between benign and malignant thyroid nodules.

[0043] In one optional implementation, the semantic vector representations of the image phenotype and the clinical phenotype are complementaryly integrated based on the association weights of the semantic correspondence. Deep coupling of cross-modal features is achieved through vector recombination driven by association weights, generating a joint feature representation including: A cross-modal complementarity metric function is constructed based on the association weights of semantic correspondences. The information gain relationship between each feature dimension in the semantic vector representation of the image phenotype and each feature dimension in the semantic vector representation of the clinical phenotype is calculated through the cross-modal complementarity metric function, and a feature dimension-level complementary weight distribution is generated. Based on the complementary weight distribution, a weighted vector concatenation is performed between the semantic vector representation of the image phenotype and the semantic vector representation of the clinical phenotype. By using the complementary weight distribution as a concatenation coefficient to adjust the expression intensity of each feature dimension in the concatenated vector, a preliminary integrated vector is generated. The initial integrated vector is subjected to a nonlinear transformation of the semantic space. By introducing the complementary weight distribution as a regularization constraint during the transformation process, the semantic correspondence structure between cross-modal features is maintained, thereby generating a joint feature representation.

[0044] After obtaining the semantic vector representations of imaging phenotypes and clinical phenotypes and establishing the semantic correspondence between them, it is necessary to complementaryly integrate the semantic vectors of the two types of phenotypes based on the association weights implied by this semantic correspondence, thereby achieving deep coupling of cross-modal features. The core of this integration process lies in the vector recombination mechanism driven by association weights, which enables imaging domain information and clinical domain information to form a mutually complementary and mutually reinforcing joint expression in a unified semantic space.

[0045] First, a cross-modal complementarity metric function is constructed based on the association weights of semantic correspondences. The design goal of this metric function is to quantify the information gain relationship between imaging phenotypes and clinical phenotypes at the feature dimension level. Specifically, assuming the semantic vector of the imaging phenotype is represented as... ,in The dimension of the semantic vector representing the image; the semantic vector of the clinical phenotype is represented as... ,in The dimension of the clinical semantic vector is represented by . The association weight matrix generated by the semantic correspondence is denoted as . Each element of the matrix The first element representing the semantic vector of the image The dimensional features and the 3rd dimension of the clinical semantic vector The strength of semantic association between dimensional features.

[0046] Cross-modal complementarity measurement function The first one used to calculate the image phenotypic semantic vector The dimensional features and the 3rd dimension of the clinical phenotype semantic vector The information gain relationship between dimensional features is calculated using the following formula: ; in The first element representing the semantic vector of the image dimensional components, The first clinical semantic vector represents the... dimensional components, To prevent division by zero by tiny constants, this function incorporates semantic association weights. The normalized measure of feature value differences can simultaneously consider semantic correspondence and numerical complementarity. When two feature dimensions are highly correlated semantically and differ numerically, their complementarity measure is high, indicating that the combination of these dimensions can provide richer information gain.

[0047] By traversing all of the image semantic vectors All dimensions and clinical semantic vectors The complete complementarity metric matrix is ​​calculated from each dimension. To generate the complementarity weight distribution at the feature dimension level, the complementarity metric matrix needs to be normalized. For the _th ... The dimensional feature, whose complementary weights to all clinical feature dimensions are calculated as follows: ; Similarly, for the clinical semantic vector of the th The dimensional features have the following complementarity weights: ; This results in a complementary weight distribution. and These respectively reflect the importance of each feature dimension of imaging phenotype and clinical phenotype in cross-modal integration.

[0048] Based on the aforementioned complementary weight distribution, a weighted vector concatenation is performed between the semantic vector representations of the image phenotype and the clinical phenotype. Traditional vector concatenation directly concatenates the two vectors along their feature dimensions, without considering the differentiated contributions of each dimension. Weighted vector concatenation, however, uses the complementary weight distribution as concatenation coefficients to adjust the expression intensity of each feature dimension in the concatenated vector. Specifically, each dimension of the image semantic vector is first weighted and scaled: ; Similarly, the clinical semantic vector is weighted and scaled: ; Then the weighted image semantic vector With weighted clinical semantic vector Perform a concatenation operation to generate a preliminary integrated vector. : ; This weighted concatenation mechanism ensures that, during the integration process, feature dimensions with higher complementarity receive stronger representational weights, thereby enabling the initial integrated vector to more fully capture the collaborative information between cross-modal features.

[0049] However, simple weighted concatenation is still a linear operation and cannot fully exploit the complex nonlinear relationships between cross-modal features. Therefore, a nonlinear semantic space transformation is needed for the initial integrated vector. This transformation is achieved through a multilayer perceptron structure, specifically a combination of two fully connected layers and activation functions. The first fully connected layer maps the initial integrated vector to the intermediate hidden space: ; in This is the weight matrix for the first layer. For bias vectors, The hidden layer dimension is represented by ReLU, which is the modified linear unit activation function. The second fully connected layer maps the hidden representations to the final joint feature space. ; in This is the weight matrix for the second layer. For bias vectors, The target dimension is the joint feature representation.

[0050] To preserve the semantic correspondence between cross-modal features during nonlinear transformation, a complementary weight distribution is introduced as a regularization constraint. Specifically, during network training, in addition to the conventional task loss function, a semantic structure preservation loss needs to be added. This loss function is designed to ensure that the information from image phenotypes and clinical phenotypes in the transformed joint feature representation still maintains the original semantic correspondence. The semantic structure preservation loss is defined as follows: ; in The regularization coefficient is . This represents the association weight in the semantic correspondence. This loss function penalizes the distance between corresponding dimensions in the joint feature representation, ensuring that the nonlinear transformation improves feature expressiveness without destroying the original cross-modal semantic correspondence structure.

[0051] Through the nonlinear transformation of the weighted vector concatenation and regularization constraints described above, the final generated joint feature representation This approach incorporates complementary information from both imaging and clinical phenotypes, and leverages nonlinear mapping to uncover deep correlations between cross-modal features while maintaining the stability of the semantic correspondence structure. This joint feature representation provides richer and more discriminative input features for subsequent classification networks, thereby improving the accuracy and robustness of benign / malignant thyroid nodules. In practical applications, the hidden layer dimensions can be adjusted according to the specific characteristics of the dataset. Target Dimension and regularization coefficient Equal hyperparameters are used to obtain the optimal feature integration effect.

[0052] In one optional implementation, a dynamic uncertainty quantification mechanism is established. By evaluating the boundary ambiguity of the probability differences among multiple candidate categories in the confidence distribution, the problematic samples where classification decisions are uncertain are identified, including: A dynamic uncertainty quantification mechanism based on probability difference evolution tracking is established. By continuously measuring the probability differences between multiple candidate categories in the confidence distribution at multiple decision times, a temporal evolution trajectory reflecting the probability differences as the decision progresses is constructed, and a dynamic evaluation matrix of boundary ambiguity is generated based on the fluctuation pattern of the temporal evolution trajectory. By using the adaptive weight allocation strategy in the dynamic uncertainty quantification mechanism, differentiated weights are assigned to the boundary ambiguity evaluation values ​​at different times based on the ambiguity change rate at different times in the dynamic evaluation matrix of boundary ambiguity, thereby generating a weighted and fused comprehensive uncertainty quantification index. Based on the combined uncertainty quantification index and the fluctuation amplitude of the time series evolution trajectory, a joint judgment is made. By identifying samples where the combined uncertainty quantification index exceeds a preset uncertainty threshold and the fluctuation amplitude of the time series evolution trajectory exceeds a preset fluctuation threshold, difficult samples are located.

[0053] After obtaining the preliminary diagnostic results and their confidence distribution from the classification network output, it is necessary to quantitatively evaluate the reliability of the diagnostic decision. The confidence distribution is usually presented as a probability vector, which contains the predicted probabilities of multiple candidate categories such as benign and malignant. In order to accurately identify the uncertainty in the classification decision, a dynamic uncertainty quantification mechanism based on probability difference evolution tracking is established. The core of this mechanism is that it does not rely on the static probability value at a single moment, but captures the dynamic changes in probability differences by continuously sampling and measuring the confidence distribution at multiple decision moments.

[0054] Specifically, multiple decision points are set up during the inference process of the classification network to sample the probability distribution. These decision points can correspond to the output states of different levels of the network, or they can be obtained through multiple forward propagations after introducing random perturbations. At each decision point... At this point, the current confidence distribution is extracted, and the probability difference between the top two candidate categories is calculated. This probability difference reflects the degree of certainty of the current decision; a larger difference indicates a clearer classification boundary, while a smaller difference indicates that the sample is near the decision boundary. By collecting probability difference values ​​at a series of decision moments, a temporal evolution trajectory is constructed. This trajectory, with time or decision stage as the horizontal axis and probability difference value as the vertical axis, visually demonstrates how the certainty of the classification decision evolves with the reasoning process.

[0055] For samples with high certainty, their temporal evolution trajectories typically exhibit stable and large values, indicating that the network can consistently provide a clear classification tendency at every decision stage. Conversely, for difficult samples, their temporal evolution trajectories often show patterns of sharp fluctuations or persistently low values, reflecting that the network's judgment of the sample wavers at different decision stages or consistently fails to form a clear tendency. Based on the fluctuation patterns of the temporal evolution trajectories, a dynamic evaluation matrix for boundary ambiguity is generated. The rows of this matrix correspond to different samples, and the columns correspond to different decision times. The matrix elements record the boundary ambiguity evaluation value of a specific sample at a specific time. The boundary ambiguity evaluation value can be calculated using the reciprocal of the probability difference value or a negative exponential function, such that the smaller the probability difference, the larger the corresponding ambiguity evaluation value.

[0056] The dynamic evaluation matrix provides rich temporal information, but the contribution of ambiguity evaluation values ​​at different decision moments to the final uncertainty determination should differ. In the early inference stage of the classification network, feature extraction is insufficient, and the probability distribution exhibits significant randomness; however, in the later stages, the network integrates more semantic information, making its output more valuable. Therefore, an adaptive weight allocation strategy is used to assign differentiated weights to the boundary ambiguity evaluation values ​​at each time step. The core of this strategy is to calculate the weights based on the rate of change of ambiguity at different times in the dynamic evaluation matrix of boundary ambiguity. The rate of change of ambiguity reflects the speed at which the ambiguity evaluation values ​​change between adjacent decision moments and can be quantified by the difference between the ambiguity evaluation values ​​at the current time and the previous time.

[0057] When the rate of change of ambiguity at a certain moment is large, it indicates a significant shift in the decision-making state at that moment, and the ambiguity assessment value at that moment should be assigned a higher weight. Conversely, if the rate of change of ambiguity is close to zero, it indicates a relatively stable decision-making state and high information redundancy at that moment, and its weight can be appropriately reduced. Weight calculation can employ a normalization scheme based on the rate of change. First, the absolute value of the rate of change of ambiguity at all decision moments is calculated, and then it is converted into weight coefficients using a softmax function or linear normalization. The sum of these weight coefficients is 1, ensuring the numerical stability of the weighted fusion process. The calculated differentiated weights are used to perform a weighted summation of the boundary ambiguity assessment values ​​at each moment in the dynamic evaluation matrix, generating a comprehensive uncertainty quantification index for each sample. This index comprehensively considers the ambiguity information and its dynamic evolution characteristics throughout the entire decision-making process, and more accurately reflects the true uncertainty level of the sample compared to a static assessment at a single moment.

[0058] After obtaining the comprehensive uncertainty quantification index, it is necessary to combine it with the fluctuation amplitude of the time-series evolution trajectory for joint judgment to improve the accuracy of difficult sample location. The fluctuation amplitude of the time-series evolution trajectory can be quantified by calculating the standard deviation or range of the probability difference values ​​in the trajectory. A large fluctuation amplitude indicates that the classification tendency of the sample differs significantly at different decision stages, and this instability is an important characteristic of difficult samples. Preset uncertainty thresholds and preset fluctuation thresholds are set as judgment criteria. The preset uncertainty threshold is used to screen samples with high comprehensive uncertainty quantification indexes, and the preset fluctuation threshold is used to screen samples with large fluctuation amplitudes in the time-series evolution trajectory. Only samples that meet both conditions are located as difficult samples. This joint judgment strategy can effectively eliminate single index anomalies caused by random noise, ensuring that the located difficult samples do indeed have uncertainty in classification decision-making.

[0059] The preset uncertainty threshold and preset fluctuation threshold can be adaptively adjusted based on the statistical characteristics of the training set. For example, by analyzing the comprehensive uncertainty quantification index and fluctuation amplitude distribution of known difficult samples in the training set, a threshold that can cover most difficult samples can be selected. Alternatively, a percentile method can be used to select samples ranking in the top certain percentage of the comprehensive uncertainty quantification index as candidates, and then perform a secondary screening based on fluctuation amplitude. For the identified difficult samples, a feature enhancement strategy based on adversarial sample generation will be triggered. By introducing adversarial perturbations into the joint feature representation, the discriminative power of the features is enhanced, thereby improving the classification network's ability to identify these difficult samples.

[0060] The entire implementation of the dynamic uncertainty quantification mechanism requires no manual intervention and can automatically complete the entire process from confidence distribution sampling, temporal evolution trajectory construction, dynamic evaluation matrix generation, adaptive weight allocation to difficult sample localization. This mechanism fully utilizes the temporal information and dynamic characteristics of the classification decision-making process. Compared to traditional methods based on a single confidence threshold, it can more accurately identify truly difficult samples requiring further processing, avoiding unnecessary enhancement operations on high-confidence samples and improving the overall efficiency and accuracy of the diagnostic system. In practical applications, the preset uncertainty threshold and preset fluctuation threshold can be flexibly adjusted according to the diagnostic criteria and risk preferences of different medical institutions, achieving fine-grained control over the sensitivity of difficult sample localization.

[0061] In one optional implementation, a dynamic uncertainty quantification mechanism based on probability difference evolution tracking is established. This involves continuously measuring the probability differences between multiple candidate categories in the confidence distribution at multiple decision points, constructing a temporal evolution trajectory reflecting the probability differences as the decision-making process progresses. Multiple decision-making sampling points are set at different stages of classification decision-making. By extracting the probability values ​​of multiple candidate categories from the confidence distribution at each decision-making sampling point, the probability interval between any two candidate categories is calculated, and the probability difference measurement value corresponding to each decision-making sampling point is generated. A time-series correlation analysis is performed on the measured probability difference values. By calculating the amount and rate of change of the measured probability difference values ​​between sampling points at adjacent decision times, time-series feature parameters reflecting the evolution of the probability competition relationship of candidate categories are extracted, and the evolution rate and trend of probability difference are quantified based on the time-series feature parameters. By using the probability difference measurement values ​​of sampling points at each decision moment as trajectory nodes, the evolution rate as the connection strength between trajectory nodes, and the evolution trend as the directional constraint of the trajectory, a temporal evolution trajectory reflecting the probability difference as the decision progresses is generated.

[0062] In the intelligent identification of thyroid nodules, the classification network’s confidence in judging samples changes dynamically at different stages of the process. By establishing a dynamic uncertainty quantification mechanism based on probability difference evolution tracking, we can accurately capture the temporal evolution of the probability competition relationship between benign and malignant categories during the classification decision-making process, thereby accurately identifying difficult samples with ambiguous decision boundaries.

[0063] During the forward propagation of a classification network, feature representations undergo multiple nonlinear transformations to gradually form the final class probability distribution. To capture this dynamic evolution, multiple decision-time sampling points are set at different depths of the network. Specifically, the hierarchical structure of the classification network is divided into several decision stages, each corresponding to an intermediate feature layer. A temporary classification head is then connected after this feature layer to extract the confidence distribution of the current stage. Assume the classification network contains... The layer, then in the first layer( ) Set up sampling points at the decision time, and map the features of each sampling point to a class probability vector through a fully connected layer. For the binary classification task of benign and malignant thyroid nodules, at the decision time... The confidence distribution of the layer extraction is denoted as ,in Represents the probability of the benign category. Let represent the probability of the malignant category, and satisfy . .

[0064] At each decision-making time point, the probability interval between the two candidate categories, benign and malignant, is calculated. This probability interval reflects the degree of certainty in judging the sample category at the current decision stage. The probability difference is defined as the absolute difference between the probabilities of the two candidate categories, i.e. .when When the value is close to 1, it indicates that the network has a high degree of certainty about the sample category at this decision-making stage; when... When the value is close to 0, it indicates that the probabilities of the two categories are similar, and there is significant uncertainty in the decision-making process. By examining all... The probability difference measurement value is calculated at each sampling point at each decision time, resulting in a probability difference sequence. This sequence depicts the deterministic evolution of classification decisions from shallow to deep levels.

[0065] To deeply analyze the temporal evolution of probability differences, a temporal correlation analysis was performed on the measured probability differences. First, the change in probability difference between sampling points at adjacent decision times was calculated. For the... Layer and First Between layers, the amount of change is defined as This change reflects the increase or decrease in decision certainty between adjacent stages: when When this indicates increased certainty in decision-making; when When this occurs, it indicates a decrease in decision-making certainty; when When the value approaches 0, it indicates that decision certainty remains stable. Further calculation of the rate of change of probability differences quantifies the speed of evolution of decision certainty. The rate of change is defined as... ,in To prevent division by zero by small constants. Rate of change. The differences in baseline probabilities at different decision-making stages were normalized, making the evolution speeds of different samples comparable.

[0066] Based on the magnitude and rate of change, temporal feature parameters reflecting the evolution of the probabilistic competition relationship among candidate categories are extracted. An evolution rate index is defined. This indicator quantifies the probability difference in the 1st... layer to the first The rate of evolution between layers. A high evolution rate indicates a dramatic change in decision certainty at that stage, corresponding to a key inflection point in the feature representation. Define evolutionary trend indicators. This indicator takes values ​​of +1, 0, or -1, corresponding to trends of increased, stable, or weakened decision certainty, respectively. By statistically analyzing the number of sign changes in the evolutionary trend throughout the decision-making process, oscillations in the decision-making process can be identified: frequent sign changes indicate that the network repeatedly oscillates between good and bad judgments, and the sample belongs to a difficult sample with high uncertainty.

[0067] After obtaining the probability difference measurements, evolution rates, and evolution trends at each decision-making time point, a temporal evolution trajectory reflecting the probability difference as the decision-making process progresses is constructed. The probability difference measurements at each decision-making time point are then... As trajectory nodes, they are indexed in the two-dimensional coordinate system according to the decision stage. The horizontal axis represents the probability difference measurement. Plotting the curve on the vertical axis creates an evolution curve of probability differences. The shape of this curve visually reflects the evolution pattern of decision certainty: a monotonically rising curve indicates that decision certainty is continuously increasing, and the sample is easily classified; a fluctuating curve indicates that the decision-making process is unstable, and the sample is near the decision boundary; a flat curve indicates that decision certainty is consistently low, and the sample is a typical difficult sample.

[0068] To further quantify the evolutionary characteristics of the trajectory, the evolution rate will be... This visualization method represents the connection strength between trajectory nodes. In the trajectory graph, connection strength is indicated by line thickness or color depth: trajectory segments corresponding to high evolution rates are marked with thick lines or dark colors, indicating drastic changes in decision certainty at this stage; trajectory segments corresponding to low evolution rates are marked with thin lines or light colors, indicating gradual changes in decision certainty at this stage. This visualization method can quickly locate key turning points in the decision-making process, providing targeted intervention locations for subsequent feature enhancement strategies.

[0069] Evolutionary trend As a directional constraint on the trajectory, the evolution direction is indicated in the trajectory diagram using arrows or gradient colors. When When using an upward arrow or warm color to indicate a positive increase in decision certainty; when When using down arrows or cool colors, it indicates a negative decrease in decision certainty; when When using horizontal arrows or neutral colors, it indicates that the decision certainty remains stable. Directional constraints can clearly identify anomalous evolution patterns in the decision-making process. For example, a sudden drop in decision certainty in deep network stages often indicates that the feature representation is affected by noise or trapped in a local optimum at that stage.

[0070] Based on the temporal evolution trajectory, a global uncertainty score is defined to quantify the overall decision difficulty of the sample. The mean of the probability difference measurement is calculated at all decision time points. with standard deviation A low mean indicates that the sample remains in a state of low certainty throughout the decision-making process, while a high standard deviation indicates that decision certainty fluctuates dramatically at different stages. Taking both mean and standard deviation into account, an uncertainty score is defined. This score reflects both the average uncertainty and evolutionary instability of the decision-making process. An uncertainty threshold is set. ,when When this happens, the sample is marked as a difficult sample, triggering subsequent adversarial sample generation and feature enhancement strategies.

[0071] By using the dynamic uncertainty quantification mechanism based on probability difference evolution tracking, the dynamic characteristics of classification decision-making can be comprehensively characterized from the perspective of temporal evolution. It can accurately locate difficult samples with ambiguous decision boundaries, providing a reliable trigger basis for subsequent discriminative enhancement and decision boundary reshaping, thereby improving the diagnostic accuracy and robustness of the intelligent identification system for thyroid nodules in complex cases.

[0072] In one optional implementation, a feature enhancement strategy based on adversarial sample generation is triggered for the difficult samples to discriminately enhance the joint feature representation. The decision boundary of the classification network is dynamically reshaped using the discriminatively enhanced joint feature representation to generate a corrected diagnostic result, including: For the joint feature representation of difficult samples, an adversarial perturbation field is generated in the feature space. The gradient distribution of the adversarial perturbation field is constructed by calculating the perturbation direction that causes the maximum change in the decision confidence of the classification network, and an adversarial enhanced feature representation containing decision-sensitive features is generated based on the gradient distribution. A comparative analysis of the feature space of the adversarial enhanced feature representation and the original joint feature representation is performed. By extracting the feature dimensions in the adversarial enhanced feature representation that cause the decision boundary to shift, a discriminative enhancement vector for the difficult sample is constructed. The discriminative enhancement vector is then fused into the joint feature representation to generate a discriminatively enhanced joint feature representation. The discriminatively enhanced joint feature representation is used as the constraint input for reshaping the decision boundary of the classification network. The distribution characteristics of the discriminatively enhanced joint feature representation in the feature space are used to adjust the decision boundary parameters of the classification network, thereby completing the dynamic reshaping of the decision boundary. Based on the reshaped decision boundary, the difficult samples are reclassified to generate a corrected diagnostic result.

[0073] like Figure 2 As shown, the method includes: Once the dynamic uncertainty quantification mechanism identifies challenging samples with uncertain classification decisions, it initiates a feature enhancement strategy based on adversarial sample generation for these samples. The core of this strategy lies in constructing an adversarial perturbation field to probe the vulnerable regions of the classification network's decision boundary, thereby extracting discriminative feature dimensions that have a crucial impact on the classification results.

[0074] Joint feature representation for difficult samples An adversarial perturbation field is generated in the feature space. The goal of constructing this perturbation field is to find a small perturbation vector. This causes the classification network's decision confidence to change to the greatest extent after adding this perturbation. By calculating the gradient of the classification network's output layer with respect to the input features, the direction in which the decision confidence changes most drastically can be obtained. Specifically, the loss function of the classification network is calculated. Compared to joint feature representation The gradient of is expressed as . Gradient vector Each component reflects the sensitivity of the corresponding feature dimension to the classification decision. Dimensions with larger absolute gradient values ​​indicate that small changes in that dimension can lead to significant fluctuations in the classification results.

[0075] When constructing adversarial perturbation fields based on gradient distributions, an improved form of the fast gradient sign method is employed. Adversarial perturbation vector. The calculation formula is: ,in The disturbance intensity coefficient is... Indicates the sign function. Disturbance intensity coefficient. The value of needs to strike a balance between ensuring the perturbation is small enough to maintain semantic consistency of the features and large enough to produce observable changes in decision-making. It is typically set to 0.1 to 0.3 times the standard deviation of each dimension of the joint feature representation. The adversarial perturbation vector... Add to the original joint feature representation to generate an adversarial enhanced feature representation. This adversarial enhancement feature representation contains key feature information that causes the decision boundary of the classification network to shift, revealing unstable regions of the current decision boundary in the feature space.

[0076] Adversarial enhancement feature representation Combined with original joint feature representation Perform a comparative analysis of the feature spaces. Calculate the feature difference vector between the two. The difference vector is directly equal to the adversarial perturbation vector. By analyzing the numerical distribution of each dimension of the difference vector, the feature dimensions that contribute most to the decision boundary offset can be identified. Specifically, the absolute value of each dimension of the difference vector is calculated. ,in This represents the feature dimension index, with a value range of 100. to Sort these absolute values ​​in descending order and select the top few. The dimension with the largest absolute value is taken as the set of decision-sensitive dimensions, among which The value of is typically 10% to 30% of the total dimensions of the joint feature representation.

[0077] The classification network represents the original joint features separately. and adversarial enhancement feature representation Perform forward propagation to obtain the corresponding classification probability distributions for both. and Calculate the KL divergence between two probability distributions. This divergence value quantifies the impact of adversarial perturbations on classification decisions. Simultaneously, the probability difference between the classes with the highest probability in the two probability distributions is calculated. This difference reflects the magnitude of change in decision confidence. When the KL divergence exceeds a preset threshold or the absolute value of the probability difference exceeds a preset threshold, it indicates that the adversarial perturbation has successfully triggered a significant shift in the decision boundary, and the extracted decision-sensitive dimension has high discriminative value at this time.

[0078] Construct discriminative enhancement vectors for difficult samples The construction of this vector is based on the contribution weights of each dimension in the decision-sensitive dimension set. For the _th ... Each dimension, and its contribution weight. The calculation comprehensively considers both the magnitude of change of this dimension under adversarial perturbation and the degree of impact of this dimension's change on the classification probability distribution. The formula for calculating the contribution weight is as follows: ,in Represents the gradient vector The Each component is a decision sensitivity dimension. The contribution weights of all decision sensitivity dimensions are normalized so that their sum is 1. Discriminative enhancement vector. The Each component is calculated as follows: For non-decision-sensitive dimensions, the corresponding discriminative enhancement vector component is set to 0.

[0079] Discriminative enhancement vector Fusion to the original joint feature representation This generates a joint feature representation with enhanced discriminative power. The fusion operation employs a weighted fusion strategy, and the fusion formula is as follows: ,in This is the fusion coefficient, used to control the strength of the influence of the discriminative enhancement vector on the original features. The value is determined based on the uncertainty score of the difficult sample. To achieve adaptive adjustment, samples with higher uncertainty scores require stronger discriminative enhancement; therefore, the fusion coefficient is set to... ,in The basic fusion coefficient typically ranges from 0.2 to 0.5. The adjustment index is typically set between 0.5 and 1.5. This adaptive fusion strategy ensures that the strength of the discriminative enhancement matches the uncertainty level of the samples, avoiding over-enhancing of samples with high certainty.

[0080] The joint feature representation after discriminative enhancement This serves as a constraint input for reshaping the decision boundary of the classification network. The goal of decision boundary reshaping is to adjust the parameters of the classification network so that it can produce more deterministic classification results based on the enhanced discriminative feature representations. Specifically, a decision boundary reshaping loss function is constructed. The loss function comprises two components: classification loss and boundary smoothing loss. The classification loss, expressed as cross-entropy, calculates the difference between the predicted probability distribution of the discriminatively enhanced joint feature representation after passing through the classification network and the true label. The boundary smoothing loss constrains the smoothness of the decision boundary in the feature space, preventing excessive distortion of the decision boundary. This loss is measured by calculating the classification consistency between the discriminatively enhanced joint feature representation and the feature representations of its neighboring samples, which are obtained through small-scale random sampling around the discriminatively enhanced joint feature representation.

[0081] The decision boundary parameters of the classification network are adjusted by analyzing the distribution characteristics of the joint feature representation in the feature space after discriminative enhancement. Gradient descent is then used to update the parameters of the classification network, with the update direction reshaping the loss function from the decision boundary. Gradient determination of network parameters. To avoid the negative impact of the decision boundary reshaping process on the classification performance of the original normal samples, a regularization constraint is introduced during parameter update. This constraint limits the shift of the network parameters relative to the original parameters. The regularization constraint adopts the L2 norm form, and the constraint term is... ,in The first classifier of the classification network One parameter, This represents the original value of the parameter before the decision boundary is reshaped. This is the regularization coefficient, typically ranging from 0.001 to 0.01. The complete optimization objective function is: .

[0082] After dynamically reshaping the decision boundary, difficult samples are reclassified based on the reshaped decision boundary. The joint feature representation with enhanced discriminative power is then applied. The data is fed into the updated classification network to obtain a new classification probability distribution. The category with the highest probability from this probability distribution is selected as the corrected diagnostic result, and the probability value corresponding to this category is recorded as the corrected confidence level. The corrected diagnostic result has higher decision certainty than the initial diagnostic result because the decision boundary reshaping process is specifically optimized for the feature distribution characteristics of difficult samples, making the decision boundary more discriminative in the feature space region where the difficult samples are located. For extremely difficult samples whose corrected confidence level is still below a preset threshold, they can be marked as samples requiring manual review, and corresponding prompts will be provided in the final identification result.

[0083] A second aspect of this invention provides a deep learning-based intelligent identification system for thyroid nodules, comprising: The image segmentation unit is used to acquire ultrasound image data of the thyroid gland to be detected, perform regional segmentation on the ultrasound image data of the thyroid gland to locate the target region of the nodule, and construct a multidimensional image feature representation of the target region of the nodule. The feature alignment unit is used to construct a cross-modal feature alignment framework. By establishing a mapping relationship between the image domain feature space and the clinical semantic feature space, the multidimensional image feature expression of the nodule target region and the patient's clinical history features are semantically coordinated and reconstructed in a unified semantic space to generate a joint feature representation containing information related to image phenotype and clinical phenotype. The classification and discrimination unit is used to input the joint feature representation into the classification network, generate a preliminary diagnosis result according to the benign and malignant discrimination rules, and output the confidence distribution corresponding to the preliminary diagnosis result; The dynamic correction unit is used to establish a dynamic uncertainty quantification mechanism. By evaluating the boundary ambiguity of the probability differences between multiple candidate categories in the confidence distribution, it locates the difficult samples with uncertainty in classification decision. For the difficult samples, it triggers a feature enhancement strategy based on adversarial sample generation to discriminately enhance the joint feature representation. The decision boundary of the classification network is dynamically reshaped using the discriminatively enhanced joint feature representation to generate a corrected diagnostic result. The result output unit is used to output the corrected diagnostic result as the final identification result and generate the decision basis visualization information corresponding to the final identification result.

[0084] A third aspect of the present invention provides an electronic device, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0085] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0086] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.

[0087] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A deep learning-based intelligent identification method for thyroid nodules, characterized in that, include: Acquire ultrasound image data of the thyroid gland to be detected, perform regional segmentation on the thyroid ultrasound image data to locate the target region of the nodule, and construct a multidimensional image feature representation of the target region of the nodule. A cross-modal feature alignment framework is constructed. By establishing a mapping relationship between the image domain feature space and the clinical semantic feature space, the multidimensional image feature expression of the nodule target region and the patient's clinical history features are semantically coordinated and reconstructed in a unified semantic space to generate a joint feature representation containing information related to image phenotype and clinical phenotype. The joint feature representation is input into a classification network, a preliminary diagnosis result is generated according to the benign / malignant discrimination rule, and the confidence distribution corresponding to the preliminary diagnosis result is output. A dynamic uncertainty quantification mechanism is established. By evaluating the boundary ambiguity of the probability differences between multiple candidate categories in the confidence distribution, difficult samples with uncertain classification decisions are located. For the difficult samples, a feature enhancement strategy based on adversarial sample generation is triggered to discriminately enhance the joint feature representation. The decision boundary of the classification network is dynamically reshaped using the discriminatively enhanced joint feature representation to generate a corrected diagnostic result. The corrected diagnostic result is output as the final identification result, and the decision basis visualization information corresponding to the final identification result is generated.

2. The method according to claim 1, characterized in that, The process of performing region segmentation on the thyroid ultrasound image data to locate the target nodule region and constructing a multidimensional image feature representation of the target nodule region includes: An adaptive boundary-aware region segmentation module is constructed. By establishing a constraint relationship between pixel-level gradient response and local texture consistency in thyroid ultrasound image data, preliminary segmentation results of nodule candidate regions and their corresponding boundary confidence distributions are generated. Based on the boundary confidence distribution, the blurred boundary regions in the preliminary segmentation results are refined and corrected. By introducing morphological prior constraints, the boundary contours are topologically optimized to generate the accurate segmentation boundary of the nodule target region. A multi-level feature decoupling and recombination framework is established. The precise segmentation boundary of the nodule target region is used as the spatial positioning benchmark. The morphological feature dimension, echo feature dimension and blood flow distribution feature dimension are decoupled and separated within the nodule target region. The decoupled features of each dimension are then recombined in a structured manner through cross-dimensional semantic association to generate a multi-dimensional image feature expression.

3. The method according to claim 1, characterized in that, A cross-modal feature alignment framework is constructed. By establishing a mapping relationship between the image domain feature space and the clinical semantic feature space, the multidimensional image feature expression of the nodule target region and the patient's clinical history features are semantically coordinated and reconstructed in a unified semantic space, generating a joint feature representation containing information on the association between image phenotype and clinical phenotype, including: The multidimensional image feature representation of the target region of the nodule is projected onto a unified semantic space through semantic embedding transformation to generate a semantic vector representation of the image phenotype. At the same time, the patient's clinical history features are semantically encoded, and clinical diagnostic information, laboratory test indicators and medical history records are mapped to the unified semantic space to generate a semantic vector representation of the clinical phenotype. In the unified semantic space, the cross-modal semantic similarity distribution between the semantic vector representation of the image phenotype and the semantic vector representation of the clinical phenotype is calculated. Based on the cross-modal semantic similarity distribution, the semantic correspondence between the image feature dimension and the clinical feature dimension is identified, and the association weight of the semantic correspondence is quantified. Based on the association weights of the semantic correspondence, the semantic vector representations of the image phenotype and the clinical phenotype are complementaryly integrated. Through vector recombination driven by association weights, deep coupling of cross-modal features is achieved, generating a joint feature representation.

4. The method according to claim 3, characterized in that, Based on the association weights of the semantic correspondence, the semantic vector representations of the image phenotype and the clinical phenotype are complementaryly integrated. Deep coupling of cross-modal features is achieved through vector recombination driven by association weights, generating a joint feature representation including: A cross-modal complementarity metric function is constructed based on the association weights of semantic correspondences. The information gain relationship between each feature dimension in the semantic vector representation of the image phenotype and each feature dimension in the semantic vector representation of the clinical phenotype is calculated through the cross-modal complementarity metric function, and a feature dimension-level complementary weight distribution is generated. Based on the complementary weight distribution, a weighted vector concatenation is performed between the semantic vector representation of the image phenotype and the semantic vector representation of the clinical phenotype. By using the complementary weight distribution as a concatenation coefficient to adjust the expression intensity of each feature dimension in the concatenated vector, a preliminary integrated vector is generated. The initial integrated vector is subjected to a nonlinear transformation of the semantic space. By introducing the complementary weight distribution as a regularization constraint during the transformation process, the semantic correspondence structure between cross-modal features is maintained, thereby generating a joint feature representation.

5. The method according to claim 1, characterized in that, A dynamic uncertainty quantification mechanism is established. By evaluating the boundary ambiguity of the probability differences among multiple candidate categories in the confidence distribution, the problematic samples with uncertain classification decisions are identified, including: A dynamic uncertainty quantification mechanism based on probability difference evolution tracking is established. By continuously measuring the probability differences between multiple candidate categories in the confidence distribution at multiple decision times, a temporal evolution trajectory reflecting the probability differences as the decision progresses is constructed, and a dynamic evaluation matrix of boundary ambiguity is generated based on the fluctuation pattern of the temporal evolution trajectory. By using the adaptive weight allocation strategy in the dynamic uncertainty quantification mechanism, differentiated weights are assigned to the boundary ambiguity evaluation values ​​at different times based on the ambiguity change rate at different times in the dynamic evaluation matrix of boundary ambiguity, thereby generating a weighted and fused comprehensive uncertainty quantification index. Based on the combined uncertainty quantification index and the fluctuation amplitude of the time series evolution trajectory, a joint judgment is made. By identifying samples where the combined uncertainty quantification index exceeds a preset uncertainty threshold and the fluctuation amplitude of the time series evolution trajectory exceeds a preset fluctuation threshold, difficult samples are located.

6. The method according to claim 5, characterized in that, A dynamic uncertainty quantification mechanism based on probability difference evolution tracking is established. By continuously measuring the probability differences between multiple candidate categories in the confidence distribution at multiple decision times, a temporal evolution trajectory reflecting the probability differences as the decision progresses is constructed, including: Multiple decision-making sampling points are set at different stages of classification decision-making. By extracting the probability values ​​of multiple candidate categories from the confidence distribution at each decision-making sampling point, the probability interval between any two candidate categories is calculated, and the probability difference measurement value corresponding to each decision-making sampling point is generated. A time-series correlation analysis is performed on the measured probability difference values. By calculating the amount and rate of change of the measured probability difference values ​​between sampling points at adjacent decision times, time-series feature parameters reflecting the evolution of the probability competition relationship of candidate categories are extracted, and the evolution rate and trend of probability difference are quantified based on the time-series feature parameters. By using the probability difference measurement values ​​of sampling points at each decision moment as trajectory nodes, the evolution rate as the connection strength between trajectory nodes, and the evolution trend as the directional constraint of the trajectory, a temporal evolution trajectory reflecting the probability difference as the decision progresses is generated.

7. The method according to claim 1, characterized in that, For the challenging samples, a feature enhancement strategy based on adversarial sample generation is triggered to discriminately enhance the joint feature representation. The decision boundary of the classification network is dynamically reshaped using the discriminatively enhanced joint feature representation to generate a corrected diagnostic result, including: For the joint feature representation of difficult samples, an adversarial perturbation field is generated in the feature space. The gradient distribution of the adversarial perturbation field is constructed by calculating the perturbation direction that causes the maximum change in the decision confidence of the classification network, and an adversarial enhanced feature representation containing decision-sensitive features is generated based on the gradient distribution. A comparative analysis of the feature space of the adversarial enhanced feature representation and the original joint feature representation is performed. By extracting the feature dimensions in the adversarial enhanced feature representation that cause the decision boundary to shift, a discriminative enhancement vector for the difficult sample is constructed. The discriminative enhancement vector is then fused into the joint feature representation to generate a discriminatively enhanced joint feature representation. The discriminatively enhanced joint feature representation is used as the constraint input for reshaping the decision boundary of the classification network. The distribution characteristics of the discriminatively enhanced joint feature representation in the feature space are used to adjust the decision boundary parameters of the classification network, thereby completing the dynamic reshaping of the decision boundary. Based on the reshaped decision boundary, the difficult samples are reclassified to generate a corrected diagnostic result.

8. A deep learning-based intelligent identification system for thyroid nodules, used to implement the method as described in any one of claims 1-7, characterized in that, include: The image segmentation unit is used to acquire ultrasound image data of the thyroid gland to be detected, perform regional segmentation on the ultrasound image data of the thyroid gland to locate the target region of the nodule, and construct a multidimensional image feature representation of the target region of the nodule. The feature alignment unit is used to construct a cross-modal feature alignment framework. By establishing a mapping relationship between the image domain feature space and the clinical semantic feature space, the multidimensional image feature expression of the nodule target region and the patient's clinical history features are semantically coordinated and reconstructed in a unified semantic space to generate a joint feature representation containing information related to image phenotype and clinical phenotype. The classification and discrimination unit is used to input the joint feature representation into the classification network, generate a preliminary diagnosis result according to the benign and malignant discrimination rules, and output the confidence distribution corresponding to the preliminary diagnosis result; The dynamic correction unit is used to establish a dynamic uncertainty quantification mechanism. By evaluating the boundary ambiguity of the probability differences between multiple candidate categories in the confidence distribution, it locates the difficult samples with uncertainty in classification decision. For the difficult samples, it triggers a feature enhancement strategy based on adversarial sample generation to discriminately enhance the joint feature representation. The decision boundary of the classification network is dynamically reshaped using the discriminatively enhanced joint feature representation to generate a corrected diagnostic result. The result output unit is used to output the corrected diagnostic result as the final identification result and generate the decision basis visualization information corresponding to the final identification result.

9. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 7.

Images