Intelligent skin lesion diagnosis system based on bimodal ultrasonic image

Abstract

The invention provides an intelligent skin lesion diagnosis system based on a bimodal ultrasonic image, and the system comprises an initial extraction module which obtains a B-mode ultrasonic image and a Doppler ultrasonic image as input, and employs two independent feature extractors to extract the initial features of two modals respectively; the bidirectional cross-modal attention module is used for taking the initial features of the two modals as context information for interaction before fusion, capturing and strengthening complementary information between the two modals, and generating enhanced features; the feature fusion module is used for evaluating the relative reliability of two modes in the current sample based on the enhanced features and generating a dynamic importance weight to obtain fusion features; the classification and recognition module inputs the fusion features into different classifiers respectively and outputs a plurality of prediction results; and the result output module is used for fusing a plurality of prediction results and carrying out dynamic weighting by utilizing uncertainty to obtain a final intelligent diagnosis result. According to the invention, accurate and reliable computer-aided diagnosis of skin lesions is realized.

Description

Technical Field

[0001] This invention relates to the field of medical image processing and computer-aided diagnosis, and in particular to an intelligent diagnostic system for skin lesions using dual-modal ultrasound images. Background Technology

[0002] Skin cancer is one of the most common malignant tumors worldwide. With accurate early diagnosis and timely intervention, most skin cancers are highly curable. Therefore, early detection of skin lesions is crucial for improving patient survival and overall prognosis. Early detection not only avoids invasive treatments such as major surgery, chemotherapy, or radiotherapy but also significantly reduces the physiological burden and economic pressure associated with advanced disease. However, the clinical diagnosis of skin lesions is highly complex and challenging. The significant heterogeneity in morphology, texture, and scale of skin lesions makes it difficult for clinicians to identify lesions accurately and promptly using only visual examination or dermoscopy. Although tissue biopsy and pathological analysis remain the current gold standard for diagnosis, they have inherent limitations such as invasiveness and time-consuming procedures. Therefore, there is an urgent clinical need for an auxiliary diagnostic technique that offers near-pathological accuracy while being non-invasive and highly efficient.

[0003] B-mode ultrasound provides structural and anatomical information about lesions, clearly showing the structure of each skin layer and assessing whether lesions have invaded the subcutaneous fat layer, cartilage, and bone tissue. It is particularly suitable for areas with thin skin and a high risk of recurrence, such as the face. However, B-mode ultrasound has limited ability to distinguish superficial, very thin lesions (such as carcinoma in situ). Furthermore, for lesions often accompanied by ulceration or keratinization (such as SCC), surface structures introduce significant acoustic artifacts, increasing the difficulty of image interpretation. Complementing this, Doppler ultrasound imaging provides functional information that B-mode lacks, especially vascularization features. Tumor growth and metastasis depend on angiogenesis; therefore, Doppler signals reflect blood perfusion within the lesion and are closely related to tumor invasiveness. Simultaneously, changes in blood perfusion around the lesion are a key early indicator for identifying sentinel lymph node metastasis. Due to the inherent differences in the imaging physics of B-mode and Doppler, the information captured by the two is highly heterogeneous but complementary. Simply superimposing or processing information from these two modalities in parallel is inefficient. Simply superimposing or using these two types of information in parallel fails to fully realize their synergistic potential. To overcome current diagnostic performance bottlenecks, it is urgent to design a fusion mechanism capable of deep modal mutual calibration and bidirectional information interaction. This mechanism would enable the structural information of the B-mode to accurately locate and constrain the blood flow signal of the Doppler, while the Doppler information would simultaneously verify and supplement the lesion margins defined by the B-mode. This type of deep synergistic mechanism is a key technological foundation for achieving near-histopathological diagnostic accuracy.

[0004] Despite significant advancements in medical image analysis using artificial intelligence (AI) technology in recent years, existing deep learning-based skin lesion diagnostic systems still face considerable challenges in model architecture and multimodal feature fusion. Many AI systems are built solely on dermoscopy images or a single ultrasound modality, limiting their diagnostic capabilities to a single information source and failing to comprehensively capture the key features required for disease differentiation. For B-mode and Doppler multimodal scenarios, common feature concatenation, summation, or weighted fusion strategies are fundamentally inadequate: they do not adequately consider the physical differences between heterogeneous modalities, lack bidirectional deep feature interaction, and struggle to achieve efficient and robust cross-modal information integration. Furthermore, medical image datasets generally exhibit significant class imbalance, particularly in classification tasks involving multiple types of skin lesions. This data imbalance often leads deep learning models to overfit to classes with larger sample sizes during training, neglecting a few clinically significant lesions (such as malignant melanoma (MM) and dermatofibrosarcoma (DFSP)). This problem directly results in low recall rates for a minority of classes, significantly increasing the risk of misdiagnosis and severely impacting the reliability and application value of diagnostic systems in real-world clinical settings. Summary of the Invention

[0005] This invention provides a dual-modal ultrasound imaging intelligent diagnostic system for skin lesions, designed to provide accurate and reliable computer-aided diagnosis of eight common skin lesions, including actinic keratosis (AK), basal cell carcinoma (BCC), Bowen's disease, external Paget's disease, squamous cell carcinoma of the skin (SCC), and seborrheic keratosis (SK).

[0006] According to a first aspect of the present invention, an integrated intelligent diagnostic system for skin lesions based on dual-modal ultrasound images is provided, comprising: an initial extraction module: acquiring B-mode ultrasound images and Doppler ultrasound images as input, and extracting initial features of the two modalities using two independent feature extractors respectively; a bidirectional cross-modal attention module: interacting the initial features of the two modalities as contextual information before fusion, capturing and enhancing complementary information between the two modalities, and generating enhanced features; a feature fusion module: evaluating the relative reliability of the two modalities in the current sample based on the enhanced features, and generating dynamic importance weights to obtain fused features; a classification and recognition module: inputting the fused features into different classifiers respectively, and outputting multiple prediction results; and a result output module: fusing multiple prediction results and dynamically weighting them using uncertainty to obtain a final intelligent diagnostic result.

[0007] According to a second aspect of the present invention, an electronic device is provided, including a processor and a memory storing a program. The program includes instructions that, when executed by the processor, cause the processor to perform the steps performed by the system of the first aspect described above.

[0008] According to a third aspect of the present invention, a computer storage medium is provided having a computer program stored thereon, which, when executed by a processor, performs the steps performed by the system of the first aspect described above.

[0009] Compared with the prior art, the present invention has the following beneficial effects: (1) This invention constructs an integrated system with high accuracy, high robustness, and self-reliable quantification capabilities. By simultaneously utilizing the structural information of the B-mode and the functional information of the Doppler, this invention achieves bimodal complementary diagnosis, significantly outperforming methods that rely solely on single-modal images. Furthermore, by leveraging cross-modal information interaction and adaptive dynamic weight calculation, this invention enables the system to dynamically adjust the contributions of the B-mode and Doppler modes based on the image quality and information reliability of the current sample. When the image quality of a certain modality is affected by artifacts, its weight is automatically reduced, thereby preventing low-quality modalities from contaminating the fusion features and greatly enhancing the robustness of the system.

[0010] (2) To address the inherent class imbalance challenge in medical image data, this invention adopts a customized optimization strategy. First, Focal Loss is used as the base loss, and by dynamically scaling the weights, the imbalance problem of rare class samples such as malignant lesions is effectively solved, making the model training more focused on difficult-to-classify samples. Second, for the prototype classifier, a prototype compactness loss is introduced. With separation loss This explicitly shapes the feature space from a geometric perspective, enhancing intra-class compactness and inter-class separability, thereby improving the stability and discriminative power of the model during clinical generalization.

[0011] (3) This invention focuses on improving clinical credibility and robustness in the decision-making stage. The system first uses temperature scaling calibration to ensure that the probabilities output by all models can truly reflect the accuracy of the models and correct the overconfidence tendency common in deep learning models. Subsequently, the uncertainty score of each model is quantified through the MC Dropout mechanism. The prediction results of multiple heterogeneous models are fused through uncertainty-weighted voting, realizing sample-level adaptive decision-making, which significantly improves the reliability of the final diagnostic results and the clinical application value. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0013] Figure 1 This is a structural block diagram of the system of the present invention.

[0014] Figure 2 This is a schematic diagram of B-mode ultrasound images and Doppler ultrasound images.

[0015] Figure 3 Grad-CAM visualization results of B-mode ultrasound images and Doppler ultrasound images.

[0016] Figure 4 This is a schematic diagram of the confusion matrix of the model on the validation set at the third fold. Detailed Implementation

[0017] To provide a clearer understanding of the technical features, objectives, and effects of the embodiments of the present invention, specific implementation methods of the embodiments of the present invention will now be described with reference to the accompanying drawings.

[0018] In this document, “exemplary” means “serving as an example, illustration or description”, and any illustrations or implementations described herein as “exemplary” should not be construed as a more preferred or advantageous technical solution.

[0019] To enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art should fall within the protection scope of the present invention.

[0020] See Figure 1 This invention provides an integrated intelligent diagnostic system for skin lesions using dual-modal ultrasound imaging, comprising: Initial extraction module 110: Acquires B-mode ultrasound images and Doppler ultrasound images as input, and uses two independent feature extractors to extract initial features for the two modes respectively; Bidirectional cross-modal attention module 120: The initial features of the two modalities interact with each other as contextual information before fusion, capturing and enhancing complementary information between the two modalities to generate enhanced features; Feature fusion module 130: Based on the enhanced features, evaluate the relative reliability of the two modalities in the current sample, generate dynamic importance weights, and obtain fused features; Classification and recognition module 140: Inputs the fused features into different classifiers and outputs multiple prediction results; Result output module 150: Integrates multiple prediction results and uses uncertainty for dynamic weighting to obtain the final intelligent diagnosis result.

[0021] Optionally, the enhanced feature is represented as:

[0022]

[0023] in, All are learnable linear projection matrices. These are the initial features for the two modes, respectively.

[0024] Optionally, the fusion feature is represented as:

[0025]

[0026] in, This represents the dynamic importance weight.

[0027] Optionally, the classifier includes a standard classifier and a prototype classifier; Focal Loss is used as the basic classification loss. The mathematical expression is:

[0028] in, This represents the model's predicted probability of the true class label. This is a focusing parameter used to dynamically adjust sample weights; when At that time, the weight of easily classified samples with high confidence It will be significantly reduced; This is a class balancing parameter used to handle differences in the number of samples between classes; Total loss function of prototype classifier Represented as:

[0029] in, For weight hyperparameters, For prototype compactness loss, For prototype separation loss, Based on the classification of losses.

[0030] Optionally, the system consists of The integration of the models yields:

[0031] It includes three groups of models: the first group consists of five models that use ConvNeXt as the feature extractor and a prototype classifier for classification; the second group consists of five models that use ConvNeXt as the feature extractor and a standard classifier for classification; and the third group consists of five models that use DenseNet201 as the feature extractor and a standard classifier for classification.

[0032] Optionally, after each model is trained, temperature scaling is used as a post-processing calibration step to smooth the probability output; Probability output Represented as:

[0033] Where T is the temperature parameter. Score for each category.

[0034] Optionally, it also includes: employing Quantifying the predictive reliability of a single model; the final predictive probability of a single model. The uncertainty score is the average of M predictions. The variance of the M predictions over the class space:

[0035]

[0036] in, The larger the value, the more unstable the model predictions and the higher the uncertainty.

[0037] Optionally, the final intelligent diagnostic result is represented as follows:

[0038]

[0039] It is the calibration probability of all models. The weighted average; As weight.

[0040] Example: This invention provides a high-precision, high-efficiency, and high-robust intelligent diagnostic system for skin lesions based on dual-modal ultrasound images. It aims to overcome the core technical limitations of existing technologies in multimodal ultrasound diagnosis of skin lesions by integrating advanced feature extraction, feature fusion mechanisms, and ensemble learning strategies.

[0041] This invention uses an internal dataset of 1954 dermatological cases collected by Shanghai Skin Disease Hospital for empirical analysis. Each case includes one ultrasound image in both B-mode and Doppler modalities. The data encompasses six disease categories: actinic keratosis (AK), basal cell carcinoma (BCC), Bowen's disease, extramammary Paget's disease, squamous cell carcinoma of the skin (SCC), and seborrheic keratosis (SK). The number of cases in each category ranges from 145 to 701, indicating a significant class imbalance.

[0042] This invention employs a unified preprocessing workflow for input ultrasound image data. First, the raw dataset is categorized into two types based on image type: standard grayscale images and Doppler color ultrasound images, which are processed separately. Standard grayscale images are converted to single-channel grayscale format; Doppler color ultrasound images retain their original color format without color conversion. Subsequently, edge cropping is performed on the grayscale images, removing 5% of the invalid regions around the edges, while color ultrasound images require no cropping. Then, interpolation is used to scale both types of images to a uniform spatial resolution of 224×224 to ensure consistent input size for subsequent model training. The images after these steps are finally saved to a designated output directory for subsequent feature extraction and model training.

[0043] Step 1 Feature extraction from dual-modal ultrasound images Step 1.1 Dual-path feature extraction for dual-modal ultrasound images This invention receives two key medical image inputs: B-mode ultrasound images. Doppler ultrasound images Using two independent encoders Extract initial features for the two modalities respectively. :

[0044] in The initial features of the B-mode image, The initial features for the Doppler image are defined. Independent encoders ensure that structural and vascular features do not interfere with each other in the early stages of extraction.

[0045] See Figure 2 The image shown is a B-mode ultrasound image of a case. Figure 2 (middle left) and Doppler ultrasound image ( Figure 2 (Right side of the middle). All images were uniformly scaled to 224*224 pixels.

[0046] Step 1.2 Cross-modal attention enhancement of bimodal features For what was obtained A bidirectional cross-modal attention module is introduced to achieve information complementarity. This module allows... and Before fusion, they interact as contextual information to capture and enhance complementary information between the two modalities, generating enhanced features. .

[0047] (1) B-mode feature enhancement: with For querying, with These are key-value pairs. The calculation method is as follows:

[0048]

[0049] (2) Doppler feature enhancement: For querying, with These are key-value pairs. The calculation method is as follows:

[0050]

[0051] in All are learnable linear projection matrices.

[0052] Step 1.3 Dynamic weight fusion of dual-path features Using a weight generator ( This generator achieves sample-level modality adaptive fusion. It is based on enhanced features. Evaluate the relative reliability of the two modalities in the current sample and generate dynamic importance weights. Thus, the fusion characteristics are obtained. :

[0053]

[0054] The weight generator is a fully connected network with two hidden layers, which receives... As input, and finally output a weight vector. ,in This allows the model to dynamically determine which modality, B-mode or Doppler, is more important for each input sample, thereby achieving adaptive feature fusion and effectively improving the system's robustness to single-modal artifacts or missing information.

[0055] In this embodiment, taking ConvNeXt as the backbone network as an example, the structural design of the weight generator is shown in Table 1.

[0056] Table 1 Weight Generator Structure Design

[0057] See Figure 3 This invention generates Grad-CAM visualization results for both B-mode and Doppler modalities to more intuitively display the model's regions of interest in different modalities, thereby verifying the consistency between the model's discrimination criteria and medical features. Figure 3 The left side of the middle image shows the Grad-CAM visualization of B-mode ultrasound images. Figure 3 The image on the right shows the Grad-CAM visualization of the Doppler ultrasound image.

[0058] Step 1.4 Classifier Model Classifier receives As input, the output is the Logits score for each category. This invention designs two classifiers: (1) Standard classifier: based on the traditional multilayer perceptron architecture of MLP, focusing on learning the hyperplane decision boundary in the feature space.

[0059] (2) Prototype Classifier: Based on metric learning, it maintains a learnable prototype matrix P, based on... The classification is performed based on the distances between the vectors and the prototype vectors in P. Each row represents a category. The ideal feature center is denoted by D, where D represents the dimension of the feature. Classification is achieved by calculating the fused features from the input. This is achieved by calculating the distances to all prototype vectors in matrix P. The classifier selects the category represented by the prototype closest to the feature as the predicted category. Furthermore, the optimization objective of the prototype classifier includes not only prediction accuracy but also explicitly promotes intra-class compactness and inter-class separability through prototype loss (see step 2.1).

[0060] In this embodiment, category C = 6. The standard classifier is a two-layer MLP structure, including BatchNorm and Dropout to support subsequent MC Dropout uncertainty estimation. Its structural design is shown in Table 2.

[0061] Table 2 Standard Classification Model Structure Design

[0062] Step 2: Training the intelligent diagnostic model Step 2.1 Design of loss function for model training (1) Basic classification loss Focal Loss Focal Loss is used as the basic classification loss. This loss significantly reduces the loss contribution of easily classified samples through a dynamic scaling mechanism, allowing the model training to focus on difficult-to-classify samples, thus addressing the severe class imbalance problem in skin lesions. Its mathematical expression is:

[0063] in This represents the model's predicted probability of the true class label. This is the focusing parameter, used to dynamically adjust sample weights. When At that time, the weight of easily classified samples with high confidence It will be significantly reduced. This is a class balancing parameter used to handle differences in the number of samples between classes.

[0064] (2) Prototype loss Prototype loss is applied only to prototype classifier models and is used for explicit optimization. The characteristic space geometry. It is divided into prototype compactness loss. Separation loss from prototype It consists of two parts. The prototype compactness loss is used to enhance intra-class compactness, causing features of samples of the same class to move closer to their prototype vectors; while the prototype separation loss is used to enhance inter-class separability, causing prototypes of different classes to move further apart.

[0065]

[0066]

[0067] in This is the prototype vector of the category to which the sample belongs. For prototype vectors of different categories, and .

[0068] Total loss function of the prototype model It is a weighted combination of various losses, and the three optimization objectives are balanced through hyperparameters:

[0069] in For weight hyperparameters.

[0070] Step 2.2 Temperature Scaling Calibration After model training, temperature scaling is used as a post-processing calibration step. This scheme learns a single temperature parameter T on an independent validation set. The original Logits are then divided by T before being applied. This smooths the probability output, thereby correcting the overconfidence tendency commonly found in deep learning models and ensuring the smoothness of the probability output. Reliability:

[0071] Step 3: Inference of Heterogeneous Integration Model Based on Dynamic Weighted Voting Step 3.1 Uncertainty estimation for a single sample use For a single model The reliability of the prediction is quantified. Maintaining this during the inference phase. Layer activation, performed M times on a single sample (here) Random forward propagation is performed, but gradient backward propagation is not. Model Final prediction probability The uncertainty score is the average of M predictions. The variance of the M predictions over the class space:

[0072]

[0073] The larger the value, the more unstable the model predictions and the higher the uncertainty.

[0074] Step 3.2 Construction of Heterogeneous Model Pool and Uncertainty-Weighted Voting The final decision-making stage integrates the predictions from multiple models and dynamically weights them using uncertainty. During model training, this invention uses a portion of the dataset as a test set; this portion is not used in training but only in the testing phase. The remaining data is trained using five-fold cross-validation, with four folds used for training and one fold used as the validation set each time. This yields five optimal models based on the five folds. This invention constructs a system containing... Heterogeneous model pool of models The dataset contains three groups of models: the first group consists of five models using ConvNeXt as the feature extractor and a prototype classifier for classification; the second group consists of five models using ConvNeXt as the feature extractor and a standard classifier for classification; and the third group consists of five models using DenseNet201 as the feature extractor and a standard classifier for classification. Each group represents the five best models obtained through five-fold training under that specific setting. By combining different feature extractors and learning paradigms, this invention aims to maximize the diversity among models and ensure that their prediction errors are decoupled. All models in the model pool have not seen data from the test set, thus eliminating data leakage issues.

[0075] The uncertainty-weighted voting mechanism is the final decision-making process in this invention. Its core idea is that the more uncertain the model, the lower its voting weight should be. The higher the value, the higher the weight. The lower:

[0076] in satisfy The weights of all 15 models are normalized to obtain the final diagnostic probability. It is the calibration probability of all models. Weighted average:

[0077] This invention selects The category with the highest probability is used as the final intelligent diagnostic result:

[0078] In this embodiment, the model structures of ConvNeXt and DenseNet are shown in Tables 3 and 4.

[0079] Table 3 ConvNeXt Model Structure Design

[0080] Table 4 DenseNet Model Structure Design

[0081] Validation results based on an internal dataset of 1954 dermatology cases collected from Shanghai Skin Disease Hospital demonstrate that the proposed method exhibits high accuracy and stability in the diagnosis of skin lesions. Even with significant differences in sample size across categories and marked class imbalance, it maintains excellent predictive performance, demonstrating good generalization ability and engineering applicability. Compared to existing intelligent ultrasound-guided methods for diagnosing skin lesions, this invention demonstrates advancements and superiority in feature representation, diagnostic accuracy, and reliability.

[0082] Step 4: Analysis of the Advantages of the Solution Step 4.1 Performance Indicator Selection Given that this invention aims to achieve accurate and reliable intelligent diagnosis of various skin lesions, and considering the significant class imbalance in the internal dataset, selecting appropriate and representative performance evaluation metrics is crucial. This invention employs the following evaluation metrics to comprehensively and objectively assess the advancement and clinical applicability of the proposed solution: (1) Accuracy: Used to measure the overall ability of the model to correctly classify, reflecting its global performance.

[0083] (2) Area under the curve (AUC): As a comprehensive performance indicator across different classification thresholds, AUC has strong robustness to class imbalance problems and can more comprehensively reflect the model's discriminative ability.

[0084] (3) F1 score: It is the harmonic average of precision and recall, which is more suitable for class imbalance scenarios and can comprehensively reflect the overall performance of the model in different classes.

[0085] (4) Macro Sensitivity: Used to evaluate the model's ability to identify positive cases. Due to the uneven distribution of data, this invention uses a macro-average method to ensure that malignant lesion categories with smaller sample sizes receive sufficient weight in the index, thereby reflecting the model's detection ability and robustness against minority classes.

[0086] (5) Macro Specificity: Used to measure the model’s performance in correctly identifying negative examples. Macro averaging ensures that the evaluation weights of different categories are consistent, thus reflecting the overall specificity of the model in multi-class tasks more fairly.

[0087] Step 4.2 Performance Result Analysis The model prediction results based on the internal dataset of Shanghai Skin Disease Hospital are shown in Table 5.

[0088] Table 5. Statistics of Model Prediction Results

[0089] The experimental results demonstrate that the proposed multimodal intelligent diagnostic system for skin lesions exhibits excellent performance in overall classification, stability, and model reliability. Firstly, the model achieves an average accuracy of 79.71% and an AUC of 0.9575, indicating high overall recognition ability across multiple skin lesion categories and maintaining stable discrimination under different discrimination thresholds. The F1 score of 0.7893 further reflects the model's good balance between precision and recall under class imbalance. Furthermore, the macro-sensitivity reaches 77.54%, and the macro-specificity reaches 95.35%, with standard deviations of only ±0.96% and ±0.29%, respectively, indicating balanced performance across different categories, particularly maintaining good detection and false positive control capabilities in categories with smaller sample sizes. In summary, these results fully demonstrate that the proposed solution not only possesses high accuracy and robustness but also maintains stable and reliable diagnostic performance under class imbalance conditions, possessing significant potential for clinical application.

[0090] Furthermore, this invention demonstrates the confusion matrix results of cross-validation at the third fold, such as... Figure 4 As shown, this is used to further demonstrate the model's actual classification performance across various categories.

[0091] Compared with the prior art, the present invention has the following beneficial effects: (1) This invention constructs an integrated system with high accuracy, high robustness, and self-reliable quantification capabilities. By simultaneously utilizing the structural information of the B-mode and the functional information of the Doppler, this invention achieves bimodal complementary diagnosis, significantly outperforming methods that rely solely on single-modal images. Furthermore, by leveraging cross-modal information interaction and adaptive dynamic weight calculation, this invention enables the system to dynamically adjust the contributions of the B-mode and Doppler modes based on the image quality and information reliability of the current sample. When the image quality of a certain modality is affected by artifacts, its weight is automatically reduced, thereby preventing low-quality modalities from contaminating the fusion features and greatly enhancing the robustness of the system.

[0092] (2) To address the inherent class imbalance challenge in medical image data, this invention adopts a customized optimization strategy. First, Focal Loss is used as the base loss, and by dynamically scaling the weights, the imbalance problem of rare class samples such as malignant lesions is effectively solved, making the model training more focused on difficult-to-classify samples. Second, for the prototype classifier, a prototype compactness loss is introduced. With separation loss This explicitly shapes the feature space from a geometric perspective, enhancing intra-class compactness and inter-class separability, thereby improving the stability and discriminative power of the model during clinical generalization.

[0093] (3) This invention focuses on improving clinical credibility and robustness in the decision-making stage. The system first uses temperature scaling calibration to ensure that the probabilities output by all models can truly reflect the accuracy of the models and correct the overconfidence tendency common in deep learning models. Subsequently, the uncertainty score of each model is quantified through the MC Dropout mechanism. The prediction results of multiple heterogeneous models are fused through uncertainty-weighted voting, realizing sample-level adaptive decision-making, which significantly improves the reliability of the final diagnostic results and the clinical application value.

[0094] As another example, embodiments of the present invention also provide an electronic device, which is an example of a hardware device that can be applied to various aspects of the present invention. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0095] The electronic device may include a processor, a communications interface, memory, and a communications bus.

[0096] The processor, communication interface, and memory communicate with each other via a communication bus. The communication interface is used to communicate with other electronic devices or servers.

[0097] The processor is used to execute programs, specifically the relevant steps in the above system embodiments.

[0098] Specifically, the program may include program code, which includes computer operation instructions.

[0099] The processor may be a CPU, an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The one or more processors included in a smart device may be of the same type, such as one or more CPUs; or they may be of different types, such as one or more CPUs and one or more ASICs.

[0100] Memory is used to store programs. Memory may include high-speed RAM, and may also include non-volatile memory, such as at least one disk drive.

[0101] When executed by a processor, the program is used to cause an electronic device to perform the steps of the system of the present invention.

[0102] Furthermore, the specific implementation of each step in the program can be found in the corresponding descriptions of the steps and units in the above system embodiments, and will not be repeated here. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices and modules described above can be referred to the corresponding process descriptions in the aforementioned system embodiments, and will not be repeated here.

[0103] An exemplary embodiment of the present invention also provides a computer storage medium storing a computer program, wherein the steps of system execution of the various embodiments of the present invention when the computer program is executed by a processor can be referred to the corresponding process descriptions in the foregoing system embodiments, and will not be repeated here.

[0104] The steps performed by the system according to embodiments of the present invention can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as a CD-ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code originally stored on a remote recording medium or a non-transitory machine-readable medium and subsequently stored on a local recording medium, downloaded via a network. Thus, the system described herein can be processed by software stored on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an ASIC or FPGA). It is understood that the computer, processor, microprocessor controller, or programmable hardware includes storage components (e.g., RAM, ROM, flash memory, etc.) capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the system described herein. Furthermore, when a general-purpose computer accesses the code used to implement the system shown herein, the execution of the code transforms the general-purpose computer into a dedicated computer for executing the system shown herein.

[0105] Specific embodiments of the invention have now been described. Other embodiments are within the scope of the appended claims. In some cases, the actions described in the claims can be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing can be advantageous.

[0106] It should be understood that although this specification is described according to various embodiments, not every embodiment contains only one independent technical solution. This way of describing the specification is only for clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

[0107] Finally, it should be noted that the above embodiments are only used to illustrate the embodiments of the present invention, and are not intended to limit the embodiments of the present invention. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the embodiments of the present invention. Therefore, all equivalent technical solutions also fall within the scope of the embodiments of the present invention, and the patent protection scope of the embodiments of the present invention should be defined by the claims.

Claims

1. A dual-modal ultrasound imaging intelligent diagnostic integrated system for skin lesions, characterized in that, include: Initial extraction module: It acquires B-mode ultrasound images and Doppler ultrasound images as input, and uses two independent feature extractors to extract initial features for the two modes respectively; Bidirectional cross-modal attention module: The initial features of the two modalities interact with each other as contextual information before fusion, capturing and enhancing complementary information between the two modalities to generate enhanced features; Feature fusion module: Based on the enhanced features, evaluate the relative reliability of the two modalities in the current sample, generate dynamic importance weights, and obtain fused features; Classification and recognition module: Inputs the fused features into different classifiers and outputs multiple prediction results; Results output module: It integrates multiple prediction results and uses uncertainty to dynamically weight them to obtain the final intelligent diagnostic result.

2. The system according to claim 1, characterized in that, The enhanced features are represented as follows: in, All are learnable linear projection matrices. These are the initial features for the two modes, respectively.

3. The system according to claim 2, characterized in that, The fusion feature is represented as follows: in, This represents the dynamic importance weight.

4. The system according to claim 1, characterized in that, The classifier includes a standard classifier and a prototype classifier; Focal Loss is used as the basic classification loss. The mathematical expression is: in, This represents the model's predicted probability of the true class label. This is a focusing parameter used to dynamically adjust sample weights; when At that time, the weight of easily classified samples with high confidence It will be significantly reduced; This is a class balancing parameter used to handle differences in the number of samples between classes; Total loss function of prototype classifier Represented as: in, For weight hyperparameters, For prototype compactness loss, For prototype separation loss, Based on the classification of losses.

5. The system according to claim 4, characterized in that, The system consists of The integration of the models yields: It includes three sets of models: The first group consists of five models that use ConvNeXt as a feature extractor and a prototype classifier for classification. The second group consists of five models that use ConvNeXt as a feature extractor and a standard classifier for classification. The third group consists of five models that use DenseNet201 as a feature extractor and a standard classifier for classification.

6. The system according to claim 5, characterized in that, After each model is trained, temperature scaling is used as a post-processing calibration step to smooth the probability output. Probability output Represented as: Where T is the temperature parameter. Score for each category.

7. The system according to claim 1, characterized in that, Also includes: use Quantify the predictive reliability of a single model; Final prediction probability of a single model The uncertainty score is the average of M predictions. The variance of M predictions over the class space: in, The larger the value, the more unstable the model predictions become, and the higher the uncertainty.

8. The system according to claim 1, characterized in that, The final intelligent diagnostic result is expressed as follows: It is the calibration probability of all models. The weighted average; As weight.

9. An electronic device, characterized in that, include: processor; Memory for stored programs; The program includes instructions that, when executed by the processor, cause the processor to perform the steps performed by the system as described in any one of claims 1-8.

10. A computer storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the steps performed by the system as described in any one of claims 1-8.

Images