Distributed model liver nodule multi-classification diagnosis method based on ultrasound contrast

Abstract

The invention relates to the technical field of liver nodule diagnosis, and discloses a distributed model liver nodule multi-classification diagnosis method based on ultrasound contrast, which comprises the following steps of: analyzing and quantifying motion information of pixels or objects in an image sequence in image processing by using optical flow; dynamic changes in a scene are revealed by capturing displacement vectors of pixel points between connection frames, the core is to analyze visual data from a time dimension, and basic support is provided for tasks such as motion perception, target tracking, behavior analysis and the like; according to the multi-classification model, two training strategies are adopted in model training and are respectively a direct training strategy and a distributed training strategy, the direct training strategy is a classification model and completes a multi-classification task, and the distributed training strategy is to train a plurality of dichotomy models and aggregate a plurality of sub-models; the image-molecule-disease forms a composite dimension reasoning framework, and the core is to introduce a molecular phenotype to connect a biological bridge of image features and clinical diagnosis.

Description

Technical Field

[0001] The present invention relates to the technical field of liver nodule diagnosis, and in particular to a distributed model liver nodule multi-classification diagnosis method based on ultrasound contrast imaging. Background Art

[0002] Accurate multiclassification is a prerequisite for the rational management of focal liver lesions (FLLs). Ultrasound is a commonly used imaging modality, but its accuracy is limited. Contrast-enhanced ultrasound (CEUS) performs better but is highly dependent on experience. Therefore, we aimed to develop a CEUS-based artificial intelligence (AI) model for the multiclassification of FLLs and evaluate its performance in a multicenter clinical trial. Summary of the Invention

[0003] (1) Technical problems solved In response to the shortcomings of the existing technology, the present invention provides a distributed model liver nodule multi-classification diagnosis method based on ultrasound contrast imaging, which has the advantages of very small differences in optical flow maps, strong anti-interference ability, improved model prediction accuracy, etc., and solves the problem of liver nodule diagnosis.

[0004] (2) Technical solution

[0005] To achieve the above-mentioned liver nodule diagnosis purpose, the present invention provides the following technical solution: a distributed model liver nodule multi-classification diagnosis method based on ultrasound contrast imaging, comprising the following steps: S1: Optical flow is used in image processing to analyze and quantify the motion information of pixels or objects in image sequences. It reveals dynamic changes in a scene by capturing the displacement vectors of pixels between connected frames. Its core function is to analyze visual data from a temporal perspective, providing fundamental support for tasks such as motion perception, target tracking, and behavior analysis. S2: Multi-classification model. Two training strategies are adopted in model training, namely direct training strategy and distributed training strategy. The direct training strategy uses one classification model to complete multi-classification tasks, while the distributed training strategy trains multiple binary classification models and aggregates multiple sub-models. S3: “Image-molecule-disease” constitutes a composite dimensional reasoning framework, the core of which is to introduce molecular phenotypes as a biological bridge connecting imaging features with clinical diagnosis.

[0006] Preferably, the optical flow can capture the change trend of two adjacent images and focus on the image itself and image changes.

[0007] Preferably, the optical flow is combined with CEUS and key frame extraction is performed.

[0008] Preferably, the optical flow method and deep learning method automatically analyze and compare CEUS videos, and use multiple biomarker information to build a non-invasive diagnostic strategy "from CEUS to biomarkers to disease".

[0009] Preferably, the distributed training is conducive to refined parameter adjustment and is passive data enhancement.

[0010] Preferably, the distributed training fuses the five sub-models into a six-classification model.

[0011] Preferably, the CEUS, biomarkers and clinical information are integrated and a multi-classification model is constructed for classifying six focal liver lesions.

[0012] Preferably, the composite dimensional framework of impact-molecule-disease composition allows the model to diagnose four pathological features of nodules based on images.

[0013] Preferably, the pathological features have a tendency distribution in different nodule types, so as to improve the overall predictive efficiency of the model by 5%, and enable the model to have a cross-modal function of "macro-micro" linkage, as well as the ability to predict patient prognosis based on molecular function diagnosis.

[0014] (3) Beneficial effects

[0015] Compared with the existing technology, the present invention provides a distributed model liver nodule multi-classification diagnosis method based on ultrasound contrast imaging, which has the following beneficial effects: 1. This invention ensures that the difference between images from the same device is stable and the difference in optical flow maps is small, so the optical flow has strong anti-interference ability; 2. This invention applies optical flow estimation to CEUS for keyframe extraction, effectively improving the accuracy of model prediction and enabling the established model to maintain good robustness in complex environments with different manufacturers, different models, and different doctors' operating habits; 3. According to the invention, when the predicted probabilities of the five sub-models are very low, the model will diagnose it as other types. Therefore, even if it encounters a disease that has never appeared in history, the model will not make such errors. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 The data collection and seven-queue division flow chart of the present invention; Figure 2 A flow chart of the experimental design of the present invention; Figure 3Performance evaluation diagrams of the disease module, biomarker module, and clinical module in the first phase of construction of the present invention; Figure 4 Performance evaluation diagrams of Model D, Model DC, Model DB, and Model DCB constructed in the second phase of the present invention; Figure 5 The performance graph of the model DCB of the present invention in the internal test set and the external test sets A, B and C; Figure 6 Figure 3 is a comparison and collaboration diagram between the DCB model of the present invention and radiologists in external test sets A, B and C. DETAILED DESCRIPTION

[0017] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the embodiments of the present invention and the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0018] A distributed model liver nodule multi-classification diagnosis method based on ultrasound contrast imaging includes the following steps: S1: Optical flow is used in image processing to analyze and quantify the motion information of pixels or objects in image sequences. It reveals dynamic changes in a scene by capturing the displacement vectors of pixels between connected frames. Its core function is to analyze visual data from a temporal perspective, providing fundamental support for tasks such as motion perception, target tracking, and behavior analysis. S2: Multi-classification model. Two training strategies are adopted in model training: direct training strategy and distributed training strategy. The direct training strategy uses a single classification model to complete multi-classification tasks, while the distributed training strategy trains multiple binary classification models and aggregates multiple sub-models. S3: “Image-molecule-disease” constitutes a composite dimensional reasoning framework, the core of which is to introduce molecular phenotypes as a biological bridge connecting imaging features with clinical diagnosis.

[0019] Optical flow is used in contrast-enhanced ultrasound (CEUS). Unlike ultrasound, CT, or MRI, CEUS is a video format. In CEUS videos, tumors do not remain static but move with time and the patient's respiration. Furthermore, a single CEUS video typically contains over 1000 frames, and CEUS image states vary significantly between different devices. Traditional CEUS keyframe extraction methods operate on the original images, but image states vary significantly between different devices, making these differences difficult to eliminate even with image normalization. In this study, we employed an optical flow method to capture the changing trends between two adjacent images. This approach has two advantages: First, the difference between images from the same device is stable, resulting in minimal differences in the optical flow map, making it highly resistant to interference. Second, traditional keyframe extraction methods destroy the temporal continuity inherent in CEUS videos. Optical flow estimation, however, focuses not only on the image itself but also on image changes, partially reflecting the temporal continuity of CEUS videos. By applying optical flow estimation to CEUS and performing keyframe extraction, we effectively improved the model's prediction accuracy and enabled our established model to maintain good robustness in complex environments with different manufacturers, models, and operating habits of different doctors.

[0020] Optical flow and deep learning methods automatically analyze and compare CEUS videos, and use a variety of biomarker information to build a non-invasive diagnostic strategy "from CEUS to biomarkers to disease." CEUS, biomarkers, and clinical information are integrated to build a multi-classification model for classifying six focal liver lesions. The model's performance is better than that of junior CEUS radiologists and comparable to that of senior CEUS / MRI radiologists. With the assistance of the model, the performance of junior CEUS radiologists can be improved to the level of senior CEUS / MRI radiologists.

[0021] Distributed training is superior to direct training for two reasons. First, it facilitates refined parameter adjustment. While adjusting some parameters can improve the diagnostic performance of liver cancer, it can reduce the diagnostic performance of hemangiomas. If a single model is used, targeted adjustments are impossible. Second, distributed training provides passive data augmentation. If a multi-classification model is trained directly using 100 cases, each binary sub-model will also be trained using 100 cases, for a total of 500 cases in distributed training. Furthermore, distributed training offers a unique advantage. In this project, we fused five sub-models into a six-classification model, adding an additional category for rare diseases. This category is crucial for the model's real-world applicability. For example, echinococcosis is a regional disease in Qinghai and Gansu. If a five-classification model achieves 100% accuracy for the five trained categories under experimental conditions, and doctors trust the model's results, then if this model is applied in Qinghai and encounters echinococcosis, it will inevitably make misdiagnoses, and doctors will find it difficult to detect these errors. In our model, when the predicted probabilities of the five sub-models are all low, the model will diagnose it as a different type. Therefore, even if it encounters a disease that has never appeared in history, the model will not make such mistakes.

[0022] In artificial intelligence for medical imaging, traditional approaches typically rely on a single-dimensional reasoning process, "image-disease." However, in this project, we incorporate a multi-dimensional reasoning process, "image-molecule-disease." Traditional diagnostic models have long adhered to this single-dimensional "image-disease" reasoning paradigm. This classic approach directly maps medical imaging features to clinical diagnostic labels by building end-to-end deep learning models. While these models demonstrate high diagnostic efficiency in specific scenarios, their inherent limitations are becoming increasingly apparent. First, the lack of an interpretable intermediate bridge between imaging features and disease representations results in a "black box" nature in the model's decision-making process. Second, single-dimensional reasoning mechanisms struggle to capture the molecular mechanisms underlying disease development, limiting their usefulness in early diagnosis and accurate classification. Third, when faced with complex diseases with high heterogeneity, these simplified models are susceptible to interference from imaging artifacts, device variability, and other factors. In this project, we innovatively constructed a multi-dimensional reasoning framework, "image-molecule-disease." The core of this paradigm shift lies in the introduction of molecular phenotypes as a biological bridge connecting imaging features with clinical diagnosis. We incorporated four pathological features of nodules based on image diagnosis, which are distributed favorably across different nodule types. This addition improved the model's overall predictive performance by approximately 5%, providing cross-modal capabilities for macro-micro linkage and the ability to predict patient prognosis based on molecular diagnostics.

[0023] CEUS videos, immunohistochemical biomarkers, and clinical information of solid FLLs (FLLs) >1 cm in diameter in adults were collected from 52 centers to construct and test the model. The model is designed to classify FLLs (focal liver lesions) into six types: hepatocellular carcinoma (HCC), hepatic metastatic carcinoma (HM), intrahepatic cholangiocarcinoma (ICC), hepatic hemangioma (HH), hepatic abscess (HA), and other types (OT). First, the disease module, biomarker module, and clinic module were constructed in the training set A and the validation set. The three modules were then aggregated into Model-DCBP (composed of the three modules: Module-Disease, Module-Biomarker, and Module-Clinic) in the training set B and the internal test set. The performance of Model-DCBP was compared with that of radiologists using CEUS and MRI (Magnetic Resonance Imaging) in three external test sets.

[0024] Results: A total of 3725 FLLs from 52 centers were divided into training set A (n=2088), validation set (n=592), training set B (n=234), internal test set (n=110), and external test sets A (n=113), B (n=276), and C (n=312). Model-DCB significantly outperformed junior CEUS radiologists (0.59-0.73) in external test sets A, B, and C (accuracies ranging from 0.85 to 0.86) and was comparable to senior CEUS radiologists (0.79-0.85) and senior MRI radiologists (0.82-0.86). In multiple subgroup analyses tailored to demographic characteristics, tumor characteristics, and ultrasound equipment, accuracy ranged from 0.79 to 0.92.

[0025] Conclusion: CEUS-based Model-DCB can accurately perform multi-classification of FLL. It is expected to benefit a wide range of populations, especially those in remote suburban areas where MRI examinations are difficult to obtain.

[0026] Impact and Significance: Ultrasound is the most commonly used imaging modality for screening focal liver lesions (FLLs), but it lacks accuracy in multiclassification, a prerequisite for appropriate management. Contrast-enhanced ultrasound (CEUS) offers improved diagnostic performance but is highly dependent on the radiologist's experience. Our developed CEUS-based model (Model-DCB) can help junior CEUS radiologists achieve diagnostic capabilities comparable to those of senior CEUS radiologists and senior MRI radiologists. The combination of ultrasound equipment, CEUS examination, and Model-DCB enables excellent diagnostic performance by junior radiologists even in remote areas.

[0027] CEUS treatment: FLLs were annotated using a combination of manual initialization and automated segmentation. A radiologist with eight years of CEUS experience outlined the lesion boundary on the arterial phase frame with the largest tumor size. This was the only manual initialization step. All subsequent steps were automated. A rectangular box was then generated that minimally covered the manually outlined area and expanded by 20 pixels to form a region of interest (ROI). Similar ROIs were then generated for all frames of the CEUS video, and time-intensity curves (TICs) based on these ROIs were generated (Supplementary Figure S1).

[0028] For each video, 80 frames were uniformly collected from the onset to the peak of the TIC curve, and 20 frames were uniformly collected from the peak to the end (Supplementary Figure S1). Then, a specially designed two-stream model was used to extract spatial and temporal information from these 100 ROIs (Supplementary Materials and Figure S2). Each model consists of a spatial branch and a temporal branch ( Figure 2 A). The spatial branch extracts spatial features using ResNet34. The temporal branch combines the pixel displacements between two adjacent frames into an optical flow map, which is then passed through ResNet18 to extract temporal features. We found that even when we downsampled each CEUS video to 100 frames, the optical flow variation between adjacent frames was too small, which misled model training, consumed excessive computing power, and sometimes even crashed our computing system (experiments were conducted using two RTX4090 Ti graphics cards). Therefore, we compared the performance of downsampling the optical flow map to 20, 15, 10, and 5 frames per video, ultimately selecting 10 frames per video as model input to ensure a balance between inference efficiency, classification accuracy, and system stability (Supplementary Figure S3).

[0029] Two-stage model building: In the first phase, three modules were constructed, namely Module-Disease, Module-Biomarker and Module-Clinic ( Figure 2 B). We use a Long Short-Term Memory (LSTM) network to integrate features extracted from the spatial and temporal branches, developing Module-Disease and Module-Biomarker. We adaptively weight the output prediction probabilities. We use a weighted sum of the spatial branch classification loss, the temporal branch classification loss, and the final classification loss as the loss function. We also weight the minority class to mitigate bias caused by imbalanced class distribution. A detailed description, function definitions, and related formulas are provided in the supplementary material.

[0030] In Module-Disease, two model building strategies were compared. Strategy A directly built a multi-classification model. Strategy B used a distributed training method to aggregate multiple binary classification sub-models to build a multi-classification model. Five sub-models were trained, including: HCC vs. non-HCC, HM vs. non-HM, ICC vs. non-ICC, HH vs. non-HH, and HA vs. non-HA. In Module-Biomarker, four binary classification sub-models were developed, including: Hep positive vs. Hep negative, GPC3 positive vs. GPC3 negative, CK7 positive vs. CK7 negative, and CK19 positive vs. CK19 negative. In Module-Clinic, all collected clinical information was filtered by the correlation coefficient with the FLL type (Supplementary Materials).

[0031] In the second stage, the corresponding modules were combined through the Multilayer Perceptron (MLP) to construct four multi-classification models, namely Model-D, Model-DC, Model-DB and Model-DCB, for performance comparison ( Figure 2 C). Furthermore, at this stage, OT cases were added as a supplementary classification to allow for classification of the six types of FLL (five common types + OT). This was achieved by utilizing an MLP. Instead of training a sub-model for OT vs. non-OT, OT was set as an additional classification in the MLP in addition to the original five common types (Supplementary Figure S4). The basic rule was that if a case had a low probability of being predicted as belonging to all five FLL types, it would be classified as OT. This is because the number of OT cases is inherently too small to support sub-model training based on CEUS videos.

[0032] Figure 1 :Data collection and seven queue division flow chart CEUS (contrast-enhanced ultrasound); FLL (focal liver lesion); HCC (hepatocellular carcinoma); HM (liver metastasis); ICC (intrahepatic bile duct carcinoma); HH (hepatic hemangioma); HA (hepatic abscess); OT (other types).

[0033] Figure 2 :Experimental design flow chart (A) During contrast-enhanced ultrasound (CEUS) processing, we extract downsampled keyframes based on the temporal intensity curve (TIC) of the CEUS video. We generate optical flow maps from these keyframe sequences and integrate latent features from the spatial and temporal branches using a long short-term memory (LSTM) network. This strategy is used to construct the disease module (Module-Disease) and biomarker module (Module-Biomarker) models, respectively.

[0034] (B) In the first phase of construction, three modules were developed. The disease module (Module-Disease) was derived by comparing direct classification of five focal liver lesion (FLL) types (Strategy A) with aggregating multiple binary classification sub-models (Strategy B). The biomarker module (Module-Biomarker) comprises four binary classification sub-models trained on four biomarkers. The clinical module (Module-Clinic) was constructed by selecting clinical information highly relevant to FLL.

[0035] (C) In the second phase, four multi-classification models were constructed: Model D (Model-D), Model DC (Model-DC), Model DB (Model-DB), and Model DCB (Model-DCB). Each model combines corresponding modules using a multilayer perceptron (MLP). Their performance was compared, and the best-performing model was selected to advance to the next phase.

[0036] (D) During the clinical testing phase, we compared the performance of the artificial intelligence (AI) model with that of six contrast-enhanced ultrasound radiologists (three junior and three senior radiologists) and two senior magnetic resonance imaging (MRI) radiologists. We explored the collaboration between the contrast-enhanced ultrasound radiologists and the model. We also assessed the model's performance in multiple subgroups.

[0037] CEUS = contrast-enhanced ultrasound; TIC = time-intensity curve; LSTM = long short-term memory; FLL = focal liver lesion; HCC = hepatocellular carcinoma; HM = liver metastasis; ICC = intrahepatic cholangiocarcinoma; HH = hepatic hemangioma; HA = liver abscess; OT = other.

[0038] Figure 3 :Performance evaluation of the disease module (Module-Disease), biomarker module (Module-Biomarker) and clinical module (Module-Clinic) in the first phase of construction.

[0039] (A) In the disease module (Module-Disease), for the classification of five focal liver lesions (FLLs) in the validation set (n=592), strategy B (green) using distributed training generally outperforms strategy A (purple) using direct training, with better performance in terms of confusion matrix, macro-AUC (Macro-AUC), accuracy, macro precision, macro recall, and macro F1 (Macro F1).

[0040] (B) In the Biomarker Module (Module-Biomarker), the ROC curves of the four biomarker sub-models in the training set A (blue) and the validation set (black) are shown.

[0041] (C) In the Clinical Module (Module-Clinic), seventeen clinical information items were selected because each had a moderate correlation coefficient (absolute value > 0.2) with at least one FLL type. The direction of the ellipse indicates a positive or negative correlation between the factor and the FLL type. The color and shape of the ellipse indicate the degree of correlation.

[0042] CEUS (contrast-enhanced ultrasound); AUC (area under the curve); FLL (focal liver lesion); HCC (hepatocellular carcinoma); HM (liver metastasis); ICC (intrahepatic cholangiocarcinoma); HH (hepatic hemangioma); HA (liver abscess); Hep (hepatocyte antigen); GPC3 (glycoprotein 3); CK7 (cytokeratin 7); CK19 (cytokeratin 19); WBC (white blood cell count); PLT (platelet count); ALT (alanine aminotransferase); AST (aspartate aminotransferase); TP (total protein); ALB (albumin); AFP (alpha-fetoprotein); CEA (carcinoembryonic antigen); CA199 (cancer antigen 199); CA125 (cancer antigen 125).

[0043] Figure 4 : Performance evaluation of Model D (Model-D), Model DC (Model-DC), Model DB (Model-DB) and Model DCB (Model-DCB) in the second phase of construction.

[0044] (A) In the internal test set (n=110), model DCB (light blue) outperforms model D (light green), model DC (light green), and model DB (dark green) in terms of confusion matrix, macro AUC (MacroAUC), accuracy (ACC), macro specificity (Macro Specificity), macro recall (Macro Recall), macro negative predictive value (Macro NPV), and macro F1 (Macro F1).

[0045] (B) Radar chart showing the different performances of the four models in the classification of six focal liver lesions (FLLs).

[0046] AUC (area under the curve); ACC (accuracy); HCC (hepatocellular carcinoma); HM (liver metastasis); ICC (intrahepatic bile duct carcinoma); HH (hepatic hemangioma); HA (hepatic abscess); OT (other types).

[0047] Model generalization and robustness experiments: Because our study population consisted of patients with FLL undergoing CEUS examination, the distribution of FLLs is not consistent with the natural distribution of FLLs. We conducted generalization experiments on our model by simulating three clinical scenarios: first, a health checkup center (with a low proportion of malignant tumors and a high proportion of benign tumors, 1:9); second, a hospital outpatient department (with an equal ratio of malignant and benign tumors, 5:5); and third, a hospital inpatient department (with a high proportion of malignant tumors and a low proportion of benign tumors, 9:1). We randomly selected cases from three external test sets based on the aforementioned ratios to form the following cohorts: a health checkup center cohort (n=100), a hospital outpatient department cohort (n=100), and a hospital inpatient department cohort (n=100). We used the model to diagnose patients in these cohorts and observed the impact of changes in FLL prevalence on model performance. This experiment was repeated 100 times (Supplementary Figure S6a). Because we used a combination of manual and automatic ROI annotation, we conducted robustness tests to assess the impact of variability in manual annotation on model performance. We randomly adjusted the ROIs manually annotated by radiologists by expanding, shrinking, moving, or combining these methods to simulate the impact of annotation variability on model performance. This experiment was repeated 100 times (Supplementary Fig. S6b).

[0048] Performance of Model-DCB in retrospective and prospective test sets: The accuracy of Model-DCB in the internal test set and external test sets A, B, and C were 0.90 (95% CI [0.85-0.95]), 0.85 (95% CI [0.78-0.92]), 0.85 (95% CI [0.80-0.89]), and 0.86 (95% CI [0.82-0.90]), respectively. Figure 5 A). The Macro-AUC, Macro-Specificity, Macro-Recall, Macro-Precision, Macro-NPV and Macro-F1 of Model-DCB in the four test sets ranged from 0.87 to 0.91, 0.96 to 0.98, 0.75 to 0.85, 0.79 to 0.87, 0.97 to 0.98 and 0.78 to 0.84 respectively ( Figure 5 B). Its accuracy across all subgroups ranged from 0.77 to 1.00 ( Figure 5 C). The classification accuracy of the six types of FLL was higher than 0.90 (HCC 0.91-0.96; HM 0.93-0.94; ICC 0.95-0.97; HH 0.96-0.99; HA 0.94-0.97; OT 0.97-1.00, Figure 5 D). These results demonstrate that Model-DCB has accurate, stable, and reliable performance in multi-type FLL classification.

[0049] Figure 5 : Performance of Model-DCB in the internal test set and external test sets A, B, and C.

[0050] (A) Receiver operating characteristic (ROC) curves and macro area under the curve (Macro AUC) of model DCB in four tests; (B) Accuracy, macro-specificity, macro-precision, macro-recall, macro-negative predictive value (Macro-NPV), and macro-F1 of the DCB model in the four test sets; (C) In subgroup analyses (including sex, age, tumor size, etiology, cirrhosis, fatty liver disease, and ultrasound equipment manufacturer), most of the DCB accuracy of the model exceeded 0.80 (red dashed line); (D) In the four test sets, the accuracy of model DCB for classifying six focal liver lesions (FLLs) exceeded 0.90 (red dashed hexagons).

[0051] AUC (area under the curve); HCC (hepatocellular carcinoma); HM (liver metastasis); ICC (intrahepatic cholangiocarcinoma); HH (hepatic hemangioma); HA (hepatic abscess); OT (other types).

[0052] Figure 6 : Comparison and collaboration between model DCB and radiologists in external test sets A, B, and C.

[0053] (A) Accuracy, macro-specificity, macro-precision, macro-recall, macro-negative predictive value (Macro-NPV), and macro-F1 of the model DCB, primary contrast-enhanced ultrasound (CEUS) radiologists (without and with AI assistance), senior CEUS radiologists (without and with AI assistance), and senior magnetic resonance imaging (MRI) radiologists were plotted for comparison.

[0054] (B) With the aid of the model DCB, the macro area under the curve (Macro AUC) of junior CEUS radiologists was significantly improved compared with the previous results.

[0055] (C) With the aid of the model DCB, the macro area under the curve (Macro AUC) of senior CEUS radiologists was slightly improved compared with the previous results.

[0056] AUC (area under the curve).

[0057] In this study, we established a large FLL database, comprising 3,725 CEUS videos and 805 biomarker results from 52 centers, to build and validate our AI model (Model-DCB) for the classification of six types of FLL. We used three independent test sets, including two single-center retrospective test sets and one multicenter prospective test set, to evaluate its diagnostic performance, achieving accuracy rates of 0.85, 0.85, and 0.86, respectively. Across all test sets, Model-DCB significantly outperformed junior CEUS radiologists (all P < 0.05) and performed comparably with senior CEUS radiologists and senior MRI radiologists (all P > 0.05). With the assistance of Model-DCB, junior CEUS radiologists significantly improved their classification accuracy (P < 0.05), reaching the level of senior radiologists.

[0058] Accurate classification of FLL is important for treatment selection, prognosis prediction, and appropriate disease management. In recent years, studies utilizing AI technology to diagnose FLL have emerged. We summarize 15 outstanding studies from 2018 to date and list them in Supplementary Table S14. Compared with previous US / CEUS-based studies, ours was the first to utilize an AI approach specifically for dynamic video analysis and had the largest sample size and multicenter prospective evaluation. Therefore, our results are more robust and reliable. Only one study had a larger data set than ours, but it was primarily used to diagnose hepatic echinococcosis rather than to classify multiple types of FLL. The robust performance of our model across 84 subgroups, including sex, age, tumor size, etiology, cirrhosis, fatty liver disease, and ultrasound device manufacturer, has not been observed in other studies and suggests that our model may have better generalization capabilities.

[0059] Previous studies using deep learning to diagnose FLL have provided us with many insights. The study by Hamm CA et al. used multiphase MRI, providing a valuable reference for CEUS in terms of processing temporal information. Compared with previous studies based on CECT or MRI, our study remains superior in terms of data volume, classification types, and prospective validation. The study by Ying et al., using CECT, is the only study with a larger data set than ours. Their AI model (LiAIDS) performed well, classifying HCC, HM, ICC, HH, and HA, similar to our Model-DCB. However, unlike ours, they did not classify the challenging and rare FLL types (such as hepatic adenoma and hepatic lymphoma). Instead, they trained the model to classify hepatic cysts, which most physicians can easily diagnose in clinical practice without the need for AI. Furthermore, compared with CECT and MRI, CEUS has the inherent advantages of real-time, radiation-free, low-cost, and short examination time.

[0060] Unlike previous studies that relied solely on the "from image to disease" strategy (Module-Disease), we also fully utilized the biomarker information of FLL and added a "from image to biomarker to disease" strategy (Module-Biomarker). The relevant results proved that our hypothesis was correct. Whether from Model-D to Model-DB (accuracy increased from 0.80 to 0.86) or from Model-DC to Model-DCB (accuracy increased from 0.84 to 0.90), adding Module-Biomarker improved the performance of FLL multi-classification ( Figure 4B). Furthermore, the intelligent application of biomarker information by Model-DCB is consistent with previous biomarker studies. For example, Hep is a well-established biomarker for HCC but is rarely found in other FLL types. Therefore, the Hep sub-model synergizes with the HCC sub-model to enhance HCC identification (Supplementary Figure S8a) and plays a decisive role when the predicted probability of the HCC sub-model approaches that of another FLL sub-model (Supplementary Figure S8b). The predictive power of biomarkers also gives Model-DCB the potential to quantitatively assess the aggressiveness of malignant FLL and predict prognosis before pathological analysis, potentially facilitating better treatment decisions.

[0061] Across three external test sets, we found that Model-DCB assistance effectively improved the diagnostic performance of CEUS radiologists at all levels, particularly for junior CEUS radiologists. With the assistance of Model-DCB, both junior and senior CEUS radiologists achieved diagnostic performance comparable to or even better than that of senior MRI radiologists, who are generally considered the "silver standard" for FLL diagnosis, second only to biopsy. This AI-assisted improvement may have significant implications for clinical practice. Globally, the number of ultrasound devices and their application areas far exceed those of MRI, and ultrasound is also a cheaper, more convenient, and faster imaging method. CEUS technology enhances the diagnostic capabilities of ultrasound, and Model-DCB further reduces the experience requirements for CEUS radiologists, enabling junior or less experienced CEUS radiologists in remote or underdeveloped areas to provide diagnostic services for FLL patients comparable to those of senior MRI radiologists. As the incidence of FLL continues to rise globally, the promotion of AI models that can assist CEUS radiologists in accurately diagnosing and managing FLL will be even more significant.

[0062] Although ultrasound is the most widely used tool for liver imaging, it still has a large imaging variability due to the numerous manufacturers and different system parameter settings. To overcome this variability, we adopted a two-stream model strategy. Since the optical flow is generated by calculating the difference between two consecutive CEUS frames, this method focuses more on the changing trend of the video rather than the video itself, thus avoiding the imaging variability in CEUS. In our study, a total of 226 ultrasound devices from nine major manufacturers participated, and Model-DCB still showed high stability in all test sets ( Figure 5 C). Supplementary Figure S5 shows two examples. CEUS images from Mindray and Siemens devices have obvious differences in color saturation, brightness, and contrast, but they are well normalized by conversion to optical flow maps.

[0063] Another unique advantage of Model-DCB is that it achieves high-accuracy classification of OT without training an OT sub-model. OT includes multiple FLL types with extremely low incidence and lacks uniform imaging features, making it nearly impossible to train an effective OT sub-model (Supplementary Figure S11). However, the distributed training strategy eliminates this requirement and instead improves the diagnostic performance of OT compensatorily by improving the diagnostic performance of other sub-models. The diagnostic capability of OT is integral to the translation of AI models from the lab to the clinic, as it prevents uncontrolled misleading diagnoses of unknown diseases by AI.

[0064] Last but not least, Model-DCB requires very limited computing power (two RTX 4090 Ti graphics cards) for training. We intentionally set this up so that any ultrasound manufacturer can easily enable its devices to independently support the model's application without significantly impacting the device's size, portability, and cost. Furthermore, Model-DCB requires almost no manual ROI definition. These two features are particularly critical for ultrasound examinations, as ultrasound radiologists need to simultaneously acquire images and make diagnoses, and ultrasound equipment is frequently moved between patient beds. Excessive computing power that cannot fit into ultrasound equipment and the heavy burden of manual annotation would make it difficult for AI models to achieve clinical translation in real-world ultrasound scenarios.

[0065] Our study has several limitations. First, the CEUS images in this study were collected at Chinese centers, and future international validation is needed. Second, a one-year prospective test set may not capture longitudinal changes or future practice, and we will continue to collect data and validate this model. Third, the distribution of FLL types in this study does not fully conform to the natural distribution but represents the distribution of CEUS patients. Fourth, the diagnosis of some benign FLLs is not based on pathology but is made based on clinical findings, laboratory results, and imaging studies.

[0066] In summary, by effectively integrating CEUS video, biomarker, and clinical information, Model-DCB achieved accurate multi-type FLL classification across multiple clinical trials, including independent retrospective and prospective multicenter trials. This improved diagnostic performance for all participating CEUS radiologists, particularly junior radiologists. We anticipate that Model-DCB's multi-classification capabilities, cross-manufacturer stability, low computational requirements, and affordability will give it significant potential for large-scale clinical application, particularly for FLL patients in remote suburban or underdeveloped areas where MRI is difficult to access, potentially benefiting a broad population.

[0067] The relevant modules involved in this system are all hardware system modules or functional modules that combine computer software programs or protocols with hardware in the existing technology. The computer software programs or protocols involved in the functional modules are themselves technologies that are well known to those skilled in the art and are not improvements to this system. The improvements to this system are the interaction or connection relationships between the modules, that is, improvements to the overall structure of the system to solve the corresponding technical problems to be solved by this system.

[0068] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A distributed model liver nodule multi-classification diagnosis method based on ultrasound contrast imaging, characterized in that: The following steps are involved: S1: Optical flow is used in image processing to analyze and quantify the motion information of pixels or objects in image sequences. It reveals dynamic changes in a scene by capturing the displacement vectors of pixels between connected frames. Its core function is to analyze visual data from a temporal perspective, providing fundamental support for tasks such as motion perception, target tracking, and behavior analysis. S2: Multi-classification model. Two training strategies are adopted in model training, namely direct training strategy and distributed training strategy. The direct training strategy uses one classification model to complete multi-classification tasks, while the distributed training strategy trains multiple binary classification models and aggregates multiple sub-models. S3: "Image-molecule-disease" constitutes a composite dimensional reasoning framework, the core of which is to introduce molecular phenotypes as a biological bridge connecting imaging features with clinical diagnosis.

2. The distributed model liver nodule multi-classification diagnosis method based on ultrasound contrast imaging according to claim 1 is characterized in that: The optical flow can capture the change trend of two adjacent images and focus on the image itself and image changes.

3. The distributed model liver nodule multi-classification diagnosis method based on ultrasound contrast according to claim 1, characterized in that: The optical flow is combined with CEUS and keyframe extraction is performed.

4. A distributed model liver nodule multi-classification diagnosis method based on ultrasound contrast imaging according to claims 1 and 3, characterized in that: The optical flow method and deep learning method automatically analyze and compare CEUS videos and use multiple biomarker information to build a non-invasive diagnostic strategy "from CEUS to biomarkers to disease." 5. The distributed model liver nodule multi-classification diagnosis method based on ultrasound contrast according to claim 1, characterized in that: The distributed training is conducive to fine-tuning parameter adjustments and is passive data enhancement.

6. A distributed model liver nodule multi-classification diagnosis method based on ultrasound contrast imaging according to claims 1 and 5, characterized in that: The distributed training merges the five sub-models into a six-classification model.

7. A distributed model liver nodule multi-classification diagnosis method based on ultrasound contrast imaging according to claims 1 and 4, characterized in that: The CEUS, biomarkers, and clinical information were integrated, and a multi-classification model was constructed for classifying six focal liver lesions.

8. The distributed model liver nodule multi-classification diagnosis method based on ultrasound contrast imaging according to any one of claim 1, characterized in that: The composite dimensional framework of impact-molecule-disease composition allows the model to diagnose four pathological features of nodules based on images.

9. The distributed model liver nodule multi-classification diagnosis method based on ultrasound contrast imaging according to any one of claim 8, characterized in that: The pathological features have a tendency to be distributed among different nodule types, which improves the overall predictive efficiency of the model by 5%, and enables the model to have cross-modal functions of "macro-micro" linkage, as well as the ability to predict patient prognosis based on molecular function diagnosis.

Images