Method for evaluating severity of pancreatitis based on hierarchical reliable evidence fusion

Abstract

The application discloses a kind of based on layered reliable evidence fusion pancreatitis severity assessment method, it constructs a layered reliable evidence fusion model, successively executes three stages of multi-view evidence generation, opinion mapping and opinion fusion: image and clinical data are generated image, clinical and pseudo view evidence, it is mapped as subjective logic opinion including belief quality and uncertainty, finally all opinions are fused using log opinion pool fusion algorithm, generate final fusion opinion;Then based on training set, the layered reliable evidence fusion model is trained;Test again, can according to the belief quality of each category in the final fusion opinion corresponding to test sample, the severity of acute pancreatitis of patient corresponding to test sample is evaluated;The advantage is that multiple-source information can be effectively coordinated, and evidence conflict is converted into uncertainty, and accurate and reliable severity assessment results are output according to the belief quality in the fusion opinion, to provide strong support for clinical decision-making.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of medical auxiliary technology, and in particular to a method for evaluating the severity of pancreatitis based on hierarchical reliable evidence fusion. BACKGROUND

[0002] Early and accurate assessment / prediction of the severity of acute pancreatitis and timely intervention in high-risk patients have become key problems that need to be solved in clinical practice.

[0003] Clinically, various scoring systems are widely relied on, including APACHE II, BISAP, Ranson, Glasgow and modified Marshall scores. These systems face significant challenges in early assessment / prediction of the severity of acute pancreatitis: they mainly rely on single-mode clinical biochemical indicators and vital sign data, and fail to fully utilize multi-modal information, resulting in limited accuracy of assessment / prediction. They generally have low positive detection rate and insufficient specificity. Some systems, such as the Ranson score, require dynamic monitoring of data for 48 hours, which is less time-efficient and difficult to meet the clinical needs of early and timely intervention.

[0004] To improve the accuracy of assessment / prediction, researchers have introduced methods combining imageomics and machine learning. These methods extract quantitative features from medical images such as CT and combine them with clinical biochemical indicators to build models. However, this approach relies on a manual feature extraction process, inevitably losing subtle and critical information in medical images, which restricts the performance of the model.

[0005] To improve the positive detection rate of assessment / prediction, in recent years, deep learning technology has been applied to multi-modal data fusion assessment / prediction, automatically extracting deep features from medical images and combining them with clinical biochemical indicators through end-to-end learning. However, this approach faces three major technical bottlenecks. First, there is a significant modal heterogeneity between clinical biochemical indicators and medical images, making it difficult for traditional fusion strategies to effectively exploit complementary information between modalities. Second, the inflammatory morphology of pancreatitis is highly heterogeneous and variable, making it difficult for the model to stably locate key pathological regions. Third, when different modal evidence is contradictory, there is a lack of effective uncertainty quantification mechanism, and the model is prone to "overconfidence" in error classification.

[0006] Although the multi-view fusion method based on evidence theory can effectively handle uncertainty, it has two major limitations in the assessment / prediction of the severity of acute pancreatitis:

[0007] First, its fusion rule assumes that the evidence of each view is basically consistent, but in reality, there are often high conflicts between clinical biochemical indicators and imaging findings, and the fusion result will produce an unreasonable "overconfidence".

[0008] Second, the lack of a unique modal conflict handling mechanism in the severity assessment / prediction of acute pancreatitis cannot convert the conflict into a reasonable uncertainty expression.

[0009] Therefore, there is an urgent need to study an intelligent assessment technology that has both uncertainty quantification capability and reliable multi-modal conflict handling mechanism. SUMMARY

[0010] The technical problem to be solved by the present application is to provide a pancreatitis severity assessment method based on hierarchical reliable evidence fusion, which can effectively fuse highly heterogeneous multi-modal data and give reasonable and cautious assessment results when data evidence conflicts occur, rather than produce overconfident wrong judgments.

[0011] The technical solution adopted by the present application to solve the above technical problems is: a pancreatitis severity assessment method based on hierarchical reliable evidence fusion, comprising the following steps:

[0012] S1, obtaining a multi-modal data set of multiple patients and constructing a training set, each training sample in the training set including standard medical image data, standard clinical table data and corresponding severity labels of the same patient;

[0013] S2, constructing a hierarchical reliable evidence fusion model, which sequentially performs the following stages:

[0014] The multi-view evidence generation stage generates multi-level image view evidence vectors, clinical view evidence vectors and pseudo view evidence vectors based on standard medical image data and standard clinical table data;

[0015] The opinion mapping stage maps each view evidence vector into a subjective logical opinion;

[0016] The opinion fusion stage fuses all subjective logical opinions based on the log opinion pool fusion algorithm to generate the final fusion opinion;

[0017] S3, training the hierarchical reliable evidence fusion model based on the training set to obtain the trained hierarchical reliable evidence fusion model;

[0018] S4, inputting any test sample into the trained hierarchical reliable evidence fusion model, and assessing the severity of acute pancreatitis of the patient according to the belief quality of each category in the final fusion opinion output.

[0019] The standard medical image data is obtained by preprocessing the medical image data in the multi-modal data set, and the preprocessing includes:

[0020] A1, performing regional rough segmentation on the medical image data based on a pre-trained nnU-Net segmentation network to obtain a rough segmentation result containing the pancreas and surrounding edema fluid region;

[0021] A2, extracting a three-dimensional region of interest of a fixed size with the centroid of the coarse segmentation result as the origin;

[0022] A3, performing image standardization processing on the three-dimensional region of interest to obtain the standard medical image data.

[0023] The standard clinical table data is obtained by preprocessing the clinical table data in the multi-modal data set, and the clinical table data includes qualitative baseline data for describing the basic condition of the patient and quantitative physiological data for reflecting the pathological and physiological state; the preprocessing includes: one-hot encoding of the qualitative baseline data, Z-score standardization and missing value filling of the quantitative physiological data.

[0024] The specific implementation process of the multi-view evidence generation stage is: (a) inputting the standard medical image data into a pre-trained 3D-Resnet network to extract image feature maps of different semantic levels from low to high, and obtaining corresponding image feature vectors through an adaptive maximum pooling layer; (b) inputting the standard clinical table data into a feature encoding network based on a Transformer architecture to obtain a clinical feature vector; (c) inputting the clinical feature vector and the image feature vector of the highest semantic level into a cross-modal complementary attention fusion module to generate a fusion feature vector; (d) sequentially passing each feature vector through a fully connected layer and a Softplus activation function to obtain image view evidence vectors, clinical view evidence vectors and pseudo view evidence vectors of each level.

[0025] The implementation process of the cross-modal complementary attention fusion module is: passing the image feature vector of the highest semantic level through a first fully connected layer, and the obtained vector is used as value V1, key K1 and query Q1 respectively; passing the clinical feature vector through a second fully connected layer, and the obtained vector is used as value V2, key K2 and query Q2 respectively; then performing vector multiplication on the transposed vectors of key K1 and query Q2, and the transposed vectors of key K2 and query Q1 respectively, to obtain vectors R1 and R2 respectively; then performing vector multiplication on vector R1 and value V1, and vector R2 and value V2 respectively, to obtain vectors R3 and R4 respectively; then performing vector addition on vector R3 and the vector output by the first fully connected layer, and vector R4 and the vector output by the second fully connected layer respectively, to obtain vectors R5 and R6 respectively; then passing vector R5 through a third fully connected layer to obtain vector R7; passing vector R6 through a fourth fully connected layer to obtain vector R8; performing vector addition on vector R7 and the image feature vector input to the first fully connected layer to obtain vector R9; finally, performing vector concatenation on vector R9 and vector R8 to obtain the fusion feature vector.

[0026] In the opinion mapping stage, first, each view evidence vector is converted into Dirichlet evidence, which includes Dirichlet distribution concentration parameters of each category, wherein the formula for converting the view evidence vector into the Dirichlet evidence is: α = e + aW, wherein α = [α1, α2], α k represents the Dirichlet distribution concentration parameter of the kth category, e represents the view evidence vector, a represents the base rate, which reflects the category prior distribution, a = [a1, a2], a k = N k / N, k = 1, 2, N k represents the number of training samples of the kth category, N represents the total number of training samples, the first category represents mild acute pancreatitis, the second category represents severe acute pancreatitis, and W represents the balance hyperparameter, W > 0; then the Dirichlet evidence is mapped into subjective logical opinions, and the mapping formula is: b k = (α k -a k W) / S, u = W / S, S = α1 + α2, wherein b k represents the belief quality of the kth category, u represents the overall uncertainty, and satisfies the constraint condition: b k ≥ 0, u ≥ 0, b1 + b2 + u = 1.

[0027] In the opinion fusion stage, first, all subjective logical opinions corresponding to the multi-level image view evidence vector are fused within the modal to generate a unified image view opinion, and is recorded as M I ; then the subjective logical opinions corresponding to the unified image view opinion M I , the clinical view evidence vector M T , and the pseudo view evidence vector M P are fused between the modal to generate the final fusion opinion, and is recorded as M Z .

[0028] The closed-form solution of the unified image view opinion M I and the final fusion opinion M Z is: M j = {b j,1 , b j,2 , u j}, wherein j is I or Z, b j,1 represents the belief quality of the first category in M j , b j,2 represents the belief quality of the second category in M j , and u j represents the overall uncertainty in M jwherein u represents the overall uncertainty in M, j is I when Ω = {I1, I2, …, In} and j is Z when Ω = {I, T, P}, n represents the number of levels, and all subjective logical opinions corresponding to the multi-level image view evidence vector are M I1 , M I2 , …, M In , M i = {b i,1 , b i,2 , u i}, b i,1 represents the belief quality of the first category in M i , b i,2 represents the belief quality of the second category in M i , u i represents the overall uncertainty in M i , c i = 1-u i = b i,1 + b i,2 , C is a normalization constant, .

[0029] Compared with the prior art, the present application has the following advantages:

[0030] 1) The hierarchical reliable evidence fusion model used in the method of the present application no longer relies on a single clinical indicator or image feature, but fully excavates and utilizes the complementary information in multi-modal (heterogeneous) data through the image view evidence vector, the clinical view evidence vector and the pseudo view evidence vector. The image view evidence vector provides morphological evidence of the organ, the clinical view evidence vector provides physiological and biochemical indicator evidence, and the pseudo view evidence vector captures the deep correlation between the two. This multi-angle and multi-source evidence collection fundamentally avoids the one-sidedness that may exist in a single information source, and provides a solid data foundation for accurately assessing the severity of acute pancreatitis.

[0031] 2) When the evidence of different modalities is contradictory (such as severe clinical indicators but mild image manifestations), the traditional fusion method will give a possibly erroneous "confident" judgment. However, the log-pooling fusion algorithm used in the method of the present application transforms the evidence conflict into a reasonable increase in prediction uncertainty, rather than simply suppressing the conflict. This makes the hierarchical reliable evidence fusion model more prudent and more in line with clinical logic when facing complex and contradictory clinical situations, thereby significantly improving the reliability of the evaluation results of high-risk cases.

[0032] 3) The final output of the layered reliable evidence fusion model used in the method of the present application is not only a "mild" or "severe" category judgment, but also an attached quantifiable "overall uncertainty" indicator. This is equivalent to providing an important "risk indicator" for the doctor's decision. When the uncertainty is high, it prompts the doctor that the current evidence is insufficient or conflicting, and further examination or consultation is needed, which is highly consistent with the cautious decision principle of clinical experts.

[0033] 4) The layered reliable evidence fusion model used in the method of the present application innovatively first applies the decision-level multi-modal fusion framework combining the log-pooling fusion algorithm with evidence theory to acute pancreatitis severity prediction, and naturally converts modality conflict into prediction uncertainty. In the face of view conflict, the layered reliable evidence fusion model shows significantly improved uncertainty compared to the view Figure One The layered reliable evidence fusion model significantly outperforms existing methods (AUC 0.928, accuracy 0.919) in 24-hour acute pancreatitis progression prediction tasks. Through Grad-Cam visualization and gradient integration analysis, the layered reliable evidence fusion model shows the ability to accurately locate the pancreatic lesion area and identify key clinical indicators such as high-sensitivity C-reactive protein and amylase, as well as potential biomarkers ignored by traditional scoring systems, providing reliable support for risk stratification and early intervention of acute pancreatitis.

[0034] 5) The layered reliable evidence fusion model used in the method of the present application addresses the representation challenge of complex heterogeneous data in acute pancreatitis severity prediction by designing a multi-level feature representation and cross-modal complementary attention fusion module, and a double-level evidence fusion strategy. At the feature level, a multi-level feature extraction network based on 3D ResNet is designed to integrate a unified image modality belief through log-pooling; at the modality level, a cross-modal complementary attention fusion mechanism is introduced to generate pseudo-views of images and clinical data, enhancing information interaction between modalities, and through log-pooling, the fusion of image, clinical data and pseudo-view data is realized, improving feature representation capability and prediction robustness.

[0035] 6) Most of the acute pancreatitis patients are mild, and severe (SAP) is a minority but has a high mortality rate. Traditional models are prone to overfitting to the majority class (mild) and ignoring the minority class (severe). In the framework of the present application, the base rate (Prior) mechanism can be set to naturally impose greater penalties on prediction errors of the minority class (severe) during the training process, thereby driving the model to actively learn and focus on key features that may develop into severe cases, effectively improving the early warning ability for high-risk patients.

[0036] 7) For the data used in the prediction of acute pancreatitis severity, the imaging machines are different, the imaging protocols are different, and the pancreas and lesion regions are variable and have large morphological differences. Therefore, the present application uses a small amount of label to roughly position the pancreas, uses a pre-trained nn-Unet to fine-tune the segmentation task of a small amount of expert annotated samples, and applies the fine-tuned model to the complete data set for automatic segmentation inference of pancreatic edema. This preprocessing procedure significantly improves the quality and consistency of the input data, effectively filters background noise (such as abdominal fat, digestive tract gas and organs unrelated to the pancreas), and enables the hierarchical reliable evidence fusion model to focus on extracting features of the pancreas and its lesion regions. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 An execution process diagram of the hierarchical reliable evidence fusion model;

[0038] Figure 2 A process diagram of region rough segmentation and three-dimensional region of interest extraction;

[0039] Figure 3 An ROC curve comparison diagram of the method of the present application and each comparative method on the task of predicting the severity of acute pancreatitis;

[0040] Figure 4 A precision-recall curve comparison diagram of the method of the present application and each comparative method on the task of predicting the severity of acute pancreatitis;

[0041] Figure 5 An uncertainty distribution diagram of the hierarchical reliable evidence fusion model (H-REFM model) on correct classification and error classification samples;

[0042] Figure 6 An uncertainty distribution diagram of the traditional Dempster-Shafer theory model on correct classification and error classification samples;

[0043] Figure 7 An uncertainty distribution diagram of the general uncertainty perception model on correct classification and error classification samples;

[0044] Figure 8 An uncertainty performance scatter plot of the hierarchical reliable evidence fusion model (H-REFM model) in the case of modal conflict;

[0045] Figure 9 An uncertainty performance scatter plot of the traditional Dempster-Shafer theory model in the case of modal conflict;

[0046] Figure 10 An uncertainty performance scatter plot of the general uncertainty perception model in the case of modal conflict;

[0047] Figure 11 Fig. 1 shows the abdominal CT cross-sectional images and their corresponding heat map overlay results of five patients with acute pancreatitis (each column corresponds to a patient), where the first row is the abdominal CT cross-sectional image showing the main slice, and the second row is the heat map overlay result showing the region of interest;

[0048] Figure 12 Fig. 2 shows the top 20 clinical indicators (features) most valuable for predicting the progression of acute pancreatitis and their normalized importance scores. DETAILED DESCRIPTION

[0049] The application will be further described in detail below with reference to the accompanying drawings.

[0050] The application proposes a pancreatitis severity assessment method based on hierarchical reliable evidence fusion, which comprises the following steps:

[0051] S1, obtaining a multi-modal data set of a plurality of patients; then based on the multi-modal data set, constructing a training set containing a plurality of training samples, each training sample containing standard medical image data and standard clinical table data of the same patient and corresponding severity labels.

[0052] Here, the multi-modal data set contains a plurality of samples, each sample corresponding to a patient and including medical image data and clinical table data and corresponding severity labels.

[0053] Further limited, the clinical table data contains qualitative baseline data for describing the basic condition of the patient to distinguish different patients and quantitative physiological data for reflecting the pathophysiological state of the patient.

[0054] Here, the medical image data is a computed tomography (CT) image sequence acquired from the abdomen of the patient, used to extract morphological features of the pancreas and surrounding tissues; the clinical table data is acquired from the clinical records and laboratory examination results after the patient is admitted to the hospital, the clinical records are qualitative baseline data, usually including but not limited to the patient's basic information such as gender, age, medical history markers, etc.; the quantitative physiological data, i.e. laboratory examination results, are continuous numerical indicators obtained through laboratory tests or instrument measurements, including but not limited to high-sensitivity C-reactive protein, indirect bilirubin, and percentage of basophilic granulocytes.

[0055] Further limited, the standard medical image data is obtained by preprocessing the medical image data in the multi-modal data set, including regional rough segmentation, three-dimensional region of interest extraction, and image standardization, the specific process is as follows:

[0056] A1, performing regional rough segmentation on the medical image data based on a pre-trained nnU-Net segmentation network to obtain a rough segmentation result containing a pancreatic and peripheral edema fluid region.

[0057] Here, the specific process of step A1 is as follows: (a) invite senior image experts to label the medical image data of a small number of samples (such as 100 samples) in the multi-modal data set (labeling by work experience), draw the outline of the pancreatic and peripheral edema fluid region by the medical image processing software ITK-SNAP, and form a labeled data set; (b) using the labeled data set to fine-tune the pre-trained nnU-Net segmentation network once to obtain pre-trained weights specific to pancreatic region segmentation; (c) using the pre-trained weights to automatically segment and infer the medical image data of all samples in the multi-modal data set to obtain the corresponding rough segmentation result.

[0058] Since the medical image data for acute pancreatitis severity assessment comes from different imaging machines, the imaging protocols are different, and the data includes upper abdominal, whole abdominal plain CT images, while considering the variability of the pancreatic and lesion regions, the morphological difference is large, therefore, it is necessary to locate the pancreatic and peripheral edema fluid region first to prevent too much abdominal irrelevant information from affecting model learning. Therefore, the present application uses a small number of samples to roughly locate the pancreatic and peripheral edema fluid region, and then uses the pre-trained nnU-Net segmentation network for automatic segmentation and inference.

[0059] A2, taking the centroid of the rough segmentation result as the origin, extracting a fixed size three-dimensional region of interest from all sample medical image data, wherein the three-dimensional region of interest has a size of 273 pixels x 168 pixels in the transverse plane and covers 25 consecutive slices in the sagittal plane.

[0060] Since the sample size for fine-tuning the pre-trained nnU-Net segmentation network is limited, the resulting segmentation result can only be used for rough positioning. To ensure that the complete organ and lesion region can be captured and the sample consistency is maintained, the rough segmentation results of all samples are evaluated to finally determine the unified three-dimensional region of interest extraction method: taking the centroid of the rough segmentation result as the origin, the size of the transverse plane is 273 pixels x 168 pixels, and in the sagittal plane, since the images come from different examinations (upper abdomen, whole abdomen), the proportion of the pancreas in the image is small, finally, in the case of ensuring that the pancreas and necessary context information are included, the longitudinal coverage range of each examination is positioned to 25 consecutive slices. This step significantly reduces the interference of irrelevant information and greatly reduces the network computing burden.

[0061] Figure 2The process diagram of regional rough segmentation and three-dimensional region of interest extraction is given. The size of the abdominal CT image sequence in the transverse plane is 512 pixels x 512 pixels, and it covers more than 100 consecutive slices in the sagittal plane. The rough segmentation result (red and green areas in the figure) is obtained by using the pre-trained nnU-Net segmentation network, and the three-dimensional region of interest is extracted with the centroid of the rough segmentation result as the origin. This process significantly improves the quality and consistency of the input data, effectively filters background noise (such as abdominal fat, digestive tract gas, and organ tissue unrelated to the pancreas), and enables the network to focus on extracting features of the pancreas and its lesion regions.

[0062] A3, to reduce the image intensity variation caused by different medical image acquisition devices (imaging machines) such as CT devices, image standardization processing is performed on the three-dimensional region of interest to obtain corresponding standard medical image data.

[0063] Here, the specific process of step A3 is: (1) applying a preset abdominal window with a window width of -100 to 240 HU to the pixel intensity values such as CT values in the three-dimensional region of interest; (2) performing Z-score standardization on the results obtained after truncation; (3) normalizing the results obtained after Z-score standardization to the [-1, 1] numerical range to obtain corresponding standard medical image data.

[0064] Further limited, the standard clinical table data is obtained by preprocessing the clinical table data in the multi-modal data set, and the preprocessing process of the clinical table data is: performing one-hot encoding processing on the qualitative baseline data; performing Z-score standardization processing on the quantitative physiological data, and then performing missing value filling processing. For the clinical table data in the multi-modal data set, the missing value filling processing uses the following method: using the median of the numerical values obtained by Z-score standardization processing of all samples in the multi-modal data set to fill the missing values. Generally, the overall missing rate is less than 10%, so the median can be used to fill the missing values. For example, if there are 100 samples and 4 missing white blood cell counts, the median of the 96 existing neutrophil absolute values is used to fill the 4 missing neutrophil absolute values.

[0065] S2, a layered reliable evidence fusion model is constructed, such as Figure 1As shown, it sequentially executes the following stages: a multi-view evidence generation stage, generating a multi-level image view evidence vector based on standard medical image data; generating a clinical view evidence vector based on standard clinical table data; and generating a pseudo view evidence vector based on the interaction features of standard medical image data and standard clinical table data. An opinion mapping stage maps the multi-level image view evidence vector, the clinical view evidence vector, and the pseudo view evidence vector into subjective logical opinions respectively, and the subjective logical opinions include belief quality of various categories and overall uncertainty. An opinion fusion stage fuses all subjective logical opinions based on a logarithmic opinion pool fusion algorithm to generate a final fusion opinion.

[0066] Further limited, as Figure 1 As shown, the specific implementation process of the multi-view evidence generation stage is as follows:

[0067] (a) input the standard medical image data into the pre-trained 3D-Resnet network to extract image feature maps of different semantic levels, wherein the semantics are from low to high; then pass the image feature maps of different semantic levels through an adaptive maximum pooling layer respectively to obtain image feature vectors of different semantic levels; (b) input the standard clinical table data into the feature encoding network based on the Transformer architecture to obtain a clinical feature vector; (c) input the clinical feature vector and the image feature vector of the highest semantic level into the cross-modal complementary attention fusion module to generate a fusion feature vector; (d) pass the image feature vectors of different semantic levels, the clinical feature vector, and the fusion feature vector through a full connection layer and a Softplus activation function respectively in sequence to obtain the image view evidence vector, the clinical view evidence vector, and the pseudo view evidence vector of each level correspondingly.

[0068] Here, considering that the pathological features of acute pancreatitis are manifested on multiple scales, such as micro-edema, local necrosis, and systemic inflammation, the shallow features of the deep convolutional network focus on local details, while the deep features capture global semantic information, therefore, the application uses a pre-trained 3D-Resnet network (specifically, a pre-trained 3D network architecture of ResNet-10) to extract hierarchical features from each convolution stage. The 3D-Resnet network is prior art, and its main structure includes an initial convolution layer and a pooling layer, followed by four convolution stages connected in sequence, which are composed of multiple residual blocks for extracting features of different levels; the pre-trained 3D-Resnet network is trained based on medical images, and the standard medical image data is input into the pre-trained 3D-Resnet network, and each of the four convolution stages outputs an image feature map of a level, and the image feature maps output by the first convolution stage to the fourth convolution stage are from low to high in semantic level.

[0069] Here, the feature encoding network based on the Transformer architecture is the existing Tab-Transformer (table transformer), which is composed of an embedding coding layer specially designed for table data, multiple standard Transformer encoder modules, and a task-related MLP (multilayer perceptron) output head. It can effectively process mixed inputs of numerical and categorical features and preserve the interaction between features. In this embodiment, the Transformer encoder module is provided with four. Clinical table data has significant predictive value, but its heterogeneity and high-dimensional feature space make it difficult for traditional methods to effectively process, so the present application uses Tab-Transformer for feature extraction and representation learning. In this embodiment, the input dimension of Tab-Transformer is 68 and the output dimension is 256.

[0070] Here, the present application designs a special evidence head (Evidence Head) for view evidence vector extraction, which includes a fully connected layer and a Softplus activation function. The present application extracts four levels of image view evidence vectors for standard medical image data, each level containing different scale information: the first level (64 dimensions) captures fine-grained texture information, the second level (128 dimensions) extracts medium-scale structural information, the third level (256 dimensions) and the fourth level (512 dimensions) focus on more abstract semantic information. These multi-level evidence can fully represent the complex pathological changes of acute pancreatitis.

[0071] In clinical practice, specialist doctors usually need to analyze medical image manifestations and clinical indicators to assess the risk of progression of acute pancreatitis, and the two information sources verify and complement each other. Based on this clinical cognitive process, the present application proposes a complementary pseudo-view generation mechanism, which realizes dynamic integration and enhancement of information through a cross-modal attention fusion (CMAF) fusion module. For example Figure 1As shown, the implementation process of the cross-modal complementary attention fusion module is as follows: the image feature vector of the highest semantic level is subjected to a first full connection layer, and the obtained vector is taken as a value V1, a key K1 and a query Q1 respectively; the clinical feature vector is subjected to a second full connection layer, and the obtained vector is taken as a value V2, a key K2 and a query Q2 respectively; then, the transposed vector of the key K1 and the query Q2 is subjected to vector multiplication to obtain a vector R1; the transposed vector of the key K2 and the query Q1 is subjected to vector multiplication to obtain a vector R2; then, the vector R1 is subjected to vector multiplication with the value V1 to obtain a vector R3; the vector R2 is subjected to vector multiplication with the value V2 to obtain a vector R4; then, the vector R3 is subjected to vector addition with the vector output by the first full connection layer to obtain a vector R5; the vector R4 is subjected to vector addition with the vector output by the second full connection layer to obtain a vector R6; then, the vector R5 is subjected to a third full connection layer to obtain a vector R7; the vector R6 is subjected to a fourth full connection layer to obtain a vector R8; the vector R7 is subjected to vector addition with the image feature vector input to the first full connection layer to obtain a vector R9; finally, the vector R9 is subjected to vector splicing with the vector R8 to obtain a fusion feature vector.

[0072] The clinical data of acute pancreatitis usually has a class proportion imbalance problem, most of the patients with acute pancreatitis will turn to mild, and only 25% of the patients will turn to severe. In the evidence theory framework, this imbalance will cause the network to tend to accumulate more evidence supporting the majority class, thereby reducing the identification ability of the minority high-risk patients who are more concerned in the clinic. Therefore, the Dirichlet evidence representation based on the class proportion imbalance is defined. As shown in Figure 1 As shown, in the opinion mapping stage, first, the multi-level image view evidence vector, the clinical view evidence vector and the pseudo view evidence vector are respectively converted into Dirichlet evidence, and the Dirichlet evidence includes the Dirichlet distribution concentration parameters of each class; then, the Dirichlet evidence is mapped into subjective logical opinions.

[0073] In the opinion mapping stage, the formula for converting the view evidence vector into Dirichlet evidence is: α = e + aW, wherein, α = [α1, α2], α1 represents the Dirichlet distribution concentration parameter of the first class, α2 represents the Dirichlet distribution concentration parameter of the second class, e represents the view evidence vector, which is the image view evidence vector or the clinical view evidence vector or the pseudo view evidence vector, a represents the base rate, which reflects the class prior distribution, a = [a1, a2], in order to ensure that all classes are given the same attention, a k = N k / N, by reducing the base rate of the minority class, a compensation mechanism is generated in the network, and the network needs a larger evidence value to reach a parameter concentration comparable to the majority class, thereby ensuring that the network can pay more attention to the minority class, k = 1, 2, N kN represents the number of training samples in the k-th category, N represents the total number of training samples, the first category represents mild acute pancreatitis, the second category represents severe acute pancreatitis, and W represents the balancing hyperparameter, W>0, which is usually set to the number of categories. In this embodiment, the number of categories is 2. Figure 1 In, α I1 , k α I2 , k α I3 , k α I4 , k α T , k α P , k The corresponding representations are the Dirichlet distribution concentration parameters of the k-th category in the Dirichlet evidence obtained by transforming the first-level, second-level, third-level, fourth-level, clinical view evidence vectors, and pseudo-view evidence vectors.

[0074] The Dirichlet distribution provides a solid probabilistic foundation for uncertainty modeling, but its parameter form is not easily intuitively understood or practically applied. Therefore, this application introduces Subjective Logic theory as an intermediate representation. This theory provides a mathematical framework for handling uncertainty and incomplete evidence, making the network decision-making process more transparent and interpretable by explicitly separating "belief" and "uncertainty." The core concept in Subjective Logic is "opinion," representing a subjective judgment of the truth of a proposition. In this application, Dirichlet evidence can be mapped to a subjective logical opinion. The formula for mapping Dirichlet evidence to a subjective logical opinion in the opinion mapping stage is: b k =(α k -a k W) / S, u=W / S, S=α1+α2, where, b k Let represent the belief quality of the k-th category, u represent the overall uncertainty, and satisfy the constraint: b k ≥0, u≥0, b1+b2+u=1, b k Direct quantification is based on the support of the k-th category based on observed evidence, while u quantifies the degree of "ignorance" caused by insufficient evidence for the view. When S is small, the overall uncertainty is high, indicating that the insufficient evidence for the view itself may require the combination of information from other views to make a decision.

[0075] The present application represents Dirichlet evidence under distribution shift for quantifying the distribution uncertainty of prediction; then maps the mutual view between Dirichlet evidence and subjective logical opinion to provide explainable uncertainty; finally, the subjective logical opinions of different views are fused by using the log-pooling fusion algorithm to transform the modal conflict into uncertainty.

[0076] Further limited, as Figure 1 shown, in the opinion fusion stage, first, all the subjective logical opinions corresponding to the multi-level image view evidence vector are fused within the modal to generate unified image view opinion, and is recorded as M I ; then the subjective logical opinions M I corresponding to the unified image view opinion M T , the subjective logical opinions M P corresponding to the clinical view evidence vector and the subjective logical opinions M Z corresponding to the pseudo view evidence vector are fused between the modal to generate the final fusion opinion, and is recorded as M Z . Here, to solve the problem of image modal over-dominance caused by multi-level evidence, the present application introduces the intra-modal fusion mechanism, and uses the log-pooling fusion algorithm to integrate the multi-level subjective logical opinions within the image modal into a single comprehensive subjective logical opinion, i.e., the unified image view opinion.

[0077] As a preferred solution, the closed-form solution of the unified image view opinion M I and the final fusion opinion M Z is: M j ={b j,1 ,b j,2 ,u j}, wherein j is I or Z, b j,1 represents the belief quality of the first category in M j , b j,2 represents the belief quality of the second category in M j , u j represents the overall uncertainty in M j , j is I when Ω={I1, I2, …, In}, j is Z when Ω={I, T, P}, n represents the level number, and in the present embodiment, n=4, all the subjective logical opinions corresponding to the multi-level image view evidence vector are M I1 , M I2 , …, M In , M i ={b i,1 ,b i,2 ,u i}, b i,1 represents the belief quality of the first category in M i , b i,2 represents the belief quality of the second category in M iu represents the second category of belief quality in M i M represents i c represents the overall uncertainty in M i c is a certainty factor, ensuring that high certainty views have more influence on the fusion result i =1-u i =b i,1 +b i,2 , C is a normalization constant .

[0078] The present application formulates the fusion problem as finding an optimal subjective logic opinion that minimizes the weighted KL divergence to all input subjective logic opinions, the KL divergence measures the information difference, and minimizing the KL divergence is equivalent to finding a fusion distribution that minimizes the information loss to all input subjective logic opinions, which ensures that the fusion result retains the key information in the original views to the greatest extent. The objective function of the logarithmic opinion pool fusion algorithm is described as: , wherein M x is an opinion variable, D KL () represents the generalized KL divergence, the present application uses the generalized KL divergence in information theory as the "distance" measure between subjective logic opinions, used to measure the information gain of M x relative to M i , M x ={b x,1 ,b x,2 ,u x}, b x,1 represents the first category of belief quality in M x , b x,2 represents the second category of belief quality in M x , u x represents the overall uncertainty in M x ; after expanding the objective function, the optimization objective function is obtained, which is described as: , the constraint conditions are: b i,k ≥0, u i ≥0, b i,1 +b i,2 +u i =1; according to the optimization objective function, a Lagrange function is constructed, which is described as: , wherein L() is the Lagrange function representation, and λ represents the Lagrange factor; solving the Lagrange function, the closed-form solution of M j is obtained.

[0079] The logarithmic opinion pool fusion algorithm has the following key properties:

[0080] Property 1 (Consistency Preservation): If all views have higher support for one class than others, the fused result will also have higher support for that class than others.

[0081] Property 2 (Natural Neglect of Uncertain Views): When a view has high uncertainty, it will have little influence on the fused result.

[0082] Property 3 (Conflict Perception and Uncertainty Enhancement): When multiple views have conflicting support for different classes with similar uncertainty, the fused result will have significantly higher uncertainty than each individual view.

[0083] Compared with the traditional Dempster-Shafer Theory (DST), the Logarithmic Opinion Pool fusion algorithm has two key advantages. Firstly, DST assigns conflicting evidence by normalization when views are highly contradictory, which often leads to counterintuitive overconfident results. In contrast, the Logarithmic Opinion Pool fusion algorithm naturally converts conflict into increased uncertainty, which is consistent with the cautious principle in clinical decision-making. Secondly, the Logarithmic Opinion Pool fusion algorithm performs weighted geometric averaging in the logarithmic space, preserving the geometric structure in the simplex space, while DST combination does not consider geometric distance, leading to possible jumps in the belief space.

[0084] The uncertainty degree of the fused result of the Logarithmic Opinion Pool fusion algorithm naturally adjusts according to the uncertainty distribution of the original views. To prevent extremely low belief quality from having an excessive impact in the logarithmic space, a smoothing factor (0.01) is introduced to ensure that the fusion operation is numerically robust and reliable.

[0085] This fusion mechanism is highly consistent with the clinical decision-making process. When key evidence is contradictory, doctors usually give more conservative opinions and seek more information rather than making arbitrary and deterministic judgments.

[0086] S3, based on the training set, training the hierarchical reliable evidence fusion model, and obtaining the trained hierarchical reliable evidence fusion model after the training, the specific process being: taking the standard medical image data and the standard clinical table data in the training set as inputs, taking the corresponding severity labels as supervision targets, training the hierarchical reliable evidence fusion model, calculating the model overall loss every round in the training process, and calculating the gradient of each model parameter layer by layer through the back propagation algorithm, updating the model weight parameters, and obtaining the pre-trained hierarchical reliable evidence fusion model after the training, wherein the model overall loss is the weighted sum of the unified image view loss, the clinical view loss, the pseudo view loss and the fusion view loss, the unified image view loss, the clinical view loss, the pseudo view loss and the fusion view loss being obtained by weighting and summing the evidence cross-entropy loss and the KL divergence regularization term of the unified image view evidence vector, the clinical view evidence vector, the pseudo view evidence vector and the fusion view evidence vector respectively, the unified image view evidence vector being obtained by mapping back the unified image view opinion obtained by fusing the subjective logical opinions of the multi-level image view evidence vectors, and the fusion view evidence vector being obtained by mapping back the final fusion opinion obtained by fusing the subjective logical opinions of the unified image view opinion, the clinical view evidence vector and the pseudo view evidence vector.

[0087] In the embodiment, the unified image view loss, the clinical view loss, the pseudo view loss and the fusion view loss are obtained in the same way, and the unified image view loss is taken as an example, the obtaining process being: (a) transforming the unified image view evidence vector by using the inverse gamma function, and calculating the cross-entropy between the transformed vector and the severity label to obtain the evidence cross-entropy loss (basic loss value); (b) converting the unified image view evidence vector into a Dirichlet evidence, the Dirichlet evidence including the concentration parameters of the Dirichlet distribution of each category; then calculating the KL divergence value between the concentration parameters of the Dirichlet distribution and the prior parameters to represent the difference between them, wherein the prior parameters are set as uniform distribution; (c) dynamically adjusting the KL divergence value weight according to the frequency of the category, and weighting the KL divergence value to obtain the KL divergence regularization term, wherein the determination process of the KL divergence value weight is: assuming that the number of training samples representing mild pancreatitis in the severity label in the training set is N1, and the number of training samples representing severe pancreatitis is N2, then if the severity label of the current training sample represents mild pancreatitis, the KL divergence value weight corresponding to the current training sample is N1 / (N1+N2); if the severity label of the current training sample represents severe pancreatitis, the KL divergence value weight corresponding to the current training sample is N2 / (N1+N2); (d) weighting and summing the evidence cross-entropy loss and the KL divergence regularization term to obtain the unified image view loss, wherein only the KL divergence regularization term is weighted, and the weighting coefficient is a pre-set hyperparameter, the value range being 0.001-0.1.

[0088] In the embodiment, the weight coefficients of the unified image view loss, the clinical view loss, the pseudo view loss and the fusion view loss correspond to 0.2, 0.2, 0.4 and 0.8.

[0089] S4, for any test sample, input it into the trained hierarchical reliable evidence fusion model to obtain the corresponding final fusion opinion, and then evaluate the severity of acute pancreatitis of the patient corresponding to the test sample according to the belief quality of each category in the final fusion opinion.

[0090] Here, the acquisition process of the test sample is: acquiring the medical image data and the clinical table data of the patient to be evaluated; then, the medical image data and the clinical table data are preprocessed respectively, and the standard medical image data and the standard clinical table data of the patient to be evaluated are obtained correspondingly, to constitute a test sample.

[0091] The medical image data of the patient to be evaluated is preprocessed by steps A1 to A3, and it should be noted that the pre-trained nnU-Net segmentation network does not need to be fine-tuned, and the pre-trained weight is directly used for automatic segmentation inference of the medical image data of the patient to be evaluated to obtain the coarse segmentation result.

[0092] The qualitative baseline data of the patient to be evaluated is one-hot encoded. The quantitative physiological data of the patient to be evaluated is Z-score standardized and then filled with missing values. The missing value filling process uses the following method: calling the median of the values obtained by Z-score standardization of the quantitative physiological data of all samples in the multi-modal data set for filling.

[0093] To further illustrate the feasibility and effectiveness of the method of the present application, experiments on the method of the present application are carried out.

[0094] Experimental preparation:

[0095] The experimental data comes from the acute pancreatitis patients admitted by the emergency department and the imaging department of a hospital from 2021 to now, and the acute pancreatitis patients with missing clinical table data or medical image data (CT image sequence) are screened out. Finally, 303 acute pancreatitis patients are included, of which 61 patients progress to severe acute pancreatitis after 48h, and the remaining 242 patients do not show obvious deterioration.

[0096] Clinical form data (including population statistics, vital signs, inflammation indicators, enzyme indicators, and other related indicators totaling 68) within 24 hours of onset and CT image sequences. The parameters of the CT image sequences are as follows: the scanning range is from the top of the diaphragm to the lower pole of the kidneys, the layer thickness and layer distance are 5.0 mm, the data is transmitted to the hospital PACS to export the original data standard line model reconstruction, the CT image sequences of all patients are obtained in the hospital PACS, the image format is DICOM, and the best window width and window level are set.

[0097] With reference to the definition standard of severe acute pancreatitis in the Atlanta classification standard in 2012, and in combination with the actual situation of data collection, the time of organ failure appearing in the course of the patient is greater than or equal to 48 hours as the prediction endpoint of severe acute pancreatitis, specifically as follows: ① Respiratory failure: PaO2 / FiO2≤300; ② Circulatory failure: Systolic blood pressure <90mmHg or mean arterial pressure <70mmHg and need to use vasoactive drugs; ③ Renal failure: serum creatinine ≥170umol / L or >3 times the baseline or urine volume <0.5ml / kg / h for more than 24 hours or anuria for 12 hours.

[0098] The data preprocessing process in the method of the application is used to preprocess the obtained medical image data and clinical form data, to obtain standard medical image data and standard clinical form data, and to form a data set, which is divided into a training set and a test set; in experiments or specific implementations, the standard medical image data in the training set can be subjected to random cropping, flipping and other enhancement operations.

[0099] The pre-trained 3D-Resnet network is obtained by pre-training on the Minist medical data set, and the data input size is 273x168x25. In the training process of the hierarchical reliable evidence fusion model (referred to as H-REFM), the batch size is 32, the Adam optimizer is used for training, the initial learning rate is 1e-4, the number of epochs is 200, and the learning rate decay rate is 1e-5. The validation strategy of the hierarchical reliable evidence fusion model is: five-fold cross-validation is used to evaluate the model performance to ensure the reliability and stability of the results, and the class proportion of each fold data division is kept consistent to avoid sampling bias.

[0100] Experimental results and analysis:

[0101] The method of the present application is compared with the current mainstream multi-view data fusion method. The comparison method includes: (1) SDGCCA based on canonical correlation analysis, derived from Moon S, Hwang J, Lee H. SDGCCA: supervised deep generalized canonical correlation analysis for multi-omics integration[J]. Journal of Computational Biology, 2022, 29(8): 892-907. (SDGCCA: Supervised Deep Generalized Canonical Correlation Analysis for Multi-omics Integration); (2) MC Dropout based on random inactivation, derived from Gal Y, Ghahramani Z. Dropout as a bayesian approximation: Representing model uncertainty in deep learning[C] / / international conference on machine learning. PMLR, 2016: 1050-1059. (Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning); (3) the latest evidence theory-based method (UA method, using a weighted fusion strategy), derived from Dong Z, Yue X, Chen Y, et al. Uncertainty-Aware Multi-view Learning for Prostate Cancer Grading with DWI[C] / / International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer Nature Switzerland, 2024: 739-748. (Uncertainty-Aware Multi-view Learning for Prostate Cancer Grading with DWI); (4) the latest evidence theory-based method (ETMC method, using the traditional DST combination rule), derived from Han Z, Zhang C, Fu H, et al. Trusted multi-view classification with dynamic evidential fusion[J].IEEE transactions on pattern analysis and machine intelligence, 2022, 45(2): 2551-2566. (Trustworthy multi-view classification based on dynamic evidence fusion).

[0102] The models in the comparative method and the method of the application were evaluated under the same data set and preprocessing conditions using five-fold cross-validation. To ensure fair comparison, all comparative methods use the same backbone network and feature extractor, and introduce pseudo views consistent with the method of the application.

[0103] Table 1 shows the performance comparison of each method on the acute pancreatitis severity prediction task (mean ± standard deviation).

[0104] Table 1 shows the performance comparison of each method on the acute pancreatitis severity prediction task

[0105]

[0106] In Table 1, AUC (Area Under the ROC Curve) is a core statistical indicator for evaluating the prediction performance of a binary classification model. Its essence is the area enclosed by the Receiver Operating Characteristic Curve (ROC Curve) and the horizontal axis (false positive rate). The value of AUC ranges from 0.5 to 1, and the closer the AUC value is to 1, the better the classification effect of the model. From Table 1, it can be concluded that the method of the application is superior to the comparative methods in all evaluation indicators, especially in accuracy and AUC, which reach 0.919 and 0.928, respectively, which is 1.6% and 2.1% higher than the second best method, i.e. comparative method 4. In terms of distinguishing between minority classes (severe acute pancreatitis), the macro-averaged precision of the method of the application reaches 0.914, which is significantly better than the comparative methods, indicating that it has outstanding advantages in identifying high-risk patients, which is of great concern in clinical practice.

[0107] Figure 3 The ROC curves of each method on the acute pancreatitis severity prediction task are shown, with the false positive rate (1-specificity) on the horizontal axis and the true positive rate (sensitivity) on the vertical axis. From Figure 3 it can be seen that the AUC value of the method of the application reaches 0.928, which is better than other comparative methods. Figure 4 The precision-recall curves of each method on the acute pancreatitis severity prediction task are shown, with the recall rate on the horizontal axis and the precision on the vertical axis. From Figure 4 it can be seen that the method of the application is significantly better than other comparative methods. Figure 3 and Figure 4 In and, H-REFM refers to the method of the application.

[0108] Ablation experiments:

[0109] To verify the effectiveness of each module in the H-REFM model in the method of the present application, a series of ablation experiments were designed, and the contribution of different modules to the performance of the model was evaluated in the experiments. Table 2 specifically compares the following model variants: (1) a single-modality model using only clinical data (Clinical); (2) a single-modality model using only CT image sequences (Imaging); (3) a baseline model that directly concatenates and fuses the features of clinical data and CT image sequences; (4) a model using a logarithmic opinion pool (LogOP-Logarithmic Opinion Pool) for multi-modal fusion; (5) a model combining multi-level feature extraction and LogOP fusion (ML+LogOP); (6) a hierarchical reliable evidence fusion model (H-REFM) containing multi-level feature extraction, cross-modal attention mechanism (CMAF), and double-level evidence fusion (intra-modality + inter-modality). All variant models use the same training strategy and evaluation process.

[0110] Table 2 Performance comparison of the method of the present application and the methods using each variant model on the task of acute pancreatitis severity prediction

[0111]

[0112] As can be seen from Table 2, the performance of single-modality models (Clinical and Imaging) is significantly lower than that of multi-modal models (Concat, LogOP, ML+LogOP, H-REFM). The Concat model has a slight improvement through direct concatenation fusion, but its performance is limited by modality heterogeneity and conflict problems. The ML+LogOP model further introduces multi-level feature extraction, improving feature representation capability, but still lacks inter-modal interaction. The H-REFM model can comprehensively address the complex heterogeneous data challenges in the task of acute pancreatitis severity prediction through multi-level feature extraction at the feature level (3D ResNet multi-level) and fusion at the modality level (image intra-modality integration + inter-modality fusion + CMAF pseudo view).

[0113] Overall uncertainty assessment:

[0114] The quality of uncertainty estimation is crucial for clinical decision support systems. Two sets of experiments were used to evaluate the uncertainty representation capability of H-REFM: (1) comparing the uncertainty distribution on correct and incorrect classification samples; (2) analyzing the uncertainty performance under modality conflict.

[0115] For the first set of experiments, the uncertainty distributions of H-REFM model, traditional Dempster-Shafer theory (Trad. DS) model and general uncertainty-aware model (UA) on correctly classified and misclassified samples are compared, which are shown in Figs. Figure 5 、 Figure 6 and Figure 7 , where green color represents the distribution of correctly classified samples and red color represents the distribution of misclassified samples. The differences of the three models in uncertainty estimation can be observed from Figure 5 、 Figure 6 and Figure 7 . To objectively quantify these differences, Cohen's d effect size is calculated between the uncertainty distributions of correctly classified and misclassified samples for each model. Cohen's d is a standardized measure to assess the degree of difference between two distributions, and it is generally considered that d>0.8 represents a large effect size. Cohen's d of H-REFM model is 2.1333, Cohen's d of Trad. DS model is 2.0088, and Cohen's d of UA model is 1.1706. Although all the three models have reached the standard of large effect size, the effect size of H-REFM model is the largest, which indicates that it performs best in distinguishing the uncertainty of correctly classified and misclassified samples, and the uncertainty of misclassified samples is significantly higher than that of correctly classified samples, which shows that it has more reliable decision confidence prediction ability.

[0116] The core contribution of the present application is to solve the problem of modality conflict in medical multi-source heterogeneous data fusion by modeling the relationship between modality conflict and uncertainty. To further understand the performance differences of different models in handling modality conflict, a second set of targeted experiments is designed to evaluate the behavior of the models when facing contradictory evidence. In the experiment, the degree of modality conflict is calculated using the following formula: conflict AB =(A b1 ×B b2 )+(A b2 ×B b1 ), where A b1 and A b2 represent the belief values of view A for the first and second categories, i.e. belief quality, and B b1 and B b2 represent the belief values of view B for the first and second categories, i.e. belief quality. This formula quantifies the degree of contradiction between two modalities in class prediction: when two modalities support the same category, the conflict value is close to 0; when they support different categories, the conflict value increases, and the maximum value is 1.

[0117] In the experiment, Pearson correlation coefficient is used to calculate the relationship between average conflict degree and uncertainty, Figure 8 ,Figure 9 and Figure 10 The scatter plots of uncertainty performance of H-REFM model, traditional Dempster-Shafer (Trad. DS) model and general uncertainty-aware model (UA) in the case of modal conflict are shown in Fig. 4, Fig. 5 and Fig. 6, respectively.

[0118] The evidence theory provides a mathematical foundation for quantifying uncertainty, while the logarithmic opinion pool fusion algorithm naturally converts view conflict into increased prediction uncertainty by performing a weighted geometric mean in the logarithmic space. From Figure 8 , Figure 9 and Figure 10 It can be observed that when the view evidence is highly conflicting, the Pearson correlation coefficient r = 0.568 of H-REFM model presents a significant positive correlation, indicating that the uncertainty of the model output increases accordingly with the increase of the degree of inter-modal evidence conflict. In contrast, the traditional DS model shows a weak positive correlation (r = 0.236), especially in the high conflict area (average conflict degree > 0.4), the uncertainty of multiple sample points still remains at a very low level (< 0.02), which verifies that the traditional DS model tends to give overconfident predictions when facing conflicting evidence. The UA model shows the weakest correlation (r = 0.080), which is almost close to no correlation, its regression line is almost parallel to the x-axis, and the uncertainty value is generally low in the whole conflict range, indicating that the model is almost insensitive to modal conflict.

[0119] Interpretability analysis:

[0120] The Grad-CAM technique is used to visualize the attention area of H-REFM model on CT images. Figure 11The abdominal CT cross-sectional images and their corresponding Grad-Cam results of 5 patients with acute pancreatitis are shown (each column corresponds to a patient), and the first row is the abdominal CT cross-sectional image showing the main slice, and the second row is the Grad-Cam result showing the model's focus area. The heat map shows that the model focuses on the pancreas and its surrounding area during the prediction process, which is highly consistent with the focus of radiologists in assessing the severity of acute pancreatitis. From the anatomical distribution, the high activation area of the heat map (yellow area) accurately covers the head, body, and tail of the pancreas and the surrounding tissues, accurately capturing the key pathological areas of acute pancreatitis. The heat maps of the first two patients with acute pancreatitis mainly focus on the head of the pancreas and the adjacent duodenum, which are often edematous and inflammatory changes in the early stage of acute pancreatitis. The heat map of the third patient with acute pancreatitis shows a more extensive focus area, covering the body of the pancreas and the surrounding retroperitoneal space, which is consistent with the imaging findings of diffuse pancreatitis. The heat map of the fourth patient with acute pancreatitis highlights the body and tail of the pancreas and the adjacent spleen, reflecting the spread of pancreatitis to the spleen. The heat map of the fifth patient with acute pancreatitis highlights the liquid accumulation area around the pancreas, which is one of the important indicators for assessing the severity of acute pancreatitis. To verify the clinical relevance of the model's focus area, two experienced radiologists (with more than 15 years of experience) were invited to conduct a blind evaluation of the heat map. 80% of the areas match the areas of interest of the radiologists.

[0121] To better understand the decision-making process of the H-REFM model and evaluate the contribution of clinical table data to the prediction of acute pancreatitis progression, the gradient integration method was used to analyze the importance of clinical table data in the experiment, and 20 clinical indicators (features) with the highest value in predicting the progression of acute pancreatitis were identified from more than 60 clinical indicators (features), and the normalized importance scores are shown as follows. Figure 12

[0122] ​Hypersensitive C-reactive protein (hs-CRP) and amylase together reflect the degree of inflammation. Liver function-related indicators such as the aspartate aminotransferase / alanine aminotransferase (AST / ALT) ratio and indirect bilirubin provide information on the involvement of the hepatobiliary system, which is closely related to the systemic response caused by pancreatitis. Among the hematological indicators, the absolute value of neutrophils, the percentage of basophils, and the percentage of eosinophils make up the immune response spectrum, reflecting the dynamic changes of different white blood cell subgroups in the disease process. The importance of these immune cell parameters indicates that a fine-grained immune function assessment may provide more accurate predictive information than the total white blood cell count. The inclusion of D-dimer as a marker of coagulation function strengthens the link between coagulation system abnormalities and the severity of pancreatitis. The importance of creatine kinase isoenzyme highlights the potential impact of pancreatitis on myocardial function, an aspect often overlooked by traditional scoring systems but of great significance in systemic inflammatory response syndrome.

[0123] The H-REFM model integrates key indicators of multiple physiological function systems of the human body, including the inflammation system (hs-CRP, amylase), the hepatobiliary system (AST / ALT ratio, indirect bilirubin), the immune system (absolute value of neutrophils, percentage of basophils, etc.), the coagulation system (D-dimer), and myocardial function (creatine kinase isoenzyme), to construct a more comprehensive prediction framework than traditional single-system evaluation. This multi-system integration method goes beyond the limitations of traditional evaluation methods such as the Ranson score, which focuses on a single physiological system, and can more comprehensively reflect the systemic impact of acute pancreatitis.

Claims

1. A method for assessing the severity of pancreatitis based on hierarchical reliable evidence fusion, characterized in that, The method comprises the following steps: S1, obtaining a plurality of multi-modal data sets of a plurality of patients and constructing a training set, each training sample in the training set comprising standard medical image data, standard clinical table data and a corresponding severity label of the same patient; S2, constructing a hierarchical reliable evidence fusion model, which sequentially performs the following stages: a multi-view evidence generation stage, generating multi-level image view evidence vectors, clinical view evidence vectors and pseudo view evidence vectors based on the standard medical image data and the standard clinical table data; The specific implementation process of the multi-view evidence generation stage is: (a) inputting the standard medical image data into a pre-trained 3D-Resnet network, extracting different semantic level image feature maps from low to high, and obtaining corresponding image feature vectors through an adaptive maximum pooling layer; (b) inputting the standard clinical table data into a feature encoding network based on a Transformer architecture to obtain a clinical feature vector; (c) inputting the clinical feature vector and the image feature vector of the highest semantic level into a cross-modal complementary attention fusion module to generate a fusion feature vector; (d) sequentially inputting each feature vector through a fully connected layer and a Softplus activation function to obtain image view evidence vectors, clinical view evidence vectors and pseudo view evidence vectors of each level; The implementation process of the cross-modal complementary attention fusion module is: inputting the image feature vector of the highest semantic level through a first fully connected layer, and the obtained vector as value V1, key K1 and query Q1 respectively; inputting the clinical feature vector through a second fully connected layer, and the obtained vector as value V2, key K2 and query Q2 respectively; then performing vector multiplication on the transposed vectors of key K1 and query Q2 and the transposed vectors of key K2 and query Q1 respectively, to obtain vectors R1 and R2 respectively; then performing vector multiplication on vector R1 and value V1 and vector R2 and value V2 respectively, to obtain vectors R3 and R4 respectively; then performing vector addition on vector R3 and the vector output by the first fully connected layer and vector R4 and the vector output by the second fully connected layer respectively, to obtain vectors R5 and R6 respectively; then inputting vector R5 through a third fully connected layer to obtain vector R7; inputting vector R6 through a fourth fully connected layer to obtain vector R8; performing vector addition on vector R7 and the image feature vector input to the first fully connected layer to obtain vector R9; finally, performing vector splicing on vector R9 and vector R8 to obtain the fusion feature vector; An opinion mapping stage for mapping each view evidence vector to a subjective logical opinion; An opinion fusion stage for fusing all subjective logical opinions based on a logarithmic opinion pool fusion algorithm to generate a final fusion opinion; S3, training the hierarchical reliable evidence fusion model based on the training set to obtain a trained hierarchical reliable evidence fusion model; S4, inputting any test sample into the trained hierarchical reliable evidence fusion model, and evaluating the severity of acute pancreatitis of the patient according to the belief quality of each category in the output final fusion opinion.

2. The method for assessing the severity of pancreatitis based on hierarchical reliable evidence fusion according to claim 1, characterized in that, The standard medical image data is obtained by preprocessing the medical image data in the multi-modal data set, and the preprocessing comprises: A1. performing regional rough segmentation on the medical image data based on a pre-trained nnU-Net segmentation network to obtain a rough segmentation result containing a pancreas and a surrounding edema effusion region; A2. extracting a fixed-size three-dimensional region of interest with the centroid of the rough segmentation result as the origin; A3. performing image standardization processing on the three-dimensional region of interest to obtain the standard medical image data.

3. The method for assessing the severity of pancreatitis based on hierarchical reliable evidence fusion according to claim 1, characterized in that, The standard clinical table data is obtained by preprocessing the clinical table data in the multi-modal data set, and the clinical table data includes qualitative baseline data for describing the basic condition of a patient and quantitative physiological data for reflecting the pathophysiological state. The preprocessing includes one-hot encoding of the qualitative baseline data, Z-score standardization of the quantitative physiological data, and missing value filling.

4. The method for assessing the severity of pancreatitis based on hierarchical reliable evidence fusion according to claim 1, characterized in that, In the opinion mapping stage, first, each view evidence vector is converted into Dirichlet evidence, which includes Dirichlet distribution concentration parameters of each category, wherein the formula for converting the view evidence vector into Dirichlet evidence is: α = e + aW, wherein α = [α1, α2], α k represents the Dirichlet distribution concentration parameter of the kth category, e represents the view evidence vector, a represents the base rate, which reflects the category prior distribution, a = [a1, a2], a k = N k / N, k = 1, 2, N k represents the number of training samples of the kth category, N represents the total number of training samples, the first category represents mild acute pancreatitis, the second category represents severe acute pancreatitis, and W represents the balance hyperparameter, W > 0; then the Dirichlet evidence is mapped into subjective logical opinions, and the mapping formula is: b k = (α k -a k W) / S, u = W / S, S = α1 + α2, wherein b k represents the belief quality of the kth category, u represents the overall uncertainty, and satisfies the constraint condition: b k ≥ 0, u ≥ 0, b1 + b2 + u = 1.

5. The method for assessing the severity of pancreatitis based on hierarchical reliable evidence fusion according to claim 4, characterized in that, In the opinion fusion stage, all subjective logical opinions corresponding to the multi-level image view evidence vectors are first fused within a modality to generate a unified image view opinion, denoted as M. I ; and then the unified image view opinion M I The subjective logical opinion M corresponding to the clinical view evidence vector T Subjective logical opinion M corresponding to the pseudo-view evidence vector P Perform intermodal fusion to generate the final fusion opinion, denoted as M. Z .

6. The method for assessing the severity of pancreatitis based on hierarchical reliable evidence fusion according to claim 5, characterized in that, The unified image view opinion M I and the final fusion opinion M Z The closed-form solution is: M j ={b j,1 ,b j,2 ,u j}, , where j is I or Z, b j,1 represents the belief quality of the first category in M j , b j,2 represents the belief quality of the second category in M j , u j represents the overall uncertainty in M j , Ω={I1, I2, …, In} when j is I, Ω={I, T, P} when j is Z, n represents the number of layers, and all subjective logical opinions corresponding to the multi-layer image view evidence vector are M I1 , M I2 , …, M In , M i ={b i,1 ,b i,2 ,u i}, b i,1 represents the belief quality of the first category in M i , b i,2 represents the belief quality of the second category in M i , u i represents the overall uncertainty in M i , c i =1-u i =b i,1 +b i,2 , and C is a normalization constant. .

Images