A method for assisting in identifying the inflammatory activity grade of ulcerative colitis based on dynamic graph multi-instance learning

Abstract

The application provides a method for identifying the inflammatory activity grade of ulcerative colitis based on dynamic graph multi-instance learning, which comprises data collection, data preprocessing, dynamic graph multi-instance learning model construction for inflammation activity diagnosis and model evaluation. The data collection mainly comprises collecting pathological images of ulcerative colitis patients, digitizing through a digital scanner, and excluding a part of pathological images containing problems such as blur, discoloration and abnormal staining, and finally determining the grading label of each pathological image by senior gastrointestinal pathology experts. The data preprocessing mainly comprises processing the digital pathological images, converting the digital pathological images into image blocks, then using a visual basic model to extract features from each image block, and finally constructing a dynamic graph multi-instance learning model for training. The application promotes the exploration of the internal relationship of the image blocks input into the WSI, and improves the prediction effect of the model.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, specifically relating to a method for assisting in the identification of the inflammatory activity level of ulcerative colitis based on dynamic graph multi-instance learning. Background Technology

[0002] Ulcerative colitis (UC) is a chronic, relapsing, and nonspecific inflammatory bowel disease (IBD). It primarily affects the mucosal layer of the colon and rectum, leading to long-term inflammation and ulceration. Severe complications, such as massive bleeding and toxic megacolon, can increase the risk of death and, in extreme cases, require total colectomy. Therefore, assessing the remission status of UC is crucial for subsequent treatment and prognosis.

[0003] Prior studies have shown that a higher degree of mucosal healing observed during endoscopy is closely associated with better clinical outcomes and prognosis in patients with ulcerative colitis (UC). Therefore, clinical goals for UC treatment typically include symptom relief and achieving mucosal healing during endoscopy. Recent studies have also indicated that persistent histological inflammation is closely associated with UC recurrence, surgical risk, and the development of colorectal cancer (CRC). The degree of histological inflammation has been identified as playing a crucial role in patient treatment. Furthermore, during histological mucosal healing, the degree of active inflammation characterized by neutrophil infiltration is most closely associated with endoscopic features and patient prognostic outcomes. Therefore, it is necessary to predict the inflammatory activity of UC.

[0004] Currently, artificial intelligence (AI) methods have been successfully applied in oncology, including pan-cancer tumor location detection, subtype classification of various diseases, prediction of tumor treatment response and prognosis, and comprehensive radiological analysis. Therefore, these AI methods provide a solid foundation for IBD research. Significant progress has been made in using AI technology to assist in the diagnosis and differential diagnosis of IBD. In the field of molecular genetics, genome-wide association studies have identified more than 240 gene loci associated with an increased risk of IBD, and these loci also help distinguish between ulcerative colitis (UC) and Crohn's disease (CD). Genetic risk scores generated using disease-related gene information from large multicenter datasets of IBD patients show a strong correlation with IBD subtypes. Furthermore, analysis of protein features in colon tissue using a support vector machine learning model can identify CD and UC with an accuracy of 76.9%. Therefore, this provides technical feasibility for diagnosis at the inflammatory level. Summary of the Invention

[0005] Purpose of the Invention: Inflammatory bowel disease (IBD) is a global disease with an increasing incidence. However, research on computational aids in the diagnosis of IBD based on pathological images is relatively scarce. Therefore, to reduce the workload of pathologists and improve the accuracy and efficiency of grading inflammatory activity in UC patients, this invention proposes a method for assisting pathologists in the accurate and consistent diagnosis and assessment of UC based on dynamic graph multi-instance learning.

[0006] Technical solution: A method for identifying the inflammatory activity level of ulcerative colitis based on dynamic graph multi-instance learning, comprising the following steps:

[0007] Step 1, Data Collection: Collect whole-section images of different ulcerative colitis (UC) levels and digitize them using a digital scanner;

[0008] Step 2, data preprocessing of the whole slice image: control the image content and quality, divide the tissue region in the whole slice image into image blocks to form the training dataset and test dataset of the model;

[0009] Step 3: Quantify the similarity between the training dataset and the external dataset, and use the visual base model UNI to filter and extract features to obtain feature vectors;

[0010] Step 4: Construct an image-level inflammatory activity prediction model for ulcerative colitis (UC). The model is trained based on the feature vectors obtained in Step 3. The Multi-Instance Learning (MIL) method of dynamic graphs is used to process the spatial relationships between different image patches. A directed graph between image patches is constructed, thereby fusing the dynamic graph features between image patches for image classification.

[0011] Step 5: Use 5-fold cross-validation to evaluate the performance of the DGMIL model for two tasks, compare it with the traditional MIL and AttMIL algorithms, and visualize it using gradient-weighted class activation mapping (Grad-CAM); the two tasks refer to determining the presence of inflammation and further subdividing the level of inflammation.

[0012] Step 1 includes: collecting whole-slice images of different ulcerative colitis (UC) levels, grading each whole-slice image by WSI, with four grades: L0 (no activity: no intraepithelial neutrophils, ulcers, or erosions); L1 (mild activity: cryptitis involving ≤25% of crypts or crypt abscesses involving ≤10% of crypts); L2 (moderate activity: cryptitis affecting >25% of crypts, crypt abscesses involving >10% of crypts, or focal, small erosions); L3 (severe activity: ulcers or extensive erosions are present); and finally, digitizing the whole-slice images using a digital scanner.

[0013] Step 2 includes: processing the tissue region individually for each whole-slice image; removing the background of the WSI using median filtering and the Otsu algorithm; and extracting the foreground for each WSI to obtain the tissue region, typically using a magnification closest to 64x downsampling. This process involves extracting the foreground and eliminating large cavities within the tissue region. The extraction method employs median filtering, the Otsu algorithm, and other morphological closure techniques, with different threshold settings used for tissue region extraction and contour delineation. Because whole-slice images (WSIs) are extremely large, typically reaching GB levels in storage, direct processing is very difficult. This invention crops the tissue region into 512×512 image blocks at the maximum magnification of 40x for the WSI. These small blocks are stored using a coordinate-based method to optimize computational resources and facilitate retrieval.

[0014] Step 3 includes: To quantify the similarity between the training dataset and the external dataset, this invention uses ψ(psi) index analysis. This non-parametric and distribution-unrestricted technique can quantify the similarity between two datasets, thereby better evaluating the model's generalization ability and robustness. To extract representative features from each image patch, this invention uses the Visual Base Model (UNI) for feature extraction for each image patch via transfer learning. The UNI transforms all input image patches into 1024-dimensional feature vectors, and then uses a separate linear layer to reduce the feature dimension of the image patches to 512 dimensions, as shown in the following formula:

[0015] h i =W h f(X)

[0016] Where X represents the input feature matrix, which here represents the feature vector of the image patch, and f(X) represents the extraction of the feature vector of the original image patch into 1024 dimensions using the UNI visual model. h This is a weight matrix, representing a linear transformation that reduces the 1024-dimensional feature vector extracted by the UNI visual model to 512 dimensions; h iThis represents the feature vector obtained after the i-th image patch passes through the visual base model UNI and a linear layer.

[0017] Step 4 includes: For all image patch labels, the WSI level labels of the whole slice image are used as the image patch labels to construct an automatic grading model of ulcerative colitis (UC) inflammatory activity at the slide level.

[0018] The automatic grading model for ulcerative colitis (UC) inflammatory activity is a Dynamic Graph-based Multiple Instance Learning (DGMIL) model. The DGMIL model includes a dynamic graph module and a multiple instance learning module (MIL). By combining the dynamic graph structure between image patches and the MIL method, the DGMIL model enhances the interactive processing of spatial information of whole slice images (WSI) and effectively mines the correlation between image patches.

[0019] The dynamic graph module first calculates the similarity score between image patches using the following formula:

[0020]

[0021] Where i ≠ j, N is the set of all image patches, h j This represents the feature vector obtained after the j-th image patch passes through the UNI and linear layers. Represents the calculation of h i h j The dot product similarity of two feature vectors is then used to obtain the similarity score w between the i-th and j-th image patches after passing the normalized exponential function softmax. i,j ,

[0022] Obtain the similarity score w i,j Then, for each image patch, the top-k image patches with the highest similarity scores are selected as the neighboring image patches of the i-th image patch, thereby enhancing the global feature representation. The formula is as follows:

[0023]

[0024] in This means selecting the k most similar images (w) to the i-th image patch from all other image patches. i,j , w represents the similarity score between the selected image patch and the i-th image patch. i,j The set of the k largest image patches is used as the neighboring image patches of the i-th image patch;

[0025] The directed topological structure between image patches is represented as follows:

[0026] d i,j =w i,j h j +(1-w i,j )h i

[0027] Where d i,j This represents the spatial embedding representation between the i-th image patch and the j-th image patch. Information flows from the k image patches with the highest similarity scores to the i-th image patch. The features of the i-th image patch are updated by combining the features of these k image patches. In the graph structure, this means that the nodes corresponding to these k image patches are connected to the nodes corresponding to the i-th image patch, with the direction from the nodes of the k image patches to the nodes of the i-th image patch.

[0028] For the i-th image patch, calculate the similarity score w between the i-th image patch and the i-th image patch selected from all image patches. i,j A linear combination of features of the set N(i) of the k largest image patches to characterize the first-order connectivity structure of the i-th image patch:

[0029]

[0030] Where τ is a weight used to guide the propagation of information from the top-k image patches to the i-th image patch, h N(i) It is the updated feature vector obtained by weighted combination of the feature vectors of the i-th image block and its neighboring image blocks;

[0031] Using dot product and summation to exchange information between nodes, it can be represented as:

[0032] h i =α(w1(h) i +h N(i) ))+β(w2(h i ⊙h N(i) ))

[0033] Where α and β represent the LeakyReLU activation functions, and w1 and w2 represent the learnable transformation matrices;

[0034] Finally, the output of the dynamic graph structure is used as input to the multi-instance learning module MIL. The label class probability of each WSI image is obtained through Softmax and MaxPooling, with the following formula:

[0035]

[0036] Where G is the updated feature vector obtained after all image patches in the dynamic graph have undergone information exchange. This is expressed as class probability.

[0037] Dynamic graph structures effectively mitigate the loss of global and spatial information caused by segmenting whole slice images (WSIs) into multiple image patches. Its core functionality allows each element in the input sequence to interact with other elements, facilitating the exploration of the intrinsic relationships between image patches input into the WSI. Furthermore, a significant advancement of dynamic graph structures lies in their ability to allow the decoder to access the entire encoded information and assign weights to the input data. This process captures the importance of each label, prioritizing them at each step when generating output labels. By covering the receptive field of the entire image, dynamic graphs utilize self-captured global and spatial features to predict the class of the WSI.

[0038] In step 4, the formula for the normalized exponential function softmax is:

[0039]

[0040] Where z i It is the i-th element in the input vector, exp(z) i ) represents z i The exponential function, the denominator part It is the sum of the exponential functions of all elements in the input vector.

[0041] In step 4, for image-level prediction, a Multi-Instance Learning (DGMIL) method combined with a dynamic graph structure is employed. The hyperparameters are configured as follows: the optimizer uses Adam, and the loss function is softmax cross-entropy, a combination of standard cross-entropy loss and softmax. The initial learning rate is set to 2e-4, and the dropout rate is 0.25. During training iterations, the loss is calculated, and the weights of the DGMIL model are updated based on the minimum loss reached at the end of each iteration. The maximum number of training iterations is set to 200, and the minimum to 50. The decision to continue training is then based on the change in the loss value. If the loss value no longer decreases after 20 consecutive training iterations, the training process is terminated.

[0042] In step 5, to evaluate the variability of the dataset and the robustness of the multi-instance learning model DGMIL, 5-fold cross-validation was used. The multi-instance learning model DGMIL was tested against two classic multi-instance learning (MIL) algorithms, MIL and AttMIL, on both internal and external test sets. In the dynamic graph structure of the network in step 4, ReLU was used as the activation function and cross-entropy as the loss function. The positional relationship between patches was quantified through learnable implicit features. The whole slice image WSI was processed using a non-overlapping block extraction method to obtain image patches of size X = {x1, x2, ..., x...}. n}, where xn This represents the nth image patch, where n is the total number of image patches, and all image patches are treated as nodes in the graph.

[0043] The multi-instance learning model DGMIL is designed for two different tasks (determining the presence of inflammation and further subdividing the level of inflammation). It summarizes image patch features through directed graph weights to predict the label of the whole slice image WSI. The training of the multi-instance learning model DGMIL uses cross-entropy loss combined with softmax to calculate the classification probability.

[0044] The multi-instance learning model DGMIL generates heatmaps to illustrate the regions of interest it focuses on during diagnosis, thus explaining the model's decision-making process. This visualization method helps to understand the degree of attention the model pays to each region and allows for comparison with pathologists' findings.

[0045] The DGMIL model of this invention handles different tasks through multiple classification branches. Specifically, after feature extraction, it inputs the features into different task branches, with each branch responsible for classifying a specific task. This design allows the model to refine its processing according to the needs of the task while sharing underlying features. It can simultaneously perform two tasks: determining the presence of inflammation and further subdividing the level of inflammation.

[0046] The present invention also provides an electronic device, including a processor and a memory, the memory storing program code that, when executed by the processor, causes the processor to perform the steps of the method.

[0047] The present invention also provides a storage medium storing a computer program or instructions that, when the computer program or instructions are run on a computer, execute the steps of the method described.

[0048] Beneficial effects: This invention employs image-level labeling, eliminating the need for pathologists to annotate inflammation levels at the pixel level, allowing them to label only at the image level, thus reducing the workload for pathologists involved in building the deep learning model. Simultaneously, the dynamic graph structure effectively mitigates the loss of global and spatial information caused by segmenting whole slice images (WSIs) into multiple image patches. Its core function enables each element in the input sequence to interact with other elements, facilitating the exploration of the intrinsic relationships between image patches input to the WSI and improving the model's predictive performance. Attached Figure Description

[0049] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, and the advantages of the present invention in the above and / or other aspects will become clearer.

[0050] Figure 1 This is a flowchart of the method of the present invention;

[0051] Figure 2 The bar chart includes the patient selection process and the number of patients and patches at each level of inflammatory activity in each cohort.

[0052] Figure 3 This is a schematic diagram illustrating the differences between datasets DTHUC and ZJH using density and cumulative distribution function (CDF) plots, as presented in this invention.

[0053] Figure 4 This is a schematic diagram illustrating the diagnostic performance of the present invention in predicting whether UC patients have inflammation through 5-fold cross-validation on internal and external test sets.

[0054] Figure 5 This is a schematic diagram illustrating the diagnostic performance of the present invention in predicting the grade IV histological activity of UC patients using DGMIL based on 5-fold cross-validation on internal and external test sets.

[0055] Figure 6 This diagram illustrates the results of comparing the diagnostic accuracy of the present invention across two tasks and two pathologists.

[0056] Figure 7 This is a prediction heatmap of the DGMIL model of this invention; Detailed Implementation

[0057] This embodiment discloses a method for identifying the inflammatory activity level of ulcerative colitis based on dynamic graph multi-instance learning, including the following steps:

[0058] Step 1: The experimental dataset used in this invention consists of whole-slice images (WSIs) of different ulcerative colitis (UC) levels collected from Nanjing Drum Tower Hospital affiliated with Nanjing Medical University as the model training set and internal test set, and UC whole-slice images collected from Zhujiang Hospital of Southern Medical University as the external test set. For convenience, the two datasets are named DTHUC and ZJH, respectively. To ensure the quality of WSIs, 33 WSIs in the DTHUC dataset were excluded due to problems such as blurring, fading, and abnormal staining. In the ZJH dataset, 26 WSIs were excluded. The final DTHUC dataset contains 570 UC WSIs, including 76 WSIs at level L0, 135 at level L1, 250 at level L2, and 109 at level L3. The training set, validation set, and test set were split using 10-fold Monte Carlo cross-validation in a 7:1:2 ratio. The ZJH dataset, used as an external test set, contains 186 UC WSIs, including 12 at level L0, 94 at level L1, 58 at level L2, and 22 at level L3. After digitizing the tissue sections, the WSIs were preprocessed into 512×512 pixel image patches at maximum magnification. All WSIs were digitized using a Leica GT450 scanner at a pixel resolution of 263 nm / pixel.

[0059] Step 2: After extracting the foreground region of the WSI image, this invention processes the foreground region into a 512×512 pixel image block, such as... Figure 1 As shown in (A), analysis of the ψ(psi) index on the training dataset and the external dataset yielded a ψ value of 0.341, indicating a moderate difference in the feature distributions between the training set and the external dataset. Figure 3 The present invention uses density and cumulative distribution function plots to visualize the differences between the DTHUC dataset and the ZJH dataset. This similarity quantification more clearly demonstrates the differences between the external dataset and the training dataset, thereby better evaluating the model's generalization ability and robustness.

[0060] Step 3: Three senior gastrointestinal pathologists jointly determine the grading label for each WSI (Warnings Indication System). Figure 1 (B) To compare the AI ​​assessment results with those of pathologists with varying levels of experience, two additional junior pathologists were also involved in the study, one with two years of experience and the other with five years. Pathologists were not required to annotate inflammation levels at the pixel level; they only needed to annotate at the image level, reducing the workload for pathologists involved in building the deep learning model.

[0061] Step 4: UNI acts as a feature extractor, transforming all image patches into 1024-dimensional structured data through the visual base model, and then fusing them to characterize each WSI.

[0062] Step 5: During the model training phase, vectorized features from the training and validation sets are constructed into a dynamic graph representation, which is then used for continuous optimization and iteration of the model in conjunction with softmax loss. The DGMIL model uses feature vector encoding based on a visual foundation model and processes the feature vectors using a dynamic graph structure, outputting a graded score using softmax. Finally, the algorithm's performance is evaluated using a test set to verify the effectiveness of the algorithm model.

[0063] To evaluate the variability of the dataset and the robustness of the algorithm model, 5-fold cross-validation was used. In the task of determining whether UC patients have inflammation (L0 is considered the non-inflammatory class, and L1, L2, and L3 are considered the inflammatory classes), the algorithm model achieved a mean accuracy (ACC) of 87.3% and a mean AUC of 0.863 (95% [CI] 0.829, 0.898) on the internal test set. The mean sensitivity was 0.913 (95% [CI] 0.866, 0.961), and the mean specificity was 0.816 (95% [CI] 0.771, 0.861). On external datasets, the model of this invention achieved a mean accuracy (ACC) of 88.7% and a mean AUC of 0.947 (95% [CI] 0.939, 0.955). The mean sensitivity was 0.889 (95% [CI] 0.837, 0.940), and the mean specificity was 0.858 (95% [CI] 0.777, 0.939). Figure 4 The table presents the results of 10-fold cross-validation on internal and external test sets in the task of determining the presence of inflammation in UC patients, including ACC, Macro-AUC, Micro-AUC, sensitivity, and specificity for each fold. In the task of grading the four histological activity levels of UC patients, the algorithm model achieved a mean ACC of 76.9%, a mean Macro-AUC of 0.827 (95% [CI] 0.803, 0.850), a mean Micro-AUC of 0.816 (95% [CI] 0.792, 0.840), a mean sensitivity of 0.770 (95% [CI] 0.705, 0.835), and a mean specificity of 0.856 (95% [CI] 0.834, 0.878) on the internal test set. On external datasets, the model of this invention achieved a mean accuracy (ACC) of 70.0%, a mean macro-AUC of 0.908 (95% [CI] 0.882, 0.935), and a mean micro-AUC of 0.898 (95% [CI] 0.869, 0.926). The mean sensitivity was 0.678 (95% [CI] 0.626, 0.731), and the mean specificity was 0.883 (95% [CI] 0.861, 0.906). Figure 5The table shows the results of 10-fold cross-validation of internal and external test sets in the four-level histological activity grading task for UC patients.

[0064] Step 6: Comparison with evaluation results from pathologists of varying experience. The main purpose of this algorithm model is to assist pathologists in making diagnoses, reduce repetitive workloads, and allow them more time to focus on other complex medical conditions. During the model training phase, each WSI was graded by three senior gastrointestinal pathologists (each with over 15 years of clinical experience) to ensure high-quality training data. To evaluate the potential application of the AI ​​model in clinical practice, this invention selected two junior pathologists (with 2-5 years of clinical experience) for result comparison. The claim that the model outperforms the pathologists' diagnostic results refers to a comparison with the diagnostic results of these two junior pathologists. Pathologist 1 has 2 years of pathological diagnosis experience, and pathologist 2 has 5 years of experience. The diagnostic accuracy of the two pathologists in the two tasks is shown in the internal and external test sets. Figure 6 Therefore, the algorithm model developed in this study can be used to guide pathologists with 2 years of experience in diagnosing the level of inflammatory activity in ulcerative colitis (UC), and can also assist pathologists with 5 years of experience in diagnosing the histological activity grading of UC.

[0065] Step 7: Comparative analysis of DGMIL with two other algorithms in computing UC pathological activity. DGMIL was compared with two classic multi-instance learning (MIL) algorithms—MIL and AttMIL—on various experimental tasks. MIL and AttMIL each had their own advantages and disadvantages in different studies, and the testing procedures for the three algorithms remained consistent.

[0066] Step 8: To provide a visual explanation of DGMIL. The visualization results are shown as follows: Figure 7 As shown, the model focuses on glandular regions, which aligns with the diagnostic priorities of pathologists. The histological activity grading criteria for pathological assessment depend on the degree of destruction of crypt glands by active inflammation, highlighting the model's ability to identify key pathological areas required for accurate diagnosis. This focus on specific regions further demonstrates that DGMIL's insights are consistent with the meticulous considerations pathologists make during assessment. In a typical case, the red highlighted areas are concentrated in the lamina propria of the colonic mucosa, the area of ​​greatest interest to pathologists and the most common site of active enteritis. In contrast, active inflammation is less prevalent in the muscularis mucosa and submucosa, thus receiving less attention from AI. The model's consistent focus on these key indicators enhances its diagnostic capabilities and aligns with the expertise of pathologists who rely on similar visual cues for accurate and comprehensive assessments. This contributes to a deeper understanding of the algorithm's decision-making process.

[0067] This invention provides a method for identifying the inflammatory activity level of ulcerative colitis based on dynamic graph multi-instance learning. Many methods and approaches exist for implementing this technical solution; the above description is merely a preferred embodiment of the invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.

Claims

1. A method for identifying the inflammatory activity level of ulcerative colitis based on dynamic graph multi-instance learning, characterized in that, Includes the following steps: Step 1, Data Collection: Collect whole-section images of different ulcerative colitis (UC) levels and digitize them using a digital scanner; Step 2, data preprocessing of the whole slice image: control the image content and quality, divide the tissue region in the whole slice image into image blocks to form the training dataset and test dataset of the model; Step 3: Quantify the similarity between the training dataset and the external dataset, and use the visual base model UNI to filter and extract features to obtain feature vectors; Step 4: Construct an image-level inflammatory activity prediction model for ulcerative colitis (UC). The model is trained based on the feature vectors obtained in Step 3. The Multi-Instance Learning (MIL) method of dynamic graphs is used to process the spatial relationships between different image patches. A directed graph between image patches is constructed, thereby fusing the dynamic graph features between image patches for image classification. Step 5: Use 5-fold cross-validation to evaluate the performance of the DGMIL model for two tasks, compare it with the traditional MIL and AttMIL algorithms, and visualize it using gradient-weighted class activation mapping; the two tasks refer to determining the presence of inflammation and further subdividing the level of inflammation. Step 4 includes: For all image patch labels, the WSI-level labels of the whole slice image are used as the image patch labels to construct an automatic grading model of ulcerative colitis (UC) inflammatory activity at the slice level; The automatic grading model for inflammatory activity in ulcerative colitis (UC) is a multi-instance learning model based on dynamic graphs, namely DGMIL, which includes a dynamic graph module and a multi-instance learning module (MIL). The dynamic graph module first calculates the similarity score between image patches using the following formula: ， in N is the set of all image patches. This represents the feature vector obtained after the j-th image patch passes through the UNI and linear layers. Representative calculation , The similarity is calculated by the dot product of two feature vectors, and then the obtained similarity is passed through a normalized exponential function (softmax) to get the similarity score between the i-th and j-th image patches. ; Get similarity score Then, for each image patch, the one with the highest similarity score is selected. The image patch is considered as a neighboring image patch of the i-th image patch, as shown by the formula: ， in This means selecting the k images with the highest similarity scores between the i-th image patch and all other image patches. , This represents the similarity score between the selected image patch and the i-th image patch. The set of the k largest image patches is used as the neighboring image patches of the i-th image patch; The directed topological structure between image patches is represented as follows: ， in This represents the spatial embedding representation between the i-th image patch and the j-th image patch. Information flows from the k image patches with the highest similarity scores to the i-th image patch. The features of the i-th image patch are updated by combining the features of the k image patches. In the graph structure, this means that the nodes corresponding to the k image patches are connected to the nodes corresponding to the i-th image patch, with the direction from the nodes of the k image patches to the nodes of the i-th image patch. For the i-th image patch, calculate the similarity score between the i-th image patch and the i-th image patch selected from all image patches. The set of the k largest image patches A linear combination of features to characterize the first-order connectivity structure of the i-th image patch: ， in It is a weight used for guidance The information from one image patch is propagated to the i-th image patch. It is the updated feature vector obtained by weighted combination of the feature vectors of the i-th image block and its neighboring image blocks; Using dot product and summation to exchange information between nodes, it can be represented as: ， in, and This represents the LeakyReLU activation function. and Represents the learnable transformation matrix; Finally, the output of the dynamic graph structure is used as input to the multi-instance learning module MIL. The label class probability of each WSI image is obtained through Softmax and MaxPooling, with the following formula: , in The updated feature vectors of all image patches are output for the dynamic graph after information exchange. Represented as class probability; In step 4, for image-level prediction, a multi-instance learning (MIL) method combined with a dynamic graph structure is adopted. The hyperparameters are configured as follows: the optimizer uses Adam, and the loss function uses softmax cross-entropy, which is a combination of standard cross-entropy loss and softmax. The initial learning rate is set to 2e-4, and the dropout rate is 0.

25. During the training iteration, the loss is calculated, and the weights of the multi-instance learning model DGMIL are updated based on the minimum loss reached at the end of each iteration. In step 5, to evaluate the variability of the dataset and the robustness of the multi-instance learning model DGMIL, 5-fold cross-validation was used. The multi-instance learning model DGMIL was tested against two multi-instance learning algorithms, MIL and AttMIL, on both internal and external test sets. In the dynamic graph structure of the network in step 4, ReLU was used as the activation function, and cross-entropy as the loss function. The positional relationships between patches were quantified using learnable implicit features. The whole-slice image WSI was obtained using a non-overlapping block extraction method. Image patches of various sizes ,in This represents the nth image patch, where n is the total number of image patches, and all image patches are treated as nodes in the graph. The multi-instance learning model DGMIL processes two tasks. It summarizes image patch features through directed graph weights to predict the label of the entire whole slice image WSI. The training of the multi-instance learning model DGMIL uses cross-entropy loss and combines softmax to calculate the classification probability. The multi-instance learning model DGMIL generates heatmaps to show the regions of interest that it focuses on during the diagnostic process, thus explaining the model's decision-making basis.

2. The method according to claim 1, characterized in that, Step 1 includes: collecting whole-slice images of different ulcerative colitis (UC) levels, grading each whole-slice image by WSI, with four levels: L0 no activity; L1 mild activity; L2 moderate activity; L3 severe activity; and finally digitizing the whole-slice images using a digital scanner.

3. The method according to claim 2, characterized in that, Step 2 includes: processing the tissue region of each full-slice image separately; removing the background of the WSI using median filtering and the Otsu algorithm; extracting the foreground of each WSI to obtain the tissue region; and then cropping the tissue region. Image blocks of a certain size.

4. The method according to claim 3, characterized in that, Step 3 includes: For each image patch, feature extraction is performed using the visual base model UNI through transfer learning. The visual base model UNI converts all input image patches into 1024-dimensional feature vectors, and then uses a separate linear layer to reduce the feature dimension of the image patch to 512 dimensions, as shown in the following formula: ， in The input feature matrix represents the feature vector of the image patch. This indicates that the feature vectors of the original image patch are extracted to 1024 dimensions using the UNI visual model. It is a weight matrix that represents a linear transformation that reduces the 1024-dimensional feature vector extracted by the visual base model UNI to 512 dimensions. This represents the feature vector obtained after the i-th image patch passes through the visual base model UNI and a linear layer.

5. The method according to claim 4, characterized in that, In step 4, the formula for the normalized exponential function softmax is: ， in It is the first in the input vector One element, express The exponential function, the denominator part It is the sum of the exponential functions of all elements in the input vector.

6. An electronic device, characterized in that, It includes a processor and a memory, the memory storing program code that, when executed by the processor, causes the processor to perform the steps of the method as described in any one of claims 1 to 5.

7. A storage medium, characterized in that, It stores a computer program or instructions that, when run on a computer, perform the steps of the method as described in any one of claims 1 to 5.

Images