Method for improving evaluation accuracy of various indexes of non-neoplastic diseases of stomach in histopathological image based on multi-task learning model

Abstract

A method based on a multi-task learning model comprises the following steps: acquiring and processing histopathological image data of gastritis through a data preparation and preprocessing step; feature extraction is performed by using a self-supervised learning pre-trained model, and image blocks are coded into high-dimensional feature vectors; and constructing a multi-task learning model, learning feature representation through a full connection layer module and an attention layer module, and outputting a classification result of each task. And carrying out model training and optimization by using an optimizer, adding multitask loss through a loss function, and dynamically adjusting the model performance. The trained model can automatically detect and grade gastritis, atrophy, acute activity, intestinal metaplasia and other pathological indexes, outputs a standardized evaluation result, and assists in pathological diagnosis. According to the method, a multi-task deep learning framework based on self-supervised learning pre-training is constructed, a traditional single-task modeling mode is broken through, deep learning framework design is driven through pathological index association, and accuracy and clinical practicability of non-neoplastic disease assessment of the stomach are remarkably improved.

Description

Technical Field

[0001] The present invention relates to tissue pathological image processing and deep learning technologies, and particularly to a method for improving the evaluation accuracy of various indicators of non-neoplastic diseases of the stomach in tissue pathological images based on a multi-task learning model. Background Art

[0002] Gastric cancer is a common malignant tumor that seriously threatens human health. According to the Correa cascade theory, the occurrence of gastric cancer usually undergoes a multi-stage evolution process of "chronic gastritis → atrophic gastritis → intestinal metaplasia → dysplasia → carcinogenesis", and Helicobacter pylori infection is considered to be one of the main driving factors. Therefore, the accurate evaluation of precancerous lesions such as gastric mucosal atrophy and intestinal metaplasia has important clinical value for the prevention and intervention of early gastric cancer. Pathological diagnosis is the globally recognized gold standard for cancer diagnosis. Traditional pathological diagnosis relies on pathologists to use a microscope to magnify tissue sections step by step and judge the degree of lesions based on experience. However, this method has certain subjectivity, the diagnostic process is time-consuming, and the consistency among doctors is low. Although pathologists have tried to grade gastritis indicators since the Sydney system was updated in 1994, this process still lacks quantitative analysis methods, which affects the standardization and objectivity of pathological evaluation.

[0003] In recent years, the rise of digital pathology technology has provided new solutions for pathological diagnosis. The digital storage of whole slide images (WSIs) combined with artificial intelligence technology has made the automation of pathological image analysis possible. However, the ultra-high resolution of WSIs results in high pixel-level annotation costs. To solve this problem, the multiple instance learning (MIL) method has been widely applied to pathological image analysis. Although the MIL method effectively reduces the annotation cost, traditional MIL frameworks usually rely on feature extractors pre-trained on natural images, and these feature extractors are not fine-tuned for pathological images, resulting in limited performance of the model in pathological tasks. In recent years, the development of self-supervised learning (SSL) technology has significantly improved the ability to extract pathological-specific features. Researchers have enabled the model to more effectively learn the feature distribution of pathological images through self-supervised pre-training on large-scale unlabeled WSIs. In addition, although deep learning technology has made significant progress in the field of pathological image analysis, existing methods focus on the evaluation of single pathological indicators and lack joint modeling of multi-stage lesions in the Correa cascade theory.

[0004] Currently, the research on using deep learning technology for the detection of non-neoplastic disease indicators in the stomach is still in its infancy, and there are many challenges and problems to be solved. First, how to utilize the correlations among inflammation, acute activity, atrophy, and intestinal metaplasia in the evaluation of non-neoplastic diseases of the stomach to jointly model a multi-task learning framework for multiple indicators. Second, how to extract pathological features through a feature extractor pre-trained based on pathological images to enhance the prediction accuracy of the downstream tasks of the model. Third, how to visualize the detection results of the model to provide guidance and assistance for the daily detection of pathological experts.

[0005] Due to the lack of a unified standard for the diagnosis of non-neoplastic indicators of the stomach by pathological experts, it is often limited by the problems of strong subjectivity, complex operation, and long time consumption. In addition, existing deep learning methods focus on the analysis of single pathological indicators and fail to fully consider the biological relevance of multi-stage lesions in the Correa cascade theory. There is an urgent need for an efficient and accurate automated pathological analysis method to improve the accuracy and stability of the evaluation of non-neoplastic lesions of the stomach.

[0006] It should be noted that the information disclosed in the above background art section is only used for the understanding of the background of the present application, and thus may include information that does not constitute the prior art known to those of ordinary skill in the art. Summary of the Invention

[0007] The main purpose of the present invention is to overcome the defects existing in the above background art, and to provide a method for improving the evaluation accuracy of various indicators of non-neoplastic diseases of the stomach in tissue pathological images based on a multi-task learning model.

[0008] To achieve the above purpose, the present invention adopts the following technical solutions:

[0009] A method for improving the evaluation accuracy of non-neoplastic diseases in tissue pathological images based on a multi-task learning model, comprising the following steps:

[0010] Data preparation and preprocessing: Obtain tissue pathological image data of gastritis, extract the foreground of the tissue region, cut the tissue samples into standardized image patches, and perform data augmentation;

[0011] Feature extraction: Use a self-supervised learning pre-trained model to extract features from the image patches, and encode the input image patches into high-dimensional feature vectors;

[0012] Multi-task learning model construction: Input the extracted features into a fully connected layer for feature representation learning, set an attention layer module for each task, and output the classification results of each task through a classification layer;

[0013] Model Training and Optimization: Use an optimizer to train the model. During the training process, use a loss function to sum up the losses of multiple tasks, dynamically optimize the model performance, evaluate the model performance, and use the trained model to automatically detect and grade the pathological indicators of gastritis, atrophy, acute activity, and intestinal metaplasia, and output standardized evaluation results to assist pathological diagnosis.

[0014] Furthermore, the feature extraction includes the following steps:

[0015] Use a pre-trained Vision Transformer (ViT) model based on self-supervised learning to encode image patches and extract high-order pathological semantic features; sequentially input the features into multiple fully connected layers, and layer by layer learn deep feature representations through non-linear activation functions to form a unified feature space suitable for multi-task learning.

[0016] Furthermore, the construction of the multi-task learning model includes the following mechanisms:

[0017] Configure an independent attention calculation module for each pathological evaluation task. The module adopts a dual-path structure combining a shared weight matrix and task-specific weights, and dynamically allocates the contribution weights of different image regions in the classification task through adaptive attention scores to achieve cross-task feature interaction and knowledge transfer.

[0018] Furthermore, the specific output of the classification layer includes:

[0019] For the four tasks of gastritis grading, atrophy determination, acute activity grading, and intestinal metaplasia grading, respectively construct independent classifiers matching the number of task categories, map the shared features to probability spaces of different dimensions through task-specific fully connected layers, and use a normalization function to output the classification prediction results of each task.

[0020] Furthermore, in the multi-task learning model, use a gated attention mechanism to globally aggregate the local image patch features, use a gated attention mechanism to globally integrate the local image patch features, generate a representation vector of the entire pathological section through learnable weight mapping and non-linear transformation, and combine random dropout regularization and activation function optimization to suppress overfitting and enhance the model's focusing ability on key pathological regions.

[0021] Furthermore, the network structure of the multi-task learning model includes the following processing:

[0022] Sequentially input the image patch features obtained through feature extraction into two fully connected layers (Fc1 and Fc2), and each fully connected layer is followed by a non-linear activation function to extract deep pathological semantic representations;

[0023] The outputs of the fully connected layer (Fc2) are respectively input into two groups of shared attention fully connected layers (Attn-Fc1 and Attn-Fc2), and combined with the set of task-independent weight parameters to generate dynamic attention scores of image regions for each pathological assessment task (Attn-Task1 to Attn-Task4).

[0024] Based on the dynamic attention scores, the features of the whole pathological section are weighted and aggregated through the attention pooling layer (Attn.pool) to highlight the semantic contributions of key regions.

[0025] The aggregated global features are respectively input into the fully connected classification layers dedicated to each task (Fc-Task1 to Fc-Task4). The classification layers adapt to the dimensional differences in the categories of gastric inflammation grading, atrophy determination, acute activity grading, and intestinal metaplasia grading, and output the classification probabilities of each task through a normalization function.

[0026] Furthermore, the use of the loss function includes:[[]]END]]

[0027] The cross-entropy loss function is used to jointly optimize the classification prediction errors of multiple pathological assessment tasks. The total loss value is the weighted sum of the independent loss values of each task, and the parameters of the shared feature extraction layer and the task-specific classification layer are synchronously updated through the backpropagation algorithm.

[0028] Furthermore, the model training and optimization specifically include:[[]]END]]

[0029] The adaptive moment estimation optimizer is used to dynamically adjust the model parameter update step size, and combined with the weight decay strategy to suppress the model complexity.

[0030] Set the upper limit of the iterative training cycle, and monitor the change trend of the validation set loss through the early stopping mechanism. When the loss value does not decrease significantly in consecutive multiple training cycles, the training is terminated in advance to avoid overfitting.

[0031] Furthermore, the model training and optimization specifically include:[[]]END]]

[0032] A hierarchical cross-validation scheme is adopted to divide the training set and the validation set, monitor the change trend of the validation set loss. When the loss value does not decrease significantly in consecutive multiple training cycles, the training is terminated, and the comprehensive diagnostic efficacy of the model is evaluated through the receiver operating characteristic curve (ROC) and the area under the curve (AUC) index.

[0033] A computer program product includes a computer program, and when the computer program is executed by a processor, it implements the method for improving the accuracy of non-neoplastic disease assessment in histopathological images based on the multi-task learning model.

[0034] The present invention has the following beneficial effects:

[0035] The present invention proposes a method for improving the accuracy of non-neoplastic disease assessment in histopathological images based on a multi-task learning model. By constructing a multi-task deep learning framework pre-trained with self-supervised learning, it effectively solves the technical bottlenecks of strong subjectivity, low efficiency in traditional pathological diagnosis, and the difficulty of single-task models in capturing the relevance of multi-stage lesions, and significantly improves the accuracy and clinical practicability of non-neoplastic disease assessment of the stomach. The model pre-trained with self-supervision on pathological images is used for feature extraction, breaking through the limitations of traditional natural image transfer models, and capturing high-order features across tissue types and staining conditions through pathological-specific semantic encoding; an innovative cross-task attention interaction mechanism is designed to jointly model the pathophysiological relevance of gastritis, atrophy, acute activity, and intestinal metaplasia, and use the shared weight matrix and task-specific parameters to jointly optimize the feature representation, realizing knowledge transfer and collaborative discrimination among multiple indicators; further combined with gated attention pooling and regularization strategies, dynamically aggregate the global semantic information of the entire pathological section, enhancing the focusing ability on key lesion areas while suppressing overfitting. Experimental verification shows that in the classification tasks of four pathological indicators, the accuracy, AUC, and F1 score of this multi-task framework significantly exceed those of the single-task benchmark model, especially in the acute activity grading task with unbalanced samples, fully verifying the effectiveness of cross-task feature correlation modeling. This method not only realizes the automation and standardization of the pathological assessment process, but also provides high-precision and interpretable auxiliary decision-making support for the early warning of gastric precancerous lesions.

[0036] Other beneficial effects in the embodiments of the present invention will be further described below. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is the overall flowchart of the method for improving the accuracy of non-neoplastic disease assessment in histopathological images based on the multi-task learning model of the present invention.

[0038] Figure 2 It is the network framework diagram of the multi-task learning model in the embodiment of the present invention.

[0039] Figure 3 It is the TSNE dimensionality reduction visualization of the contrast feature extractor in the embodiment of the present invention.

[0040] Figure 4 It is the whole-slide-level attention heatmap in the embodiment of the present invention.

[0041] Figure 5 It is the receiver operating characteristic curve (ROC) of the four pathological assessment tasks (gastritis grading, atrophy determination, acute activity grading, intestinal metaplasia grading) in the single-task and multi-task frameworks in the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0042] The following provides a detailed description of the embodiments of the present invention. It should be emphasized that the following description is merely exemplary and not intended to limit the scope of the present invention and its applications.

[0043] In addition, the terms "first" and "second" are only used for descriptive purposes and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the embodiments of the present invention, "a plurality" means two or more unless otherwise specifically defined.

[0044] Referring to Figure 1 , an embodiment of the present invention provides a method for improving the accuracy of non-neoplastic disease assessment in histopathological images based on a multi-task learning model, including the following steps:

[0045] Data preparation and preprocessing: Obtain histopathological image data of gastritis (such as H&E stained pathological image data), extract the foreground of the tissue region, cut the tissue sample into standardized image patches, and perform data augmentation;

[0046] Feature extraction: Use a self-supervised learning pre-trained model to extract features from the image patches and encode the input image patches into high-dimensional feature vectors;

[0047] Multi-task learning model construction: Input the extracted features into a fully connected layer for feature representation learning, set an attention layer module for each task, and output the classification results of each task through a classification layer;

[0048] Model training and optimization: Use an optimizer to train the model. During the training process, use a loss function to sum up the losses of multiple tasks, dynamically optimize the model performance, evaluate the model performance, and use the trained model to automatically detect and grade the pathological indicators of gastritis, atrophy, acute activity, and intestinal metaplasia, and output standardized evaluation results to assist pathological diagnosis.

[0049] The method of the present invention constructs a multi-task deep learning framework based on self-supervised learning pre-training, breaks through the traditional single-task modeling paradigm, and significantly improves the accuracy and clinical practicability of non-neoplastic disease assessment of the stomach by driving the design of the deep learning framework through pathological index association.

[0050] In some embodiments, the feature extraction includes the following steps: Use a pre-trained vision Transformer (ViT) model based on self-supervised learning to encode the image patches and extract high-order pathological semantic features; Input the features into multiple fully connected layers in sequence, and layer by layer learn the deep feature representation through a non-linear activation function to form a unified feature space suitable for multi-task learning.

[0051] In some embodiments, the construction of the multi-task learning model includes the following mechanism: an independent attention calculation module is configured for each pathological assessment task, and the module adopts a dual-path structure combining a shared weight matrix and task-specific weights, and dynamically allocates the contribution weights of different image regions in the classification task through an adaptive attention score, so as to achieve cross-task feature interaction and knowledge transfer.

[0052] In some embodiments, the output of the classification layer specifically includes: for the four tasks of gastritis grading, atrophy determination, acute activity grading, and intestinal metaplasia grading, independent classifiers matching the number of task categories are respectively constructed, and the shared features are mapped to probability spaces of different dimensions through task-specific fully connected layers, and a normalization function is used to output the classification prediction results of each task.

[0053] In some embodiments, in the multi-task learning model, a gated attention mechanism is used to globally aggregate the local image patch features, and a gated attention mechanism is used to globally integrate the local image patch features, and a representation vector of the whole pathological section is generated through learnable weight mapping and non-linear transformation, and combined with random dropout regularization and activation function optimization, overfitting is suppressed and the focusing ability of the model on key pathological regions is enhanced.

[0054] As Figure 2 shown, in some embodiments, the network structure of the multi-task learning model includes the following processing: the image patch features obtained through feature extraction are sequentially input into two fully connected layers (Fc1 and Fc2), and each fully connected layer is followed by a non-linear activation function for extracting deep pathological semantic representations; the outputs of the fully connected layer (Fc2) are respectively input into two groups of shared attention fully connected layers (Attn-Fc1 and Attn-Fc2), and combined with a set of task-independent weight parameters, dynamic attention scores of image regions are generated for each pathological assessment task (Attn-Task1 to Attn-Task4); based on the dynamic attention scores, the features of the whole pathological section are weighted and aggregated through an attention pooling layer (Attn.pool) to highlight the semantic contributions of key regions; the aggregated global features are respectively input into the fully connected classification layers (Fc-Task1 to Fc-Task4) exclusive to each task, and the classification layer adapts the dimensions according to the category differences of gastritis grading, atrophy determination, acute activity grading, and intestinal metaplasia grading, and outputs the classification probabilities of each task through a normalization function. Through the above network structure design, the collaborative optimization of feature sharing and specific modeling among multiple tasks is realized, and the discrimination accuracy of the model for non-neoplastic lesions of the stomach is significantly improved; at the same time, the dynamic attention mechanism combined with hierarchical feature aggregation enhances the semantic focusing ability of the model on key pathological regions, providing high-robustness automated assessment support for clinical diagnosis.

[0055] In some embodiments, the use of the loss function includes: using a cross-entropy loss function to jointly optimize the classification prediction errors of multiple pathological evaluation tasks, where the total loss value is the weighted sum of the independent loss values of each task, and the parameters of the shared feature extraction layer and the task-specific classification layer are synchronously updated through the backpropagation algorithm.

[0056] In some embodiments, the model training and optimization specifically include: using an adaptive moment estimation optimizer to dynamically adjust the model parameter update step size, and combining a weight decay strategy to suppress the model complexity; setting an upper limit on the iterative training cycle, and monitoring the change trend of the validation set loss through an early stopping mechanism. When the loss value does not decrease significantly in consecutive multiple training cycles, the training is terminated in advance to avoid overfitting.

[0057] In some embodiments, the model training and optimization specifically include: adopting a hierarchical cross-validation scheme to divide the training set and the validation set, monitoring the change trend of the validation set loss, terminating the training when the loss value does not decrease significantly in consecutive multiple training cycles, and evaluating the comprehensive diagnostic efficacy of the model through the receiver operating characteristic curve (ROC) and the area under the curve (AUC) index.

[0058] The present invention constructs a multi-task evaluation system for gastric non-neoplastic diseases for the first time, breaking through the traditional single-task modeling paradigm. Through the design of a deep learning framework driven by the association of pathological indicators, a multi-task deep learning model for evaluating gastric non-neoplastic diseases is constructed. The present invention not only realizes the objective quantitative evaluation of pathological features such as gastritis, but also further models based on the potential associations between different pathological indicators. Experimental results show that this association modeling based on pathophysiological mechanisms significantly improves the generalization ability of the model, and verifies the advantages of the multi-task learning framework in the multi-classification experiment of four tasks. Compared with the single-task model, the multi-task framework has improved performance in the evaluation of the multi-classification task of 4 indicators, especially in complex tasks, which indicates that the cross-task knowledge transfer mechanism helps to capture the implicit feature associations between different pathological indicators and further enhances the discriminative ability of the model. At the same time, the multi-task model can better learn the deep representation of the model and better complete the prediction on complex tasks. The achievements of the present invention not only provide a reusable technical framework for the artificial intelligence-assisted diagnosis of digestive tract diseases, but also provide a new means for the early screening and warning of gastric precancerous lesions, including providing high-precision and interpretable auxiliary decision-making support, and have important clinical value.

[0059] The following further describes specific embodiments, algorithm examples and experimental verifications of the present invention.

[0060] A method for improving the evaluation accuracy of non-neoplastic diseases in tissue pathological images based on a multi-task learning model includes:

[0061] Step S1: Construct a rich H&E staining image dataset containing gastritis, atrophy, acute activity, and intestinal metaplasia indicators. Cut tiles from the whole-slide images (WSIs) of the dataset. This process may require manual participation to screen and correct the data. Finally, perform data preprocessing, including image enhancement, noise removal, color normalization, etc., to improve the quality and consistency of the images.

[0062] Step S2: Use the Virchow model based on DINOv2 self-supervised learning (Vision Transformer-Huge architecture, ViT-H / 14, with 632M parameters) for feature extraction. This model is trained on 1.5 million H&E stained pathological sections with mixed magnifications (5× / 10× / 20× / 40×), significantly improving the generalization ability across tissue types and staining conditions. After freezing the feature extractor, the input image patches are encoded into 2560-dimensional feature vectors to capture high-order pathological semantic information. Then, in the feature space, aggregate the information of all tissue regions in each slide by extending attention-based pooling to multiple tasks. Based on these tasks, the classification layer of the network outputs the final prediction for the entire WSI. Input the obtained features into two fully connected layers, and then pass through a multi-task learning network based on the gated attention mechanism to give prediction results for 4 tasks simultaneously.

[0063] Step S3: Use the labeled training dataset to train the multi-task learning model. Update the model's parameters through the backpropagation algorithm and gradient descent optimizer so that the model can fully learn the feature representation. Use an independent test dataset to evaluate the trained model, and calculate metrics such as the accuracy, recall rate, and F1 score of the model. Optimize the model's performance according to the evaluation results, and at the same time compare the results of each single-task learning.

[0064] More specifically, a method for improving the accuracy of non-neoplastic disease assessment in tissue pathological images based on a multi-task learning model includes the following specific steps:

[0065] Step S1-1: Obtain H&E pathological image data of gastritis. This batch of data contains 984 whole-slide images (WSIs).

[0066] Step S1-2: Use the Otsu algorithm based on the HSV threshold to achieve foreground extraction of tissue regions, effectively separating background noise and non-target regions. Subsequently, on the premise of maintaining the original anatomical structure features, cut the tissue samples under a 20-fold magnification field of view into non-overlapping standardized image patches of 256×256 pixels, and perform feature enhancement on the dataset to increase data diversity.

[0067] After feature extraction in step S2-1, the model first adds two fully connected layers, fc1 and fc2. After each fully connected layer, there is an activation layer, allowing the model to learn the deep feature representation of the pathological images, and then input the feature embedding of a patch.

[0068] In step S2-2 of multi-task learning, for each task t, there is an attention layer module respectively, and each layer has weight coefficients, which are shared among all tasks. At the same time, for each task, there is an independent set of weight coefficients. During training, an attention score a is assigned to each patch in different tasks respectively. k,t , a high attention score indicates high information content and importance in the classification task.

[0069] In step S2-3, for each different task, different classification layers are used to output the classification results. The first task is a 3-classification problem regarding mild, moderate, and severe gastritis. The second task is a 2-classification problem regarding the presence or absence of atrophy. The third task is a 4-classification problem regarding the absence, mild, moderate, and severe of acute activity. The fourth task is a 4-classification problem regarding the absence, mild, moderate, and severe of intestinal metaplasia. The loss function uses cross-entropy loss, and the loss of backpropagation is the sum of the losses of the 4 tasks.

[0070] In step S2-4 of the contrastive single task, the standard Gated Attention-based Multiple Instance Learning (GatedABMIL) architecture is used to aggregate patch-level features into WSI-level features. First, a fully connected layer is used to uniformly map the input feature dimension to 512, and the dimension of the hidden layer is modeled using 384. A dropout of 0.25 is used as a regularization method during the input mapping process. The relu activation function is used for non-linear activation in both the input mapping and the hidden layer mapping.

[0071] In step S3-1, the model training uses the Adaptive Moment Estimation optimizer (Adam), the initial learning rate is set to 0.001, and the weight decay coefficient is set to 1e-5 to suppress overfitting. The total number of training epochs is 100, and the model performance is dynamically optimized through the early stopping strategy (EarlyStopping): when the cross-entropy loss value no longer significantly decreases for 50 consecutive epochs (patience = 50), the training is terminated.

[0072] In step S3-2, the training data is input into the model, and forward propagation is performed through each layer of the model to obtain the prediction results. Calculate the loss based on the prediction results of the model and the true labels, calculate the gradient of the loss, and update the parameters of the model using the strategy of the backpropagation algorithm and the optimizer.

[0073] Step S3-3 is to ensure the statistical balance of data partitioning. The experiment adopts a stratified five-fold cross-validation scheme. In the external validation, the original dataset is divided into a training set and a test set at a ratio of 4:1, and then the training set is further divided into a training set and a validation set at a ratio of 7:1 internally, so as to ensure the consistency of the data distribution in each fold. In the evaluation of the experimental results, the receiver operating characteristic curve (ROC) and the area under the curve (AUC) are used to quantify the diagnostic efficacy of the model, and the accuracy (Accuracy), sensitivity (Sensitivity), and specificity (Specificity) are calculated simultaneously as comprehensive evaluation indicators. All statistical analyses are implemented based on the Python scikit-learn library. The performance of the multi-task model on each task is comprehensively evaluated through these indicators.

[0074] Further, the method specifically includes the following steps:

[0075] Step S1-1 obtains the H&E pathological image data of gastritis. This batch of data contains 984 whole slide images (WSIs).

[0076] Step S1-2 uses the Otsu algorithm based on the HSV threshold to implement the foreground extraction of the tissue region, effectively separating the background noise and non-target regions. Subsequently, on the premise of maintaining the original anatomical structure features, the tissue samples under a 20-fold magnified field of view are cut into non-overlapping standardized image patches of 256×256 pixels, and the dataset is subjected to feature enhancement to increase data diversity.

[0077] In step S2-1, after feature extraction, the model first adds two fully connected layers fc1 and fc2. The parameters of the two fully connected layers are W1∈R 2560×512 , b1∈R 512 and W1∈R 512×512 , b1∈R 512 . After each fully connected layer, there is an activation layer, allowing the model to learn the deep feature representation of the pathological image, and then input the feature embedding z k , z k ∈R 2560 .

[0078] h k = ReLU(W2(ReLU(W1z k +b1))+b2)

[0079] In step S2-2, in multi-task learning, for each task t, there is an attention layer module respectively, and each layer has weight coefficients W1, W2∈R 384×512 , shared by all tasks, and at the same time, for each task, there is an independent set of weight coefficients w a,t ∈R1×384 During training, an attention score a is assigned to each patch separately for different tasks. k,t A high attention score indicates high information content and importance in the classification task.

[0080]

[0081] In step S2-3, for each different task, different classification layers are used to output classification results.

[0082] p t = softmax(W cls,t v p,t + b cls,t )

[0083] where W cls,t refers to the classification layer parameters for different tasks, and b cls,t refers to the bias parameters for different tasks. The first task is a 3-classification problem regarding mild, moderate, and severe gastritis. The second task is a 2-classification problem regarding the presence or absence of atrophy. The third task is a 4-classification problem regarding the absence, mild, moderate, and severe acute activity. The fourth task is a 4-classification problem regarding the absence, mild, moderate, and severe intestinal metaplasia. Therefore, the classification layer parameters for these 4 specific tasks are W cls,1 ∈ R 3×512 , W cls,2 ∈ R 2×512 , W cls,3 ∈ R 4×512 and W cls,4 ∈ R 4×512 . The loss function uses cross-entropy loss, and the loss in backpropagation is the sum of the losses of the 4 tasks.

[0084]

[0085] where C t is the number of classes for the t-th task, y ic is the true label indicator, and p ic is the model prediction probability.

[0086] In step S2-4, in the compared single task, a standard Gated Attention-based Multiple Instance Learning (GatedABMIL) architecture is used to aggregate patch-level features into WSI-level features. First, a fully connected layer is used to uniformly map the input feature dimension to 512, and the dimension of the hidden layer is modeled using 384. A dropout of 0.25 is used as a regularization method during the input mapping process. The relu activation function is used for non-linear activation in both the input mapping and the hidden layer mapping.

[0087] In step S3-1, the model training uses the Adaptive Moment Estimation optimizer (Adam). The initial learning rate is set to 0.001, and the weight decay coefficient is set to 1e-5 to suppress overfitting. The total number of training epochs is 100. The model performance is dynamically optimized through the early stopping strategy (EarlyStopping): when the cross-entropy loss value no longer significantly decreases for 50 consecutive epochs (patience = 50), the training is terminated.

[0088] In step S3-2, the training data is input into the model, and forward propagation is performed through each layer of the model to obtain the prediction results. The loss is calculated based on the prediction results of the model and the true labels. By calculating the gradient of the loss, the parameters of the model are updated using the backpropagation algorithm and the optimizer strategy.

[0089] In step S3-3, to ensure the statistical balance of data partitioning, the experiment adopts a stratified five-fold cross-validation scheme. In the external validation, the original dataset is divided into a training set and a test set at a ratio of 4:1, and then the training set is further divided into a training set and a validation set at a ratio of 7:1 internally, so as to ensure the consistency of the data distribution in each fold. In the evaluation of the experimental results, the Receiver Operating Characteristic curve (ROC) and the Area Under the Curve (AUC) are used to quantify the diagnostic efficacy of the model, and the accuracy (Accuracy), sensitivity (Sensitivity), and specificity (Specificity) are calculated simultaneously as comprehensive evaluation indicators. All statistical analyses are implemented based on the Python scikit-learn library. The performance of the multi-task model on various tasks is comprehensively evaluated through these indicators.

[0090] The superiority of the present invention is discussed from quantitative and qualitative indicators. The quantitative indicators include ACC, AUC, and F1-Score, and these indicators can all be used to evaluate the model performance. The results are shown in Table 1 below.

[0091] Table 1

[0092]

[0093] The above table shows the performance of the Virchow feature extraction model for four multi-classification tasks (gastritis grading GC, atrophy determination GA, acute activity grading AC, and intestinal metaplasia grading IM) under single-task learning and multi-task learning frameworks in three evaluation metrics, including accuracy, AUC, and F1-score, and compares the index results of multi-task and single-task. All metrics were cross-validated 5-fold for each task, and the mean and standard deviation were reported. Single-task benchmark test: The performance of each task is relatively poor (GC: 72.46% ± 0.03, GA: 78.46% ± 0.02, AC: 60.16% ± 0.06, IM: 78.56% ± 0.07). Among them, the performance of the acute activity (AC) task is limited due to the imbalance of sample categories. In multi-task learning, compared with single-task learning, the accuracy of all tasks has been improved (GC: +1.52%, GA: +1.72%, AC: +6.09%, IM: +5.29%). Multi-task learning is significantly superior to single-task learning in terms of accuracy, especially in the GA and IM tasks, where the improvement in accuracy is the most obvious. In addition, multi-task learning also shows good performance in other metrics such as AUC and F1-score, indicating that multi-task learning improves the comprehensive ability of the model by sharing knowledge of different tasks.

[0094] Figure 3 TSNE dimensionality reduction visualization of the contrast feature extractor of the embodiment of the present invention. As Figure 3 can be seen from the comparison of the TSNE dimensionality reduction results of the four feature extractors Virchow, Conch, Prov, and Uni, Virchow has the highest discrimination between different categories and the lowest confusion rate between different categories. Figure 4 Full-slide level attention heat map of the embodiment of the present invention. As Figure 4 The heat map analysis shows that in the gastritis grade assessment task, the single-task model tends to focus on glandular structures or marginal morphological features, while the multi-task joint modeling shows a significant shift in attention to the stromal region, which is closely related to the fact that more attention is paid to inflammatory cells (such as lymphocytes and plasma cells) during clinical gastritis grading. Compared with the single-task learning model, the multi-task learning model has a more accurate attention range and a wider coverage area for atrophy, which is in line with the main concerns of atrophy in clinical practice - glandular reduction and fibrotic regions. The acute activity task mainly focuses on neutrophils, and the multi-task model not only has a wider attention range but also is more accurate in the attention to neutrophils. The analysis of intestinal metaplasia usually focuses on the presence of goblet cells in gastric glands. The single-task model can identify the feature of gastric glands but fails to accurately identify the goblet cells among them; while multi-task learning significantly improves the recognition accuracy of goblet cells.

[0095] As Figure 5As shown, by plotting the receiver operating characteristic curves (ROC) of four pathological assessment tasks (gastritis grading, atrophy determination, acute activity grading, and intestinal metaplasia grading) under single-task and multi-task frameworks, the performance differences of different classification strategies were systematically demonstrated. Each subplot uses a standardized coordinate system, with the false positive rate (FPR) on the horizontal axis and the true positive rate (TPR) on the vertical axis. The area under the curve (AUC) is marked in the form of a colored filled area. The results show that multi-task learning presents significant advantages, and the AUC values of all tasks are higher than those in the single-task learning framework, which is most significant in the performance of the acute activity grading task. This result indicates that multi-task learning achieves better classification performance of the model through knowledge sharing between tasks.

[0096] The present invention is developed based on the clinical pain points of gastric biopsy pathological images. Centering around four non-neoplastic pathological indicators of gastritis, atrophy, acute activity, and intestinal metaplasia, a multi-task deep learning model for evaluating non-neoplastic gastric diseases is constructed. The present invention not only realizes the objective quantitative evaluation of pathological features such as gastritis, but also further models based on the potential associations between different pathological indicators. The experimental results show that this association modeling based on pathophysiological mechanisms significantly improves the generalization ability of the model and verifies the advantages of the multi-task learning framework in the multi-classification experiments of four tasks. Compared with the single-task model, the multi-task framework has improved performance in the evaluation of the multi-classification task of 4 indicators, especially in complex tasks, which indicates that the cross-task knowledge transfer mechanism helps to capture the implicit feature associations between different pathological indicators and further enhances the discriminative ability of the model. At the same time, the multi-task model is more capable of learning the deep representation of the model and better completing the prediction on complex tasks.

[0097] The present invention constructs a multi-task evaluation system for non-neoplastic gastric diseases for the first time. Its innovation lies in breaking through the traditional single-task modeling paradigm and driving the design of the deep learning framework through pathological indicator associations. This achievement not only provides a reusable technical framework for the artificial intelligence-assisted diagnosis of digestive tract diseases, but also provides a new idea for the early warning of gastric precancerous lesions, with important application value and clinical transformation potential.

[0098] The present invention has the following beneficial effects:

[0099] 1. The weights of the Virchow model for self-supervised pre-training on a large number of pathological images are introduced to extract features from gastric pathological images, effectively solving the problem of poor prediction performance of the feature extractor migrated from natural images on pathological images, thus greatly improving the accuracy of the downstream tasks of the model;

[0100] 2. A multi-task learning framework is constructed to jointly model the pathological correlations of inflammation degree, acute activity, atrophy, and intestinal metaplasia. The accuracy of the multi-task model is stronger than that of each single-task model in the prediction of non-neoplastic indicators of the stomach;

[0101] 3. Once trained and optimized, the multi-task learning model can rapidly process large amounts of H&E image data for gastritis, enabling automated detection of gastritis, atrophy, acute activity, and intestinal metaplasia. This significantly improves the efficiency of pathological diagnosis, reduces the workload of physicians, and effectively assists them in diagnosis.

[0102] The present invention is designed based on the clinical pain points of gastric biopsy pathological images, and focuses on the four major non-tumor pathological indicators of gastritis, atrophy, acute activity and intestinal metaplasia to construct a multi-task deep learning model for the evaluation of non-tumorous gastric diseases. Through the introduction of artificial intelligence technology, not only the objective quantitative evaluation of pathological characteristics such as gastritis is achieved, but also modeling is further carried out based on the potential correlation between different pathological indicators. In addition, the present invention innovatively uses the Virchow model pre-trained by DINOv2 self-supervised learning as a feature extractor, which significantly improves the generalization ability of the model in pathological images.

[0103] In terms of clinical application value, the model significantly reduces subjective bias in pathological film reading through standardized output. In addition, in tests at primary medical institutions, pathologists only need to review cases determined to be abnormal by the model, which can shorten film reading time and improve diagnostic efficiency. The model's visual heat map further reveals the pathological areas that the model focuses on, which provides an explainable basis for clinical decision-making. This achievement not only provides a reusable technical framework for AI-assisted diagnosis of gastrointestinal diseases, but also enhances the consistency of gastric non-tumor index evaluation, effectively screening the risk and possibility of gastric precancerous lesions, and has important scientific research value and clinical translation potential.

[0104] The design and implementation method of a multi-task learning model based on feature extraction of a large pathological model is designed. A multi-task learning module is designed to jointly learn the feature representations of gastritis, atrophy, acute activity, and intestinal metaplasia, revealing their internal correlations, thereby improving the detection performance of the model on the four indicators and guiding the evaluation of non-tumor diseases.

[0105] An embodiment of the present invention further provides a storage medium for storing a computer program, which at least performs the above method when executed.

[0106] An embodiment of the present invention further provides a control device, comprising a processor and a storage medium for storing a computer program; wherein the processor is configured to execute at least the method described above when executing the computer program.

[0107] An embodiment of the present invention further provides a processor, which executes a computer program and at least performs the method described above.

[0108] The storage medium can be implemented by any type of non-volatile storage device, or a combination thereof. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a ferromagnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The storage medium described in the embodiments of the present invention is intended to include, but not limited to, these and any other suitable types of memories.

[0109] In several embodiments provided by the present invention, it should be understood that the disclosed systems and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed with each other can be through some interfaces. The indirect coupling or communication connection of devices or units can be electrical, mechanical, or other forms.

[0110] The units described above as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they can be located in one place, or distributed to multiple network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0111] In addition, in each embodiment of the present invention, the functional units can all be integrated in one processing unit, or each unit can be separately used as a unit, or two or more units can be integrated in one unit; the above integrated units can be implemented in the form of hardware, or in the form of a combination of hardware and software functional units.

[0112] Those of ordinary skill in the art will understand that all or part of the steps to implement the above method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps including those of the above method embodiments; and the aforementioned storage medium includes: various media that can store program codes, such as removable storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs.

[0113] Alternatively, if the above integrated units are implemented in the form of software function modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solutions of the embodiments of the present invention, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in the various embodiments of the present invention. And the aforementioned storage medium includes: various media that can store program codes, such as removable storage devices, ROM, RAM, magnetic disks, or optical discs.

[0114] The methods disclosed in the several method embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments.

[0115] The features disclosed in the several product embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new product embodiments.

[0116] The features disclosed in the several method or device embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0117] The above content is a further detailed description of the present invention in combination with specific preferred implementation manners. It cannot be determined that the specific implementation of the present invention is only limited to these descriptions. For those skilled in the technical field to which the present invention belongs, without departing from the concept of the present invention, several equivalent substitutions or obvious variations can be made, and as long as the performance or use is the same, they should all be regarded as falling within the protection scope of the present invention.

Claims

1. A method for improving the accuracy of non - neoplastic disease assessment in histopathological images based on a multi - task learning model, characterized in that, It includes the following steps: Data preparation and preprocessing: Obtain the histopathological image data of gastritis, extract the foreground of the tissue area, cut the tissue samples into standardized image patches, and perform data augmentation; Feature extraction: Use a self-supervised learning pre-trained model to extract features from the image patches, and encode the input image patches into high-dimensional feature vectors; Multi-task learning model construction: Input the extracted features into a fully connected layer for feature representation learning, set an attention layer module for each task, and output the classification results of each task through the classification layer; Model training and optimization: Use an optimizer to train the model. During the training process, use a loss function to sum up the losses of multiple tasks, dynamically optimize the model performance, evaluate the model performance, and use the trained model to automatically detect and grade the pathological indicators of gastritis, atrophy, acute activity, and intestinal metaplasia, and output standardized evaluation results to assist pathological diagnosis.

2. The method according to claim 1, characterized in that The feature extraction includes the following steps: Use a pre-trained Vision Transformer (ViT) model based on self-supervised learning to encode the image patches and extract high-order pathological semantic features; Input the features into multiple fully connected layers in sequence, and learn the deep feature representations layer by layer through a non-linear activation function to form a unified feature space suitable for multi-task learning.

3. The method according to claim 1, wherein The multi-task learning model construction includes the following mechanisms: Configure an independent attention calculation module for each pathological evaluation task. The module adopts a dual-path structure that combines a shared weight matrix and task-specific weights, and dynamically allocates the contribution weights of different image regions in the classification task through adaptive attention scores to achieve cross-task feature interaction and knowledge transfer.

4. The method according to claim 1, wherein The specific output of the classification layer includes: For the four tasks of gastritis grading, atrophy determination, acute activity grading, and intestinal metaplasia grading, construct independent classifiers that match the number of task categories, map the shared features to probability spaces of different dimensions through task-specific fully connected layers, and use a normalization function to output the classification prediction results of each task.

5. The method according to claim 1, wherein In the multi-task learning model, use a gated attention mechanism to globally aggregate the local image patch features, use a gated attention mechanism to globally integrate the local image patch features, generate a representation vector of the entire pathological section through learnable weight mapping and non-linear transformation, and combine random dropout regularization and activation function optimization to suppress overfitting and enhance the model's focusing ability on key pathological regions.

6. The method according to any one of claims 1 to 5, characterized in that, The network structure of the multi-task learning model includes the following processing: Input the image patch features obtained through feature extraction into two fully connected layers (Fc1 and Fc2) in sequence, and each fully connected layer is followed by a non-linear activation function to extract deep pathological semantic representations; The output of the fully connected layer (Fc2) is respectively input into two groups of shared attention fully connected layers (Attn-Fc1 and Attn-Fc2), and combined with a set of task-independent weight parameters to generate dynamic attention scores for the image regions of each pathological evaluation task (Attn-Task1 to Attn-Task4); Based on the dynamic attention scores, the features of the whole pathological section are weighted and aggregated through an attention pooling layer (Attn.pool) to highlight the semantic contributions of key regions; The aggregated global features are respectively input into the fully-connected classification layers dedicated to each task (Fc-Task1 to Fc-Task4). The classification layers adapt the dimensions according to the category differences of gastritis grading, atrophy determination, acute activity grading, and intestinal metaplasia grading, and output the classification probabilities of each task through a normalization function.

7. The method according to any one of claims 1 to 6, characterized in that, The use of the loss function includes: The cross-entropy loss function is used to jointly optimize the classification prediction errors of multiple pathological assessment tasks. The total loss value is the weighted sum of the independent loss values of each task, and the parameters of the shared feature extraction layer and the task-specific classification layer are synchronously updated through the backpropagation algorithm.

8. The method according to any one of claims 1 to 7, characterized in that, The model training and optimization specifically include: An adaptive moment estimation optimizer is used to dynamically adjust the model parameter update step size, and a weight decay strategy is combined to suppress the model complexity; Set the upper limit of the iterative training cycle, and monitor the change trend of the validation set loss through an early stopping mechanism. When the loss value does not decrease significantly in consecutive training cycles, the training is terminated in advance to avoid overfitting.

9. The method according to any one of claims 1 to 8, characterized in that The model training and optimization specifically include: A stratified cross-validation scheme is adopted to divide the training set and the validation set, monitor the change trend of the validation set loss, terminate the training when the loss value does not decrease significantly in consecutive training cycles, and evaluate the comprehensive diagnostic efficacy of the model through the receiver operating characteristic curve (ROC) and the area under the curve (AUC) indicators.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method for improving the accuracy of non-neoplastic disease assessment in tissue pathological images based on the multi-task learning model according to any one of claims 1 to 9.

Images