Animal Pathology Whole-Slide Image Analysis System Based on Weakly Supervised Learning and Graph Neural Networks

Abstract

This invention provides an animal pathology auxiliary diagnostic system based on weakly supervised learning and graph neural networks. The system is trained on large-scale unlabeled pathological slides (WSI) and a small number of region-labeled samples. First, the entire pathological image is tiled. Each image block generates a feature vector through a feature extraction network, and these vectors are then reconstructed into a two-dimensional feature tensor based on spatial coordinates, achieving a complete representation of the spatial structure. Subsequently, an attention-driven multi-instance learning (MIL) model is employed, relying only on slide-level labels or a very small number of region annotations to perform preliminary classification of the entire slide and automatically generate an attention heatmap, focusing on the key regions on which the model relies for discrimination. Building upon this, a graph neural network (GCN) further integrates tile features, attention weights, and (if applicable) image block-level annotation information to deeply mine the spatial relationships between tissue structures, achieving accurate detection of tumor regions, boundary refinement, and aggregation of key pathological features.

Description

Technical Field

[0001] This application belongs to the field of image analysis, specifically relating to an animal pathology whole-slice image analysis system based on weakly supervised learning and graph neural networks. Background Technology

[0002] With increased awareness of pet health and advancements in veterinary technology, the average lifespan of dogs and cats is gradually increasing. However, cancer has become one of the leading causes of death in dogs and cats. Currently, the veterinary pathology diagnostic system suffers from a severe imbalance between supply and demand, particularly in the pathology diagnostic stage, where the number of professional veterinary pathologists is far from sufficient. This results in long and inefficient pathology testing cycles, hindering the early detection and precise treatment of pet diseases.

[0003] In the existing veterinary pathology diagnostic process, from the veterinarian submitting the sample, to the pathology laboratory preparing the slides, then the pathologist reviewing the slides and making a diagnosis, before finally returning to the clinician, this process is complex and lengthy, typically taking several days to a week, which is insufficient to meet the needs of rapid clinical decision-making. Furthermore, the accuracy of pathological diagnosis highly depends on the individual experience and training level of the pathologist; subjective judgments among different experts can vary significantly, affecting diagnostic consistency. Finally, due to the complexity of clinical pathology reports, which must include basic animal information, medical history, sample description, diagnosis, pathological diagnostic opinion, subsequent treatment recommendations, and references, the reading of pathological images and the writing of reports often consume a significant amount of the pathologist's time, further limiting their ability to handle complex cases and conduct research.

[0004] In recent years, artificial intelligence (AI) and machine learning (ML) technologies have demonstrated significant value in the field of medical image analysis, driving the development of automated and intelligent assisted diagnosis. However, intelligent applications in veterinary pathology are still in their early stages, especially the automated analysis of whole-slide images (WSI), which faces numerous technical challenges. Existing deep learning-based medical image analysis methods are mainly focused on human medicine and rely heavily on large amounts of labeled data and fully supervised learning models. In veterinary pathology, however, the limited sample size and high labeling costs make it difficult to obtain sufficient high-quality training data, thus restricting the practical effectiveness and generalizability of fully supervised models. Furthermore, whole-slide images have extremely high resolution and massive data volumes (often tens of gigabytes), making it difficult for traditional convolutional neural networks (CNNs) and other methods to process them efficiently without sacrificing accuracy, while also incurring high computational and storage costs.

[0005] Existing intelligent image recognition technologies, especially in the field of whole-slice image analysis of animal pathology, have the following main drawbacks: Strong dependence on annotation: Existing methods rely heavily on a large amount of pixel-level and region-level fine-grained annotation data, resulting in high cost and low efficiency in obtaining model training data, making them unsuitable for veterinary pathology scenarios where annotations are scarce.

[0006] Limited generalization ability of models: Existing deep learning models are limited by the distribution of training data and have difficulty adapting to different species, different tissue types and diverse pathological manifestations, resulting in insufficient generalization ability.

[0007] High computational resource consumption: When processing ultra-high resolution WSI, traditional fully supervised methods often need to divide the image into a large number of small tiles for tile-by-tile analysis, which not only brings huge computational costs, but also makes it difficult to capture global spatial relationships, affecting diagnostic accuracy and efficiency.

[0008] Lack of modeling of global structure and spatial relationships: Existing methods such as convolutional neural networks are better at extracting local features, but they are difficult to effectively integrate and understand the spatial distribution and complex tissue structure relationships in the whole image, resulting in one-sided diagnostic results and difficulty in supporting complex case analysis and subtype discrimination.

[0009] Clinical application is immature: The current market lacks a mature and readily applicable intelligent analysis system for whole-slice veterinary pathology images, which makes it difficult to meet the urgent clinical needs for efficient, accurate and consistent diagnosis.

[0010] Therefore, there is an urgent need for an intelligent image analysis method that can effectively model the spatial structural relationships of the entire pathological slide under weak or minimal annotation conditions, while also ensuring high efficiency and accuracy, in order to improve the efficiency and quality of animal pathological diagnosis. Summary of the Invention

[0011] This invention provides an animal pathology auxiliary diagnostic system based on weakly supervised learning and graph neural networks, aiming to improve the efficiency of pet tumor diagnosis and pathology reporting, significantly shorten the pathology diagnosis cycle, and reduce the workload of pathologists. The system is embedded in an integrated cloud-based digital slide management system, realizing the digitization, intelligentization, and automation of the entire pathology diagnosis process.

[0012] The computer vision method of this invention adopts a multi-level feature extraction and fusion architecture, with weakly supervised learning and graph neural networks as the core, to achieve efficient and automated analysis of high-resolution whole-slice pathological images. It mainly includes the following core sub-modules: image preprocessing and feature extraction module (Module A), weakly supervised learning and preliminary classification module (Module B), graph neural network and hybrid supervised feature fusion module (Module C), and visualization and auxiliary decision output module (Module C).

[0013] The main functions implemented by this module include: Tumor interpretation and classification on slides: Using weakly supervised learning techniques, the AI ​​is trained on only unlabeled pathological slides, enabling it to classify the entire slide, such as determining whether a tumor is present.

[0014] Lesion area detection: Based on graph neural networks, the tumor area in the pathological section is accurately located and the invasive characteristics of the tumor, such as tumor size, resection margin, and necrotic area, are identified.

[0015] Specific implementation steps and procedures (1) Pathological image acquisition and uploading Veterinary clinics or pathology laboratories use digital slide scanners to perform high-resolution scans of tissue samples and upload the resulting whole-slide image data (WSI) to a cloud system.

[0016] (2) Computer vision AI for pathological analysis This system is trained on large-scale unlabeled pathological slides (WSI) and a small number of region-labeled samples. First, the entire pathological image is tiled. Each image block generates a feature vector through a feature extraction network, which is then reconstructed into a two-dimensional feature tensor based on spatial coordinates, achieving a complete representation of the spatial structure. Subsequently, an attention-driven multi-instance learning (MIL) model is employed, relying only on slide-level labels or a very small number of region annotations to perform preliminary classification of the entire slide and automatically generate an attention heatmap, focusing on the key regions upon which the model's discrimination depends.

[0017] Building upon this foundation, a graph neural network (GCN) is used to further integrate tile features, attention weights, and (if applicable) image block-level annotation information to deeply explore the spatial relationships between tissue structures, achieving accurate detection of tumor regions, boundary refinement, and aggregation of key pathological features. For lesion determination, the model outputs a tumor confidence score for the entire image, and the system automatically determines the presence of a tumor using a threshold of 0.5.

[0018] Finally, the system overlays the normalized attention heatmap output by GCN onto the original slices to achieve high-brightness visualization of the tumor region. Based on the heatmap distribution, the system can automatically identify tumor boundaries and calculate key indicators such as the maximum tumor diameter and resection status (distance between the tumor boundary and the resection margin), assisting pathology experts in generating standardized reports efficiently and accurately.

[0019] Beneficial effects This invention's AI-assisted pathology system addresses key challenges in the current field of veterinary pathology diagnosis, including a shortage of expert resources, lengthy diagnostic cycles, strong diagnostic subjectivity, significant repetitive work, and limited AI applications. It provides an innovative solution integrating intelligent image recognition. Its core technical features and advantages are reflected in the following aspects: The system uses a computer vision AI model to automatically analyze the entire pathological slide, quickly locate the tumor area and mark key lesion features, significantly reducing the time experts spend manually reviewing the slides and improving overall work efficiency.

[0020] We introduce a weakly supervised learning mechanism to support model training using a large amount of unlabeled historical slice data, reducing labeling costs and improving the usability of the algorithm in actual clinical practice.

[0021] It supports automatic quantification of pathological features (such as tumor radius and tumor boundary measurement), replacing manual calculations and improving the consistency and objectivity of analysis results. Attached Figure Description

[0022] Figure 1 The present invention illustrates the extraction of features from full-slice image blocks and patch images, as well as the acquisition of a preliminary tumor detection heatmap through multi-instance weakly supervised learning based on an attention mechanism. Figure 2 This invention demonstrates the fusion of a graph neural network with hybrid supervised features to further detect tumor regions. Detailed Implementation

[0023] The AI ​​pathological image analysis workflow of this invention enables automatic analysis and target detection of high-resolution whole-slice images, including the following core steps (see appendix for details). Figure 2 ): 1. Full-slice image preprocessing and feature extraction Because whole-slice pathological images have ultra-high resolution, direct deep learning computation is computationally expensive and requires a large amount of GPU memory. Therefore, this invention employs a block-patching image slicing method to preprocess the whole-slice images, specifically including: Full-slice image patching: Due to the ultra-high resolution of full-slice images (typically up to 1 billion pixels), traditional neural networks struggle to process them directly. This invention first divides the full-slice image into blocks, segmenting the large full-slice image into a series of small patch images. Each patch image is set to a size of 224×224 pixels, and each patch image block is processed independently to reduce computational complexity. Each small block records its physical spatial coordinates within the original image, ensuring spatial consistency during subsequent feature reconstruction. This step facilitates efficient parallel processing of large images, laying the foundation for feature extraction and spatial relationship modeling.

[0024] Pathological image feature extraction: Features of each patch image were extracted using the pathological image basic model (H-Optimus-0), generating compact feature vectors (1536×1 dimension). The feature vectors of all patches were precisely mapped to a two-dimensional feature tensor based on their original spatial coordinates, forming a structured full-patch feature grid. This feature tensor fully preserves the spatial distribution and adjacency relationships between patches, possessing global spatial representation capabilities, and is used for subsequent neural network analysis of the lesion region.

[0025] 2. Preliminary Analysis of Multi-Instance Weakly Supervised Learning Based on Attention Mechanism The system utilizes a Multiple Instance Learning (MIL) neural network framework, requiring only slice-level labels or a minimal number of region annotations to achieve preliminary classification of the entire slice (e.g., determining the presence of a tumor). During model training and inference, each patch is assigned an attention weight output by an attention sub-network, representing its contribution to the overall classification result. The features of all patch images are used to calculate a weighted average feature based on the attention weights. This feature integrates the image features of all patch images and is used to characterize the overall image features of the entire slice. During training, this overall feature vector is further input into a linear layer and outputs the detection result. This output result is compared with the true overall slice annotation (e.g., the tumor type of the slice) to calculate the loss, and the model completes one round of training by backpropagating the loss function gradient. Based on the attention weights of the patch images, the model adaptively generates a two-dimensional attention heatmap, intuitively reflecting the areas the model focuses on, providing a reference for subsequent lesion localization and interpretation. This step significantly reduces the dependence on pixel-level annotations and improves the algorithm's generalization ability.

[0026] 3. Graph Neural Network and Hybrid Supervised Feature Fusion for Tumor Region Detection To further enhance the understanding of the spatial relationships between tissue structure and lesions, the system uses the features of all small patches as nodes and constructs a spatial relationship graph based on spatial adjacency. A graph neural network is then used to process the spatial relationship graph of pathological slides to enhance the model's understanding of the overall slide structure information. The specific process is as follows: (1) Constructing the graph structure: A graph data structure is constructed based on the feature vectors of the patch images. Each patch image serves as a node, and edges are established between neighboring patch images. The input features of each node consist of three parts: The small feature vectors generated in the aforementioned feature extraction stage; Attention weights generated by the MIL model; If there is a region label, the corresponding small block is marked as 1 or 0; if there is no label, it is set to 0.5 by default, forming mixed supervision information.

[0027] (2) Graph Neural Network Inference: The graph neural network model analyzes the entire graph, stacks three graph convolutional layers, and then performs pooling to generate the overall classification result of the pathological slices (e.g., tumor / non-tumor). It calculates the loss between the model output and the overall annotation of the real slices, and completes one round of model training by backpropagating the gradient of the loss function. The output weights of the attention mechanism pooling layer are used to score the importance of different regions, and the lesion region is detected with the patch image as the smallest unit.

[0028] (3) Attention Mechanism Pooling Layer: A graph convolutional network combined with an attention mechanism pooling layer is used to perform multi-layer reasoning on the spatial structure graph. This structure can effectively integrate spatial and semantic information across the entire area, enabling accurate detection of lesion regions, boundary refinement, and aggregation of key pathological features. The hybrid supervision strategy significantly alleviates the bottleneck caused by the scarcity of high-quality annotations.

[0029] (4) Detection map output: The graph neural network finally outputs the normalized attention value (range 0 to 1) of each small block, generating a heatmap of the lesion area. The system automatically overlays the heatmap onto the original slice image, intuitively highlighting the suspected lesion area, and can automatically delineate the tumor outline according to the set threshold.

[0030] 4. Detection and quantitative analysis of lesion areas (1) Lesion area detection: The two-dimensional attention heatmap is processed morphologically to generate tumor detection, which is used for precise segmentation of the lesion area. The segmented image is displayed in the slide viewing user interface of this invention in the form of region annotation, guiding pathologists to quickly determine the location of the lesion in the whole slide image.

[0031] (2) Expert fine-tuning: During the training of the graph neural network, veterinary pathology experts are allowed to provide a small amount of manually labeled data for mixed supervised training, and fine-tuning of the graph neural network and the attention mechanism pooling layer is performed to improve the detection accuracy.

[0032] (3) Key pathological feature analysis: For the detected lesion areas, fine-grained pathological features are extracted by combining the target detection deep neural network, including: Quantitative calculation of tumor area: Based on the detected tumor map, the system calculates the area of ​​the tumor and further analyzes the tumor boundary, tumor radius, and tumor invasion depth.

[0033] Results visualization: Tumor region detection results and margin resection status detection results are superimposed on the original full-slice pathology image in the form of a semi-transparent heatmap for verification by pathology experts.

[0034] Performance verification The system of this invention has undergone thorough performance validation on a dataset of whole pathological slides collected in actual veterinary clinical scenarios. The dataset covers 21 major tumor types in dogs and cats, including mast cell tumors, pilomyeloma / basal cell tumors, breast cancer, mammary adenoma, soft tissue sarcoma, histiocytoma, lipoma, squamous cell carcinoma, pilomatoma, funnel-shaped cyst, anal gland adenoma, anal gland epithelioma, meibomian gland / sebaceous gland hyperplasia or adenoma, meibomian gland / sebaceous gland epithelioma, apocrine gland carcinoma, etc. in dogs, as well as mast cell tumors, pilomyeloma / apocrine gland adenoma, breast cancer, soft tissue sarcoma, especially lymphomas of the gastrointestinal type, lipomas, squamous cell carcinoma, and funnel-shaped cysts in cats, totaling 3870 high-resolution whole-slide pathological images.

[0035] During model evaluation, a 5-fold cross-validation scheme was employed to ensure the reliability and generalization ability of the experiments. The system automatically classifies whole-slice images, achieving an overall slice classification accuracy of 90.8% and a class-weighted average accuracy of 90.2%. The AUC (Area Under the Receiver Operating Characteristic) distribution for individual tumor types ranged from 0.912 to 0.987, with an average AUC of 0.953, demonstrating the broad adaptability and high recognition accuracy of the method for different tumor types.

[0036] For certain tumor types, such as canine mast cell tumors, 20 images with manually annotated regions by pathology experts were introduced during the training phase to support the hybrid supervision mode. The classification accuracy of the system for this subtype increased from 95.5% to 98.2%, demonstrating the significant enhancement of the model's discriminative power by the hybrid supervision mechanism.

[0037] In terms of inference efficiency, the system takes an average of 5.6 minutes to process, extract features, and infer AI models per full slice image on a server environment configured with an NVIDIA Tesla T4 16GB GPU, 4-core vCPU, and 32GB of memory. This meets the efficiency requirements of digital pathology slide reading in actual clinical and high-throughput batch analysis scenarios.

[0038] Experimental results show that the intelligent analysis system for whole animal pathology slides of the present invention has reached the international advanced level in terms of automatic slide classification, lesion identification accuracy and reasoning efficiency, and has large-scale application value in actual veterinary pathology diagnosis.

[0039] Innovation The core innovation of this invention is reflected in the following two aspects: I. Innovative Application of Weakly Supervised Learning in Whole-Slide Analysis of Animal Pathology Traditional animal pathology image analysis often relies on manual block-by-block annotation or fully supervised learning methods, which consume a large amount of annotation resources. In contrast, this invention, based on the Multiple Instance Learning (MIL) framework, can automatically identify and locate regions in high-resolution whole-slice images (WSI) with only slice-level labels or a very small number of region annotations.

[0040] This technical approach is the first to systematically introduce weakly supervised learning methods into the field of animal pathology, enabling large-scale, low-annotation-cost intelligent diagnosis. It effectively improves the model's adaptability and generalization ability to real clinical scenarios, and solves practical problems such as the scarcity of animal pathology data annotations, diverse categories, and high labor costs.

[0041] Especially in the long-tail field of animal pathology, this method greatly reduces the data dependency threshold and lays the foundation for the popularization of intelligent methods.

[0042] II. Original Technical Approach of Hybrid Supervision Mode Combining Graph Neural Networks and Attention Pooling Layers The more core innovation lies in the invention's unique hybrid supervision information fusion and spatial structure modeling scheme: (1) This system introduces a hybrid supervised learning mechanism (i.e., simultaneously using slice-level weak labels and a very small number of region-level labels) into the full slice automatic analysis process for the first time, effectively integrating the two types of information, providing multi-level and progressive supervision signals for model training, and greatly improving the model's discrimination ability and generalization performance in scenarios with few labels. (2) The system innovatively uses graph neural networks (GCN) for spatial reasoning of the entire pathological image, taking the features of each small image patch as nodes and constructing a global graph structure by combining spatial adjacency relationships. Through multi-layer graph convolution operations, it can fully capture and integrate the complex spatial relationships of tumors and tissue structures throughout the entire image, which not only improves the accuracy of lesion detection, but also greatly enhances the refinement of lesion boundaries and the aggregation of complex pathological features.

[0043] (3) The system further integrates a pooling layer based on an attention mechanism, dynamically focusing the model's attention on the key regions with the highest diagnostic value, suppressing noise interference, and significantly improving the accuracy and stability of the final detection and discrimination. This multi-module collaborative spatial semantic fusion scheme is not only the first of its kind in the field of animal pathology, but also a cutting-edge innovation in the entire computational pathology industry, solving the problems of information fragmentation and spatial structure modeling in ultra-high resolution image analysis. Beneficial effects Compared with the prior art, the present invention has the following significant advantages: Significantly reduces labeling dependency: Through weak supervision and hybrid supervision strategies, high-performance models can be trained with only a small amount of manual labeling, significantly saving data preparation and labeling costs.

[0044] Adapting to real-world scenarios for animal pathology data: The innovative method fully addresses issues such as scarce annotations and strong case heterogeneity in the field of animal pathology, making the algorithm more universal and valuable for wider application.

[0045] Significantly improves the accuracy and efficiency of lesion detection: Based on spatial modeling and attention pooling mechanism of graph neural network, it realizes global understanding and accurate identification of complex tissue structures and lesion areas, greatly improving diagnostic consistency and expert work efficiency.

[0046] Fully automated, integrated process: It realizes a fully intelligent closed loop of automatic classification at the whole slice level, lesion area detection, visualization and report generation, and promotes the intelligentization of animal pathology into the practical stage.

Claims

1. A pathological classification method based on weakly supervised learning and graph neural networks, characterized in that, include: The original whole slice image is divided into blocks to obtain patch image blocks of the original whole slice image: Features are extracted from each patch image block of the original full-slice image to obtain the feature vector of each patch image block; The feature vectors of each patch image block are input into the trained multi-instance learning neural network model, and the attention weights of each patch image block are output. The attention weights represent the contribution of the feature vectors of each patch image block to the classification result. Based on the attention weight of each patch image block, the feature vector of each patch image block is weighted and calculated to obtain the overall feature vector of the original full slice image. The overall feature vector integrates the features of each patch image block. The overall feature vector of the original whole slice image is input into the trained linear classifier, and the output is the first preliminary classification result of whether there are lesions and the type of lesions in the original whole slice image.

2. The method according to claim 1, characterized in that, Prior to pathological classification, a multi-instance learning neural network model and a linear classifier are trained as follows: The training sample full slice image is divided into blocks to obtain each patch image block of the training sample full slice image. The training sample full slice image has a slice-level label, which is used to characterize the lesion type of the training sample full slice image. Features are extracted from each patch image block of the full slice image of the training sample to obtain the feature vector of each patch image block; The feature vectors of each patch image block are input into the multi-instance learning neural network model to be trained, and the attention weights of each patch image block are output. Based on the attention weight of each patch image block, the feature vector of each patch image block is weighted and calculated to obtain the overall feature vector of the full slice image of the training sample. The overall feature vector of the full slice image of the training sample is input into the linear classifier to be trained, and the output is the first preliminary classification result of whether there is a lesion in the full slice image of the training sample; Loss is calculated based on the initial classification results and the slice-level labels of the full slice images of the training samples, and the training process is completed by backpropagating the gradient of the loss function.

3. The method according to claim 1, characterized in that, The method further includes: Based on the physical spatial coordinates of each patch image block in its original full-slice image, the feature vectors of each patch image block are mapped to a two-dimensional feature tensor to form a structured full-slice feature grid of multiple original full-slice images. The attention weights of each patch image block are superimposed onto the full-slice feature grid to obtain a first attention weight heatmap. The first attention weight heatmap is used to characterize the second preliminary classification result of the lesion region in the original slice image.

4. The method according to claim 1 or 3, characterized in that, The method further includes: The feature vectors of each patch image block and the attention weights of each patch image block output by the trained multi-instance learning neural network model are used as graph nodes and input into the trained graph neural network model to output a second attention weight heatmap. The second attention weight heatmap is used to characterize the final classification result of the lesion region in the original slice image.

5. The method according to claim 4, characterized in that, Before pathological classification, the graph neural network model is trained and obtained as follows: The feature vectors of each patch image block in the full slice image of the training sample and the attention weights of each patch image block output by the multi-instance learning neural network model at the end of training are used as graph nodes and input into the graph neural network model to be trained. The output is the final classification result of the lesion region in the full slice image of the training sample. The loss is calculated based on the final classification result and the slice-level labels of the full slice images of the training samples, and the training process is completed by backpropagating the gradient of the loss function.

6. The method according to claim 5, characterized in that, The at least some training sample full-slice images also have region-level labels, which are used to characterize the lesion regions in the patch image blocks of the training sample full-slice images and are input as graph nodes into the graph neural network model to be trained for training.

7. The method according to claim 5, characterized in that, The graph neural network model includes three graph convolutional layers and one attention pooling layer.

8. The method according to claim 6, characterized in that, The maximum 10% of the training sample full-slice images have region-level labels.

9. The method according to any one of claims 1-8, characterized in that, It also includes: determining at least one of the following based on the detected lesion area: the area of ​​the lesion region, the boundary of the lesion region, the radius of the lesion, and the depth of lesion infiltration.

10. The method according to claim 9, characterized in that, Also includes: The lesion area and margin status are overlaid on the original whole slice image in the form of a semi-transparent heat map and visualized.

11. A computer device, characterized in that, include: A processor and a memory, the memory storing machine-readable instructions executable by the processor, the processor executing the machine-readable instructions stored in the memory, wherein when the machine-readable instructions are executed by the processor, the processor performs the steps of the method as described in any one of claims 1 to 10.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a computer device, performs the steps of the method as described in any one of claims 1 to 10.

Images