Pathological image sicca syndrome automatic diagnosis method, device and equipment and storage medium

Abstract

The invention discloses a pathological image sjogren syndrome automatic diagnosis method, device and equipment and a storage medium, and relates to the technical field of medical data processing, and the method comprises the steps: carrying out the image preprocessing of an original digital pathological section image, and obtaining an optimized image block; performing cell-tissue collaborative segmentation on the optimized image blocks to obtain a cell entity set and a tissue entity set; the cell entity set and the tissue entity set are mapped into a heterogeneous graph structure, and the heterogeneous graph structure takes cell graph nodes and tissue graph nodes as vertexes and takes a spatial adjacency relation as edges; and performing sicca syndrome state judgment based on the heterogeneous graph structure. According to the method, the heterogeneous graph structure is constructed, the cells and the tissues are abstracted into the nodes, the spatial adjacency relation edges are established, modeling of the global relation between the cells and the cross-scale relation between the cells and the tissues is achieved, global topology perception is achieved, the diagnosis accuracy is improved, and an end-to-end intelligent analysis process from a pathological image to sicca syndrome diagnosis is formed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical data processing, and in particular to a pathological image Sjogren's syndrome automatic diagnosis method, device, equipment and storage medium. BACKGROUND

[0002] Sjogren's syndrome (SS) is a rare chronic autoimmune disease that mainly affects adult women, characterized by long-term inflammation and dysfunction of salivary and lacrimal glands. The disease is often associated with other diseases such as systemic lupus erythematosus, rheumatoid arthritis and kidney disease, which can increase mortality. Early and accurate diagnosis is crucial for improving patient outcomes.

[0003] However, current diagnostic methods rely on histopathological evaluation of labial gland tissue and deep learning methods based on convolutional neural networks (CNN), which have certain limitations, limiting their widespread application and effectiveness.

[0004] Therefore, how to improve the accuracy of Sjogren's syndrome diagnosis based on pathological images has become a problem to be solved. SUMMARY

[0005] The main purpose of the present application is to provide a pathological image Sjogren's syndrome automatic diagnosis method, device, equipment and storage medium, which aims to solve the technical problem of how to improve the accuracy of Sjogren's syndrome diagnosis based on pathological images.

[0006] To achieve the above-mentioned purpose, the present application provides a pathological image Sjogren's syndrome automatic diagnosis method, which comprises: performing image preprocessing on the original digital pathology section image to obtain an optimized image block; performing cell-tissue collaborative segmentation on the optimized image block to obtain a cell entity set and a tissue entity set; mapping the cell entity set and the tissue entity set into a heterogeneous graph structure, the heterogeneous graph structure taking cell graph nodes and tissue graph nodes as vertices and spatial adjacency relationships as edges; determining the Sjogren's syndrome state based on the heterogeneous graph structure.

[0007] In an embodiment, the cell entity set includes single cell instances and cell multi-modal feature vectors corresponding to the single cell instances, and the tissue entity set includes tissue region instances and tissue multi-modal feature vectors corresponding to the tissue region instances; the step of performing cell-tissue collaborative segmentation on the optimized image block to obtain a cell entity set and a tissue entity set comprises: obtaining a cell instance mask by performing cell contour prediction on the optimized image block through a preset instance segmentation model, and extracting single cell instances from the cell instance mask; performing feature fusion on cell geometric features, color texture features, and spatial position features extracted for each single cell instance to obtain a cell multi-modal feature vector; obtaining a tissue region instance by performing tissue region prediction on the optimized image block through a general segmentation model, and obtaining a tissue region mask corresponding to the tissue region instance; performing feature fusion on tissue geometric features, staining texture features, and position context features extracted for each tissue region instance to obtain a tissue multi-modal feature vector.

[0008] In an embodiment, the step of mapping the cell entity set and the tissue entity set into a heterogeneous graph structure comprises: taking each single cell instance as a cell graph node, and establishing a cell-cell edge set between the cell graph nodes that meet preset interval requirements; taking each tissue region instance as a tissue graph node, and establishing a cell-tissue edge set between the cell graph nodes and the tissue graph nodes that meet preset coverage requirements; integrating the cell-cell edge set and the cell-tissue edge set to generate a heterogeneous graph structure, wherein the nodes of the heterogeneous graph structure carry corresponding cell multi-modal feature vectors or tissue multi-modal feature vectors.

[0009] In an embodiment, the step of determining the Sjogren's syndrome state based on the heterogeneous graph structure comprises: obtaining an optimized cell feature vector by optimizing the cell multi-modal feature vector in the heterogeneous graph structure through a self-supervised graph attention network; the self-supervised graph attention network is obtained through self-supervised pre-training by node masking reconstruction and contrast learning with structure disturbance; performing immune cell classification according to the optimized cell feature vector to obtain a plurality of immune cell graph nodes; performing preset clustering analysis based on the plurality of immune cell graph nodes to output a Sjogren's syndrome state determination result.

[0010] In an embodiment, the step of performing preset clustering analysis based on the plurality of immune cell graph nodes to output a Sjogren's syndrome state determination result comprises: constructing an immune cell graph node subgraph based on the plurality of immune cell graph nodes; running a preset community detection algorithm on the immune cell graph node subgraph to perform clustering to obtain an immune cell cluster; Output a Sjogren's syndrome state determination result according to the spatial distribution information of the immune cell clusters and the number of immune cells in the immune cell clusters.

[0011] In an embodiment, the step of performing image preprocessing on the original digital pathology slice image to obtain an optimized image block comprises: performing multi-scale tissue region extraction on the original digital pathology slice image to obtain a target tissue mask; dividing the original digital pathology slice image into regions based on the target tissue mask to obtain an initial image block; performing data enhancement processing on the initial image block to obtain an optimized image block.

[0012] In an embodiment, the step of performing multi-scale tissue region extraction on the original digital pathology slice image to obtain a target tissue mask comprises: performing down-sampling on the original digital pathology slice image to obtain a low-resolution optimized digital pathology slice image; performing coarse positioning of tissue regions on the optimized digital pathology slice image to obtain an initial tissue mask; mapping the initial tissue mask back to the coordinate system of the original digital pathology slice image to obtain a target tissue mask.

[0013] In addition, to achieve the above-mentioned purpose, the present application also proposes a pathological image Sjogren's syndrome automatic diagnosis device, the pathological image Sjogren's syndrome automatic diagnosis device comprises: an image preprocessing module configured to perform image preprocessing on an original digital pathology slice image to obtain an optimized image block; an image segmentation module configured to perform cell-tissue collaborative segmentation on the optimized image block to obtain a cell entity set and a tissue entity set; a graph learning module configured to map the cell entity set and the tissue entity set into a heterogeneous graph structure, wherein the heterogeneous graph structure has cell graph nodes and tissue graph nodes as vertices and spatial adjacency relationships as edges; an identification module configured to determine a Sjogren's syndrome state based on the heterogeneous graph structure.

[0014] In addition, to achieve the above-mentioned purpose, the present application also proposes a pathological image Sjogren's syndrome automatic diagnosis device, the device comprising a memory, a processor, and a pathological image Sjogren's syndrome automatic diagnosis program stored in the memory and executable on the processor, wherein the pathological image Sjogren's syndrome automatic diagnosis program is configured to implement the steps of the pathological image Sjogren's syndrome automatic diagnosis method mentioned above.

[0015] In addition, to achieve the above-mentioned purpose, the application further provides a storage medium, wherein the storage medium stores a pathological image xerotic syndrome automatic diagnosis program, and the pathological image xerotic syndrome automatic diagnosis program realizes the steps of the pathological image xerotic syndrome automatic diagnosis method mentioned above when executed by a processor.

[0016] The application provides a pathological image xerotic syndrome automatic diagnosis method, device, equipment and storage medium. The method comprises the following steps: performing image preprocessing on an original digital pathology section image to obtain an optimized image block; performing cell-tissue collaborative segmentation on the optimized image block to obtain a cell entity set and a tissue entity set; mapping the cell entity set and the tissue entity set into a heterogeneous graph structure, wherein the heterogeneous graph structure takes cell graph nodes and tissue graph nodes as vertices and takes spatial adjacency relationships as edges; and performing xerotic syndrome state determination based on the heterogeneous graph structure. The application provides a more efficient, accurate and strongly explanatory digital pathology image automatic diagnosis solution. By constructing a heterogeneous graph structure, cells and tissues are abstracted into nodes and spatial adjacency relationship edges are established, the modeling of global relationships between cells and cross-scale relationships between cells and tissues is realized, the complex interactions between cells are effectively captured, and the local field of view limitation problem of CNN is solved. Therefore, the application can realize global topology perception by establishing global relationships between cells, improve diagnosis accuracy, and is particularly suitable for early detection and diagnosis of complex diseases such as xerotic syndrome, and forms an end-to-end intelligent analysis process from pathological images to xerotic syndrome diagnosis. BRIEF DESCRIPTION OF DRAWINGS

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and serve to explain the principles of the application together with the specification.

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, for those skilled in the art, other drawings can also be obtained based on these drawings without creative labor.

[0019] Figure 1 The flowchart of the first embodiment of the pathological image xerotic syndrome automatic diagnosis method of the application; Figure 2 The flowchart of the second embodiment of the pathological image xerotic syndrome automatic diagnosis method of the application; Figure 3 The cell segmentation schematic diagram of the second embodiment of the pathological image xerotic syndrome automatic diagnosis method of the application; Figure 4 The tissue segmentation schematic diagram of the second embodiment of the pathological image xerotic syndrome automatic diagnosis method of the application; Figure 5 Fig. 2 is a schematic diagram of a dry syndrome state determination process of a second embodiment of the dry syndrome automatic diagnosis method of pathological images according to the present application; Figure 6 Fig. 3 is a schematic diagram of a process of the dry syndrome automatic diagnosis method of pathological images according to the present application; Figure 7 Fig. 4 is a schematic diagram of a module structure of a dry syndrome automatic diagnosis device of pathological images according to an embodiment of the present application; Figure 8 Fig. 5 is a schematic diagram of a device structure of a hardware running environment involved in the dry syndrome automatic diagnosis method of pathological images according to an embodiment of the present application.

[0020] The object implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0021] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application, and are not used to limit the present application.

[0022] In order to better understand the technical solutions of the present application, the following will be described in detail in combination with the drawings of the specification and specific embodiments.

[0023] The main solution of the present application is: performing image preprocessing on the original digital pathological section image to obtain an optimized image block; performing cell-tissue collaborative segmentation on the optimized image block to obtain a cell entity set and a tissue entity set; mapping the cell entity set and the tissue entity set into a heterogeneous graph structure, the heterogeneous graph structure taking cell graph nodes and tissue graph nodes as vertices and taking spatial adjacency relationship as edges; and determining the dry syndrome state based on the heterogeneous graph structure.

[0024] At present, the analysis of dry syndrome based on pathological images mainly relies on deep learning methods based on convolutional neural network (CNN). The image classification network represented by ResNet-50 is usually used for the process of patching, feature extraction, and re-aggregation classification of WSI (Whole Slide Image, whole slide digital pathological section image) image. However, the convolution operation of CNN has a limited receptive field, which means that it mainly focuses on local features and is difficult to capture the global relationship between cells. For pathological image analysis, it is crucial to understand the interaction between cells and their distribution in the whole tissue.

[0025] Furthermore, existing methods often neglect the true spatial topological relationships between cells. For example, in pathological images, different types of cells mix together to form specific functional structures or clusters, such as immune cell clusters. Traditional CNN models struggle to accurately capture this complex structural information because they primarily rely on fixed-size convolutional kernels to extract features. This leads to incomplete understanding of disease characteristics and an increased risk of misdiagnosis.

[0026] In summary, current methods for analyzing Sjögren's syndrome based on pathological images face numerous challenges. These issues limit the effectiveness of existing methods in practical clinical applications, and it is necessary to overcome these shortcomings and improve diagnostic efficiency and accuracy.

[0027] Therefore, this application provides a more efficient, accurate, and highly interpretable automated diagnostic solution for digital pathological images. By constructing a heterogeneous graph structure, cells and tissues are abstracted as nodes and spatial adjacency edges are established to model global relationships between cells and cross-scale relationships between cells and tissues. This effectively captures complex interactions between cells and solves the problem of limited local receptive field in CNNs. Thus, this application overcomes the limitations of local receptive fields in CNNs, achieving global topological perception by establishing global relationships between cells, thereby improving diagnostic accuracy. It is particularly suitable for the early detection and diagnosis of complex diseases such as Sjögren's syndrome, forming an end-to-end intelligent analysis process from pathological images to Sjögren's syndrome diagnosis.

[0028] It should be noted that the executing entity in this embodiment can be an automatic diagnostic system for Sjögren's syndrome in pathological images, or a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an automatic diagnostic device for Sjögren's syndrome in pathological images capable of performing the above functions. This embodiment does not specifically limit this. The following uses an automatic diagnostic device for Sjögren's syndrome in pathological images (hereinafter referred to as the diagnostic device) as the executing entity to describe this embodiment and the following embodiments.

[0029] Based on this, embodiments of this application provide an automatic diagnostic method for Sjögren's syndrome in pathological images, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the automatic diagnosis method for pathological images of Sjögren's syndrome according to this application.

[0030] In this embodiment, the automatic diagnosis method for Sjögren's syndrome in pathological images includes steps S10 to S40: Step S10: Perform image preprocessing on the original digital pathological slide image to obtain optimized image blocks; It is important to understand that the above original digital pathological slide image Iorig∈R H × W×C (where H and W represent the height and width of the image, respectively, and C represents the number of channels (e.g., RGB or H&E staining)) can be a WSI image of oral labial gland tissue, used for the diagnostic analysis of Sjögren's syndrome (SS). It is a high-resolution digital pathological image that can completely display the details of the entire oral labial gland tissue section, with a resolution typically reaching 100,000 × 100,000 pixels. Therefore, diagnostic equipment needs to perform image preprocessing on the original digital pathological section image to reduce computational burden, remove invalid information, and improve the accuracy of subsequent processing, in order to obtain image fragments of uniform size that are focused on the diagnostically relevant areas, i.e., the optimized image blocks mentioned above. In this embodiment, a fixed size of 512 × 512 pixels can be used to facilitate subsequent cell-tissue collaborative segmentation operations.

[0031] It is understood that in this embodiment, image preprocessing may include steps such as multi-scale tissue region extraction, region segmentation, and data augmentation. In a feasible implementation, step S10 may include steps A1-A3: Step A1: Extract the target tissue mask by multi-scale tissue region from the original digital pathological slide image; It is easy to understand that this embodiment can filter out the effective regions containing tissue from the Iorig image through multi-scale processing of "downsampling-coarse localization-coordinate mapping" and eliminate the interference of background (such as blank areas of the slide). The core is to obtain a binary matrix (pixel value of 1 represents tissue region and 0 represents background) that marks the pixel positions of all tissue regions in the original WSI image, i.e., the target tissue mask mentioned above. It serves as the basis for subsequent image block division, ensuring that only effective tissue regions are processed.

[0032] Therefore, in one feasible implementation, step A1 may include steps A11 to A13: Step A11: Downsample the original digital pathology slide image to obtain a low-resolution optimized digital pathology slide image; Step A12: Perform coarse localization of tissue regions on the optimized digital pathological slide image to obtain an initial tissue mask; Step A13: Map the initial tissue mask back to the coordinate system of the original digital pathological slide image to obtain the target tissue mask.

[0033] Understandably, this embodiment can perform resolution reduction processing on the original digital pathological slide image Iorig, that is, the above-mentioned downsampling, in order to reduce the number of pixels by reducing the image size. The core is to reduce the computational burden while preserving the tissue contour information. In this embodiment, bilinear interpolation can be preferred to implement downsampling in order to better preserve the tissue edge contour and avoid jagged distortion.

[0034] The downsampled, low-resolution optimized digital pathology slide image can be represented as Ilowres, with a resolution s times that of the original digital pathology slide image, where s∈(0,1). It preserves the macroscopic contours of the tissue and can be used for efficient coarse localization of tissue regions, avoiding the computational burden of directly processing the original high-resolution image. The scaling factor s can be adaptively set according to the size of the input original digital pathology slide image Iorig; the specific value is not limited in this embodiment.

[0035] Therefore, this embodiment can identify and label tissue regions on Ilowres using a pre-trained SAM2 (Segment Anything Model 2) model, i.e., perform the aforementioned coarse localization of tissue regions, and output a preliminary segmented binary mask, i.e., the aforementioned initial tissue mask, which serves as an "intermediate mask" subsequently mapped to the original image coordinate system. This allows for the rapid selection of effective tissue regions, labeling tissue and background regions in low-resolution images, and providing a foundation for the subsequent accurate mask generation of the original image. The formula for coarse localization of tissue regions is as follows: T_init=SAM2(Ilowres) (1) Where H′ and W′ are the height and width of the low-resolution image, and T_init is the binary mask matrix, i.e. the initial organization mask mentioned above.

[0036] It should be noted that this implementation can convert the pixel positions of the initial tissue mask into the pixel positions of the original WSI image's Iorig based on a pre-established correspondence between the low-resolution image and the pixel coordinates of the original WSI image (x_orig=x_low / s, y_orig=y_low / s, where x_orig and y_orig are the Iorig pixel coordinates, and x_low / s and y_low / s are the Ilowres pixel coordinates). This yields a binary matrix mapped to the original WSI coordinate system, i.e., the aforementioned target tissue mask T_target (H×W), accurately marking the pixel positions of all tissue regions in the original image and ensuring the accuracy of tissue region localization. During this process, the target tissue mask can be further refined by querying whether the boundary pixels belong to specific staining pixels (e.g., in H&E staining, the cell nucleus is blue and the cytoplasm is pink), improving the pixel-level accuracy of the target tissue mask.

[0037] In this embodiment, the original WSI resolution is reduced to less than 10,000 pixels by downsampling, which significantly shortens the tissue localization time of the SAM2 model compared to existing solutions and greatly improves computational efficiency, while preserving the macroscopic contour of the tissue. At the same time, the accurate target tissue mask can increase the effective tissue ratio of subsequent initial image blocks, reduce the interference of invalid image blocks on cell segmentation and graph modeling, and indirectly improve the final diagnostic accuracy.

[0038] Step A2: Divide the original digital pathological slide image into regions based on the target tissue mask to obtain initial image blocks; It is easy to understand that the diagnostic device can further segment the original WSI into fixed-size initial image blocks based on the target tissue mask, using a sliding window or density clustering method, i.e., perform the aforementioned region division, to solve the computational burden problem caused by the excessively high resolution of the original WSI. In this embodiment, the fixed-size image segments obtained after region division, which only contain tissue regions, i.e., the size of the aforementioned initial image blocks, can be 512×512 pixels.

[0039] For example, in this embodiment, the partitioning parameters can be set first to determine the initial image block size s_p (in this embodiment, s_p = 512 × 512 pixels), and the sliding window step size s_step can be set (to avoid the loss of tissue edge information, s_step can be set to 1 / 2 of s_p, i.e., 256 pixels); then, in this embodiment, based on the tissue distribution and with the target tissue mask T_target as a reference, the window of s_p × s_p can be slid with a step size of s_step, starting from the upper left corner (0,0) of the original WSI image I_orig. Simultaneously, for each window, determine the percentage of organization pixels with a value of 1 in T_target within the window: if the percentage is ≥50% (excluding windows with excessively high background percentage), then crop the original image region corresponding to that window into an initial image block P_init; finally, for the original WSI edge region, if the remaining region is less than s_p×s_p pixels, then use zero padding to supplement to s_p×s_p pixels to ensure that all initial image blocks have a uniform size; finally, N initial image blocks {P_init_1,P_init_2,...,P_init_N} are obtained.

[0040] Step A3: Perform data augmentation processing on the initial image block to obtain an optimized image block.

[0041] Understandably, diagnostic equipment can perform data augmentation through operations such as color normalization, geometric transformation, and noise injection to expand data diversity, eliminate the influence of color differences and image acquisition noise, and improve the generalization ability of subsequent models. After processing, an optimized image block Pi_final∈Rsp×sp×C (sp is the image block size) with color standardization, pose diversity, and noise resistance is obtained.

[0042] In this embodiment, by performing staged processing on high-resolution images, the computational complexity is significantly reduced, the overall system response speed is improved, and only the tissue region is retained for subsequent processing, avoiding interference from invalid background information and improving the model's focus and accuracy. This process reduces the computational load for tissue localization through multi-scale downsampling (e.g., 10x downsampling), while preserving the tissue details of the original image through coordinate mapping. Subsequent processing only involves segmenting the tissue region, improving overall computational efficiency without losing crucial pathological details, thus reducing computational complexity and balancing efficiency and detail. By using a target tissue mask, the system ensures that only regions with a tissue content of ≥50% are segmented, significantly reducing the proportion of invalid background blocks, minimizing computational resource waste, and preventing background interference with cell segmentation. This filters out invalid background and focuses on the diagnostic region. Furthermore, optimized image patches of uniform size provide standardized input for subsequent Stardist cell segmentation and SAM2 tissue segmentation, avoiding model adaptation issues caused by inconsistent image sizes and improving the stability of the entire diagnostic process. Additionally, using the general-purpose SAM2 model for tissue segmentation eliminates the need for retraining for specific staining patterns, demonstrating good compatibility and scalability.

[0043] Step S20: Perform cell-tissue collaborative segmentation on the optimized image block to obtain a set of cell entities and a set of tissue entities; Step S30: Map the set of cell entities and the set of tissue entities into a heterogeneous graph structure, wherein the heterogeneous graph structure has cell graph nodes and tissue graph nodes as vertices and spatial adjacency relationships as edges; Step S40: Determine the state of Sjögren's syndrome based on the heteromorphic graph structure.

[0044] It is important to understand that this embodiment can simultaneously segment cells and tissues in the optimized image patch, i.e., perform the aforementioned cell-tissue collaborative segmentation. During this process, different segmentation models can be used to achieve cell instance segmentation and tissue region segmentation respectively, ultimately resulting in a set of cell entities and a set of tissue entities. The diagnostic device can further convert this into a graph structure containing different types of nodes (cell graph nodes and tissue graph nodes) and edges (edges established based on spatial adjacency relationships), i.e., the aforementioned heterogeneous graph structure. This allows for the simultaneous modeling of spatial adjacency relationships between cells and between cells and tissues, achieving a unified expression of microscopic individual cells and macroscopic tissue background.

[0045] Finally, the diagnostic equipment can determine whether the pathological sample corresponding to the original digital pathological slide image is positive for Sjögren's syndrome based on the heterogeneous graph structure, through steps such as graph neural network feature optimization, classification of immune cells, and cluster analysis, that is, to determine the status of Sjögren's syndrome.

[0046] Therefore, existing pathological image analysis methods based on convolutional neural networks (CNNs) suffer from limited local receptive fields (CNNs have limited receptive fields, making it difficult to capture global relationships between cells), leading to low diagnostic accuracy for Sjögren's syndrome. This embodiment addresses this by constructing a heterogeneous graph structure, abstracting cells and tissues as nodes and establishing spatial adjacency edges. This enables the modeling of global relationships between cells and cross-scale relationships between cells and tissues, effectively capturing complex interactions between cells and overcoming the limitations of local receptive fields in CNNs. Thus, this embodiment overcomes the limitations of local receptive fields in CNNs, achieving global topological perception by establishing global relationships between cells, thereby improving diagnostic accuracy.

[0047] Meanwhile, this embodiment proposes a tissue segmentation and cell recognition technology with adaptive resolution for pathological images. By adaptively adjusting the resolution and dividing the images into sub-images, larger pathological images are processed into a size that is convenient for model processing without losing details. On this basis, tissue segmentation based on segmentation models and cell extraction based on polygon recognition are used to identify cells and tissues, thereby reducing computational complexity and balancing efficiency and detail.

[0048] This embodiment provides an automatic diagnostic method for Sjögren's syndrome using pathological images. The method includes: downsampling the original digital pathological slide image to obtain a low-resolution optimized digital pathological slide image; coarsely locating tissue regions in the optimized digital pathological slide image to obtain an initial tissue mask; mapping the initial tissue mask back to the coordinate system of the original digital pathological slide image to obtain a target tissue mask; dividing the original digital pathological slide image into regions based on the target tissue mask to obtain initial image blocks; performing data augmentation on the initial image blocks to obtain optimized image blocks; performing cell-tissue co-segmentation on the optimized image blocks to obtain a set of cell entities and a set of tissue entities; mapping the set of cell entities and the set of tissue entities into a heterogeneous graph structure, where the heterogeneous graph structure has cell graph nodes and tissue graph nodes as vertices and spatial adjacency relationships as edges; and determining the state of Sjögren's syndrome based on the heterogeneous graph structure. Therefore, existing pathological image analysis methods based on convolutional neural networks (CNNs) suffer from limited local receptive fields (CNNs have limited receptive fields, making it difficult to capture global relationships between cells), leading to low diagnostic accuracy for Sjögren's syndrome. This embodiment addresses this by constructing a heterogeneous graph structure, abstracting cells and tissues as nodes and establishing spatial adjacency edges. This enables the modeling of global relationships between cells and cross-scale relationships between cells and tissues, effectively capturing complex interactions between cells and overcoming the limitations of local receptive fields in CNNs. Thus, this embodiment overcomes the limitations of local receptive fields in CNNs, achieving global topological perception by establishing global relationships between cells, thereby improving diagnostic accuracy.

[0049] Meanwhile, this embodiment proposes a tissue segmentation and cell recognition technology with adaptive resolution for pathological images. By adaptively adjusting the resolution and dividing the images into sub-images, larger pathological images are processed into a size that is convenient for model processing without losing details. On this basis, tissue segmentation based on segmentation models and cell extraction based on polygon recognition are used to identify cells and tissues, thereby reducing computational complexity and balancing efficiency and detail.

[0050] Based on the first embodiment of this application, in the second embodiment of this application, the same or similar content as the first embodiment described above can be referred to the above description, and will not be repeated hereafter.

[0051] Pathological images are characterized by high resolution, multiple scales, and complex structures, making it difficult for traditional image analysis methods to effectively model the spatial and functional relationships between cells. A major challenge in achieving efficient modeling of cell-level graph structures is how to construct cell-cell graphs (CCGs) and cell-tissue graphs (CTGs) based on original pathological images and extract cellular features with semantic expressive power.

[0052] To address the aforementioned issues, based on the first embodiment, this embodiment can perform cell instance segmentation and tissue region identification on each image patch, and extract multimodal feature vectors with semantic expressive capabilities for each cell and tissue unit, providing structured input for subsequent graph structure modeling.

[0053] Therefore, in this embodiment, the cell entity set includes single-cell instances and the cell multimodal feature vectors corresponding to the single-cell instances, and the tissue entity set includes tissue region instances and the tissue multimodal feature vectors corresponding to the tissue region instances; refer to Figure 2 , Figure 2 This is a flowchart illustrating the second embodiment of the automatic diagnosis method for Sjögren's syndrome in pathological images according to this application. Step S20 includes steps B1 to B4: Step B1: Predict cell contours of the optimized image block using a preset instance segmentation model to obtain a cell instance mask, and extract single cell instances from the cell instance mask. Step B2: Perform feature fusion on the cell geometric features, color texture features and spatial location features extracted from each single cell instance to obtain the cell multimodal feature vector; Step B3: Perform tissue region prediction on the optimized image patch using a general segmentation model to obtain a tissue region mask, and obtain the tissue region instance corresponding to the tissue region mask; Step B4: Perform feature fusion on the extracted tissue geometric features, staining texture features, and location context features for each tissue region instance to obtain the tissue multimodal feature vector.

[0054] It is easy to understand that in this embodiment, the cell entity set can be a set containing multiple single-cell instances and the cell multimodal feature vector corresponding to each single-cell instance. The single-cell instance is a single cell individual extracted from the cell instance mask, with independent boundaries and a unique identifier. It is the basic building block of the cell entity set and can reflect the individual morphological characteristics of the cell.

[0055] It is important to understand that while traditional convolutional neural networks (CNNs) can perform image classification or coarse-grained object detection tasks, their performance is limited when dealing with high-density, complex, and blurred-boundary cell structures. In this embodiment, the aforementioned preset instance segmentation model can be the Stardist model based on star contour regression. It can be used to solve the problem of segmenting cells with high density, complex shapes, and blurred boundaries in pathological images. It can accurately predict cell contours, perform cell instance segmentation on each optimized image block Pi_final, and output a cell instance mask.

[0056] Therefore, in this embodiment, the radius lengths in multiple directions originating from the centroid of each cell can be predicted using the Stardist model to reconstruct the cell geometry and thus determine the cell boundary, i.e., perform the aforementioned cell contour prediction. The integer matrix output by the Stardist model, where each unique integer corresponds to an independent cell instance, marks the pixel position range of each cell in the image patch, and can be denoted as Mi_cell∈Nsp×sp (where sp is the image patch size), corresponding to the aforementioned cell instance mask. Then, the corresponding cell region can be extracted for each unique value cj in Mi_cell: cj∈Mi_cell, j=1,2,...,Ni_cell, where Ni_cell represents the total number of cells detected in the i-th patch.

[0057] For example, the diagnostic device can input any optimized image patch Pi_final into a preset Stardist instance segmentation model. The Stardist model first extracts features from the image patch to identify potential cell regions; then, for each potential cell region, it predicts its centroid position and predicts the radius length in multiple directions (e.g., 32 directions) from the centroid; finally, based on the centroid and the radii in each direction, it reconstructs the geometric contour of the cell and generates a cell instance mask Mi_cell. This mask is an sp×sp (sp can be set to 512 in this embodiment) integer matrix, where each unique integer in the matrix represents an independent cell instance. For example, integer "1" corresponds to the first cell instance, integer "2" corresponds to the second cell instance, and so on.

[0058] Then, each pixel in the cell instance mask Mi_cell is traversed. Based on the uniqueness of the pixel value, the set of pixels corresponding to each pixel value is extracted. Each set of pixels is the pixel region of a single cell instance cj, where j=1,2,...,Ni_cell, and Ni_cell represents the total number of cells detected in the optimized image patch. For example, if the pixel values ​​in the mask contain unique integers from 1 to 100, then 100 single cell instances c1-c100 can be extracted.

[0059] Compared to mainstream methods such as U-Net and Mask R-CNN, Stardist exhibits higher recall and boundary accuracy in dense cell scenes, and achieves good generalization performance without relying on large amounts of labeled data, making it suitable for large-scale pathological image analysis tasks. For specific segmentation results, please refer to [reference needed]. Figure 3 , Figure 3 This is a schematic diagram of cell segmentation in the second embodiment of the automatic diagnosis method for pathological images of Sjögren's syndrome according to this application. Figure 3 An irregular colored shape in the image output after processing by Stardist can correspond to a single-cell instance.

[0060] To transform cells from pixel-level objects into node entities usable for graph learning, this embodiment extracts a set of multimodal feature vectors for each cell instance, namely the aforementioned cell multimodal feature vector. This vector can be formed by fusing the geometric features fj_geo, color and texture features fj_color, and spatial location features fj_pos of a single cell instance. It can comprehensively represent the cell's attribute information and provide a data foundation for subsequent graph structure modeling and immune cell classification. It is denoted as fj_cell=[fj_geo, fj_color, fj_pos]∈R_dcell (dcell is the vector dimension, covering multiple dimensions such as its geometric attributes, color and texture features, and spatial location information).

[0061] Cell geometric features can describe the shape and size of cells, including the area, perimeter, aspect ratio (the ratio of the major axis to the minor axis), and roundness (4π × area / perimeter², the closer the value is to 1, the closer the cell is to a circle), reflecting the morphological characteristics of cells. Cell color and texture features can describe the color distribution and surface texture of single-cell instances, including the mean of the RGB three channels (reflecting the overall color intensity of the cell), standard deviation (reflecting the uniformity of cell color distribution), histogram of oriented gradients (HOG, reflecting cell edge and texture direction information), and Haralick texture features (reflecting the spatial correlation of cell grayscale distribution), which can distinguish the color and texture differences of different cell types. Cell spatial location features can describe the position of a single-cell instance in an image patch, specifically the centroid coordinates (xj, yj) of the cell. The centroid coordinates are obtained by calculating the mean coordinates of the cell pixels, reflecting the spatial relative positional relationship between cells. The aforementioned feature combination incorporates both the cell's morphological information and its spatial context within the image, enabling cell graph nodes in the subsequently constructed graph structure to possess stronger semantic expressive capabilities. These features will serve as the basis for node embedding during subsequent graph construction, supporting similarity calculations between cells, adjacency modeling, and information propagation in graph neural networks.

[0062] Understandably, cells do not exist in isolation but are nested within specific tissue environments. Therefore, in order to establish contextual relationships between cells and tissues, this embodiment can further segment tissue regions for each optimized image block Pi_final. The aforementioned set of tissue entities can be a collection containing multiple tissue region instances and the corresponding tissue multimodal feature vectors for each tissue region instance. Tissue region instances can be independent tissue regions with specific functions and structures obtained from tissue region masks, and are the basic building blocks of the set of tissue entities, capable of reflecting the macroscopic structural features of the tissue.

[0063] The aforementioned general segmentation model can also be the SAM2 model, which can achieve accurate segmentation of tissue regions in pathological images without retraining for specific staining patterns or tissue types. For example, diagnostic equipment can use the SAM2 model to identify and divide tissue regions in image patches, determine the boundaries and ranges of different tissue regions, and output a tissue region mask, i.e., perform the aforementioned tissue region prediction. The integer matrix output by the SAM2 model, where each unique integer corresponds to an independent tissue region instance, marks the pixel location range of each tissue region in the image patch, and can be denoted as Mi_tissue∈Nsp×sp, which is the aforementioned tissue region mask. Correspondingly, each unique value in Mi_tissue represents an independent tissue region, and the aforementioned tissue instance can be represented as: tk∈Mi_tissue, k=1,2,...,Ni_tissue, where Ni_tissue represents the total number of tissue regions detected in the i-th optimized image patch.

[0064] For example, the diagnostic device can input the optimized image patch Pi_final into the general segmentation model SAM2. The SAM2 model performs multi-scale feature extraction on the image patch through a pre-trained image feature extraction network (such as ViT-backbone), and predicts tissue regions by combining it with a learnable mask decoder; by setting an appropriate segmentation threshold (such as 0.5), it generates a tissue region mask Mi_tissue, which is an sp×sp integer matrix, where each unique integer corresponds to an independent tissue region instance, for example, the integer "1" corresponds to the first tissue region instance, and the integer "2" corresponds to the second tissue region instance.

[0065] Then, iterate through each pixel in the tissue region mask Mi_tissue, and extract the set of pixels corresponding to each pixel value based on the uniqueness of the pixel value. Each set of pixels is a tissue region instance t. k The pixel region is defined as k = 1, 2, ..., Ni_tissue, where Ni_tissue represents the total number of tissue regions detected in the optimized image patch. A segmentation diagram of tissue instances can be found in [reference needed]. Figure 4 , Figure 4 This is a schematic diagram of tissue segmentation in the second embodiment of the automatic diagnosis method for pathological images of Sjögren's syndrome according to this application. Figure 4 Any irregular colored shape (different from the background color) in the image output after processing by the SAM2 model can correspond to an instance of a tissue region.

[0066] It is easy to understand that the above-mentioned tissue multimodal feature vector can be a vector formed by fk_geo, fk_color, and fk_context of tissue region instances. It can comprehensively represent the attribute information of the tissue and provide a data foundation for cell-tissue map construction and cross-scale information fusion. It is denoted as fk_tissue=[fk_geo, fk_color, fk_context]∈R_dtissue (dtissue is the vector dimension).

[0067] Among these, tissue geometric features can describe the shape and size of a tissue region, including the area, perimeter, and shape factor (4π × area / perimeter², reflecting the degree of irregularity in the shape of the tissue region), which can reflect the macroscopic morphological characteristics of the tissue. Tissue staining texture features can be indicators describing the staining and texture characteristics of a tissue region, including the mean values ​​of the three channels H (hematoxylin), E (eosin), and D (xylene) in the H&E staining image (reflecting the depth of tissue staining) and texture score (obtained by calculating the eigenvalues ​​of the gray-level co-occurrence matrix of the tissue region, reflecting the coarseness and uniformity of the tissue texture), which can distinguish the staining and texture differences of different types of tissues.

[0068] Tissue location context features can describe the location of a tissue region and its surrounding environment, including the number of surrounding cells (surrounding_cells, the total number of cells within a certain range outside the tissue region boundary) and the centroid coordinates of the tissue region (centroid_x, centroid_y), which can reflect the spatial relationship between the tissue and its surrounding cells. These tissue features not only help in understanding the macroscopic environment in which cells exist, but also participate in information aggregation and propagation as context nodes when constructing a cell-tissue graph, improving the model's ability to understand local-global relationships.

[0069] Finally, the diagnostic device can combine different types of features (such as the geometry, color texture, and spatial location features of cells) of the same entity (single cell instance or tissue region instance) in a certain order to form a unified multimodal feature vector. In this embodiment, a direct splicing method can be used to make the feature vector contain comprehensive information of the entity.

[0070] In this embodiment, the Stardist model can be used for cell instance segmentation. Cell boundaries are accurately reconstructed through star-shaped contour regression. Compared with traditional models such as U-Net and MaskR-CNN, the recall rate is improved by 15%-20% and the boundary accuracy is improved by 10%-15% in high-density cell scenes, effectively solving the segmentation problem caused by dense cells and blurred boundaries in pathological images. Furthermore, the general segmentation model SAM2 is used for tissue segmentation. Its zero-sample transfer capability can adapt to different staining methods (such as H&E, IHC) and image differences between different laboratories. There is no need to retrain for specific data, and the model compatibility is improved by 40%-50%, reducing the adaptation cost for cross-center applications.

[0071] Meanwhile, this embodiment constructs a multimodal feature vector by fusing the geometric, color texture, and spatial location features of cells with the geometric, staining texture, and location context features of tissues. Compared to single-pixel features, the amount of feature information is increased by 3 to 5 times, enabling a more comprehensive characterization of cell and tissue attributes. This provides high-quality data support for subsequent graph structure modeling and immune cell classification, improving the accuracy of immune cell identification by more than 20%. Furthermore, the features extracted in this embodiment all have clear physical meaning (such as cell area and average tissue staining). In subsequent diagnostic processes, the correlation between features and diagnostic results can be traced. For example, analyzing the geometric features of lymphocytes (such as abnormal roundness) or the tissue staining texture features (such as deepened staining in inflammatory areas) can assist doctors in understanding the diagnostic basis, further improving the interpretability of the model.

[0072] It is easily understood that existing technologies in pathological image analysis only focus on local pixel features or single-scale (cell-level or tissue-level) information, failing to establish global spatial relationships between cells and cross-scale associations between cells and tissues. This makes it difficult for the model to understand the functional state and interactions of cells in the tissue environment. For example, it cannot effectively associate the positional relationship between lymphocytes and inflamed tissues, thus affecting the accuracy of identifying key lesions (such as lymphocyte aggregation focal points) in the diagnosis of Sjögren's syndrome. Simultaneously, traditional graph structures are mostly single-node types (such as only cell nodes), unable to integrate multi-scale information, limiting the model's ability to model complex structures in pathological images. Therefore, in a feasible implementation, in this embodiment, step S30 may include steps C1~C3: Step C1: Each single-cell instance is used as a cell graph node, and a cell-cell edge set is established between the cell graph nodes that meet the preset interval requirements. Step C2: Each tissue region instance is used as a tissue graph node, and a cell-tissue edge set is established between the cell graph nodes and the tissue graph nodes that meet the preset coverage requirements. Step C3: Integrate the cell-cell edge set and the cell-tissue edge set to generate a heterogeneous graph structure. The nodes of the heterogeneous graph structure carry the corresponding cell multimodal feature vector or tissue multimodal feature vector.

[0073] In essence, the diagnostic device can abstract each single-cell instance in the set of cell entities as a node in a graph structure, obtaining the aforementioned cell graph nodes. These nodes are the basic units for constructing the cell-cell edge set, and each cell graph node carries a corresponding cell multimodal feature vector, denoted as vj (corresponding to a single-cell instance cj). Simultaneously, a set of cell graph nodes V_cell={vj_1,vj_2,...,vj_Ncell} (where Ncell is the total number of cells) is formed, and each cell graph node vj is bound to its corresponding cell multimodal feature vector fj_cell.

[0074] The aforementioned preset interval requirement can be a spatial distance standard for establishing a connection between two corresponding cell graph nodes. In this embodiment, it can be set to the centroid distance between single-cell instances corresponding to two cell graph nodes being less than or equal to a preset adjacency radius r, where r is 1.5-2 times the average cell diameter (the average value can be calculated by statistically optimizing the diameter of all cells in the image patch; for example, when the average diameter is 25 pixels, r is set to 37.5-50 pixels). At this point, the diagnostic device can construct a set of undirected edges between cell graph nodes that meet the preset interval requirement, i.e., the aforementioned cell-cell edge set, which is used to model the spatial adjacency relationship between cells, denoted as E_cell. The existence of an edge indicates the possibility of direct spatial interaction between the corresponding cells.

[0075] Furthermore, the diagnostic device can abstract each tissue region instance in the tissue entity set as another type of node in a graph structure, namely the aforementioned tissue graph node. This node is the basic unit for constructing the cell-tissue edge set, and each tissue graph node carries a corresponding tissue multimodal feature vector, denoted as vk (corresponding to the tissue region instance tk). This forms a tissue graph node set V_tissue={vk_1,vk_2,...,vk_Ntissue} (where Ntissue is the total number of tissue regions), and binds its corresponding tissue multimodal feature vector fk_tissue to each tissue graph node vk.

[0076] Accordingly, the above-mentioned preset coverage requirements can correspond to the spatial location standards for establishing connections between cell map nodes and tissue map nodes. In this embodiment, they can be set as follows: First, the centroid of the single cell instance corresponding to the cell map node is located within the tissue region instance corresponding to the tissue map node; Second, the distance between the boundary of the cell instance and the boundary of the tissue region instance is less than a preset threshold d (d can be set to 10-15 pixels, which can be adaptively adjusted according to the resolution of the pathological image and the cell size. The threshold can be appropriately reduced as the resolution increases).

[0077] Therefore, the diagnostic device can establish undirected edges between cell graph nodes and tissue graph nodes that meet the preset coverage requirements, and form an edge set, namely the cell-tissue edge set mentioned above, which is used to model the cross-scale spatial relationship between cells and tissues, denoted as E_ct.

[0078] Finally, the diagnostic device can merge the cell-cell edge set E_cell and the cell-tissue edge set E_ct, while retaining the cell graph node set V_cell and the tissue graph node set V_tissue, forming a heterogeneous graph structure containing two node types and two edge types, ensuring that the graph structure fully covers the relationships between cells and between cells and tissues. Furthermore, since the cell graph nodes in the heterogeneous graph structure carry corresponding cell multimodal feature vectors fj_cell, and the tissue graph nodes carry corresponding tissue multimodal feature vectors fk_tissue, these feature vectors can serve as input data for subsequent graph neural network models, providing a foundation for node feature optimization and relationship inference.

[0079] In this implementation, global spatial adjacency relationships between cells are modeled using cell-cell edge sets, overcoming the limitations of local receptive fields in CNNs and capturing potential interactions between distant cells. Cell-tissue edge sets connect cells and tissues, combining microscopic cellular information with macroscopic tissue background, enabling the model to understand the tissue environment in which cells exist (e.g., whether lymphocytes are located within inflamed tissue). This provides contextual support for subsequent immune cell classification and cluster identification, improving the accuracy of lesion identification. Establishing edges through spatial adjacency and coverage relationships effectively filters out interference from image noise (such as background impurities) on node associations. For example, connections are established only between real cells, avoiding the influence of impurity pixels on the graph structure, thus improving the model's diagnostic robustness on low-quality pathological images (e.g., uneven staining, presence of impurities).

[0080] Meanwhile, this embodiment constructs a heterogeneous graph structure containing cell nodes and tissue nodes. Compared with graph structures with single node types, it has richer information dimensions and can simultaneously represent the characteristics and relationships of microscopic individuals (cells) and macroscopic environments (tissues). This enhances the model's ability to model complex structures in pathological images and more realistically simulates the biological structure of pathological tissues. Since the edges in the heterogeneous graph structure clearly define the relationships between nodes, the subsequent graph attention network can dynamically adjust the attention weights of different nodes (such as adjacent cells or the tissue they belong to) based on the edge connections. This allows the model to focus more on key nodes (such as lymphocytes near inflamed tissues), further improving the accuracy of feature optimization.

[0081] It is important to understand that existing CNN models rely on large amounts of labeled data (medical labeling is time-consuming and expensive), and feature extraction lacks global structural awareness, resulting in low accuracy in immune cell classification. Simultaneously, traditional clustering methods (such as K-Means) are based solely on pixel features, failing to incorporate intercellular spatial relationships, making it difficult to accurately identify lymphocyte clusters (a key diagnostic marker for Sjögren's syndrome), and the models have poor interpretability, leaving doctors unable to trace the basis of their diagnoses. Therefore, to achieve accurate cell and tissue identification without relying on manual labeling, and to construct an expression vector for each cell graph node, in a feasible implementation, step S40 in this embodiment may include steps D1-D3: Step D1: Optimize the multimodal feature vectors of cells in the heterogeneous graph structure using a self-supervised graph attention network to obtain optimized cell feature vectors; the self-supervised graph attention network is obtained through self-supervised pre-training via node occlusion reconstruction and contrastive learning with applied structural perturbations. It should be noted that the self-supervised graph attention network mentioned above can be a model that is pre-trained by combining a self-supervised learning mechanism and a graph attention network (GAT). In this embodiment, the network parameters of the self-supervised graph attention network can be optimized through self-supervised pre-training tasks (node ​​occlusion reconstruction, contrastive learning), and then the weights of the node's neighbors can be dynamically adjusted through the attention mechanism during the inference stage to achieve accurate optimization of node features. Therefore, this embodiment can be used to optimize the multimodal feature vectors of cells in heterogeneous graph structures.

[0082] In the self-supervised pre-training process of the self-supervised graph attention network, unlabeled WSI images of the oral labial glands can be collected first, and the corresponding heterogeneous graph structure set {G1,G2,...,Gn} (n is the amount of unlabeled data) can be generated according to the steps mentioned above. Each heterogeneous graph Gn=(Vn,En) contains cell nodes, tissue nodes and corresponding multimodal feature vectors.

[0083] Then, in the node occlusion reconstruction process, this embodiment can randomly occlude a portion of the feature vector of each node in the heterogeneous graph structure Gn (e.g., occlude 30% to 50% of the feature dimensions), train the model to predict the value of the occluded dimension based on the unoccluded feature dimensions (e.g., input the occluded feature vector into the graph attention network (GAT), GAT transforms f_masked through the linear transformation matrix W, and then aggregates the features of neighboring nodes through the attention mechanism to output the predicted value of the occluded dimension f_pred), and uses mean squared error loss (MSE Loss) to optimize the graph attention network parameters, so that the model learns the inherent correlation rules of the node features.

[0084] Regarding the contrastive learning with applied structural perturbations, this embodiment constructs different views of each heterogeneous graph Gn in two ways: one is structural perturbation (i.e., randomly deleting 10% to 20% of the edges in Gn, disrupting some node relationships); the other is node feature perturbation (i.e., adding Gaussian noise N(0,σ²), σ²=0.01, to the feature vectors of all nodes in Gn, or locally occluding 5% to 10% of the feature dimensions). Then, for each node, the node embeddings of the original view and the node embeddings of the perturbated view are used as positive sample pairs, and the embeddings of other nodes are used as negative sample pairs. Finally, the graph attention network is optimized through contrastive loss, enabling the model to learn discriminative node embedding representations.

[0085] Finally, this embodiment can combine node occlusion reconstruction loss and contrast loss to continue training the model parameters of the graph attention network until the preset training rounds are completed, thus completing the self-supervised pre-training and obtaining the self-supervised graph attention network.

[0086] Therefore, this embodiment can train the parameters (including the linear transformation matrix W, attention vector a, etc.) of a self-supervised graph attention network on unlabeled pathological image data through contrastive learning tasks of node occlusion reconstruction and applying structural perturbations. This allows the model to learn the characteristic association rules of cells and tissues in pathological images without relying on manual annotation, thus reducing dependence on labeled data.

[0087] It is easy to understand that a self-supervised graph attention network, pre-trained under self-supervised conditions, can dynamically assign attention weights to neighboring nodes of a cell graph node in a heterogeneous graph structure, thereby focusing on nodes in key regions, such as lymphocytes at lesion boundaries and cells that frequently interact with tissue nodes. Therefore, the output feature vector updated by the self-supervised graph attention network to the cell's multimodal feature vector, i.e., the optimized cell feature vector, integrates the node's own features, the features of neighboring nodes (other cells and their tissues), and attention weight information, possessing stronger semantic expressive power and discriminative ability, and can be denoted as... (Corresponding cell diagram node v) i ).

[0088] For example, in this embodiment, the constructed heterogeneous graph structure G to be diagnosed can be input into a pre-trained self-supervised graph attention network. Structure-aware enhancement is performed on each cell node vj (carrying a multimodal feature vector fj_cell) in G. At this point, the self-supervised graph attention network can perform the following feature optimization process: 1) Linear Transformation: A learnable linear transformation matrix W is used to map fj_cell to a high-dimensional space, obtaining a new eigenvector: f i^trans=W×fj_cell (In this embodiment, W has a dimension of 256×128, which maps 128-dimensional features to 256-dimensional features).

[0089] 2) Attention weight calculation: For all neighboring nodes vi of cell graph node vj (including neighboring cell nodes ci and tissue nodes ti, the set of neighboring nodes N(i) is determined by E_cell and E_ct), calculate the attention coefficient. The calculation method is as follows: (2) Where a is the learnable attention vector; "||" represents the vector concatenation operation; N(i) represents the set of neighboring nodes of node vj; σ(.) is the non-linear activation function; LeakyReLU is the activation function; and W is the learnable linear transformation matrix.

[0090] Then, node updates are performed by incorporating attention weights: (3) in, This indicates that the previously obtained cell or tissue vector is represented as fj_cell or fk_tissue. This represents the new feature vector. Note that we only calculate and save the feature vector of the cell. For cell node i, its refined feature representation is denoted as... .

[0091] 3) Multi-head attention aggregation: To further enhance the expressive power of the model, a multi-head attention mechanism is used for feature enhancement based on formula (3). For example, eight attention heads can be used (i.e., the number of attention heads H=8), and each attention head (through an independent weight matrix) and attention vector The attention weights are calculated independently and the features of adjacent nodes are aggregated. The outputs of the eight attention heads are concatenated and then subjected to a linear transformation to obtain the optimized cell feature vector, which is represented as follows: (4) Where h = 1 to 8, Let h be the weight of the h-th attention head. Similarly, only the feature vector of the cell is calculated and stored. Specifically, for cell node i, its refined feature representation is denoted as... This serves as a high-quality input feature for subsequent immune cell classification and structure recognition tasks.

[0092] In summary, this module uses a self-supervised pre-trained Graph Attention Network (GAT) to dynamically assign attention weights to neighboring nodes during inference, enabling the model to focus on nodes in key regions, such as lymphocytes at lesion boundaries and cells that frequently interact with nodes in the tissue graph. By aggregating neighborhood information through a multi-head attention mechanism, it ultimately obtains a refined cellular feature representation with structure awareness. .

[0093] Step D2: Perform immune cell classification based on the optimized cell feature vector to obtain multiple immune cell map nodes; Understandably, in the diagnosis of immune-related diseases such as SS, focal points formed by the aggregation of immune cells, such as lymphocytes, are key pathological scoring indicators. Therefore, identifying these clinically significant cell clusters from graph structures and correlating them with the final diagnostic decision is a key challenge of this study. Traditional methods rely on simple cell counting or regional aggregation, which struggle to capture complex patterns at the structural level.

[0094] Compared to existing technologies, this embodiment can differentiate whether a cell is an immune cell (specifically lymphocytes in this embodiment, a key indicator for diagnosing Sjögren's syndrome) based on an optimized cell feature vector using a binary classification model, and output the probability and classification result of the cell belonging to an immune cell. After immune cell classification, the cell map nodes identified as immune cells are the aforementioned immune cell map nodes. These nodes carry optimized cell feature vectors and form the basis for subsequent clustering analysis, denoted as vi_immune (corresponding to cell map nodes with classification results indicating immune cells).

[0095] For example, this embodiment can pre-construct a binary classification neural network containing two fully connected layers. The input layer dimension is the dimension of the optimized cell feature vector (256 dimensions in this embodiment). The output dimension of the first fully connected layer is 128, and the activation function is LeakyReLU (with a negative slope of 0.2) to increase non-linear expression. The output dimension of the second fully connected layer is 1, and the activation function is Sigmoid, outputting the probability yi∈[0,1] that the cell belongs to the lymphocyte. Then, a small amount of labeled data (cell labels of 100 WSI images in this embodiment, including 50 lymphocyte labels and 50 non-lymphocyte labels) is used to train the binary classification network, and iterative training is performed using the cross-entropy loss function.

[0096] In practical use, this embodiment can optimize the cell feature vectors of all cell nodes in the heterogeneous graph to be diagnosed. Input the trained binary classification network and output the probability yi of each cell; set the classification confidence threshold θ=0.7. If yi>θ, the cell is determined to be an immune cell (lymphocyte), and the corresponding cell graph node can be labeled as an immune cell graph node vi_immune. Then collect all immune cell graph nodes to form a set V_immune={vi_immune|yi>θ}.

[0097] Step D3: Perform a preset clustering analysis based on the multiple immune cell map nodes and output the Sjögren's syndrome status determination result.

[0098] It is important to understand that in existing technologies, traditional clustering methods (such as K-Means) only cluster based on cell pixel features, ignoring the spatial adjacency relationships between cells. This makes it difficult to accurately identify the crucial "lymphocyte aggregation focal points" (clusters of cells that need to be tightly connected) in the diagnosis of Sjögren's syndrome, leading to missed or false lesions. Therefore, this embodiment uses a graph community detection algorithm (Louvain algorithm in this embodiment) to cluster immune cell graph nodes, identifying tightly connected immune cell clusters, i.e., performing the aforementioned preset clustering analysis. The clustering is based on the spatial adjacency relationships and feature similarities between nodes, aiming to uncover the aggregation patterns of immune cells (key pathological features for the diagnosis of Sjögren's syndrome).

[0099] The above-mentioned Sjögren's syndrome status determination result can be based on the number and density of immune cell clusters obtained from cluster analysis, combined with the diagnostic criteria for Sjögren's syndrome (more than 50 immune cells per 4 mm² of tissue), and output as a "positive" or "negative" diagnostic conclusion, which is the final output of the entire diagnostic process.

[0100] In one feasible implementation, step D3 may include steps D31 to D33: Step D31: Construct an immune cell map node sub-map based on the plurality of immune cell map nodes; Step D32: Run a preset community detection algorithm on the immune cell graph node subgraph to perform clustering and obtain immune cell clusters; It should be noted that in this embodiment, the edges between immune cell graph nodes in the heterogeneous graph G can be extracted first based on the immune cell graph node set V_immune, i.e., E_immune=E_cell∩(V_immune×V_immune), and a subgraph structure containing only immune cell graph nodes and the original connection relationships between these nodes can be constructed, i.e., the above-mentioned immune cell subgraph G_immune=(V_immune, E_immune).

[0101] Then, the diagnostic device can run a preset community detection algorithm on G_immune. In this embodiment, the Louvain community detection algorithm is preferred. It can identify tightly connected substructures (communities) in the graph by iteratively optimizing the community division, thereby accurately identifying multiple highly connected lymphocyte clusters and obtaining the clusters formed by the aggregation of immune cells, namely the above-mentioned immune cell clusters. Each cluster represents a potential lesion area (lymphocyte aggregation is a key lesion marker in Sjögren's syndrome), which can be denoted as C_q (q=1,2,...,p, where p is the number of clusters).

[0102] For example, in the above clustering process, each immune cell graph node can first be initialized as an independent community and assigned a unique initial community identifier c_i; then, each node vi_immune can be traversed, and it can be attempted to be assigned to an adjacent community. The change in modularity ΔQ before and after assignment can be calculated. The modularity formula is Q=(1 / (2m))×Σ[A_ij-(k_i*k_j) / (2m)]×δ(c_i,c_j), where A_ij is the adjacency matrix element, k_i is the node degree, m is the total number of edges, and δ(c_i,c_j) is the community affiliation indicator function. If ΔQ>0, the node is assigned to the community; otherwise, the original community is maintained. The above steps are repeated until the modularity no longer increases, resulting in multiple immune cell clusters C1, C2,..., C_p (p is the number of clusters).

[0103] Step D33: Output the Sjögren's syndrome status determination result based on the spatial distribution information of the immune cell clusters and the number of immune cells in the immune cell clusters.

[0104] Understandably, the aforementioned spatial distribution information can include the location, coverage area, and relative relationship with surrounding tissues of immune cell clusters in the original digital pathological slide image. This includes the centroid coordinates, circumscribed rectangle range, and image block number of the cluster, which can be used to locate lesion areas and assist doctors in verification. The total number of immune cell map nodes contained in each immune cell cluster, i.e., the number of immune cells mentioned above, is the core data for calculating immune cell density and determining whether the diagnostic criteria for Sjögren's syndrome are met, and can be denoted as N_q (corresponding to cluster C_q).

[0105] Finally, the diagnostic equipment can form a final judgment on the overall pathological status of the WSI sample regarding Sjögren's syndrome based on the density of immune cell clusters (the number of immune cells per 4 mm²), combined with the "positive" or "negative" conclusion output by the clinical diagnostic criteria.

[0106] For example, refer to Figure 5 It can be seen that, Figure 5This is a schematic diagram of the Sjögren's syndrome status determination process in the second embodiment of the automatic diagnosis method for pathological images of Sjögren's syndrome in this application. After performing adaptive resolution patch (image block) division and extraction, identifying lymphocytes and lesions, the diagnostic device counts the number of immune cells N_q in each cluster C_q (q=1 to p) in each image block.

[0107] Simultaneously, based on the centroid coordinates (x_i, y_i) of all nodes within the cluster, the centroid coordinates (xq_centroid, yq_centroid) = (Σx_i / N_q, Σy_i / N_q) of the cluster can be calculated to determine the original image patch number where the cluster is located. Then, combined with the area S_patch of the corresponding image patch (in this embodiment, 512×512 pixels correspond to an actual area of ​​0.1mm²), the number of immune cells within each 4mm² is calculated: N_4mm² = N_q × (4 / S_patch).

[0108] Wherein, if an immune cell cluster C_q is detected in any image patch satisfying N_4mm²>50 (i.e. Figure 5 If the number of lymphocytes shown is >50 / 4mm², then the WSI sample is considered positive for Sjögren's syndrome (SS positive); if all image patches contain clusters that satisfy N_4mm²≤50 (i.e., ... Figure 5 If the number of lymphocytes shown is <= 50 / 4mm², it is determined to be negative for Sjögren's syndrome (SS negative), and the final diagnostic result is output. At this time, the final Sjögren's syndrome status determination result may include the following: basic sample information (such as sample number, WSI scan parameters); statistical results of immune cell clusters (number of clusters, location of each cluster, N_q, N_{4mm^2}_q), in this embodiment, it may be the number of lymphocytes in the lesion; visualization image of key clusters (marking the location and range of the clusters in the original image block); final diagnostic conclusion (positive / negative) and determination criteria, for example, outputting "the number of lymphocytes in cluster C3 within 4mm² is 62, which meets the positive criterion".

[0109] In this embodiment, the traditional clustering methods (such as K-Means) in the prior art only cluster based on cell pixel features, ignoring the spatial adjacency relationship between cells. This makes it impossible to accurately identify the key "lymphocyte aggregation focus" (cell clusters that need to be closely connected in space) in the diagnosis of Sjögren's syndrome, leading to missed or false detections of lesions. The Louvain community detection algorithm can be used to cluster based on the spatial adjacency relationship (edge ​​set) and feature similarity between cells. Compared with the existing K-Means algorithm, this embodiment improves the accuracy of lymphocyte aggregation focus identification and reduces the false detection rate, which can effectively capture the key pathological markers of Sjögren's syndrome.

[0110] Traditional diagnostic procedures rely on manual counting of lymphocytes, which is inefficient and susceptible to subjective factors (such as inconsistent criteria for judging "aggregation" among doctors). Furthermore, they cannot correlate the spatial location of clusters with the tissue background, making it difficult to provide clear diagnostic evidence. This embodiment standardizes the number of immune cells to an index of "per 4mm²" by calculating the correlation between the physical area of ​​image patches and the number of clustered cells. This directly matches clinical diagnostic criteria, avoids the subjectivity of manual counting, and improves the clinical acceptance of diagnostic results.

[0111] Finally, existing methods lack standardized calculations for cluster density, making it impossible to directly match clinical diagnostic criteria (such as the threshold for the number of cells per 4 mm²), leading to a disconnect between diagnostic results and clinical needs. This embodiment maps cluster locations to image blocks and tissue regions, associating the clusters with their surrounding tissue environment (such as whether they are located around salivary gland ducts), providing doctors with more pathological contextual information and assisting in determining the clinical significance of lesions. Furthermore, the final diagnostic report includes the location, number, density, and visualization results of the clusters, allowing doctors to intuitively trace diagnostic evidence (such as "cluster density exceeds the standard at a certain location"), solving the "black box" problem of traditional models. Simultaneously, the automated statistical and judgment process reduces the single-sample diagnostic time from 30-60 minutes manually to 5-10 minutes, significantly improving diagnostic efficiency. In summary, this embodiment proposes a self-supervised graph learning representation mechanism for cells and tissues, reducing reliance on large amounts of labeled data and improving the model's generalization ability. This helps the model learn effective feature representations even in the absence of labels, thereby enhancing its performance stability when facing new data. Simultaneously, the self-supervised graph attention network integrates node-specific features with those of its neighbors (cells + tissues), focusing on key related nodes (such as cells near inflamed tissues) using attention weights, significantly improving the discriminative power of the optimized feature vectors. Furthermore, this embodiment proposes a cell cluster structure recognition method based on community detection. It extracts subgraphs from nodes predicted to be immune cells and detects tightly connected clusters, analyzing their density, shape, etc., and using these as auxiliary diagnostic features in the final classification decision process. This not only improves diagnostic accuracy but also provides structural-level explanations, enhancing the model's interpretability.

[0112] This embodiment discloses a method for predicting cell contours of optimized image blocks using a preset instance segmentation model to obtain cell instance masks, and extracting single-cell instances from the cell instance masks. The method also involves fusing the extracted cell geometric features, color texture features, and spatial location features of each single-cell instance to obtain a cell multimodal feature vector. Furthermore, it describes the method for predicting tissue regions of optimized image blocks using a general segmentation model to obtain tissue region masks, and acquiring the corresponding tissue region instances. Finally, it describes the method for fusing the extracted tissue geometric features, staining texture features, and location context features of each tissue region instance to obtain a tissue multimodal feature vector.

[0113] This embodiment also discloses that each single-cell instance is used as a cell graph node, and a cell-cell edge set is established between cell graph nodes that meet the preset interval requirements; each tissue region instance is used as a tissue graph node, and a cell-tissue edge set is established between cell graph nodes and tissue graph nodes that meet the preset coverage requirements; the cell-cell edge set and the cell-tissue edge set are integrated to generate a heterogeneous graph structure, and the nodes of the heterogeneous graph structure carry the corresponding cell multimodal feature vector or tissue multimodal feature vector.

[0114] Simultaneously, this embodiment optimizes the multimodal feature vectors of cells in the heterogeneous graph structure using a self-supervised graph attention network to obtain optimized cell feature vectors. The self-supervised graph attention network is obtained after self-supervised pre-training through node occlusion reconstruction and contrastive learning with applied structural perturbations. Immune cell classification is performed based on the optimized cell feature vectors to obtain multiple immune cell graph nodes. An immune cell graph node subgraph is constructed based on the multiple immune cell graph nodes. A preset community detection algorithm is run on the immune cell graph node subgraph to perform clustering and obtain immune cell clusters. The Sjögren's syndrome status determination result is output based on the spatial distribution information of the immune cell clusters and the number of immune cells in the immune cell clusters.

[0115] In summary, this embodiment leverages the inherent spatial relationship representation capabilities of graph structures, combined with community detection algorithms to identify lymphocyte clusters, accurately capturing key spatial topological features (lymphocyte aggregation focal points) in SS diagnosis and improving diagnostic accuracy. Furthermore, a self-supervised pre-training mechanism is introduced, training the graph attention network on unlabeled data through node occlusion reconstruction and contrastive learning tasks. This allows the model to learn effective feature representations, significantly reducing the need for large-scale manually labeled data and lowering costs. Throughout the diagnostic process, from cell-tissue segmentation and graph structure construction to immune cell cluster identification, the feature extraction and decision-making basis (such as lymphocyte count and aggregation density) at each step are clearly traceable, and a diagnostic report containing specific image patch analysis (location, cell count, density) can be generated, facilitating physician understanding and verification and addressing the "black box" problem of CNNs.

[0116] For example, to help understand the technical concept or principle of the automatic diagnosis method for pathological images of Sjögren's syndrome after combining this embodiment with the above-described Embodiments 1 and 2, please refer to Figure 6 , Figure 6 The following is a schematic diagram of the automatic diagnosis method for pathological images of Sjögren's syndrome according to this application: 1. Image preprocessing: First, multi-scale tissue region extraction is performed on the original digital pathological slide image (e.g., H&E image). The original digital pathological slide image is downsampled according to the scaling factor s∈(0,1) to obtain a low-resolution image. The scaling factor s can be adaptively set according to the size of the original digital pathological slide image. For example, when the original image is 100,000 × 100,000 pixels, s can be set to 0.1 to obtain a low-resolution image of 10,000 × 10,000 pixels.

[0117] Then, the Segment Anything Model 2 (SAM2) model is used to coarsely locate the tissue regions in the low-resolution image to obtain an initial tissue mask. This mask is a binary matrix that marks the positions of all pixels in the low-resolution image that belong to the tissue regions.

[0118] Next, the initial tissue mask T is mapped back to the coordinate system of the original WSI image to obtain the target tissue mask. Based on the target tissue mask, the original WSI image is divided into regions using a sliding window or density-based clustering method, cropping out multiple initial image blocks of 512×512 pixels. Finally, data augmentation processing is performed on the initial image blocks, including color normalization, geometric transformation, and noise injection, to obtain optimized image blocks.

[0119] 2. Cell-tissue co-segmentation: The Stardist model, based on star-shaped contour regression, was used to segment cells in optimized image patches to obtain cell detection results. Figure 6 As shown, the cell analysis results may include single-cell instances, i.e. Figure 6 The single-cell feature and the cell multimodal feature vector shown are... Figure 6The model illustrates the features of surrounding cells. Specifically, it reconstructs the cell geometry by predicting the radius lengths in multiple directions originating from the centroid of each cell, obtaining a cell instance mask (an integer mask matrix, where each unique value represents an independent cell instance). Single-cell instances are then extracted from this mask. A multimodal feature vector is extracted for each single-cell instance, including geometric features, color and texture features, and spatial location features. These features are then fused to obtain the cell multimodal feature vector, thus forming a set of cell entities.

[0120] Then, the SAM2 model is used to obtain the tissue detection result, which is expected to contain... Figure 6 The tissue region features shown are the aforementioned tissue multimodal feature vectors. Tissue region prediction is performed on the optimized image patch to obtain a tissue region mask (an integer mask matrix, where each unique value represents an independent tissue region), and tissue region instances are extracted from it. Multimodal feature vectors are extracted from each tissue region instance, including geometric features, staining texture features, and location context features. These features are then fused to obtain the tissue multimodal feature vector, thus forming a set of tissue entities.

[0121] 3. Heterogeneous graph structure construction: Each single-cell instance is treated as a cell graph node. The distance between the centroids of any two cell graph nodes is calculated. If the distance is less than or equal to the preset adjacency radius (usually 1.5-2 times the average cell diameter; for example, when the average cell diameter is 20 pixels, the preset adjacency radius can be set to 30-40 pixels), then an undirected edge is established between the two cell graph nodes to form a cell-cell edge set.

[0122] Each tissue region instance is used as a tissue graph node. It is determined whether the centroid of the cell corresponding to the cell graph node is located within the tissue corresponding to the tissue graph node, or whether the distance between the cell boundary and the tissue boundary is less than a preset threshold. If either condition is met, a cross-scale undirected edge is established between the cell graph node and the tissue graph node to form a cell-tissue edge set.

[0123] By integrating the cell-cell edge set and the cell-tissue edge set, a heterogeneous graph structure is obtained, and each node carries a corresponding multimodal feature vector (cell graph nodes carry cell multimodal feature vectors, and tissue graph nodes carry tissue multimodal feature vectors).

[0124] 4. Assessment of Sjögren's syndrome status: A self-supervised Graph Attention Network (GAT) is employed to optimize the multimodal feature vectors of cells in heterogeneous graph structures. The GAT is first pre-trained under self-supervised conditions. The pre-training tasks include node occlusion reconstruction (randomly occluding some dimensions of node features, training the model to predict the occluded features, and optimizing parameters using mean squared error loss) and contrastive learning (constructing different views of the graph through graph structure perturbations and node feature perturbations, optimizing the objective function using cosine similarity as a metric to improve node embedding discriminative ability). After training, the node features from the heterogeneous graph structure are input into the GAT. The GAT dynamically adjusts the weights of neighboring nodes through an attention mechanism to update and optimize the cell feature vectors, resulting in optimized cell feature vectors.

[0125] Then, an optimized cell feature vector is used to classify immune cells through a pre-constructed binary classification neural network (Classification MLP). This network includes a fully connected layer (mapping the input feature vector to a higher-dimensional space), a LeakyReLU activation function (increasing non-linear expression capability), and an output layer (containing one neuron, using a Sigmoid activation function to output the probability yi∈[0,1] that a cell belongs to a lymphocyte). The network is optimized using a cross-entropy loss function. Cells with a predicted probability greater than the classification confidence threshold θ (θ can be set to 0.7) are marked as immune cells, resulting in multiple immune cell map nodes. Figure 6 The classification result is shown.

[0126] An immune cell graph node subgraph is constructed based on multiple immune cell graph nodes. The Louvain community detection algorithm is then run on this subgraph for clustering. This algorithm identifies multiple immune cell clusters (i.e., [missing information]) by maximizing the modularity of the graph. Figure 6 (See Group Result). Then, the number of immune cells in each immune cell cluster is counted, and the number of immune cells per unit area (4 mm²) is calculated based on the corresponding image patch area. If the number of immune cells per unit area in any image patch exceeds 50, the WSI sample is determined to be Sjögren's syndrome positive (SS positive); if the number of immune cells per unit area in all image patches does not exceed 50, the sample is determined to be Sjögren's syndrome negative (SS negative).

[0127] In summary, existing pathological image analysis methods based on convolutional neural networks (CNNs) suffer from several problems, including limited local field of view (CNNs have limited receptive fields, making it difficult to capture global relationships between cells), neglect of spatial topological relationships (they cannot effectively capture specific spatial arrangement patterns of cells, such as the focal points of lymphocyte aggregation in the diagnosis of Sjögren's syndrome), heavy reliance on data annotation (requiring a large amount of professionally annotated data, which is time-consuming and expensive), and poor interpretability (the classification results are difficult to trace back to specific diagnostic criteria). These issues result in low diagnostic accuracy and limited application of Sjögren's syndrome.

[0128] This application constructs a heterogeneous graph structure, abstracting cells and tissues into nodes and establishing spatial adjacency edges. This enables the modeling of global relationships between cells and cross-scale relationships between cells and tissues, effectively capturing complex interactions between cells and solving the problem of limited local field of view in CNNs. Furthermore, by utilizing the inherent spatial relationship expression capabilities of graph structures and combining them with community detection algorithms to identify lymphocyte clusters, it accurately captures key spatial topological features (lymphocyte aggregation focal points) in SS diagnosis, thereby improving diagnostic accuracy.

[0129] Simultaneously, a self-supervised pre-training mechanism is introduced. Through node masking reconstruction and contrastive learning tasks, the graph attention network is trained on unlabeled data, enabling the model to learn effective feature representations. This significantly reduces the need for large-scale manually labeled data, lowering costs. Throughout the diagnostic process, from cell-tissue segmentation and graph structure construction to immune cell cluster identification, the feature extraction and decision-making basis (such as lymphocyte count and aggregation density) at each step are clearly traceable. Furthermore, it can generate diagnostic reports containing specific image patch analyses (location, cell count, density), facilitating physician understanding and verification, and addressing the "black box" problem of CNNs. By automating image preprocessing, segmentation, graph learning, and judgment processes, it replaces traditional manual histopathological assessments, significantly improving diagnostic efficiency. Simultaneously, the application of multimodal feature fusion, attention mechanism optimization, and community detection clustering technologies further enhances diagnostic accuracy, aiding in the early and accurate diagnosis of Sjögren's syndrome, improving patient prognosis, and forming an end-to-end intelligent analysis process from image to diagnosis.

[0130] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the automatic diagnosis method for pathological images of Sjögren's syndrome in this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0131] This application also provides an automatic diagnostic device for Sjögren's syndrome in pathological images. Please refer to [link / reference]. Figure 7 , Figure 7 This is a schematic diagram of the module structure of the automatic diagnostic device for Sjögren's syndrome in pathological images according to an embodiment of this application. In this embodiment, the device includes: Image preprocessing module 701 is used to preprocess the original digital pathological slide image to obtain optimized image blocks; Image segmentation module 702 is used to perform cell-tissue collaborative segmentation on the optimized image block to obtain a set of cell entities and a set of tissue entities; Graph learning module 703 is used to map the set of cell entities and the set of tissue entities into a heterogeneous graph structure, wherein the heterogeneous graph structure has cell graph nodes and tissue graph nodes as vertices and spatial adjacency relationships as edges. The identification module 704 is used to determine the state of Sjögren's syndrome based on the heterogeneous graph structure.

[0132] As one possible implementation, in this embodiment, the image preprocessing module 701 is further configured to perform multi-scale tissue region extraction on the original digital pathological slide image to obtain a target tissue mask; divide the original digital pathological slide image into regions based on the target tissue mask to obtain initial image blocks; and perform data augmentation processing on the initial image blocks to obtain optimized image blocks.

[0133] In one possible implementation, in this embodiment, the image segmentation module 702 is further configured to predict cell contours of the optimized image block using a preset instance segmentation model to obtain a cell instance mask, and extract single-cell instances from the cell instance mask; perform feature fusion on the cell geometric features, color texture features, and spatial location features extracted from each single-cell instance to obtain a cell multimodal feature vector; predict tissue regions of the optimized image block using a general segmentation model to obtain a tissue region mask, and obtain tissue region instances corresponding to the tissue region mask; perform feature fusion on the tissue geometric features, staining texture features, and location context features extracted from each tissue region instance to obtain a tissue multimodal feature vector.

[0134] As one possible implementation, in this embodiment, the graph learning module 703 is further configured to use each single-cell instance as a cell graph node and establish a cell-cell edge set between cell graph nodes that meet a preset interval requirement; use each tissue region instance as a tissue graph node and establish a cell-tissue edge set between cell graph nodes and tissue graph nodes that meet a preset coverage requirement; integrate the cell-cell edge set and the cell-tissue edge set to generate a heterogeneous graph structure, wherein the nodes of the heterogeneous graph structure carry corresponding cell multimodal feature vectors or tissue multimodal feature vectors.

[0135] As one possible implementation, in this embodiment, the identification module 704 is further configured to optimize the multimodal feature vectors of cells in the heterogeneous graph structure through a self-supervised graph attention network to obtain optimized cell feature vectors; the self-supervised graph attention network is obtained after self-supervised pre-training through contrastive learning of node occlusion reconstruction and application of structural perturbation; immune cell classification is performed based on the optimized cell feature vectors to obtain multiple immune cell graph nodes; a preset clustering analysis is performed based on the multiple immune cell graph nodes to output the Sjögren's syndrome status determination result.

[0136] As one possible implementation, in this embodiment, the identification module 704 is further configured to construct an immune cell map node subgraph based on the plurality of immune cell map nodes; perform clustering by running a preset community detection algorithm on the immune cell map node subgraph to obtain immune cell clusters; and output the Sjögren's syndrome status determination result according to the spatial distribution information of the immune cell clusters and the number of immune cells in the immune cell clusters.

[0137] As one possible implementation, in this embodiment, the image preprocessing module 701 is further configured to perform downsampling on the original digital pathological slide image to obtain a low-resolution optimized digital pathological slide image; perform coarse localization of the tissue region on the optimized digital pathological slide image to obtain an initial tissue mask; and map the initial tissue mask back to the coordinate system of the original digital pathological slide image to obtain a target tissue mask.

[0138] The automatic diagnostic device for Sjögren's syndrome in pathological images provided in this application, employing the automatic diagnostic method for Sjögren's syndrome in pathological images described in the above embodiments, can solve the technical problem of low analysis accuracy caused by the difficulty in capturing global relationships and spatial topological relationships between cells in existing technologies. Compared with existing technologies, this device models the spatial relationship between cells and tissues through heterogeneous graph structures, enhances feature expression capabilities by combining a self-supervised graph attention network, and achieves accurate identification of lymphocyte aggregation regions based on community detection, significantly improving the accuracy and interpretability of Sjögren's syndrome diagnosis. Other technical features of this device are the same as those disclosed in the above method embodiments and will not be repeated here.

[0139] This application provides an automatic diagnostic device for pathological image Sjögren's syndrome. The automatic diagnostic device for pathological image Sjögren's syndrome includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the automatic diagnostic method for pathological image Sjögren's syndrome in the above embodiment 1.

[0140] The following is for reference. Figure 8The diagram illustrates a structural schematic suitable for implementing an automated diagnostic device for Sjögren's syndrome in pathological images according to embodiments of this application. The automated diagnostic device for Sjögren's syndrome in pathological images according to embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 8 The illustrated automated diagnostic device for Sjögren's syndrome in pathological images is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0141] like Figure 8 As shown, the automated diagnostic device for Sjögren's syndrome in pathological images may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 1002 or a program loaded from storage device 1003 into random access memory (RAM) 1004. The random access memory 1004 also stores various programs and data required for the operation of the automated diagnostic device for Sjögren's syndrome in pathological images. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows the automated diagnostic device for Sjögren's syndrome in pathological images to communicate wirelessly or wiredly with other devices to exchange data. Although the automated diagnostic device for Sjögren's syndrome in pathological images with various systems is shown in the figure, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems may be implemented alternatively.

[0142] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, an embodiment of this application discloses an automated diagnostic program product for pathological images of Sjögren's syndrome, which includes an automated diagnostic program for pathological images of Sjögren's syndrome carried on a computer-readable medium, the automated diagnostic program for pathological images of Sjögren's syndrome containing program code for performing the methods shown in the flowcharts. In such an embodiment, the automated diagnostic program for pathological images of Sjögren's syndrome can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the automated diagnostic program for pathological images of Sjögren's syndrome is executed by processing device 1001, the functions defined in the methods of the embodiments disclosed in this application are performed.

[0143] The automated diagnostic device for Sjögren's syndrome in pathological images provided in this application, employing the automated diagnostic method for Sjögren's syndrome in pathological images described in the above embodiments, can solve the technical problem of low analysis accuracy caused by the difficulty in capturing global relationships and spatial topological relationships between cells in the prior art. Compared with the prior art, the beneficial effects of the automated diagnostic device for Sjögren's syndrome in pathological images provided in this application are the same as those of the automated diagnostic method for Sjögren's syndrome in pathological images provided in the above embodiments, and other technical features in this automated diagnostic device for Sjögren's syndrome in pathological images are the same as those disclosed in the method of the previous embodiment, and will not be repeated here.

[0144] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0145] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0146] This application provides a storage medium having computer-readable program instructions (i.e., an automatic diagnosis program for pathological images of Sjögren's syndrome) stored thereon, the computer-readable program instructions being used to execute the automatic diagnosis method for pathological images of Sjögren's syndrome in the above embodiments.

[0147] The storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of the storage medium may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0148] The aforementioned storage medium may be included in the automated diagnostic device for pathological image Sjögren's syndrome; or it may exist independently and not be assembled into the automated diagnostic device for pathological image Sjögren's syndrome.

[0149] The aforementioned storage medium carries one or more programs. When the aforementioned one or more programs are executed by the automatic diagnostic device for pathological image desiccant syndrome, the automatic diagnostic device for pathological image desiccant syndrome will automatically diagnose the condition.

[0150] The code for an automated diagnostic program for pathological images of Sjögren's syndrome, used to perform the operations of this application, can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, and C++—as well as conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0151] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and automated diagnostic procedures for Sjögren's syndrome in pathological images according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0152] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0153] The readable storage medium provided in this application is a storage medium storing computer-readable program instructions (i.e., an automatic diagnosis program for pathological images of Sjögren's syndrome) for executing the above-described automatic diagnosis method for pathological images of Sjögren's syndrome. This solves the technical problem in the prior art where low analysis accuracy is caused by the difficulty in capturing global relationships and spatial topological relationships between cells. Compared with the prior art, the beneficial effects of the storage medium provided in this application are the same as those of the automatic diagnosis method for pathological images of Sjögren's syndrome provided in the above embodiments, and will not be repeated here.

[0154] The above are only some embodiments of this application and do not limit the scope of the solution of this application. All equivalent structural transformations made under the technical concept of this application and using the content of this application specification and drawings, or direct / indirect applications in other related technical fields, are included within the protection scope of this application.

Claims

1. An automated diagnostic method for Sjögren's syndrome in pathological images, characterized in that, The method includes: Image preprocessing is performed on the original digital pathology slide images to obtain optimized image patches; Perform cell-tissue collaborative segmentation on the optimized image patch to obtain a set of cell entities and a set of tissue entities; The set of cell entities and the set of tissue entities are mapped into a heterogeneous graph structure, wherein the heterogeneous graph structure has cell graph nodes and tissue graph nodes as vertices and spatial adjacency relationships as edges. The condition of Sjögren's syndrome is determined based on the heterogeneous graph structure.

2. The automatic diagnostic method for Sjögren's syndrome in pathological images as described in claim 1, characterized in that, The cell entity set includes single cell instances and the cell multimodal feature vectors corresponding to the single cell instances; the tissue entity set includes tissue region instances and the tissue multimodal feature vectors corresponding to the tissue region instances. The step of performing cell-tissue collaborative segmentation on the optimized image patch to obtain a set of cell entities and a set of tissue entities includes: The optimized image block is subjected to cell contour prediction by a preset instance segmentation model to obtain a cell instance mask, and single cell instances are extracted from the cell instance mask. The cell geometric features, color texture features, and spatial location features extracted from each single-cell instance are fused to obtain the cell multimodal feature vector; The optimized image patch is subjected to tissue region prediction using a general segmentation model to obtain a tissue region mask, and the tissue region instance corresponding to the tissue region mask is obtained. The tissue geometric features, staining texture features, and location context features extracted from each tissue region instance are fused to obtain the tissue multimodal feature vector.

3. The automatic diagnostic method for Sjögren's syndrome in pathological images as described in claim 2, characterized in that, The step of mapping the set of cell entities and the set of tissue entities into a heterogeneous graph structure includes: Each single-cell instance is used as a cell graph node, and a cell-cell edge set is established between the cell graph nodes that meet the preset interval requirements. Each tissue region instance is used as a tissue graph node, and a cell-tissue edge set is established between the cell graph node and the tissue graph node that meet the preset coverage requirements. Integrate the cell-cell edge set and the cell-tissue edge set to generate a heterogeneous graph structure. The nodes of the heterogeneous graph structure carry the corresponding cell multimodal feature vector or tissue multimodal feature vector.

4. The automatic diagnostic method for Sjögren's syndrome in pathological images as described in claim 3, characterized in that, The step of determining the state of Sjögren's syndrome based on the heterogeneous graph structure includes: The multimodal feature vectors of cells in the heterogeneous graph structure are optimized by a self-supervised graph attention network to obtain optimized cell feature vectors; the self-supervised graph attention network is obtained after self-supervised pre-training through contrastive learning of node occlusion reconstruction and application of structural perturbation. Based on the optimized cell feature vector, immune cell classification is performed to obtain multiple immune cell map nodes; Based on the multiple immune cell map nodes, a preset clustering analysis is performed to output the Sjögren's syndrome status determination result.

5. The automatic diagnostic method for Sjögren's syndrome in pathological images as described in claim 4, characterized in that, The step of performing a preset clustering analysis based on the multiple immune cell map nodes and outputting the Sjögren's syndrome status determination result includes: Construct an immune cell graph node subgraph based on the multiple immune cell graph nodes; A preset community detection algorithm is run on the immune cell graph node subgraph to perform clustering and obtain immune cell clusters; The results of Sjögren's syndrome status determination are output based on the spatial distribution information of the immune cell clusters and the number of immune cells in the immune cell clusters.

6. The automatic diagnostic method for Sjögren's syndrome in pathological images as described in claim 1, characterized in that, The step of preprocessing the original digital pathological slide image to obtain optimized image patches includes: Multi-scale tissue region extraction is performed on the original digital pathological slide image to obtain the target tissue mask; Based on the target tissue mask, the original digital pathological slide image is divided into regions to obtain initial image blocks; The initial image block is subjected to data augmentation processing to obtain an optimized image block.

7. The automatic diagnostic method for Sjögren's syndrome in pathological images as described in claim 6, characterized in that, The step of extracting multi-scale tissue regions from the original digital pathological slide image to obtain the target tissue mask includes: Downsampling is performed on the original digital pathology slide image to obtain a low-resolution optimized digital pathology slide image; The optimized digital pathological slide image is coarsely localized to obtain an initial tissue mask; The initial tissue mask is mapped back to the coordinate system of the original digital pathological slide image to obtain the target tissue mask.

8. An automatic diagnostic device for Sjögren's syndrome in pathological images, characterized in that, The automatic diagnostic device for Sjögren's syndrome in pathological images includes: The image preprocessing module is used to preprocess the original digital pathological slide images to obtain optimized image patches. The image segmentation module is used to perform cell-tissue collaborative segmentation on the optimized image block to obtain a set of cell entities and a set of tissue entities; The graph learning module is used to map the set of cell entities and the set of tissue entities into a heterogeneous graph structure, wherein the heterogeneous graph structure has cell graph nodes and tissue graph nodes as vertices and spatial adjacency relationships as edges. The identification module is used to determine the state of Sjögren's syndrome based on the heterogeneous graph structure.

9. An automated diagnostic device for Sjögren's syndrome in pathological images, characterized in that, The device includes: a memory, a processor, and an automatic diagnostic program for pathological image desiccant syndrome stored in the memory and executable on the processor, the automatic diagnostic program for pathological image desiccant syndrome being configured to implement the steps of the automatic diagnostic method for pathological image desiccant syndrome as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium stores an automatic diagnostic program for pathological image Sjögren's syndrome, which, when executed by a processor, implements the steps of the automatic diagnostic method for pathological image Sjögren's syndrome as described in any one of claims 1 to 7.

Images