A thyroid pathological image auxiliary analysis system

Abstract

The present application belongs to the technical field of medical image analysis, and particularly relates to a thyroid cell pathology image auxiliary analysis system. The system of the present application comprises an input module for inputting a thyroid cell pathology section image; a cell detection module for detecting a target cell through a neural network model; a semantic classification module for performing semantic classification on the target cell to obtain a classification label; and a knowledge graph search module for searching in a knowledge graph according to the classification label and performing textual interpretation on an obtained diagnosis result. The system of the present application can perform high-accuracy auxiliary diagnosis on thyroid papillary carcinoma. Meanwhile, the diagnosis result can provide a diagnosis basis and reference literature according to the knowledge graph, so that the AI auxiliary pathology diagnosis has public credibility. Therefore, the present application has a good application prospect.

Description

Technical Field

[0001] This invention belongs to the field of medical image analysis technology, specifically relating to a thyroid pathology image-assisted analysis system. Background Technology

[0002] In recent years, the incidence of various malignant tumors has been increasing year by year worldwide, with the incidence of thyroid malignant tumors showing a particularly significant increase. The detection rate of thyroid nodules in the physical examination population is 24.4%-33.22%. Fine needle aspiration cytology (FNAC) under ultrasound guidance is recognized as the most specific and accurate diagnostic method for thyroid nodules. Due to its advantages such as minimal invasiveness, cost-effectiveness, and convenience, it has gradually become the main preoperative examination method for thyroid nodules in clinical practice.

[0003] In the current field of pathology, pathological diagnosis has begun to deeply integrate with artificial intelligence (AI) technology. The aim is to leverage AI's powerful learning and computational capabilities to reduce the repetitive workload and subjective judgment of pathologists. Previous research and most pathological AI have employed deep learning-based convolutional neural networks (CNNs) for image analysis, with numerous studies reported in the field of thyroid FNAC. CNNs excel at image processing, but their drawback lies in the "black box" effect, making it difficult to summarize observed patterns. This is a significant shortcoming of "second-generation AI," leading to difficulties for both pathologists and patients in fully trusting AI-generated diagnoses. Therefore, it is necessary to address the issue of uninterpretable AI diagnostic results. Consequently, selecting appropriate model algorithms and constructing a thyroid pathology image-assisted analysis system with good interpretability and high predictive performance remains a pressing problem in this field.

[0004] GNN is a graph data structure, and GNN technology is a technique for running neural network algorithms directly on graph data structures. Knowledge graphs, a concept proposed by Google in 2012, are essentially knowledge bases of semantic networks, and can also be understood as multi-relation graphs composed of nodes and edges. However, there are currently no reports on the application of GNNs and knowledge graphs in thyroid pathology image analysis. Summary of the Invention

[0005] To address the problems of existing technologies, this invention provides a thyroid pathology image-assisted analysis system.

[0006] A thyroid pathology image-assisted analysis system, comprising:

[0007] The input module is used to input images of thyroid cell pathology slides;

[0008] The cell detection module is used to detect target cells using a neural network model;

[0009] The semantic classification module is used to perform semantic classification on target cells and obtain classification labels;

[0010] The knowledge graph search module is used to search the knowledge graph based on category tags and provide textual interpretations of the diagnostic results.

[0011] Preferably, the neural network model is selected from the Cascade-R-CNN model.

[0012] Preferably, the target cell is a papillary thyroid carcinoma cell, a multinucleated macrophage, a normal thyroid cell, or other background cells.

[0013] Preferably, the semantic classification is implemented using the GINet graph reasoning method based on semantic features.

[0014] Preferably, the classification label includes at least one of the following labels: ground-glass nucleus, nuclear groove, intranuclear pseudo-inclusion body, multinucleated macrophage.

[0015] Preferably, the method for constructing the knowledge graph includes:

[0016] Step 1: Search for literature related to thyroid pathology images;

[0017] Step 2: Extract entities and relationships from the documents;

[0018] Step 3: Perform knowledge fusion to solve entity disambiguation and coreference disambiguation problems for equivalent instances, equivalent classes / subclasses, and equivalent attributes / sub-attributes;

[0019] Step 4: Store the data using the Neo4j graph database.

[0020] Preferably, in step 2, the spaCy library of PyTorch is used for entity extraction and relation extraction.

[0021] Preferably, in step 4, the storage structure consists of four elements: tags, nodes, relationships, and attributes.

[0022] The present invention also provides a computer-readable storage medium having stored thereon a computer program for implementing the above-described thyroid pathology image-assisted analysis system.

[0023] This invention constructs a system based on a neural network model, a graph neural network, and a knowledge graph, and further optimizes the algorithm selection for each model. Specifically, the neural network model segments thyroid pathological images into small image blocks containing target cells; furthermore, based on the graph neural network, it can not only accurately identify target cells, but also, compared with other AI-assisted pathological diagnosis studies, classify the identified tumor cells by karyotype using GINet, and then use the knowledge graph to provide textual explanations of the classification results. Therefore, the system of this invention can not only diagnose the pathological classification of papillary thyroid carcinoma (PTC), but also provide diagnostic evidence and references, giving credibility to AI-assisted pathological diagnosis. Thus, this invention has excellent application prospects.

[0024] Obviously, based on the above description of the present invention, and according to common technical knowledge and conventional methods in the field, various other modifications, substitutions, or alterations can be made without departing from the basic technical concept of the present invention.

[0025] The following detailed embodiments further illustrate the above-described content of the present invention. However, this should not be construed as limiting the scope of the present invention to the following examples. All technologies implemented based on the above-described content of the present invention fall within the scope of the present invention. Attached Figure Description

[0026] Figure 1 The loss function curve of the Casdade-R-CNN model in Example 1;

[0027] Figure 2 The results are from the semantic loss weight ablation experiment in Example 1;

[0028] Figure 3 The training / loss curves for the semantic classification model in Example 1 are shown. Detailed Implementation

[0029] It should be noted that the algorithms for data acquisition, transmission, storage and processing steps not specifically described in the embodiments, as well as the hardware structures and circuit connections not specifically described, can all be implemented using content already disclosed in the prior art.

[0030] Example 1: Thyroid Pathology Image-Assisted Analysis System

[0031] This embodiment provides a thyroid pathology image-assisted analysis system, including:

[0032] The input module is used to input images of thyroid cell pathology slides;

[0033] The cell detection module is used to detect target cells using the Cascade-R-CNN model;

[0034] The semantic classification module uses the GINet graph reasoning method based on semantic features to perform semantic classification of target cells and obtain classification labels.

[0035] The knowledge graph search module is used to search the knowledge graph based on category tags and provide textual interpretations of the diagnostic results.

[0036] The target cells are papillary thyroid carcinoma cells, multinucleated macrophages, normal thyroid cells, or other background cells. The classification labels include at least one of the following labels: ground-glass nucleus, nuclear groove, intranuclear pseudoinclusion body, and multinucleated macrophage.

[0037] The training of the above model and the construction of the knowledge graph are carried out according to the following steps:

[0038] Step A, Nuclear Labeling: According to the Bthesda Reporting System for Thyroid Cytopathology, the main nuclear characteristics of papillary thyroid carcinoma include: enlarged nuclei, crowded nuclei, overlapping nuclei, nuclear atypia, ground-glass nuclei, nuclear grooves, and intranuclear pseudo-inclusion bodies; background components include multinucleated macrophages. In QuPath 3.0 software, labels for ground-glass nuclei, nuclear grooves, intranuclear pseudo-inclusion bodies, and multinucleated macrophages were added to label the nuclei with the corresponding characteristics. Additionally, a "Tumour" label was set to label tumor cells exhibiting only nuclear enlargement or nuclear atypia.

[0039] Step B, Literature Annotation: Enter keywords such as "thyroid," "papillary carcinoma," "cytology," and "pathology" into a search engine and use "and" or "or" combinations to search for literature. Literature inclusion criteria: The literature content must contain direct or indirect descriptions of the pathological morphology of papillary carcinoma cells, such as nuclear crowding, nuclear overlap, nuclear enlargement, nuclear pleomorphism, ground-glass nuclei, nuclear grooves, intranuclear pseudo-inclusion bodies, background cells, and multinucleated macrophages. Semantic types should include: tumor cells, nuclear crowding, nuclear overlap, nuclear enlargement, nuclear pleomorphism, ground-glass nuclei, nuclear grooves, intranuclear pseudo-inclusion bodies, background cells, and multinucleated macrophages; semantic relationships should include: nuclear characteristics, arrangement, and belonging to. After annotating according to the defined semantic types, associate the annotated text content.

[0040] Step C: The computer reads in the digitized slice image file that has been labeled and classified cytologically, and trains the Cascade-R-CNN model and the graph inference method GINet.

[0041] In this embodiment, the labeled training set data is imported into the Cascade-R-CNN model for training. We obtained mAP values ​​of 0.89 and 0.73 for the training set and 0.87 and 0.69 for the validation set at IoU of 0.5 and 0.75, respectively (Table 1). The test loss and training loss decrease with increasing iterations. Figure 1 The curve indicates that the model training effect is good.

[0042] Table 1. mAP values ​​of the Casdade-R-CNN model on the training and validation sets.

[0043]

[0044] *Note: mAP@.5 / .75 is the average mAP value for IoU = 0.5 / 0.75.

[0045] The training set was imported into the GINet model for image classification training, and mIoU and Dice were used to evaluate the model. To investigate the necessity and effectiveness of semantic loss, the weight hyperparameters λ = {0.2, 0.4, 0.6, 0.8, 1.0} were set according to general standards for training. Figure 2 The results show that the best training results are achieved when λ is 0.2, with mIoU of 0.5614 and Dice of 0.6470. Furthermore, during training, the training loss shows a continuous decreasing trend with increasing iterations, indicating that the model training effect is good. Figure 3 ).

[0046] Step D, Knowledge Graph Construction: Data is primarily sourced from relevant literature, and entity and relation extraction is performed using the PyTorch spaCy library. After knowledge acquisition, knowledge fusion is used to resolve entity disambiguation and coreference disambiguation issues related to equivalent instances, equivalence classes / subclasses, and equivalent attributes / subattributes. Finally, the Neo4j graph database is used for storage, with a storage structure consisting of four elements: labels, nodes, relations, and attributes.

[0047] The trained Cascade-R-CNN model, GINet model, and constructed knowledge graph information were imported into CDSS software. The internal linking module of CDSS software performed joint analysis on the data from each step, diagnosed it as PTC, and was able to determine the relevant semantic type and literature description, with a consistency rate of 88.84%.

[0048] The technical solution of the present invention will be further illustrated by the following experiments.

[0049] Example 1: Optimization of the Target Cell Detection Model

[0050] I. Experimental Methods

[0051] From data collected from hospitals, nearly 1,000 thyroid cell samples were labeled using segmentation model pre-labeling and manual inspection and correction, and the training set and test set were divided in an 8:2 ratio.

[0052] Using the dataset described above, we trained the Cascade-R-CNN model and other comparative models. During training, all variables except the model and pre- and post-processing variables remained unchanged, and random factors such as the model's parameter initialization scheme were kept constant.

[0053] II. Experimental Results

[0054] The performance of the Cascade-R-CNN model and other comparative models is shown in Table 2.

[0055]

[0056] As can be seen from the data in the table above, among various existing neural network models, the Cascade-R-CNN model has the best predictive performance for the target cell detection task of this invention.

[0057] As can be seen from the above embodiments and experimental examples, this invention provides a system that combines a target cell detection model, a target cell semantic classification model, and a knowledge graph, enabling highly accurate auxiliary diagnosis of papillary thyroid carcinoma. Furthermore, the diagnostic results can provide diagnostic evidence and references based on the knowledge graph, thus enhancing the credibility of AI-assisted pathological diagnosis. Therefore, this invention has excellent application prospects.

Claims

1. A thyroid pathology image-assisted analysis system, characterized in that, include: The input module is used to input images of thyroid cell pathology slides; The cell detection module is used to detect target cells through a neural network model; the neural network model is selected from the Cascade-R-CNN model; the target cells are papillary thyroid carcinoma cells, multinucleated macrophages, normal thyroid cells, or other background cells. The semantic classification module is used to perform semantic classification on target cells and obtain classification labels; the semantic classification is implemented using the GINet graph reasoning method based on semantic features. The knowledge graph search module is used to search the knowledge graph based on category tags and provide textual interpretation of the diagnostic results. The methods for constructing the knowledge graph include: Step 1: Search for literature related to thyroid pathology images; Step 2: Extract entities and relationships from the documents; Step 3: Perform knowledge fusion to solve entity disambiguation and coreference disambiguation problems for equivalent instances, equivalent classes / subclasses, and equivalent attributes / sub-attributes; Step 4: Store the data using the Neo4j graph database; In step 2, the spaCy library of PyTorch is used for entity extraction and relation extraction; in step 4, the storage structure consists of four elements: labels, nodes, relations, and attributes.

2. The thyroid pathology image-assisted analysis system according to claim 1, characterized in that: The classification labels include at least one of the following labels: ground-glass nucleus, nuclear groove, intranuclear pseudo-inclusion body, multinucleated macrophage.

3. A computer-readable storage medium, characterized in that, It stores a computer program for implementing the thyroid pathology image-assisted analysis system as described in claim 1 or 2.

Images