A method for classifying and diagnosing diseases of a liver based on a CT image of the liver using a neural network

Abstract

A method based on an r-IBS-introduced neural network for automatically determining the liver's condition from three common diseases—hepatocellular carcinoma (HCC), intrahepatic cholangiocarcinoma (ICC), and fatty liver (FAT)—using liver CT images includes the following steps: First, construct an r-IBS-based training classifier. Use the classifier to determine the disease type for each liver CT image. Acquire liver CT images and process them using a snake model. Input the images into the trained classification neural network model. The neural network outputs the classification results for the aforementioned diseases.

Description

Technical Field

[0001] This invention relates to deep learning, convolutional neural networks, information similarity, statistical analysis theory, category memory, in the classification direction, the K-nearest neighbor algorithm (KNN), information-based similarity (IBS), and applications of contrastive learning. Background Technology

[0002] Liver cancer is a serious health problem worldwide. According to data released by the International Agency for Research on Cancer, there were nearly 2.5 million new cases of liver cancer globally in 2022, accounting for about 12% of all new cancer cases worldwide, ranking relatively high.

[0003] Today, imaging examinations also play a vital role in the diagnosis of liver diseases. Imaging techniques such as ultrasound, CT, and MRI can clearly show changes in the morphology and structure of the liver, helping doctors to accurately diagnose and stage liver diseases. In addition, liver elastography can also determine the extent of liver fibrosis, providing important evidence for early intervention and treatment of liver diseases.

[0004] With the rapid development of artificial intelligence, convolutional neural networks have become increasingly powerful and remain dominant in the field of computer vision, achieving tremendous success. However, liver imaging datasets differ from most commonly used classification datasets, possessing many unique characteristics. They tend to be small in size, contain significant noise, are difficult to interpret, and have uneven data distribution. Therefore, existing network architectures suitable for conventional classification datasets may not be adequate for diagnosing liver diseases and abnormalities.

[0005] Currently, CT imaging is widely used in liver detection. However, medical images place high demands on the network's ability to discover features with high specificity and fine granularity. Therefore, in addition to developing a new algorithm based on the concept of information similarity, which we call r-IBS (reflection IBS), we also implemented KNN online learning for natural animal image classification tasks. Furthermore, we applied the network to medical CT image datasets to explore and validate its classification performance on this task. Traditional manual classification methods are time-consuming and labor-intensive, leading to increasing workload for doctors and failing to meet the needs of large-scale medical imaging applications.

[0006] If we can use computers to automatically classify the types of diseases in the dataset by putting CT images into the network, doctors can perform a little verification without having to put in as much effort as before, which can greatly reduce the workload of medical staff.

[0007] KNN Online Learning: The KNN algorithm can easily handle new data. When new training samples arrive, they only need to be added to the training dataset, without retraining the entire model. This makes the KNN algorithm very suitable for online learning scenarios. When new data arrives, the algorithm can immediately use this new data for prediction without requiring complex adjustments to the model. Unlike many other machine learning algorithms, this means that in online learning environments, when new data arrives, the KNN algorithm can immediately start using this data for prediction without waiting for the model training to complete.

[0008] This invention utilizes a novel neural network model based on liver CT images to classify diseases. The novel network structure mainly comprises four parts: a traditional convolutional neural network feature extraction module, a novel r-IBS (reflection IBS) algorithm module inspired by information similarity, an online KNN classification module, and a loss function. The feature extraction module primarily uses a CNN model. CNN is a machine learning method frequently used in image analysis. Its inventors were initially inspired by the structure of the human visual cortex and it is generally used for image recognition and classification problems, achieving excellent results in these tasks and winning outstanding awards in various competitions. The memory processing module is a similarity-based memory network module. This module mainly simulates the human memory storage and updating mechanism, and is responsible for generating memories, updating memories, and modifying input images during training. The KNN classification module implements online learning of KNN, offering advantages such as rapid adaptation to new data, real-time feedback and adjustment, data stream processing, and high resource utilization. The loss function will be introduced later. Summary of the Invention

[0009] This invention solves the above-mentioned problems: The network model for predicting common liver diseases based on CT images proposed in this invention can help patients reduce the damage to the liver caused by X-ray examinations and reduce the workload of doctors. At the same time, the prediction accuracy of this network model is also excellent.

[0010] This invention solves the above-mentioned problems: The algorithm model proposed in this invention is a novel classification network based on similarity calculation and category memory and an online KNN classifier, which helps to automatically diagnose common liver diseases, greatly reducing the workload of medical staff, while improving the accuracy of predicting common liver diseases, enabling patients to achieve early detection and early treatment, and reducing the greater damage to the liver caused by late detection.

[0011] This invention solves the aforementioned problems: the category memory storage unit and online KNN classifier proposed in this invention can continuously modify the memory based on the input image during training. On the one hand, this method ensures the continuous updating of the category memory, and each memory continuously absorbs features of the specific category image input to the network. On the other hand, by feeding the k largest IBSsub values ​​(k outputs) of the KNN classification into the KNN classifier, and re-weighting the probability values ​​of the three categories according to different categories, the network's prediction accuracy can be continuously improved. This allows the computer to address the problem that less experienced doctors are less accurate in their diagnoses than more experienced doctors.

[0012] This invention addresses the aforementioned problems: the neural network-based algorithm model proposed in this invention utilizes memory to help the network classify CT images. To increase resource utilization, we introduce online KNN learning. This invention solves the efficiency problem; online KNN algorithms typically improve efficiency through optimization strategies, such as maintaining a data structure to quickly find the nearest neighbors. The method described in this invention, based on a neural network, for classifying liver CT images of three common diseases—hepatocellular carcinoma (HCC), intrahepatic cholangiocarcinoma (ICC), and fatty liver (FAT)—comprising the following steps:

[0013] Step 1 involved acquiring CT images of the liver. The study included 163 subjects, comprising 90 cases of hepatocellular carcinoma and 73 cases of intrahepatic cholangiocarcinoma. Furthermore, to accurately classify patients as having liver cancer, 70 cases of fatty liver were also included as a control group alongside the liver cancer cases.

[0014] Step 2: The obtained liver CT images are segmented. Most of the images are used as the network training set, and a small number of images are used as the network validation set. The original CT images in the dataset are whole abdominal images. In order to make the training objective clear, we segment and remove pixels of other parts that are not related to the liver. We use the semi-automatic snake model method combined with the Unet network to automatically segment the data. The initial model used by snake was provided by radiologists from the cooperating hospital.

[0015] Step 3: Before the images are input into the network, they need to be uniformly adjusted to ensure that the input size is correct. Of course, they also need to be standardized according to the overall mean and standard deviation of the dataset, transforming the image data to the range [0, 1], transforming the mean of the data distribution to 0, and the standard deviation to 1, so that the data conforms to the standard normal distribution. This can speed up the convergence of the model and improve the training accuracy.

[0016] Step 4: Establish a CT image feature extraction module based on convolutional neural network. Input the training set images of liver CT into the model, fix the output dimension of the feature extraction network to H (default H=2048), output the features extracted from the image, and pass them to the next part of the network.

[0017] Step 5: Establish an r-IBS-based network classifier module (B*C*M). The number of r-IBS similarity calculation modules corresponds to the number of classes in the network classification module. In this invention, we take three disease categories as an example, therefore we consider there to be three r-IBS similarity calculation modules. B is the batch size, M is the size of each memory set (memory dimension) (default value = 16), and C is the number of categories we want to classify; in this task, we consider C to be 3. Perform r-IBS similarity calculations on the extracted image feature data module and the pre-stored memory module based on information similarity heuristics to help update and modify the memory.

[0018] Step 6: Construct a classifier to determine the category of liver disease in the input image. The classifier is composed of the IBSsub value and KNN classification combination.

[0019] Step 7: Complete the training and fine-tuning of the overall module, and evaluate it on the test set. After the test is passed, apply the network module to more predictive diagnoses of liver CT images.

[0020] Furthermore, in step 5, the r-IBS module can be subdivided into several steps: x and y represent two systems. In this network, x represents the features extracted from the input image, typically 2048 dimensions, and y represents the initialization memory stored in the network itself. Then, x and y are mapped and encoded using the same encoder, and the encoded data is mapped to fixed-dimensional channels. The values ​​on these channels represent the performance of the input in that channel. The similarity and dissimilarity of the two systems are measured by the energy magnitude and direction of the channels on the mapping plane. We believe that if two systems are completely similar, the energy values ​​of their projected channels should be exactly the same, and the projection feature directions should be identical. When the directions are completely opposite, and the sum of the absolute values ​​of the energy values ​​is higher, the two systems are more dissimilar.

[0021] Rx = Encoder(x)

[0022] Ry = Encoder(y)

[0023] The specific calculation process for r-IBS is shown in the following formula:

[0024]

[0025]

[0026] Px_log=-Px·logPx

[0027] Py_log = -Py·logPy

[0028]

[0029] IBSsub=∑|Rx-Ry|·Fk

[0030] IBSplus=Σ|Rx+Ry|·Fk

[0031] Where R represents the projection vector obtained after the input is processed by the unified encoder. The sign of R indicates the feature direction on a certain plane after all features are projected onto it. If they belong to the same class, their directions should be consistent; if they belong to different classes, they should be opposite. The absolute value of R represents the energy magnitude. P represents the ratio of the energy magnitudes of each channel of the projection vector, thus characterizing the information density of the projection vector. Fk is the ratio of the sum of the information entropies of the two projection vectors across each channel. The smaller the information entropy, the more stable the system, and the lower the uncertainty of the channel values ​​after projection. Finally, we subtract the R values ​​of the two systems to calculate IBSsub, which represents the similarity between the two systems, and add the R values ​​of the two systems to calculate IBSplus, which represents the dissimilarity between the two systems. In practical applications, we use IBSsub as the similarity value r-NorIBS, and normalize IBSsub using the dissimilarity value IBSplus to obtain r-IBS. The value range of r-IBS is 0 to 1.

[0032] The loss function mainly consists of two parts: Losssoftmax and LossIBS, where LossIBS is:

[0033]

[0034] Loss = Losssoftmax + LossIBS

[0035] The overall loss function consists of cross-entropy loss and IBS loss. The IBS loss is designed to constrain the memory maintained by each IGM unit to the image of its corresponding class, ensuring that the memory maintained by each IGM is of the same class as itself and that crosstalk does not occur. Losssoftmax is the traditional classification loss. Although the two losses have different meanings, they are essentially the same in form after using r-IBS as the output probability.

[0036] Step 6, further examining the similarity calculation process. After obtaining the IBSsub similarity, we borrow the idea of ​​the K-Nearest Neighbors algorithm to select the k IBSsub values ​​with the largest values. The choice of k is crucial. Below, we use a pseudocode example to illustrate the idea of ​​KNN classification:

[0037]

[0038] Compared with the original technology, the beneficial effects of this invention are as follows: by extracting image features through convolutional neural networks, introducing category memory and online learning KNN algorithm, the network is helped to extract image features from CT datasets and perform classification prediction. This solves the problems of small CT image data volume, high noise, and image feature extraction, and can efficiently predict diseases. With experienced doctors ensuring the dataset meets the standards, training the network can achieve excellent classification results, solving the problems of heavy workload, slow speed, and inconsistent accuracy in diagnosis for doctors.

[0039] Now, when a new data point needs to be classified, the model calculates the distance between this data point and all samples in the window, and selects the K closest samples (K is another preset constant). Then, the model looks at the categories of these K samples and selects the category that appears most frequently as the predicted category for the new data point, thus greatly improving efficiency. Attached Figure Description

[0040] In order to more clearly and understandably illustrate the technical solutions in the embodiments of the present invention or the prior art, I will now briefly introduce the accompanying drawings used in the description of the embodiments or the prior art.

[0041] Figure 1 The overall model architecture diagram of the neural network for the KNN classification algorithm based on memory networks provided in this embodiment of the invention;

[0042] Figure 2 This is a schematic diagram of the r-IBS calculation process; Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described clearly and in detail below with reference to the accompanying drawings and specific examples.

[0044] The solution adopted by this invention to solve the problem consists of four parts:

[0045] The first part involves constructing a classifier to determine the type of liver disease in each CT image: hepatocellular carcinoma (HCC), intrahepatic cholangiocarcinoma (ICC), or fatty liver disease (FAT). Liver CT images, segmented and confirmed by professional radiologists, are labeled with the disease type. Each liver CT image is standardized, with uniform image size and grayscale range. This processed liver CT image dataset is used as input to a neural network training model, and the corresponding labels are used as the output. The model is then trained as follows: [Diagram of the neural network model is shown below]. Figure 1 As shown, the trained neural network model with high classification accuracy is used as a classifier to determine the type of liver disease in a patient's CT scan, and it is deployed on a computer host or server.

[0046] The second part involves using a standard dose of CT to scan the liver area, acquiring complete CT images of the liver from the patient being examined, and having a professional radiologist segment the CT images to include only the liver area. The extracted images are then used for training, employing image preprocessing methods similar to those used in the training set, namely, completing and scaling the CT images to standardize their size and grayscale range.

[0047] The third part uses each liver CT image as input to the neural network, and uses the network's output as the judgment result for which type of disease the CT image belongs to.

[0048] It should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This way of describing the specification is only to describe an example more clearly. Those skilled in the art should regard the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

[0049] The detailed descriptions of the ideas and methods listed above are merely specific examples of feasible implementations of the present invention and are not intended to limit the scope of protection of the present invention. The scope of protection of the present invention should include this innovative approach of solving problems using the methods and network structures of the present invention. All equivalent implementations or modifications made without departing from the spirit of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for classifying a CT image of a liver based on a neural network, the method comprising: inputting the CT image into a neural network; and outputting a result of the classifying the CT image based on the neural network. The method comprises the following steps: Step 1, collect the CT image of the abdomen, and ask the radiologist to segment the CT image of the liver part, and then another doctor confirms the accuracy of the segmentation, so as to obtain a reliable CT image of the liver; Step 2, divide the obtained CT image of the liver, most of the images as a network training set, and a small part of the images as a network verification set; Step 3, pre-process the training set and the verification set; Step 4, establish a liver CT image feature extraction module based on a convolutional neural network; Step 5, establish a network classifier module B*C*M based on r-IBS, wherein B is the batch size, M is the size of each memory set, and C is the number of expected categories; the network classifier module has several r-IBS similarity calculation modules corresponding to the number of categories; x and y represent two systems, x represents the features extracted from the input image, which is generally 2048-dimensional in the network, and y represents the initialized memory stored in the network itself; then x and y are mapped and translated by using the same Encoder, and are projected onto a fixed-dimensional channel after translation; the value on the channel represents the performance of the input on the channel; the similarity and dissimilarity are measured by measuring the energy size and direction of the channel on the mapping plane; if the two systems are completely similar, the energy value of each channel after projection should be completely consistent, and the projection feature direction should be completely the same; when the directions are completely opposite, and the absolute value of the energy value is higher, the two systems are more dissimilar: Rx=Encoder(x) Ry=Encoder(y) The specific r-IBS calculation process is shown in the following formula: Px_log=-px logpx Py_log=-py logpy IBSsub = ∑ |Rx - Ry| * Fk IBSplus = ∑ |Rx+Ry| * Fk Wherein R represents the projection vector obtained after the input is uniformly encoded, the positive and negative of R represent the feature direction of the projection vector on the plane after all features are projected onto the plane, if they belong to the same class, the directions should be consistent, if they belong to different classes, the directions should be opposite, the absolute value of R represents the energy size, P represents the energy size ratio of each channel of the projection vector, which represents the information density of the projection vector, Fk is the ratio of the sum of information entropy of the two projection vectors on each channel, the smaller the information entropy, the more stable the system, and the lower the uncertainty of the channel value after projection, finally, the R values of the two systems are subtracted to calculate IBssub, which represents the similarity between the two systems, and the R values of the two systems are added to calculate IBSplus, which represents the dissimilarity between the two systems, IBSsub is used as the similarity value r-NorIBS, and IBSplus is used to normalize IBSsub to obtain rIBS, the value range of rIBS is 0 to 1; Wherein the loss function mainly comprises two parts, Losssoftmax and LossIBS, wherein LossIBS is: Loss=Losssoftmax+LossIBS Step 6, the classifier is constructed to determine the category of liver disease of the input picture, and the classifier is composed of the value of IBSsub and KNN classification; Step 7, the total loss function is used to complete the training and fine-tuning of the whole module, and the evaluation test is carried out on the test set, and after the test is qualified, the whole module is applied to more prediction and diagnosis of liver CT pictures.

2. The method for disease classification and diagnosis of liver CT images based on neural networks according to claim 1, characterized in that... In step 1, the liver CT pictures are collected, 163 research objects include 90 cases of hepatocellular carcinoma and 73 cases of intrahepatic cholangiocarcinoma, and 70 cases of fatty liver are set as a control group for liver cancer cases. 3.The method of claim 1, wherein the method is based on a neural network for classifying a disease of a liver from a CT image. In step 2, the obtained liver CT pictures are divided, most of the pictures are used as network training set, and a small part of the pictures are used as network verification set, and the original CT image in the data set is the whole abdominal image. 4.The method of claim 1, wherein the method is based on a neural network for classifying a disease of a liver from a CT image, characterized in that In step 3, in order to make the training target clear, the pixels of other parts irrelevant to the liver are segmented and removed, and the semi-automatic snake model method is used in combination with Unet network to automatically segment the data.

5. The method of claim 1, wherein the method is based on a neural network for classifying a liver disease from a CT image of the liver. In step 4, the liver CT picture feature extraction module based on convolutional neural network is established, the training set of liver CT pictures is input into the CT picture feature extraction module, the output dimension of the fixed feature extraction network is H, H = 2048, the extracted features of the output picture are transmitted to the next network.

6. The method of claim 1, wherein the method is based on a neural network for classifying a liver disease from a CT image of the liver. In step 6, the classifier is constructed to determine the category of liver disease of the input picture, and the value of IBSsub is selected based on the idea of K nearest neighbor algorithm.

7. The method of claim 1, wherein the method is based on a neural network for classifying a liver disease from a CT image of the liver. The total loss function in step 7 is cross entropy loss Losssoftmax and IBS loss, wherein the IBS loss is to make each IGM unit maintain the memory and the corresponding image of its own category to produce constraint, so that the memory maintained by each IGM is of the same category as itself, and cross talk does not occur.

Images