Automatic liver tumor identification and measurement method based on CT (Computed Tomography) image

Abstract

The invention discloses a liver tumor automatic identification and measurement method based on a CT (Computed Tomography) image, which belongs to the technical field of medical image processing, and comprises the following steps of: firstly, preprocessing the CT image of the abdomen of a patient, and inputting the preprocessed image into a liver component decomposition network to obtain decomposed liver component images I1-I4; inputting the liver component images I1-I4 into the liver tumor segmentation network, and outputting a tumor segmentation result; and finally, inputting a segmentation result into a tumor measurement and analysis module for tumor analysis, and calculating detailed information of the tumor. By means of the mode, the problems that a traditional method is poor in applicability and robustness, result accuracy is difficult to guarantee, and model interpretability is lacked are solved. The method is high in applicability and robustness and high in accuracy, and the model has interpretability.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a method for automatically identifying and measuring liver tumors based on CT images. Background Art

[0002] In liver tumor surgery, tumor identification, segmentation, and measurement based on preoperative CT images are crucial and directly determine the surgical plan. However, manual tumor labeling, including tumor identification, segmentation, and measurement, requires highly specialized medical expertise and is a laborious, repetitive task, hindering the high-quality and efficient performance of liver tumor surgery. Therefore, developing automated methods for liver tumor identification, segmentation, and measurement is highly valuable.

[0003] Currently, commonly used methods for liver tumor recognition and segmentation include: threshold segmentation, region growing, edge detection, machine learning methods such as support vector machines, and deep learning methods represented by convolutional neural networks. Deep learning methods are currently the most mainstream and efficient methods. However, conventional end-to-end deep learning methods have some defects and shortcomings in automatically identifying liver tumors for the following reasons: 1. Medical imaging features are complex. Tumor morphology is diverse, presenting different shapes, with blurred boundaries and poor contrast. Liver tumors also have a wide range of density (including low density, equal density, or high density) and are highly variable (different density areas may exist within a tumor, usually caused by factors such as bleeding, necrosis, and calcification within the tumor).

[0004] 2. Noise and artifacts in CT images interfere with tumor identification, such as quantum noise, motion artifacts, metal artifacts, etc.

[0005] 3. Individual differences. There are certain differences in the liver anatomical structure of different patients, including the size, shape, position and blood vessel distribution of the liver. This makes some liver variations or deformities, vascular bifurcations, swelling, etc. difficult to accurately identify tumors.

[0006] 4. Liver tumor annotated data is scarce. Manually annotating liver tumors is difficult and labor-intensive, so high-quality annotated data is very scarce.

[0007] Therefore, traditional deep learning methods that are completely driven by labeled data will have problems with poor applicability and robustness in liver tumor recognition and segmentation tasks, making it difficult to guarantee the accuracy of the results, and also lacking model interpretability.

[0008] To solve the above problems, the present invention designs an automatic liver tumor recognition and measurement method based on CT images, which improves the robustness, accuracy and interpretability of the model by introducing a liver component knowledge graph and a cascade network model. Summary of the Invention

[0009] To address the problems of poor applicability and robustness of traditional methods, difficulty in ensuring the accuracy of results, and lack of model interpretability, the present invention provides a method for automatic liver tumor recognition and measurement based on CT images. The present invention first decomposes the patient's abdominal CT image into liver components, decomposing multiple liver component maps, including: a complete liver map, a normal liver tissue map, a high-density tissue map, and a low-density tissue map; all component maps are then input into a liver tumor segmentation network for tumor recognition and segmentation, and finally achieve tumor measurement and analysis.

[0010] To achieve the above objectives, the present invention is implemented through the following technical solutions: A method for automatically identifying and measuring liver tumors based on CT images comprises the following steps: first, after image preprocessing, a patient's abdominal CT image is input into a liver component decomposition network. The liver component decomposition network effectively decomposes the liver image into components, effectively separating normal liver tissue from abnormal liver tissue, and obtaining decomposed liver component images I1-I4. The liver component images I1-I4 are then input into a liver tumor segmentation network. The liver tumor segmentation network extracts and analyzes features of the input liver component images I1-I4, segments the tumor area in the liver, and outputs the tumor segmentation result. Finally, the segmentation result is input into a tumor measurement and analysis module for tumor analysis to calculate detailed information about the tumor.

[0011] Furthermore, it includes the model training phase and the model inference phase; Among them, the model training stage is used to train the liver component decomposition network and the liver tumor segmentation network; The model inference phase performs the following steps: First, the patient's abdominal CT image is preprocessed and then input into the liver component decomposition network to obtain the decomposed liver component images I1-I4. Then, the liver component images I1-I4 are input into the liver tumor segmentation network to segment the liver tumor and output the tumor segmentation results. Finally, the segmentation results are input into the tumor measurement and analysis module for tumor analysis and calculation of detailed tumor information.

[0012] Furthermore, the detailed information of the tumor includes the major diameter, minor diameter, average radius, average density, maximum density, minimum density, maximum cross-sectional area, total volume and total mass of each tumor.

[0013] Furthermore, the model inference phase includes four steps: 1.1、Data preprocessing; 1.2. Liver component decomposition: Input the abdominal CT image into the liver component decomposition network to obtain liver component images I1-I4; 1.3. Tumor segmentation: Input the liver component images I1-I4 into the liver tumor segmentation network to obtain the tumor segmentation results; 1.4. Tumor measurement and analysis: The tumor measurement and analysis module measures and analyzes the tumor segmentation results and outputs detailed information about the tumor.

[0014] Furthermore, step 1.1, data preprocessing, specifically the following steps: the patient's abdominal CT image will be pre-parsed from the DICOM format data and normalized, specifically by resampling the image to a uniform resolution, which is the median resolution of all image resolution distributions, and normalizing the image value range to [0, 1].

[0015] Furthermore, the methods used in the tumor measurement and analysis module include: three-dimensional connected domain detection, internal cavity filling, major and minor diameter calculation, area and volume calculation, and density and mass calculation.

[0016] Furthermore, the model training phase includes four steps: 2.1. Data preprocessing: Preprocess the patient's abdominal CT images; 2.2, data augmentation; 2.3 Model training The preprocessed abdominal CT image is input and sequentially input into the liver component decomposition network and the liver tumor segmentation network. The liver component decomposition network and the liver tumor segmentation network are then forward-derived and the network weight gradients are calculated to obtain the output of the liver component decomposition network, i.e., the liver component images I1-I4, and the output of the liver tumor segmentation network, i.e., the segmented tumor mask P. During the gradient backpropagation and weight update phase, the liver component decomposition network and the liver tumor segmentation network are performed alternately: First, the weights of the liver component decomposition network are frozen, and the segmentation loss value L2 between the segmented tumor mask P and the labeled tumor gold standard T is calculated. Then, the liver tumor segmentation network is gradient backpropagated and the network weights are updated. Next, the weights of the liver component decomposition network are unfrozen, and the weights of the liver tumor segmentation network are frozen. The mutual information loss is calculated using the liver component images I1-I4 and the input abdominal CT images, and the loss value L1 is accumulated. The similarity SSIM loss value L0 is then calculated for the liver component images I1-I4 and the manually decomposed liver component images J1-J4. Finally, the L1 and L0 loss values are added to the previously calculated segmentation loss value L2 to obtain the final loss value L3. The liver component decomposition network is then gradient-backed and the network weights are updated. After the liver component decomposition network and the liver tumor segmentation network have completed gradient backpropagation and network weight update, they enter the next training cycle. Each training cycle samples different data for training. 2.4. Model Validation: After training for a specified number of rounds, use the validation data to test the effectiveness of the current model weights, and select the model weights with the best performance to save.

[0017] Furthermore, step 2.2, data augmentation, specifically includes the following steps: performing random image flipping, random image translation, random image rotation, random image elastic deformation, and random image contrast adjustment on the pre-processed abdominal CT image.

[0018] Furthermore, the segmentation loss value L2 = Dice loss value + CE cross entropy loss value.

[0019] Furthermore, the main body of the liver component decomposition network is a 3D UNet network. The abdominal CT image is input, and the deep features are extracted by the 3D UNet network and decoded into a component weight map. The component weight map has the same size as the input abdominal CT image and is converted into a component weight probability map through a softmax activation function. Finally, the input abdominal CT image is multiplied by each weight probability map respectively to obtain the decomposed liver component images I1-I4. The backbone of the liver tumor segmentation network is a UNet network structure. The liver component images I1-I4 are input, downsampled and encoded three times in sequence, and finally decoded three times. In the middle, cross-layer connections are used to connect the same-scale features. The output of the final decoding layer is then activated by a sigmoid function to obtain the final output tumor segmentation result, i.e., the segmented tumor mask P.

[0020] Compared with existing technologies, this invention offers the following advantages: 1) It is highly robust. Compared with traditional standalone liver tumor segmentation networks, this invention introduces a liver component decomposition network before the segmentation network, effectively filtering out other useless information and achieving higher learning efficiency and accuracy.

[0021] 2) The present invention has strong applicability. Compared with traditional single liver tumor segmentation networks, the liver component decomposition network of the present invention outputs normalized image components, making the input domain of the liver tumor segmentation network simple and single, making it applicable to CT images of various image qualities.

[0022] 3) High accuracy. This invention fully incorporates prior information on liver tumor components, including liver tissue extent, normal liver tissue composition, high-density components, and low-density components. This improves the effectiveness of model feature engineering and makes the output more accurate.

[0023] 4) The model of the present invention is interpretable. The liver component images I1-I4 of the liver component decomposition network of the present invention are intuitively interpretable. Liver component images I1-I4 sequentially reflect the tissue range of the liver, normal liver tissue, high-density liver tissue, and low-density liver tissue. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive effort.

[0025] Figure 1 This is a flow chart of the method for automatically identifying and measuring liver tumors based on CT images of the present invention; Figure 2 This is a schematic diagram of the model training phase in the present invention; Figure 3 Schematic diagram of the liver component decomposition network in the present invention; Figure 4 Schematic diagram of the structure of the liver tumor segmentation network in the present invention. DETAILED DESCRIPTION

[0026] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0027] Example 1: In some embodiments, please refer to the accompanying drawings of the specification. Figure 1-2, a method for automatic recognition and measurement of liver tumors based on CT images, including a model training stage and a model inference stage; The model inference phase performs the following steps: First, after image preprocessing, the patient's abdominal CT image is input into the liver component decomposition network to obtain the decomposed liver component images I1-I4. Then, the liver component images I1-I4 are input into the liver tumor segmentation network to segment the liver tumor, and the tumor segmentation results are output. Finally, the segmentation results are input into the tumor measurement and analysis module for tumor analysis, and detailed information of the tumor is calculated, including the major diameter, minor diameter, average radius, average density, maximum density, minimum density, maximum cross-sectional area, total volume, and total mass of all tumors.

[0028] The model inference phase consists of four steps: 1.1 Data Preprocessing: The patient's abdominal CT images are pre-parsed from DICOM format data and normalized. Specifically, the images are resampled to a uniform resolution, which is the median resolution of all image resolution distributions, and the image range is normalized to [0, 1].

[0029] 1.2. Liver component decomposition: Input the abdominal CT image into the liver component decomposition network to obtain liver component images I1-I4; 1.3. Tumor segmentation: Input the liver component images I1-I4 into the liver tumor segmentation network to obtain the tumor segmentation results; 1.4. Tumor Measurement and Analysis: The tumor measurement and analysis module measures and analyzes the tumor segmentation results and outputs the major diameter, minor diameter, average radius, average density, maximum density, minimum density, maximum cross-sectional area, total volume, and total mass of all tumors.

[0030] Among them, the methods used in the tumor measurement and analysis module include: three-dimensional connected domain detection, internal cavity filling, major and minor diameter calculation, area and volume calculation, density and mass calculation, etc.

[0031] The present invention decomposes the patient's abdominal CT images into components and identifies and segments tumors based on the decomposed components. This network design that decomposes the components first and then segments the tumor can effectively improve the recognition ability and segmentation accuracy of liver tumors. At the same time, the decomposed component graph also provides a feature analysis of the model's recognition and segmentation of tumors, which facilitates the doctor's review and correction.

[0032] During the model training phase, a liver component knowledge graph is introduced to guide the training of a cascade network, comprising a liver component decomposition network and a liver tumor segmentation network. The liver component decomposition network effectively decomposes liver images into components, effectively separating normal and abnormal liver tissue. The liver tumor segmentation network extracts and analyzes features of the input liver components to segment the tumor area within the liver.

[0033] The model training phase is used to train the liver component decomposition network and the liver tumor segmentation network. This training process is an end-to-end training method and includes four steps: 2.1. Data preprocessing: Preprocess the patient's abdominal CT images. The preprocessing method refers to step 1.1.

[0034] 2.2. Data augmentation: The pre-processed abdominal CT images are subjected to random image flipping, random image translation, random image rotation, random image elastic deformation and random image contrast adjustment.

[0035] 2.3 Model training The preprocessed abdominal CT image is input and sequentially input into the liver component decomposition network and the liver tumor segmentation network. The liver component decomposition network and the liver tumor segmentation network are then forward-derived and the network weight gradients are calculated to obtain the output of the liver component decomposition network, i.e., the liver component images I1-I4, and the output of the liver tumor segmentation network, i.e., the segmented tumor mask P. During the gradient backpropagation and weight update phase, the liver component decomposition network and the liver tumor segmentation network are performed alternately: First, the weights of the liver component decomposition network are frozen, and the segmentation loss value L2 between the segmented tumor mask P and the labeled tumor gold standard T is calculated (segmentation loss value L2 = Dice loss value (Dice Loss) + CE cross entropy loss value (CE Loss)). Then, the liver tumor segmentation network is gradient backpropagated and the network weights are updated.

[0036] Secondly, the weights of the liver component decomposition network are unfrozen, and the weights of the liver tumor segmentation network are frozen. The mutual information loss (MI Loss) of the liver component images I1-I4 and the input abdominal CT images are respectively calculated and accumulated to obtain the loss value L1. The similarity SSIM loss value L0 of the liver component images I1-I4 and the manually decomposed liver component images J1-J4 is then calculated. Finally, the L1 and L0 loss values are added to the segmentation loss value L2 calculated previously to obtain the final loss value L3. The liver component decomposition network is then gradient backpropagated and the network weights are updated.

[0037] After the liver component decomposition network and the liver tumor segmentation network have completed gradient backpropagation and network weight update, they enter the next training cycle. Each training cycle samples different data for training.

[0038] Among them, liver component images I1-I4 are interpretable and represent: liver tissue range, normal liver tissue area, liver high-density area, and high-viscera low-density area, respectively. Manually decomposed liver component images J1-J4 are manually labeled with liver tissue range, normal liver tissue area, liver high-density area, and high-viscera low-density area.

[0039] 2.4. Model Validation: After training for a specified number of rounds, use the validation data to test the effectiveness of the current model weights, and select the model weights with the best performance to save.

[0040] Embodiment 2: In some embodiments, as Figure 3 As shown in the figure, the main body of the liver component decomposition network is a classic 3D UNet network. The abdominal CT image is input, and the deep features are extracted through the 3D UNet network and decoded into a component weight map. The component weight map has the same size as the input abdominal CT image and is converted into a component weight probability map through a softmax activation function. Finally, the input abdominal CT image is multiplied by each weight probability map respectively to obtain the decomposed liver component images I1-I4.

[0041] Embodiment 3: In some embodiments, as Figure 4 As shown in the figure, the skeleton of the liver tumor segmentation network is a UNet network structure. The liver component images I1-I4 are input, downsampled and encoded three times in sequence, and finally decoded three times. In the middle, cross-layer connections are used to connect the same-scale features. The output of the final decoding layer is then activated by a sigmoid function to obtain the final output tumor segmentation result, that is, the segmented tumor mask P.

[0042] The present invention has the following technical effects: 1) This method is highly robust. Compared to traditional standalone liver tumor segmentation networks, this method introduces a liver component decomposition network before the segmentation network, effectively filtering out other useless information and achieving higher learning efficiency and accuracy.

[0043] 2) The present invention has strong applicability. Compared with traditional single liver tumor segmentation networks, the liver component decomposition network of the present invention outputs normalized image components, making the input domain of the liver tumor segmentation network simple and single, making it applicable to CT images of various image qualities.

[0044] 3) High accuracy. This invention fully incorporates prior information on liver tumor components, including liver tissue extent, normal liver tissue composition, high-density components, and low-density components. This improves the effectiveness of model feature engineering and makes the output more accurate.

[0045] 4) The model of the present invention is interpretable. The liver component images I1-I4 of the liver component decomposition network of the present invention are intuitively interpretable. Liver component images I1-I4 sequentially reflect the tissue range of the liver, normal liver tissue, high-density liver tissue, and low-density liver tissue.

[0046] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for automatic identification and measurement of liver tumors based on CT images, characterized in that: The method includes the following steps: first, after image preprocessing, the patient's abdominal CT image is input into a liver component decomposition network. The liver component decomposition network effectively decomposes the liver image into components and effectively separates normal liver tissue from abnormal liver tissue to obtain decomposed liver component images I1-I4. Then, the liver component images I1-I4 are input into a liver tumor segmentation network. The liver tumor segmentation network extracts and analyzes the input liver component images I1-I4, segments the tumor area in the liver, and outputs the tumor segmentation result. Finally, the segmentation result is input into a tumor measurement and analysis module for tumor analysis to calculate detailed information of the tumor.

2. The method for automatic liver tumor recognition and measurement based on CT images according to claim 1, characterized in that: Includes model training phase and model inference phase; Among them, the model training stage is used to train the liver component decomposition network and the liver tumor segmentation network; The model inference phase performs the following steps: First, the patient's abdominal CT image is preprocessed and then input into the liver component decomposition network to obtain the decomposed liver component images I1-I4. Then, the liver component images I1-I4 are input into the liver tumor segmentation network to segment the liver tumor and output the tumor segmentation results. Finally, the segmentation results are input into the tumor measurement and analysis module for tumor analysis and calculation of detailed tumor information.

3. The method for automatic liver tumor recognition and measurement based on CT images according to claim 1 or 2, characterized in that: The detailed information of the tumor includes the major axis, minor axis, average radius, average density, maximum density, minimum density, maximum cross-sectional area, total volume and total mass of each tumor.

4. The method for automatic liver tumor recognition and measurement based on CT images according to claim 1, characterized in that: The model inference phase consists of four steps: 1.1、Data preprocessing; 1.

2. Liver component decomposition: Input the abdominal CT image into the liver component decomposition network to obtain liver component images I1-I4; 1.

3. Tumor segmentation: Input the liver component images I1-I4 into the liver tumor segmentation network to obtain the tumor segmentation results; 1.

4. Tumor measurement and analysis: The tumor measurement and analysis module measures and analyzes the tumor segmentation results and outputs detailed information about the tumor.

5. The method for automatic liver tumor recognition and measurement based on CT images according to claim 4, characterized in that: Step 1.1, data preprocessing, the specific steps are as follows: the patient's abdominal CT image will be pre-parsed from the DICOM format data and normalized. Specifically, the image will be resampled to a uniform resolution, which is the median resolution of all image resolution distributions, and the image value range will be normalized to [0, 1].

6. The method for automatic liver tumor recognition and measurement based on CT images according to claim 4, characterized in that: The methods used in the tumor measurement and analysis module include: three-dimensional connected domain detection, internal cavity filling, major and minor diameter calculation, area and volume calculation, and density and mass calculation.

7. The method for automatic liver tumor recognition and measurement based on CT images according to claim 1, characterized in that: The model training phase includes four steps: 2.

1. Data preprocessing: Preprocess the patient's abdominal CT images; 2.2, data augmentation; 2.3 Model training The preprocessed abdominal CT image is input and sequentially input into the liver component decomposition network and the liver tumor segmentation network. The liver component decomposition network and the liver tumor segmentation network are then forward-derived and the network weight gradients are calculated to obtain the output of the liver component decomposition network, i.e., the liver component images I1-I4, and the output of the liver tumor segmentation network, i.e., the segmented tumor mask P. During the gradient backpropagation and weight update phase, the liver component decomposition network and the liver tumor segmentation network are performed alternately: First, the weights of the liver component decomposition network are frozen, and the segmentation loss value L2 between the segmented tumor mask P and the labeled tumor gold standard T is calculated. Then, the liver tumor segmentation network is gradient backpropagated and the network weights are updated. Next, the weights of the liver component decomposition network are unfrozen, and the weights of the liver tumor segmentation network are frozen. The mutual information loss is calculated using the liver component images I1-I4 and the input abdominal CT images, and the loss value L1 is accumulated. The similarity SSIM loss value L0 is then calculated for the liver component images I1-I4 and the manually decomposed liver component images J1-J4. Finally, the L1 and L0 loss values are added to the previously calculated segmentation loss value L2 to obtain the final loss value L3. The liver component decomposition network is then gradient-backed and the network weights are updated. After the liver component decomposition network and the liver tumor segmentation network have completed gradient backpropagation and network weight update, they enter the next training cycle. Each training cycle samples different data for training. 2.

4. Model Validation: After training for a specified number of rounds, use the validation data to test the effectiveness of the current model weights, and select the model weights with the best performance to save.

8. The method for automatic liver tumor recognition and measurement based on CT images according to claim 7, characterized in that: Step 2.2, data augmentation, specifically the following steps: performing random image flipping, random image translation, random image rotation, random image elastic deformation, and random image contrast adjustment on the pre-processed abdominal CT image.

9. The method for automatic liver tumor recognition and measurement based on CT images according to claim 7, characterized in that: Segmentation loss value L2 = Dice loss value + CE cross entropy loss value.

10. The method for automatic liver tumor recognition and measurement based on CT images according to claim 1, characterized in that: The liver component decomposition network is based on a 3D UNet network. It takes an abdominal CT image as input, extracts deep features through the 3D UNet network, and decodes them into component weight maps. The component weight maps have the same size as the input abdominal CT image and are converted into component weight probability maps through a softmax activation function. Finally, the input abdominal CT image is multiplied by each weight probability map to obtain the decomposed liver component images I1-I4. The backbone of the liver tumor segmentation network is a UNet network structure. The liver component images I1-I4 are input, sequentially downsampled and encoded multiple times, and finally decoded multiple times. In the middle, cross-layer connections are used to connect the same-scale features. The output of the final decoding layer is then activated by a sigmoid function to obtain the final output tumor segmentation result, i.e., the segmented tumor mask P.

Images