Intelligent evaluation system and method for residual liver volume based on CT image
By cross-training multiple deep learning models and hyperparameter experience sets, and combining CT images for precise segmentation and residual liver volume assessment, the problem of precise segmentation and residual liver prediction in existing technologies has been solved, achieving efficient and accurate residual liver volume assessment and postoperative assessment support.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHEJIANG UNIV
- Filing Date
- 2022-05-25
- Publication Date
- 2026-07-31
AI Technical Summary
Existing technologies lack accurate segmentation and residual liver prediction schemes based on deep learning. Manual parameter tuning is time-consuming and labor-intensive, and hyperparameter optimization is inefficient, making it difficult to achieve efficient residual liver volume assessment.
Multiple deep learning models and hyperparameter experience sets are used to select the best model and hyperparameter configuration through cross-training. Combined with CT images, accurate segmentation and residual liver volume assessment are performed, and the residual liver volume is calculated using a segmentation algorithm.
It achieves highly accurate assessment of residual liver volume, simplifies the hyperparameter optimization process, reduces time and effort costs, provides reliable postoperative assessment support, and improves the efficiency and accuracy of surgical procedures.
Smart Images
Figure CN115311195B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of residual liver volume assessment technology, and in particular relates to an intelligent assessment system and method for residual liver volume based on CT images. Background Technology
[0002] With the continuous development of surgical techniques and deep learning technology, we have reached a truly precise era in pathological liver resection. Computed tomography (CT) provides comprehensive information for the diagnosis and treatment of liver tumors, forming a solid foundation for precise segmentation of liver tumors. Predicting the volume of residual liver after surgical resection can help assess the patient's postoperative liver quality and function, and also helps determine whether sufficient functional residual liver volume has been preserved after radical tumor resection. This has very important guiding significance for the patient's surgical outcome and prognosis.
[0003] Precise segmentation refers to the process of identifying the boundaries of different liver segments, which are substructures of the liver. This involves "segmenting" the liver into its eight major segments (hepatic substructures) from a "complete" liver region, or "segmenting" the liver into two types of parts: normal liver tissue and tumor-related liver tissue. To achieve precise segmentation, improve the accuracy of liver resection, and assess the postoperative residual liver ratio, researchers have conducted extensive studies. For example, Chinese patent application CN202111085064.2 discloses a method for characterizing the residual liver ratio after liver resection. This method uses three-dimensional reconstruction to simulate resection, combining region calculation and eight-segment liver identification functions, and embedding Mimics software to achieve liver model reconstruction and characterization of the residual liver ratio. However, to date, there is no deep learning-based solution for precise segmentation and residual liver prediction.
[0004] Deep learning is a type of machine learning motivated by the creation and simulation of neural networks that mimic the human brain's analytical learning processes. It imitates the mechanisms of the human brain to interpret data such as images, sounds, and text. Deep learning discovers distributed feature representations of data by combining low-level features to form more abstract high-level representations of attribute categories or features. The concept of deep learning was proposed by Hinton et al. in 2006, who introduced an unsupervised greedy layer-by-layer training algorithm based on Deep Belief Networks (DBNs), offering hope for solving optimization problems related to deep structures. Subsequently, multi-layer autoencoders were proposed for deep structures. Furthermore, the Convolutional Neural Network (CNN) proposed by Lecun et al. was the first truly multi-layer learning algorithm, utilizing spatial relative relationships to reduce the number of parameters and improve training performance.
[0005] In typical deep learning, after designing the model, the hyperparameters of each sub-component need to be controlled. Hyperparameters are parameters whose values are set before learning begins, rather than parameters obtained through training data. Typically, hyperparameters need to be manually optimized during the learning process to select an optimal set of hyperparameters for the learning machine to improve learning performance and effectiveness. However, manual hyperparameter tuning is very time-consuming and labor-intensive. Currently, there are two main non-manual methods for hyperparameter optimization: grid search and random search. The first method requires trying every possible combination of hyperparameters and then selecting the optimal set, which is obviously inefficient. The second method, randomly selecting parameter combinations, is more efficient than the first, but its performance is unstable and can result in extremely poor hyperparameter combinations. Summary of the Invention
[0006] The purpose of this invention is to address the above-mentioned problems by providing an intelligent assessment system and method for residual liver volume based on CT images.
[0007] To achieve the above objectives, the present invention adopts the following technical solutions:
[0008] A method for intelligent assessment of residual liver volume based on CT images includes the following steps:
[0009] S1. Multiple different deep learning models and hyperparameter experience sets are prepared;
[0010] S2. Use the hyperparameter experience set to adjust the hyperparameters of each deep learning model, and use training samples to train the deep learning models under different hyperparameter configurations separately;
[0011] S3. Select the deep learning model with the best training performance and its hyperparameter configuration;
[0012] S4. Use the trained deep learning model to accurately segment CT images and identify the spatial location of tumors;
[0013] S5. Assess the volume of the remaining liver based on the physician's resection plan.
[0014] In the above-mentioned intelligent assessment method for residual liver volume based on CT images, in step S1, a 2DU-Net network, a normal 3DU-Uet network running at full image resolution, and a cascaded network consisting of two 3DU-Nets are prepared, as well as an empirical set of hyperparameters for each of the aforementioned deep learning models.
[0015] In the above-mentioned intelligent assessment method for residual liver volume based on CT images, in step S2, a deep learning model is trained using a CT imaging dataset with liver segment annotation and tumor spatial annotation.
[0016] In the above-mentioned intelligent assessment method for residual liver volume based on CT images, a portion or all of the labeled CT imaging datasets are randomly selected for cross-training in step S2 to select the deep learning model and its hyperparameter configuration with the best training effect.
[0017] In the above-mentioned intelligent assessment method for residual liver volume based on CT images, in step S3, the training effect is judged based on the test results during the training process, and the smallest test error indicates the best training effect.
[0018] In the above-mentioned intelligent assessment method for residual liver volume based on CT images, in step S3, after selecting the optimal deep learning model and hyperparameter configuration, the selected deep learning model is trained using all or the remaining labeled CT imaging dataset to optimize the model parameters.
[0019] In the above-mentioned intelligent assessment method for residual liver volume based on CT images, the CT images also indicate the involvement of each liver segment by the tumor.
[0020] In step S4, the trained deep learning model accurately segments CT images, identifies the spatial location of the tumor, and identifies liver segments that are not involved by the tumor and / or are involved by the tumor.
[0021] In the above-mentioned intelligent assessment method for residual liver volume based on CT images, in step S1, the hyperparameter experience set is a combination of hyperparameters selected from historical experience. Different deep learning models correspond to multiple different hyperparameter combinations, or these hyperparameter combinations are applied to different deep learning models at the same time.
[0022] In the above-mentioned intelligent assessment method of residual liver volume based on CT images, in step S5, the physician designs a resection plan based on the accurate segmentation results of step S4, the spatial location of the tumor, and the extent to which each liver segment is involved by the tumor.
[0023] In the above-mentioned intelligent assessment method for residual liver volume based on CT images, step S5 uses a segmented algorithm to assess the residual liver volume:
[0024] S51. Eliminate the portion of liver that was removed in the resection plan;
[0025] S52. Separate the liver into segments after cutting, calculate the area of each layer of each liver segment, and obtain the volume of the corresponding liver segment by summing the areas of each layer;
[0026] S52. The residual liver volume is obtained by summing the values of each liver segment.
[0027] A smart liver remnant volume assessment system based on CT images is proposed, which assesses the volume of the remnant liver using the method described above.
[0028] The advantages of this invention are:
[0029] 1. This solution proposes to use deep learning to accurately segment liver CT images, leveraging the advantages of deep learning to achieve higher accuracy in segmentation;
[0030] 2. By combining experience and training methods to select models and optimize hyperparameters for deep learning models, manual parameter tuning is simplified and the efficiency of hyperparameter optimization is improved. This reduces the time, effort, and case requirements for using deep learning to predict residual liver, enabling deep learning to be better applied to residual liver prediction.
[0031] 3. By performing pixel-level calculations on the segmentation results, the estimated volume of the residual liver after anatomical hepatectomy can be obtained, achieving a good evaluation effect;
[0032] 4. Using segmented and surface-accumulated methods to calculate the residual liver volume can improve the accuracy of residual liver volume measurement, providing doctors with reliable postoperative predictive assessments for resection plans. This helps doctors adjust and determine surgical plans more efficiently and accurately, thereby improving surgical outcomes. Attached Figure Description
[0033] Figure 1 This is a model architecture diagram of the system of the present invention;
[0034] Figure 2 This is a flowchart of the evaluation process for the system of the present invention;
[0035] Figure 3 This is a cascaded network architecture diagram in the model architecture diagram;
[0036] Figure 4 This is a diagram illustrating the five-fold cross-validation.
[0037] Figure 5 Image showing the result of segmenting the liver into eight segments before tumor resection;
[0038] Figure 6 Another perspective of the liver segmentation results before tumor resection;
[0039] Figure 7 Two isolated liver tumors were identified;
[0040] Figure 8 This refers to the residual liver segment after tumor resection.
[0041] Figure 9 Another perspective of the remaining liver segment after tumor resection. Detailed Implementation
[0042] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.
[0043] like Figure 1 and Figure 2 As shown, this embodiment discloses an intelligent liver remnant volume assessment system based on CT images, used for liver volume segmentation and remnant liver volume assessment. A dataset is prepared in advance as a training sample. In this embodiment, it is a CT imaging dataset that has been labeled with liver segments, tumor spatial labels, and the tumor involvement status of each liver segment.
[0044] The system specifically assesses the volume of the residual liver through the following steps:
[0045] S1. Prepare multiple different deep learning models and hyperparameter experience sets;
[0046] This solution prepares three types of networks: a 2DU-Net network, a normal 3DU-Uet network operating at full image resolution, and a cascaded network consisting of two 3DU-Nets. The 2DU-Net network is a standard 2DU-Net, and the 3DU-Uet is essentially the same as a 2D U-Net, the only difference being that the 2D operations are replaced with 3D operations; for example... Figure 3 As shown, in the cascaded network, the first network operates on the downsampled image, while the second network adjusts the result of the first network across the entire image's pixels.
[0047] These hyperparameter experience sets consist of several hyperparameter combinations selected by the user from historical experience. For example, for the 2DU-Net network, there are 55 hyperparameter combinations. The user can select several sets of hyperparameter combinations with the smallest historical test error based on experience, such as four sets. During training, the 2DU-Net network using these four sets of hyperparameters is trained respectively.
[0048] Different deep learning models correspond to multiple different hyperparameter combinations. For example, there might be 20 hyperparameter combinations, with 4 corresponding to 2DU-Net, 8 to the standard 3DU-Net, and 8 to cascaded networks. Alternatively, these hyperparameter combinations can be used simultaneously for different deep learning models, meaning all 20 combinations can be used to train, test, and compare three networks to select the optimal network and its parameter configuration. Of course, some hyperparameter combinations can also be shared by several networks, while others can be used exclusively by a single network.
[0049] S2. Use the hyperparameter experience set to adjust the hyperparameters of each deep learning model, and use training samples to train the deep learning models under different hyperparameter configurations separately;
[0050] Specifically, a portion of the labeled CT imaging dataset is randomly selected for cross-training in the training process described above to select the deep learning model and its hyperparameter configuration that yields the best training results. The number of samples randomly selected is determined by those skilled in the art; for example, 1 / 5 of the samples may be randomly selected. Fewer samples result in higher selection efficiency, while more samples result in better selection. Therefore, the entire CT imaging dataset can also be used, although this is less efficient. Technicians can determine the appropriate datasets for model selection and hyperparameter combination selection based on their needs.
[0051] Furthermore, this scheme employs a five-fold cross-training method, where the training samples are divided into five equal parts. One part is used for testing in each experiment, while the rest are used for training. The average value is calculated after five experiments. Figure 4 In the first experiment, the first set was used as the test set, and the rest were used as the training set. In the second experiment, the second set was used as the test set, and the rest were used as the training set, and so on.
[0052] S3. Select the deep learning model with the best training performance and its hyperparameter configuration;
[0053] After training, the network with the smallest test error and its corresponding set of hyperparameters are selected to form the network model of the system. The selected deep learning model is then trained using all or the remaining labeled CT imaging dataset samples to optimize the model parameters.
[0054] S4. Use the trained deep learning model to accurately segment CT images and identify the spatial location of the tumor, while identifying liver segments that are not involved by the tumor and / or are involved by the tumor; liver segments involved by the tumor need to be completely resected.
[0055] Figure 5 and Figure 6 The images show the results of segmenting the liver into eight segments in patient A before tumor resection using this system's model. Figure 7 This is an ex vivo demonstration of two identified liver tumors.
[0056] S5. Assess the volume of the remaining liver based on the physician's resection plan. Figure 8 and Figure 9 These are illustrations of the liver segments involved in the removal of liver tumors according to the resection plan provided by the physician.
[0057] Specifically, based on the precise segmentation results of step S4, the spatial location of the tumor, and the extent to which each liver segment is involved by the tumor, the physician designs a resection plan.
[0058] Furthermore, the residual liver volume was evaluated using a segmented algorithm, the specific process of which is as follows:
[0059] S51. Eliminate the portion of liver that was removed in the resection plan;
[0060] S52. Separate the liver into segments after cutting, calculate the area of each layer of each liver segment, and obtain the volume of the corresponding liver segment by summing the areas of each layer;
[0061] S52. The residual liver volume is obtained by summing the values of each liver segment.
[0062] By performing pixel-level calculations on the segmentation results, the estimated volume of the residual liver after anatomical hepatectomy can be obtained, achieving good results.
[0063] The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to substitute them, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.
Claims
1. A method for intelligent assessment of residual liver volume based on CT images, characterized in that, Includes the following steps: S1. Prepare multiple different deep learning models and hyperparameter experience sets, wherein the hyperparameter experience sets are several fixed combinations of hyperparameters selected from historical experiments; S2. Adjust the hyperparameters of each deep learning model using the hyperparameters in the hyperparameter experience set, and train the deep learning models under different hyperparameter configurations using the CT imaging dataset with liver segment annotation, tumor spatial annotation, and annotation of tumor involvement in each liver segment. S3. Select the deep learning model with the best training performance and its hyperparameter configuration; S4. Use the trained deep learning model to accurately segment CT images, identify the spatial location of tumors, and identify liver segments that are not involved by the tumor and / or are involved by the tumor. S5. Based on the precise segmentation results of step S4, the spatial location of the tumor, and the extent to which each liver segment is involved by the tumor, the physician designs a resection plan. The residual liver volume was assessed using a segmented algorithm based on the surgeon's resection plan. S51. Eliminate the portion of liver that was removed in the resection plan; S52. Separate the liver into segments after cutting, calculate the area of each layer of each liver segment, and obtain the volume of the corresponding liver segment by summing the areas of each layer; S53. The residual liver volume is obtained by summing the segments of the liver.
2. The intelligent assessment method for residual liver volume based on CT images according to claim 1, characterized in that, In step S1, a 2DU-Net network, a normal 3DU-Uet network running at full image resolution, and a cascaded network consisting of two 3DU-Nets are prepared, along with an empirical set of hyperparameters for each of the aforementioned deep learning models.
3. The intelligent assessment method for residual liver volume based on CT images according to claim 2, characterized in that, Randomly select a portion or all of the labeled CT imaging datasets for cross-training in step S2 to select the deep learning model and its hyperparameter configuration that achieves the best training effect.
4. The intelligent assessment method for residual liver volume based on CT images according to claim 3, characterized in that, In step S3, the training effect is judged based on the test results during the training process, and the smallest test error indicates the best training effect.
5. The intelligent assessment method for residual liver volume based on CT images according to claim 4, characterized in that, In step S3, after selecting the optimal deep learning model and hyperparameter configuration, the selected deep learning model is trained using all or the remaining labeled CT imaging datasets to optimize the model parameters.
6. The intelligent assessment method for residual liver volume based on CT images according to claim 1, characterized in that, In step S1, the hyperparameter experience set consists of several combinations of hyperparameters selected from historical experience. Different deep learning models correspond to multiple different hyperparameter combinations, or these hyperparameter combinations are applied to different deep learning models simultaneously.