Apparatus and method for automated liver tumor segmentation and staging using 3D u-net in ct images

Abstract

The present invention relates to a device and method for automatic liver tumor segmentation and staging based on 3D U-Net using CT images, and more specifically, to a technology that receives abdominal CT images and automatically performs image preprocessing, image enhancement, image segmentation, surface mesh generation, staging analysis, and staging classification. The present invention includes a technology that receives CT images through an image receiving unit and improves image quality by performing pixel value normalization, linear scaling, background removal, noise reduction, and contrast enhancement in an image preprocessing unit. Ultimately, the present invention reduces the workload of medical staff through automated image analysis and staging classification, and enables faster and more precise diagnosis in clinical settings.

Description

Technology Field

[0001] The present invention relates to an apparatus and method for automatically segmenting liver tumors and classifying their stages based on CT images, and more specifically, to a technology for segmenting the liver, liver blood vessels, and liver tumors using 3D U-Net, generating a surface mesh, and automatically determining the stage of progression of hepatocellular carcinoma by applying TNM staging criteria. Background Technology

[0002] Hepatocellular carcinoma (HCC) is a disease with a high cancer mortality rate worldwide, making early diagnosis and accurate staging essential for establishing a treatment plan. While curative treatments such as liver resection or liver transplantation are possible for HCC detected early, treatment options are limited and the prognosis is poor in advanced stages.

[0003] The staging of HCC is primarily based on the TNM staging system and is determined by considering the size and number of liver tumors and the presence of major vascular invasion.

[0004] Previously, it was common practice for radiologists to directly interpret CT images to determine the presence and stage of tumors; however, recently, research utilizing deep learning-based image segmentation models to automatically segment the liver and liver tumors in CT images has been actively conducted. While there have been many studies on liver tumor segmentation using 2D U-Net, 3D U-Net models that reflect the characteristics of 3D medical imaging data are reported to provide more precise segmentation performance.

[0005] Previous studies have frequently performed individual tasks such as determining whether liver tumors are benign or malignant, segmenting the tumor, and evaluating lymph nodes and metastasis. However, there is a relative lack of research that goes beyond simple segmentation to perform CT image-based automated staging, and most studies have had limitations due to a lack of accurate quantitative criteria for staging.

[0006] Some studies have proposed methods to automatically determine the TNM stage of liver tumors using deep learning models, but most studies have been limited to models utilizing limited features rather than comprehensive staging models that comprehensively consider the location of the lesion, tumor size, and vascular invasion.

[0007] Since the diagnosis and staging of HCC based on CT images often rely on the subjective judgment of radiologists, results may vary depending on the experience and proficiency of the reader; furthermore, due to the complexity of the images, it can be difficult to clearly distinguish tumor boundaries, posing a risk of missing minute lesions.

[0008] In addition, the process of medical staff visually reviewing hundreds of CT images is time-consuming and poses a problem of reduced efficiency in clinical environments requiring rapid decision-making.

[0009] Existing 2D U-Net-based models have limitations in that they fail to adequately reflect spatial continuity in 3D medical images, and since existing studies often perform segmentation at the 2D CT slice level, there is a lack of accurate 3D structural analysis using continuous CT tomographic images.

[0010] Some studies have utilized 3D U-Net, but it lacked the capability to quantitatively evaluate tumor size and vascular invasion. Existing studies have primarily focused on improving the segmentation accuracy of liver tumors, and there is a relative lack of research that automatically performs staging classification.

[0011] In particular, studies on automatically classifying TNM stages by comprehensively evaluating factors such as vascular invasion, tumor size, and number have been limited. Furthermore, some studies have the limitation of requiring tissue biopsies for accurate staging, thus requiring a method to perform reliable staging using only non-invasive CT images.

[0012] Existing deep learning-based liver tumor segmentation studies have focused primarily on pixel-wise segmentation, but there has been a lack of intuitive staging analysis techniques using 3D modeling. Additionally, while surface meshes can be used to quantitatively analyze the spatial shape of liver tumors and their contact relationship with blood vessels, existing studies have lacked this capability.

[0013] In particular, there is a lack of T4 staging classification based on blood vessel diameter and algorithms that determine the stage by automatically analyzing tumor size and number. Prior art literature

[0014] Korean Published Patent No. 10-2024-0167458 "Automatic segmentation of kidney tumors in CT abdominal images using deep learning and automatic classification device and method using hu (Houncfield Unit) change features and radiomics features according to contrast level based on segmentation region" Korean Registered Patent No. 10-2756982 "Segmentation and rigid body registration device and method of 3D liver CT images for automatic prediction of fatty liver disease" Korean Registered Patent No. 10-1954858 "Method and device for segmenting lung tumors in chest CT images using weighted voting-based elliptical propagation and probabilistic map of surrounding organs" The problem to be solved

[0015] The present invention aims to solve the problem that CT image-based liver tumor diagnosis and staging relies on the subjective judgment of radiologists.

[0016] The present invention aims to reduce the time required for medical personnel to analyze hundreds of CT images and to enable rapid and consistent staging classification.

[0017] The present invention aims to solve the problem that 2D-based image segmentation techniques do not sufficiently reflect the continuity of 3D medical images, and to perform more precise liver tumor segmentation by applying 3D U-Net.

[0018] The present invention aims to provide a function for automatically determining the TNM stage by comprehensively analyzing tumor size, number, and the presence of major vascular invasion.

[0019] The present invention aims to quantitatively analyze the spatial relationship between liver tumors and blood vessels using a surface mesh and to perform a more intuitive and reliable staging classification.

[0020] The present invention aims to provide an algorithm for determining the stage by automatically analyzing the T4 staging classification based on blood vessel diameter, tumor size, and number.

[0021] The present invention aims to perform highly reliable staging classification using only non-invasive CT images and to enable comprehensive staging of hepatocellular carcinoma without tissue biopsy. means of solving the problem

[0022] An automatic liver tumor segmentation and staging device according to one embodiment may include an image receiving unit that receives an abdominal CT image, an image preprocessing unit that normalizes pixel values, performs linear scaling, and removes backgrounds for the input abdominal CT image, an image augmentation unit that applies a data augmentation technique to the preprocessed image, an image segmentation unit that segments the liver, hepatic vessels, and liver tumors in the augmented image, a mesh generation unit that generates a surface mesh based on the segmented liver, hepatic vessel, and liver tumor regions, a staging analysis unit that analyzes the number, size, and contact status with major blood vessels of liver tumors using the generated surface mesh, and a staging classification unit that applies TNM (Tumor, Node, Metastasis) staging criteria and classifies the progression stage of liver tumors based on the analyzed contact status.

[0023] A method of operation for an automatic liver tumor segmentation and staging classification device according to one embodiment may include: receiving an abdominal CT image; a preprocessing step of normalizing pixel values ​​of the input abdominal CT image, performing linear scaling, and removing the background; applying a data augmentation technique to the preprocessed image; segmenting the liver, hepatic vessels, and liver tumors in the augmented image; generating a surface mesh based on the segmented liver, hepatic vessel, and liver tumor regions; analyzing the number, size, and contact with major blood vessels of liver tumors using the generated surface mesh; and applying TNM staging criteria based on the analyzed contact status and classifying the progression stage of the liver tumor. Effects of the invention

[0024] According to one embodiment, in CT image-based liver tumor diagnosis and staging, subjective judgment by a radiologist can be minimized and consistent results can be provided.

[0025] According to one embodiment, clinical efficiency can be improved by reducing the workload of medical staff and enabling rapid diagnosis through automatic analysis of CT images.

[0026] According to one embodiment, by utilizing 3D U-Net to reflect the spatial continuity of 3D medical images, liver and liver tumors can be segmented with higher accuracy than existing 2D-based image segmentation techniques.

[0027] According to one embodiment, objective and quantitative staging classification can be performed by automatically analyzing tumor size, number, and vascular invasion and applying TNM staging criteria.

[0028] According to one embodiment, a surface mesh can be used to visually analyze whether there is contact between a liver tumor and major blood vessels, thereby enabling more precise staging.

[0029] According to one embodiment, a function can be provided to automatically determine whether the blood vessel diameter is 5 mm or more and to classify it as T4 stage if the tumor has invaded major blood vessels.

[0030] According to one embodiment, a highly reliable staging assessment of hepatocellular carcinoma can be performed based solely on CT images without a tissue biopsy, enabling accurate diagnosis in a non-invasive manner.

[0031] According to one embodiment, automated disease analysis can be performed more quickly than the existing manual disease evaluation method, and can be utilized in real-time at the medical site.

[0032] According to one embodiment, the accuracy of weapon classification can be improved as data is accumulated by continuously training a deep learning model.

[0033] According to one embodiment, the system is optimized to be applicable in various medical environments, making it easy to use in hospitals and research institutions. Brief explanation of the drawing

[0034] FIG. 1 is a drawing illustrating an automatic liver tumor segmentation and staging classification device according to one embodiment. Figure 2 is a diagram illustrating the process of segmentation and staging of hepatocellular carcinoma according to one embodiment. Figure 3 is a diagram illustrating the process of generating training data according to one embodiment. Figure 4 is a diagram illustrating an image preprocessing process according to one embodiment. FIG. 5 is a diagram illustrating the network architecture of a 3D U-Net according to one embodiment. Figure 6 is a diagram illustrating the training results according to one embodiment. FIG. 7 is a diagram illustrating the result of generating a surface mesh according to one embodiment. Figure 8 is a diagram illustrating the results of liver tumor division according to one embodiment. FIG. 9 is a diagram illustrating the staging process of hepatocellular carcinoma according to one embodiment. FIG. 10 is a diagram illustrating the operation method of an automatic liver tumor segmentation and staging classification device according to one embodiment. Specific details for implementing the invention

[0035] Specific structural or functional descriptions of embodiments according to the concept of the present invention disclosed herein are provided merely for the purpose of explaining embodiments according to the concept of the present invention, and embodiments according to the concept of the present invention may be implemented in various forms and are not limited to the embodiments described herein.

[0036] Embodiments according to the concept of the present invention may be subject to various modifications and may take various forms; therefore, embodiments are illustrated in the drawings and described in detail in this specification. However, this is not intended to limit the embodiments according to the concept of the present invention to specific disclosed forms, and includes modifications, equivalents, or substitutions that fall within the spirit and scope of the present invention.

[0037] Terms such as "first" or "second" may be used to describe various components, but said components should not be limited by said terms. For the sole purpose of distinguishing one component from another, for example, without departing from the scope of rights according to the concept of the present invention, the first component may be named the second component, and similarly, the second component may be named the first component.

[0038] When it is stated that one component is "connected" or "connected" to another component, it should be understood that while it may be directly connected or connected to that other component, there may also be other components in between. Conversely, when it is stated that one component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between. Expressions describing the relationships between components, such as "between," "exactly between," or "directly adjacent to," should be interpreted in the same way.

[0039] The terms used herein are used merely to describe specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, terms such as “comprising” or “having” are intended to specify the existence of the described features, numbers, steps, actions, components, parts, or combinations thereof, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0040] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art to which the present invention pertains. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this specification.

[0042] Hereinafter, embodiments will be described in detail with reference to the attached drawings. However, the scope of the patent application is not limited or restricted by these embodiments. Identical reference numerals in each drawing indicate identical components.

[0043] FIG. 1 is a drawing illustrating an automatic liver tumor segmentation and staging classification device (100) according to one embodiment.

[0044] The liver tumor automatic segmentation and staging device (100) includes a plurality of components that receive an abdominal CT image and perform preprocessing, data augmentation, image segmentation, surface mesh generation, staging analysis, and staging classification.

[0045] The liver tumor automatic segmentation and staging device (100) includes an image receiving unit (110), an image preprocessing unit (120), an image enhancement unit (130), an image segmentation unit (140), a mesh generation unit (150), a staging analysis unit (160), and a staging unit (170), and each component operates in conjunction with each other to perform staging of hepatocellular carcinoma based on CT images.

[0046] The image receiving unit (110) is a component that receives abdominal CT images to automatically classify the stage of hepatocellular carcinoma. The image receiving unit (110) performs the function of receiving abdominal CT image data from a medical image storage system (PACS), a hospital server, a cloud-based image storage, an external storage device (DICOM file), etc. The image receiving unit (110) stores the received abdominal CT image data while maintaining the original resolution and transmits it to a subsequent processing step without loss to improve the accuracy of image processing. The image receiving unit (110) may include a function to convert the CT image to a specific resolution and format (DICOM, NIfTI, etc.) if necessary, while maintaining the CT image metadata (shooting information, resolution, patient information, shooting conditions, etc.).

[0047] The image preprocessing unit (120) preprocesses image data to improve the quality of the abdominal CT image received from the image receiving unit (110) and to maximize the accuracy of image segmentation.

[0048] The image preprocessing unit (120) performs the function of normalizing the pixel values ​​of the CT image to between -140 HU and 260 HU. The image preprocessing unit (120) linearly scales the pixel values ​​of the CT image to convert them into values ​​in the range of 0 to 1 and converts them into a form suitable for input to machine learning and deep learning models. The image preprocessing unit (120) includes the function of removing background areas to leave only the liver, hepatic vessel, and liver tumor areas that are the subjects of analysis. The image preprocessing unit (120) applies an anisotropic diffusion filter to remove noise within the image, thereby performing the function of removing unnecessary noise while maintaining major boundaries. The image preprocessing unit (120) performs histogram equalization to improve the contrast of the CT image, thereby facilitating the distinction between cancerous lesions and normal tissues.

[0049] The image augmentation unit (130) is a component that receives an abdominal CT image preprocessed by the image preprocessing unit (120), applies a data augmentation technique to increase the diversity of the training data, and improves the generalization performance of the deep learning model. The image augmentation unit (130) transforms the input abdominal CT image to generate new training samples, thereby preventing the problem of overfitting to specific conditions and enabling training that reflects various clinical changes.

[0050] The image augmentation unit (130) applies various augmentation techniques including geometric transformations, pixel intensity transformations, augmenting noise, and composite augmentation. To perform geometric transformations, the image augmentation unit (130) performs the function of applying at least one of flip, rotate, zoom, translation, affine transformation, and perspective transformation to the input abdominal CT image.

[0051] The image enhancement unit (130) includes the function of applying histogram equalization, gamma correction, contrast stretching, intensity shifting, and color space transformation to perform pixel intensity transformation. The image enhancement unit (130) performs the function of generating training samples similar to CT images taken in various lighting environments by adjusting the brightness, contrast, saturation, and gamma of the image.

[0052] The image augmentation unit (130) performs the function of adding Gaussian noise, Salt and Pepper noise, Speckle noise, and Poisson noise to reflect the noise characteristics of the image. The image augmentation unit (130) enables the model to learn noise that may occur in a specific CT scanning environment and contributes to building a deep learning model that is robust to noise.

[0053] The image augmentation unit (130) may adopt a mixed augmentation method that combines and applies various augmentation techniques to perform complex augmentation, and performs the function of generating an optimal training dataset by adjusting the application probability of each augmentation technique. The image augmentation unit (130) includes a quality assurance step to evaluate whether the augmented image maintains structural consistency with the original image, and includes a function to adjust the normalized augmentation intensity so that the image is not damaged during the augmentation process.

[0054] The image augmentation unit (130) performs the function of converting the augmented CT image into training data and providing it as input to the 3D U-Net model in the image segmentation unit (140), and can adjust the augmentation technique so that it does not distort specific disease patterns. The image augmentation unit (130) can also support a User-Defined Augmentation option to optimize prediction performance by reflecting specific augmentation conditions set by medical staff.

[0055] The image segmentation unit (140) is a component that receives an abdominal CT image augmented by the image augmentation unit (130) and segments the liver, hepatic vessel, and tumor using 3D U-Net. The image segmentation unit (140) accurately identifies the location of the liver, hepatic vessel, and tumor within the CT image and segments them into independent regions to generate data for subsequent processing steps.

[0056] The image segmentation unit (140) performs the function of automatically segmenting multiple organ and lesion regions by applying a 3D U-Net-based deep learning model. The image segmentation unit (140) includes the function of extracting features from each axis (Axial, Coronal, Sagittal) of the CT image using a multi-channel input structure to process the input CT image in a three-dimensional space. The image segmentation unit (140) performs the process of extracting hierarchical features in the encoder stage of the 3D U-Net and generating a high-resolution segmentation map in the decoder stage, thereby playing a role in maximizing segmentation accuracy while maintaining spatial information of the image.

[0057] The image segmentation unit (140) includes a function to extract the boundaries of the liver, liver blood vessels, and liver tumors more precisely by applying a skip connection that merges the feature map of the encoder stage in the decoder stage. The image segmentation unit (140) includes a function to improve segmentation performance by utilizing a multi-scale feature map so that accurate segmentation can be performed even when the shape of the tumor is irregular or the boundaries are unclear.

[0058] The image segmentation unit (140) performs the function of optimizing loss functions such as cross-entropy loss, dice loss, and category distance loss to accurately identify the location of the liver, liver blood vessels, and liver tumors within the image. The image segmentation unit (140) also includes the function of ensuring optimal segmentation performance by automatically adjusting kernel size and padding values ​​according to the resolution and area size of the CT image.

[0059] The image segmentation unit (140) transmits mask data of the segmented liver, liver blood vessels, and liver tumors to the surface mesh generation unit (150) and performs the function of constructing a 3D model for staging analysis and staging classification based thereon. The image segmentation unit (140) includes a visual feedback function so that medical staff can review the segmentation results, and may provide a function to calculate quantitative indicators (e.g., Dice Similarity Coefficient, IoU) to evaluate the segmentation accuracy.

[0060] The mesh generation unit (150) is a component that generates a surface mesh based on the regions of the liver, liver blood vessels, and liver tumors segmented by the image segmentation unit (140). The mesh generation unit (150) receives pixel-based 3D binary mask data provided by the image segmentation unit (140) as input and converts it into a polygon-based 3D structure.

[0061] The mesh generation unit (150) generates a surface mesh of the liver, liver blood vessels, and liver tumor by applying the Flying Edges algorithm and performs the function of extracting a continuous surface structure based on voxel data. The mesh generation unit (150) may apply Laplacian smoothing or elliptical smoothing techniques to remove unnecessary faces or vertices during the mesh generation process and to maintain a smoother surface.

[0062] The mesh generation unit (150) generates a surface mesh and then calculates the normal vector of the mesh to more clearly define the boundary features of the liver tumor and performs the function of optimizing the mesh structure. The mesh generation unit (150) provides data to the staging analysis unit (160), which analyzes the number, size, and contact with major blood vessels of the liver tumor based on the surface mesh, and enables medical staff to intuitively visualize the generated 3D mesh.

[0063] The staging analysis unit (160) is a component that analyzes the number, size, and contact with major blood vessels of liver tumors using the surface mesh generated by the mesh generation unit (150). The staging analysis unit (160) performs the function of identifying the boundaries of liver tumors in the generated surface mesh and calculating the volume, maximum diameter, and centroid coordinates of each tumor.

[0064] The staging analysis unit (160) calculates the total tumor volume by performing volume integration on the triangular mesh of the tumor based on vertex and face information of the surface mesh to analyze the size of the liver tumor. The staging analysis unit (160) also includes a function to calculate the Euclidean distance between all points on the mesh to measure the maximum diameter of each tumor and to set the longest distance as the maximum diameter.

[0065] The staging analysis unit (160) performs the function of determining whether there is contact between the liver tumor and major blood vessels by referring to the surface mesh of the blood vessels to calculate the minimum distance between the liver tumor and the blood vessels, and determining it as vascular invasion if it is below a specific threshold. The staging analysis unit (160) measures the diameter of the blood vessels to determine the importance of the blood vessels, and analyzes whether the liver tumor is in contact with major blood vessels by checking whether the diameter of major vessels is 5 mm or more.

[0066] The staging analysis unit (160) transmits information on the size, number, and vascular invasion of the analyzed liver tumors to the staging classification unit (170) so that the stage of the hepatocellular carcinoma can be finally classified. The staging analysis unit (160) also provides location information of the tumors based on the analyzed data, thereby supporting medical staff in more accurately identifying the spatial distribution of the liver tumors.

[0067] The staging unit (170) is a component that classifies the progression stage of a liver tumor by applying TNM (Tumor, Node, Metastasis) staging criteria based on the size, number, and contact with major blood vessels of the liver tumors analyzed by the staging analysis unit (160). The staging unit (170) evaluates the size of the liver tumor (T-Stage), the presence of lymph node metastasis (N-Stage), and the presence of distant metastasis (M-Stage), respectively, and then performs the function of determining the final stage by synthesizing this information.

[0068] The staging unit (170) applies TNM criteria based on the maximum diameter and number of tumors provided by the staging analysis unit (160) to determine the T-Stage. The staging unit (170) performs the function of classifying as T1a if the tumor size is 2cm or less and is a single lesion, and as T1b or T2 if there is a single tumor or multiple tumors exceeding 2cm and 5cm or less. The staging unit (170) also includes the function of setting to T3 if there is a single tumor exceeding 5cm or multiple tumors within the liver, and checking for vascular invasion and setting to T4 if there is vascular invasion.

[0069] The staging classification unit (170) checks for metastasis to regional lymph nodes to determine the N-Stage. If lymph node invasion information is not detected in the staging analysis unit (160), it is set to N0, and if there is lymph node metastasis, it performs the function of classifying it as N1.

[0070] The staging unit (170) checks the major blood vessel contact information of the tumor and whether there is distant metastasis provided by the staging analysis unit (160) to determine the M-Stage. The staging unit (170) determines that there is a high probability of distant metastasis if the liver tumor has invaded major blood vessels, and performs the function of setting it to M1 if the tumor has metastasized to organs other than the liver (e.g., lungs, bones, brain).

[0071] The staging classification unit (170) determines the final stage by synthesizing the analysis results of T-Stage, N-Stage, and M-Stage, and performs the function of converting this into stages (Stage I, II, III, IV). The staging classification unit (170) provides the staging determination results to the staging analysis unit (160) and medical staff, and may include a function to correct by referring to previous clinical data and patient records to increase the reliability of the staging determination. The staging classification unit (170) supports the visualization of the finally determined staging information so that medical staff can intuitively check it, and links the staging information with a medical database so that it can be utilized in establishing a treatment plan for the patient.

[0072] FIG. 2 is a diagram (200) illustrating the process of segmentation and staging of hepatocellular carcinoma according to one embodiment.

[0073] Figure 2 illustrates the entire process performed by an automatic liver tumor segmentation and staging device that automatically classifies the stage of hepatocellular carcinoma in stages. The automatic liver tumor segmentation and staging device includes image preprocessing, image enhancement, image segmentation, surface mesh generation, staging analysis, and staging stages, and each stage is interconnected to perform automated staging.

[0074] The flow shown in Figure 2 schematically visualizes the process of segmenting the liver, hepatic vessels, and tumors from an input CT image and determining the stage of hepatocellular carcinoma based on this.

[0075] The image receiver receives an abdominal CT image and transmits it to the image preprocessing unit. To improve the quality of the input abdominal CT image, the image preprocessing unit performs normalization of pixel values ​​to between -140 HU and 260 HU, applies linear scaling, and removes the background. To remove noise within the CT image, the image preprocessing unit applies an anisotropic diffusion filter and performs histogram equalization to enhance image contrast. Additionally, the image preprocessing unit performs the function of removing unnecessary background information to maximize learning and segmentation performance while maintaining image resolution.

[0076] The image augmentation unit receives preprocessed CT images as input and performs the function of applying data augmentation techniques. The image augmentation unit applies various augmentation techniques to ensure diversity in the training data and maximize the generalization performance of the model. By applying geometric transformation techniques, the image augmentation unit performs flip, rotation, zoom, translation, affine transformation, and perspective transformation.

[0077] In addition, the image enhancement unit performs intensity transformations to apply gamma correction, contrast stretching, and intensity shifting, and adjusts to reflect various shooting conditions and anatomical changes.

[0078] The image augmentation unit performs the function of adding Gaussian noise, Salt and Pepper noise, Speckle noise, and Poisson noise to reflect the noise characteristics of CT images, thereby enabling the construction of a deep learning model robust to noise.

[0079] The image segmentation unit receives CT images augmented by the image augmentation unit and performs the function of segmenting the liver, hepatic vessels, and liver tumors using 3D U-Net. The image segmentation unit applies a 3D U-Net-based deep learning model to automatically segment multiple organ and lesion regions, and applies skip connections in the decoder stage to multi-scale feature maps extracted in the encoder stage to enable more precise extraction of the boundaries of the liver, hepatic vessels, and liver tumors. To improve segmentation accuracy, the image segmentation unit performs the function of optimizing loss functions such as cross-entropy loss, dice count loss, and category distance loss, and ensures optimal segmentation performance by automatically adjusting kernel size and padding values ​​according to the resolution and area size of the CT image.

[0080] The mesh generation unit performs the function of generating a surface mesh based on liver, liver blood vessel, and liver tumor data segmented by the image segmentation unit. The mesh generation unit generates surface meshes of the liver, liver blood vessels, and liver tumors by applying the Flying Edges algorithm, and provides intuitive visualization using 3D modeling technology. In addition, the mesh generation unit performs the function of calculating the mesh normal vector and optimizing the mesh structure to more clearly identify the boundaries of each tumor.

[0081] The Staging Analysis Unit performs the function of analyzing the number, size, and contact with major blood vessels of liver tumors using the surface mesh generated by the Mesh Generation Unit. The Staging Analysis Unit calculates the maximum diameter, volume, and minimum distance to major vessels of each tumor to extract key information necessary for staging classification. The Staging Classification Unit performs the function of determining the final stage by applying TNM (Tumor, Node, Metastasis) staging criteria based on the information analyzed by the Staging Analysis Unit. The Staging Classification Unit classifies the final stage by analyzing the size and number of tumors to determine T-Stage, evaluating the presence of lymph node metastasis to determine N-Stage, and assessing the presence of distant metastasis to determine M-Stage. The Staging Classification Unit determines the progression stage of hepatocellular carcinoma based on TNM staging criteria, provides the staging results to medical staff, and supports intuitive analysis through visual feedback.

[0082] Figure 3 is a diagram illustrating the process of generating training data according to one embodiment.

[0083] Figure 3 shows the entire process of generating a training dataset using an automatic liver tumor segmentation and staging device that automatically classifies the stage of hepatocellular carcinoma in steps.

[0084] The liver tumor automatic segmentation and staging device includes a series of processes that, after acquiring original CT images, segment the liver, hepatic vessels, and liver tumors, and integrate the data to convert it into training data. The flow shown in FIG. 3 visually illustrates the steps of original image acquisition (310), liver region segmentation (320), and hepatic vessel and liver tumor labeling integration (330) performed during the training data generation process.

[0085] The image receiving unit (110) performs the function of receiving an abdominal CT image and transmitting it to the image preprocessing unit (120). The image receiving unit (110) receives abdominal CT image data from a medical image storage system (PACS), a hospital server, a cloud-based image storage, an external storage device (DICOM file), etc., and performs the function of storing the received CT image data while maintaining the original resolution. The image receiving unit (110) includes the function of converting to a specific resolution and format (DICOM, NIfTI, etc.) when necessary while maintaining the metadata of the CT image (shooting information, resolution, patient information, shooting conditions, etc.). The original CT image (310) includes a cross-section of the patient's abdomen, and the pixel values ​​of the CT image are expressed in Hounsfield Units (HU).

[0086] The image segmentation unit (140) receives the CT image augmented by the image enhancement unit (130) and performs the function of segmenting the liver, hepatic vessel, and liver tumor using 3D U-Net. The image segmentation unit (140) identifies the liver within the CT image and applies a 3D U-Net-based deep learning model to accurately extract the outer boundary of the liver. The 3D U-Net model extracts multi-layered features from the input 3D CT image and applies skip connections in the decoder stage to enable more precise segmentation of the boundaries of the liver, hepatic vessel, and liver tumor. The image segmentation unit (140) performs the function of aligning the liver segmentation results with the original pixel coordinates of the CT image and outputting the segmented liver region in the form of a separate binary mask.

[0087] The image segmentation unit (140) includes a function to improve the accuracy of liver region segmentation by optimizing loss functions such as cross-entropy loss, dice count loss, and category distance loss. The liver region segmentation result (320) is displayed as a red-colored overlay, highlighting the liver region of the CT image.

[0088] The mesh generation unit (150) performs the function of generating a surface mesh based on liver, liver blood vessel, and liver tumor data segmented by the image segmentation unit (140). The mesh generation unit (150) generates a surface mesh of the liver, liver blood vessel, and liver tumor by applying the Flying Edges algorithm and provides intuitive visualization by utilizing 3D modeling technology. The mesh generation unit (150) performs the function of calculating the normal vector of the mesh and optimizing the mesh structure to more clearly identify the boundaries of each tumor. The image segmentation unit (140) segments the liver blood vessel and liver tumor individually and combines them with a liver region mask (320) to generate final training data (330) containing multi-class labels. Each color represents a specific anatomical structure, for example, liver tumors are displayed in green and liver blood vessels in pink.

[0089] The staging analysis unit (160) performs the function of analyzing the number, size, and contact with major blood vessels of liver tumors using the surface mesh generated by the mesh generation unit (150), and transmits the analyzed data to the staging classification unit (170) so that the stage can be determined.

[0090] The image segmentation unit (140) performs the function of aligning the segmented data of the liver, liver blood vessels, and liver tumors with the CT image and converting them to the format of the training data. The training data is stored as a training dataset so that a deep learning model can automatically recognize the liver, liver blood vessels, and liver tumors in the CT image. The staging analysis unit (160) performs the function of evaluating the consistency of the morphological features of the segmented region and the actual liver structure within the CT image to verify the quality of each training data. The verified training data is trained in the staging classification unit (170) and used as a dataset for automatic staging of hepatocellular carcinoma.

[0091] Figure 4 is a diagram illustrating an image preprocessing process according to one embodiment.

[0092] FIG. 4 illustrates the entire process of an automatic liver tumor segmentation and staging device that automatically classifies the stage of hepatocellular carcinoma, receiving and preprocessing an abdominal CT image step by step. The automatic liver tumor segmentation and staging device performs a series of image preprocessing steps, such as image reception, pixel value normalization, linear scaling, background removal, noise reduction, and contrast enhancement, to maximize the accuracy of subsequent image analysis and learning. The flow shown in FIG. 4 is a structure that visually explains how the preprocessing process improves image quality by comparing the original CT image (410) acquired by the image receiving unit (110) with the preprocessed result image (420) performed by the image preprocessing unit (120).

[0093] The image receiving unit (110) performs the function of receiving an abdominal CT image and transmitting it to the image preprocessing unit (120). The image receiving unit (110) receives abdominal CT image data from a medical image storage system (PACS), a hospital server, a cloud-based image storage, or an external storage device (DICOM file). The image receiving unit (110) stores the received CT image data while maintaining the original resolution and performs the function of preserving image metadata (shooting information, resolution, patient information, shooting conditions, etc.). The image receiving unit (110) converts the image data to a specific resolution and format (DICOM, NIfTI, etc.) as needed to optimize it for subsequent processing. The original CT image (410) represents the abdominal CT image obtained through this receiving process, and is in a state where no separate normalization, contrast adjustment, or noise removal has been applied in the original state. The original CT image (410) has pixel values ​​expressed in Hounsfield Units (HU), and the liver, hepatic vessel, and liver tumor exist in a state where they are not clearly distinguished.

[0094] The image preprocessing unit (120) performs multiple preprocessing steps to improve the quality of the abdominal CT image received from the image receiving unit (110) and to maximize the accuracy of image segmentation. The image preprocessing unit (120) performs the function of normalizing the pixel values ​​of the input CT image to between -140 HU and 260 HU to increase the visibility of major anatomical structures such as the liver and tumors. The image preprocessing unit (120) converts the pixel values ​​into values ​​in the range of 0 to 1 by linear scaling, thereby enabling the deep learning model to interpret the input data more effectively.

[0095] The image preprocessing unit (120) performs the function of removing unnecessary background areas within the image to leave only the liver, liver blood vessels, and liver tumors that are the subjects of analysis. In the background removal process, an algorithm is applied to identify the outer edges of the patient's body and surrounding tissues within the image and to mask areas outside the Region of Interest (ROI). The image preprocessing unit (120) applies an anisotropic diffusion filter to remove noise within the CT image. The anisotropic diffusion filter performs the role of removing unnecessary noise while maintaining the main boundaries of the image, and helps preserve structural information within the image.

[0096] The image preprocessing unit (120) performs histogram equalization to improve the contrast of the image. Histogram equalization serves to increase the contrast between cancerous lesions and normal tissues by adjusting the distribution of pixel values ​​within the CT image. Through this process, the boundary between the liver tumor and surrounding tissues can be distinguished more clearly in the CT image. The preprocessed CT image (420) represents the result of undergoing this series of image preprocessing processes, and when compared to the original CT image (410), the outlines of major anatomical structures are seen more clearly, and the contrast between the tumor and surrounding tissues is improved. The image preprocessing unit (120) plays a key role in optimizing the quality of the image to increase the accuracy of subsequent image enhancement, image segmentation, and staging analysis processes.

[0097] FIG. 5 is a diagram (500) illustrating the network architecture of a 3D U-Net according to one embodiment.

[0098] Figure 5 shows the structure of the 3D U-Net used to perform image segmentation by an automatic liver tumor segmentation and staging device that automatically classifies the stage of hepatocellular carcinoma in stages.

[0099] The liver tumor automatic segmentation and staging device utilizes a 3D U-Net-based deep learning model to precisely segment the liver, hepatic vessels, and liver tumors within the CT image in the image segmentation unit (140). The image segmentation unit (140) uses the 3D U-Net model to extract multi-layered features and applies skip connections in the decoder stage to generate segmentation results with high accuracy. FIG. 5 explains the network structure of this 3D U-Net by dividing it into the encoder stage and the decoder stage, and visually illustrates the flow of each operation.

[0100] The image segmentation unit (140) receives 3D medical data from a CT image and transmits it to the first layer of the 3D U-Net network. The input layer processes CT image data converted into 3 channels (3D Volume) and performs a first convolution operation to extract low-level features. The first convolution operation performed in the input layer applies a filter of kernel size 3×3×3 to extract local features within the image and performs Batch Normalization to improve learning stability. Subsequently, a Rectified Linear Unit (ReLU) is applied as an activation function to add non-linearity, enabling the network to learn more complex patterns.

[0101] In the encoder stage, image features are extracted by successively applying 3D convolution operations, the ReLU activation function, and Batch Normalization. The first encoder layer learns the anatomical features of the input image by applying 3D convolution filters and analyzes spatial patterns based on this. After convolution, batch normalization is performed to normalize the distribution of internal activation values, and a non-linear activation function is applied to enhance the model's expressive power. At the end of each encoder block, Max Pooling is performed, reducing the size of the feature map by half while lowering the resolution to retain only essential information. The reason for performing Max Pooling is to maximize the model's training efficiency by removing unnecessary information to extract high-level features and reducing the amount of computation. In the encoder stage, the number of feature map channels gradually increases, generating feature maps with 64, 128, 256, and 512 channels, respectively. This structure is designed to allow the network to learn increasingly complex features, enabling more abstract representations as the layers deepen.

[0102] The bottleneck layer compresses high-level features extracted in the encoder stage and refines them for use in the subsequent decoder stage. In the bottleneck layer, 3D convolution operations, the activation function (ReLU), and batch normalization are applied sequentially to learn more complex features. The bottleneck layer is located at the center of the network and plays a crucial connecting role in transmitting information learned in the encoder stage to the decoder stage.

[0103] In the decoder stage, up-convolution operations are performed to gradually restore the resolution of the feature map, and skip connections are applied to preserve low-level features extracted in the encoder stage.

[0104] Upsampling is the reverse process of max pooling, which increases the size of the feature map and restores it to a resolution close to the original. After upsampling, 3D convolution is applied to relearn the features, and the model's performance is improved by applying an activation function (ReLU) and batch normalization.

[0105] Skip Connections directly connect encoder and decoder layers with the same resolution, ensuring that low-level spatial information is not lost. This enables the model to perform precise segmentation while maintaining fine boundaries.

[0106] The final output layer of the decoder performs 3D convolution to ultimately generate probability maps for the liver, liver blood vessels, and liver tumors. Each probability map represents the probability of belonging to a specific class (e.g., liver, liver blood vessels, liver tumor), and finally outputs the segmented result by assigning classes at the pixel level. The final output layer of the decoder performs 1×1×1 convolution operations to generate probability maps for multi-class prediction, and assigns the liver, liver blood vessel, and liver tumor regions to different classes, respectively.

[0107] In the output layer, a softmax activation function is applied to calculate the probability for each class for each pixel, and the pixel is classified into the class with the highest probability value. The output segmentation result is transmitted to subsequent analysis stages (mesh generation unit (150), weapon analysis unit (160), weapon classification unit (170)) through the image segmentation unit (140), and a final weapon classification is performed based on this.

[0108] Figure 6 is a diagram showing the result of image segmentation performed by an automatic liver tumor segmentation and staging device that automatically classifies the stage of hepatocellular carcinoma.

[0109] In FIG. 6, by comparing the original CT image (610) and the segmented result image (620) processed through the image segmentation unit (140), it is possible to visually confirm whether the trained 3D U-Net model effectively segments the liver, hepatic vessel, and tumor.

[0110] The image segmentation unit (140) performs the function of segmenting the liver, liver blood vessels, and liver tumors using a 3D U-Net-based deep learning model. The 3D U-Net model extracts multi-layered features from the input CT image and generates precise segmentation results using an encoder and decoder structure. It performs precise segmentation that reflects high-level features while maintaining low-level feature information by applying skip connections.

[0111] The original CT image (610) represents the state before image segmentation is performed, in which the liver, liver blood vessels, and liver tumor within the CT image are not clearly distinguished. The original CT image (610) includes pixel values ​​expressed in Hounsfield Units (HU), and it is difficult to clearly identify the boundary between the lesion and surrounding tissues through direct visual analysis alone.

[0112] The segmentation result image (620) shows the result of the image segmentation unit (140) automatically segmenting the liver, hepatic vessels, and liver tumors within the CT image using a trained 3D U-Net model. The red area represents the liver, the green area represents the hepatic vessels, and the pink area represents the liver tumors, and each structure is visualized so that they can be clearly distinguished. The image segmentation unit (140) maximizes segmentation accuracy by optimizing loss functions such as cross-entropy loss, dice count loss, and category distance loss.

[0113] The staging analysis unit (160) performs the function of analyzing the number, size, and contact with major blood vessels of liver tumors using the surface mesh generated by the mesh generation unit (150). The staging analysis unit (160) includes the function of checking whether the diameter of the blood vessel is 5 mm or more to determine whether there is contact between the liver tumor and the major blood vessel, and based on this, the staging classification unit (170) determines the final stage.

[0114] The staging unit (170) performs the function of classifying the progression stage of hepatocellular carcinoma by applying TNM (Tumor, Node, Metastasis) staging criteria. The staging unit (170) analyzes the size and number of tumors to determine the T-Stage, evaluates whether there is lymph node metastasis to determine the N-Stage, and determines whether there is distant metastasis to determine the M-Stage to determine the final stage. The staging unit (170) performs automated staging based on data analyzed from the segmentation result image (620) and ultimately enables intuitive feedback to be provided to medical staff.

[0115] FIG. 6 is configured to allow visual evaluation of the performance of the image segmentation unit (140) by comparing the original CT image (610) and the segmentation result image (620). Through FIG. 6, it can be confirmed that the 3D U-Net model can precisely segment the liver, liver blood vessels, and liver tumors, and the reliability of subsequent staging analysis and staging classification stages can be improved.

[0116] FIG. 7 visually illustrates the entire process of a liver tumor automatic segmentation and staging device (700) that automatically classifies the stage of hepatocellular carcinoma, segmenting the liver, hepatic vessel, and liver tumor in a CT image, and then generating a surface mesh.

[0117] FIG. 7 describes a process of improving the accuracy of weapon analysis based on 3D modeling by comparing surface meshes (710, 720) generated through a mesh generation unit (150).

[0118] The mesh generation unit (150) performs the function of generating a 3D surface mesh based on liver, liver blood vessel, and liver tumor data segmented by the image segmentation unit (140). In the surface mesh generation process, the Flying Edges algorithm is utilized to allow for more intuitive visualization of the boundaries of each organ. The Flying Edges algorithm operates by utilizing volume data of the CT image to generate a surface along the boundary of each pixel, thereby efficiently constructing a 3D model.

[0119] The surface mesh (710) represents a 3D model containing the liver, hepatic vessels, and liver tumor, with major structures distinguished by color. The liver is displayed in light yellow, the hepatic vessels in green, and the liver tumor in red. The surface mesh (710) is configured to allow for an intuitive analysis of the positional relationship between the vascular structures and the tumor within the liver.

[0120] The surface mesh (720) represents the results of additional staging analysis performed on the surface mesh (710) and includes a function to quantitatively analyze the relationship between liver tumors and vascular structures. The staging analysis unit (160) analyzes the number of tumors, tumor size, and vascular invasion of liver tumors using the surface mesh, and based on this, the staging classification unit (170) automatically determines the stage according to the TNM staging criteria.

[0121] The staging analysis unit (160) calculates the maximum diameter, volume, and minimum distance to major vessel of each tumor, and performs the function of determining whether the diameter of the vessel is 5 mm or more to check for contact with the vessel. In the staging analysis process, if the diameter of the vessel is 5 mm or more, it is classified as stage IV, and stages T1 to T3 are determined according to the size of the tumor.

[0122] The staging unit (170) determines the final stage by applying TNM staging criteria. The T-Stage is determined based on the size and number of tumors, the N-Stage evaluates whether there is lymph node metastasis, and the M-Stage determines whether there is distant metastasis to classify the final stage. The staging unit (170) performs automated staging based on data analyzed from the surface mesh (720) and enables intuitive feedback to be provided to medical staff.

[0123] FIG. 7 is configured to visually demonstrate that the reliability of the staging analysis process using a surface mesh can be improved by comparing the surface mesh (710) and the surface mesh (720). It can be seen that precise analysis using a 3D surface mesh can improve the accuracy of staging classification of hepatocellular carcinoma and help in establishing a subsequent treatment plan.

[0124] FIG. 8 is a drawing (800) illustrating the liver tumor division result according to one embodiment.

[0125] Figure 8 shows the entire process of an automatic liver tumor segmentation and staging device that automatically classifies the stage of hepatocellular carcinoma, segmenting the liver, hepatic vessel, and liver tumor based on CT images step by step.

[0126] FIG. 8 is configured to evaluate how precisely a trained 3D U-Net model segments the liver, liver blood vessels, and liver tumors by comparing the original CT image (810), the segmented result image (820), the original Ground Truth (830), and the overlay image (840).

[0127] The image receiving unit (110) performs the function of receiving an abdominal CT image and transmitting it to the image preprocessing unit (120). The image receiving unit (110) receives data from a medical image storage system (PACS), a hospital server, a cloud-based image storage, etc., and stores it while maintaining the original resolution. The original CT image (810) represents the abdominal CT image obtained through this process and refers to the state before preprocessing or segmentation is applied.

[0128] The image preprocessing unit (120) performs preprocessing processes such as pixel value normalization, contrast enhancement, and noise reduction on the original CT image (810). Through this, optimized input data is generated so that the image segmentation unit (140) can segment the liver, liver blood vessels, and liver tumors more accurately.

[0129] The image segmentation unit (140) performs the function of precisely segmenting the liver, liver blood vessels, and liver tumors using a 3D U-Net-based deep learning model. The 3D U-Net model automatically identifies each anatomical structure in the CT image by extracting multi-layered features, and generates high-accuracy segmentation results by utilizing encoder and decoder structures. The segmented result image (820) shows the results performed by the image segmentation unit (140), and allows one to confirm that each structure (liver, liver blood vessels, liver tumor) is individually segmented.

[0130] The existing Ground Truth (830) is a set of correct data manually labeled by medical staff and serves as reference data when performing staging analysis of actual hepatocellular carcinoma. This Ground Truth (830) is compared with the segmented result image (820) to serve as a standard for evaluating how similarly the model segmented the actual structure. The overlay image (840) is the result of visually comparing the segmented result image (820) and the existing Ground Truth (830) by overlaying them, allowing for clear identification of the areas accurately segmented by the model and the areas where errors occurred.

[0131] The image segmentation unit (140) maximizes segmentation accuracy by optimizing loss functions such as cross-entropy loss, dice loss, and category distance loss. Additionally, the mesh generation unit (150) generates a 3D surface mesh of the liver, liver blood vessels, and liver tumors based on the segmented result image (820), and uses this to analyze the number, size, and contact with major blood vessels of liver tumors in the staging analysis unit (160).

[0132] The staging analysis unit (160) evaluates the tumor size, number of tumors, and vascular invasion of liver tumors based on a surface mesh, and utilizes this to determine the final stage according to TNM staging standards in the staging classification unit (170). The staging classification unit (170) automatically classifies the stage of hepatocellular carcinoma by considering T-Stage, N-Stage, and M-Stage, allowing medical staff to check the staging classification results more intuitively.

[0133] FIG. 8 is configured to allow visual evaluation of the performance of the image segmentation unit (140) by comparing the original CT image (810), the segmented result image (820), the existing Ground Truth (830), and the overlay image (840). Through this, it can be confirmed that the 3D U-Net model can precisely segment the liver, liver blood vessels, and liver tumors, and the reliability of the subsequent staging analysis and staging classification stages can be improved.

[0134] FIG. 9 is a diagram illustrating the staging process of hepatocellular carcinoma according to one embodiment.

[0135] In particular, FIG. 9 shows the entire process in steps of an automatic liver tumor segmentation and staging device (900) that automatically classifies the stage of hepatocellular carcinoma, quantitatively analyzing the size and vascular invasion of the liver tumor based on a surface mesh, and applying TNM staging criteria (Tumor, Node, Metastasis).

[0136] FIG. 9 visually explains the process in which the staging analysis unit (160) evaluates the characteristics of a tumor using a surface mesh generated by the mesh generation unit (150), and the staging classification unit (170) automatically determines the stage.

[0137] The staging analysis unit (160) performs the function of analyzing the number of tumors, maximum tumor diameter, and vascular invasion based on a surface mesh. The staging analysis unit (160) first evaluates whether the liver tumor has invaded a major blood vessel using a vessel diameter determination module at the invasion point. The vessel diameter determination module at the invasion point identifies the contact area between the liver tumor and the major blood vessel (hepatic vessel) through the surface mesh and determines whether the diameter of the vessel is 5 mm or more. If the diameter of the vessel is 5 mm or more, it is considered that vascular invasion has occurred, and the tumor is classified as Stage IV. If the diameter of the vessel is less than 5 mm, an additional staging classification process is performed.

[0138] The staging analysis unit (160) measures the maximum diameter of a tumor using a tumor size evaluation module for tumors without vascular invasion. The tumor size evaluation module performs the function of determining whether the diameter of the tumor is 5 cm or more by using tumor size information extracted from a surface mesh. If the tumor size is 5 cm or more, the tumor is classified as T3 stage (Stage III). If the tumor size is less than 5 cm, additional analysis is performed for staging evaluation.

[0139] The staging analysis unit (160) checks whether it is a single tumor using a single tumor diameter evaluation module (940) when the tumor size is less than 5 cm. The single tumor diameter evaluation module (940) performs the function of determining whether it is a single tumor by checking the number of tumors extracted from the surface mesh. If a single tumor exists and the tumor size is less than 2 cm, it is classified as Stage I. If it is a single tumor but the size is 2 cm or more, or if multiple tumors exist, the tumor is classified as Stage II.

[0140] The staging unit (170) determines the final stage based on the results of the analysis of tumor size and vascular invasion performed by the staging analysis unit (160), and automatically determines the stage of progression of hepatocellular carcinoma by applying TNM staging criteria. The staging unit (170) first evaluates whether there is vascular invasion, and if there is invasion, it determines Stage IV, and if there is no invasion, it determines the stage according to the size of the tumor. If the tumor size is 5 cm or more, it is classified as Stage III; if it is a single tumor less than 2 cm, it is classified as Stage I; and otherwise, it is classified as Stage II.

[0141] The staging unit (170) automatically performs this staging classification process and supports medical staff in objectively evaluating the stage of hepatocellular carcinoma and optimizing the patient's treatment plan. The staging analysis unit (160) and the staging unit (170) automatically classify the stage of the liver tumor based on information analyzed from the surface mesh, thereby enabling a more quantitative evaluation than the existing manual analysis method. FIG. 9 illustrates the automated staging classification process based on the surface mesh through the invasion point blood vessel diameter determination module, the tumor size evaluation module, and the single tumor diameter evaluation module (940), and is configured to allow visual confirmation that the system can improve the reliability of the staging evaluation through objective data-based analysis.

[0142] FIG. 10 is a diagram illustrating the operation method of an automatic liver tumor segmentation and staging classification device that automatically classifies the stage of hepatocellular carcinoma according to one embodiment.

[0143] Figure 10 illustrates the entire process in which an automatic liver tumor segmentation and staging device receives an abdominal CT image and performs image preprocessing, image enhancement, image segmentation, surface mesh generation, staging analysis, and final staging determination step by step.

[0144] Each step is performed by components such as an image receiving unit (110), an image preprocessing unit (120), an image enhancement unit (130), an image segmentation unit (140), a mesh generation unit (150), a staging analysis unit (160), and a staging classification unit (170), and finally, the stage of hepatocellular carcinoma is automatically determined by applying the TNM staging standard (Tumor, Node, Metastasis Staging System).

[0145] For the step (1001) of receiving abdominal CT images, the image receiving unit (110) receives abdominal CT image data and performs a preparation process for subsequent processing. The image receiving unit (110) performs the function of receiving abdominal CT images from a medical image storage system (PACS), a hospital server, a cloud-based medical database, an external storage device (DICOM file), etc.

[0146] The image receiving unit (110) stores the received CT image data while maintaining the original resolution and pixel information, and includes a function to preserve metadata (shooting information, resolution, patient ID, shooting conditions, etc.) for accurate image processing in subsequent steps.

[0147] The image preprocessing unit (120) performs a preprocessing step of normalizing the pixel values ​​of the input abdominal CT image, performing linear scaling, and removing the background (1002).

[0148] The abdominal CT image data received from the image receiver (110) is converted into a form that is easy to analyze by performing processes such as normalization, linear scaling, background removal, noise reduction, and contrast enhancement.

[0149] The image preprocessing unit (120) normalizes pixel values ​​to between -140 HU and 260 HU and linearly transforms them into a range of 0 to 1 to adjust them so that machine learning and deep learning models can process them. The image preprocessing unit (120) includes a function to remove unnecessary background areas within the CT image and adjust it to retain only the liver, hepatic vessel, and tumor, which are the subjects of analysis. Additionally, the image preprocessing unit (120) performs histogram equalization to improve image contrast and applies an anisotropic diffusion filter to minimize noise within the image.

[0150] The image augmentation unit (130) applies a data augmentation technique to the preprocessed image (1003).

[0151] The CT image data refined in the image preprocessing unit (120) is input, and data augmentation is performed. The image augmentation unit (130) applies various augmentation techniques, such as geometric transformations, pixel intensity transformations, and noise injection, to maximize the generalization performance of model learning. The image augmentation unit (130) performs axis flipping, rotation, zooming, translation, and affine transformation, and generates a training dataset reflecting various CT image environments through gamma correction, contrast stretching, and the addition of Gaussian noise.

[0152] The image segmentation unit (140) segments the liver, hepatic vessel, and liver tumor in the augmented image (1004).

[0153] Data generated by the image augmentation unit (130) is input, and the liver, liver blood vessels, and liver tumors are segmented using a 3D U-Net-based deep learning model. The image segmentation unit (140) extracts multi-layered features using an encoder and decoder structure, and generates precise segmentation results by applying skip connections to reflect low-level and high-level information. Finally, the image segmentation unit (140) classifies the liver, liver blood vessels, and liver tumors at the pixel level, generates a probability map for each class, and outputs the segmented regions.

[0154] The mesh generation unit (150) generates a surface mesh based on the divided liver, liver blood vessel and liver tumor regions (1005).

[0155] A surface mesh is generated based on the segmentation results generated by the image segmentation unit (140). The mesh generation unit (150) forms a surface along the boundaries of the 3D volume data by applying the Flying Edges algorithm and converts it into an intuitive 3D model. The mesh generation unit (150) visualizes the liver, liver blood vessels, and liver tumors in different colors in the surface mesh and is configured to allow for precise analysis of the positional relationship between the liver tumor and major blood vessels.

[0156] The staging analysis unit (160) analyzes the number, size, and contact with major blood vessels of liver tumors using the generated surface mesh (1006).

[0157] Using the surface mesh generated by the mesh generation unit (150), the number of tumors, tumor size, and vascular invasion of major blood vessels are analyzed. The staging analysis unit (160) analyzes the contact area between the tumor and the blood vessel by utilizing the invasion point blood vessel diameter determination module (162) to determine whether there is vascular invasion, and classifies it as Stage IV if the diameter of the blood vessel is 5 mm or more. Additionally, the staging analysis unit (160) uses the tumor size evaluation module (164) to classify it as Stage III if the maximum diameter of the tumor is 5 cm or more, Stage I if it is a single tumor less than 2 cm, and Stage II otherwise.

[0158] The staging section (170) applies TNM staging criteria based on the analyzed contact status and classifies the stage of progression of the liver tumor (1007).

[0159] The final stage is determined by applying the TNM Staging System based on the results of the tumor size and vascular invasion analysis performed by the staging analysis unit (160). The staging classification unit (170) first checks for vascular invasion; if there is vascular invasion, it is classified as Stage IV, and if there is no vascular invasion, the stage is determined based on the size and number of tumors. The staging classification unit (170) finally provides the classified stage information to the medical staff, and can establish a treatment plan by quantitatively evaluating the stage of hepatocellular carcinoma.

[0161] The device described above may be implemented as a hardware component, a software component, and / or a combination of a hardware component and a software component. For example, the device and components described in the embodiments may be implemented using one or more general-purpose or special-purpose computers, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing unit may execute an operating system (OS) and one or more software applications executed on said operating system. Additionally, the processing unit may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing unit may be described as being used as a single unit, but those skilled in the art will understand that the processing unit may include multiple processing elements and / or multiple types of processing elements. For example, the processing unit may include multiple processors or one processor and one controller. In addition, other processing configurations, such as parallel processors, are also possible.

[0162] Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or command the processing unit independently or collectively. Software and / or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal wave so as to be interpreted by the processing unit or to provide instructions or data to the processing unit. Software may be distributed over networked computer systems and may be stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

[0163] The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either alone or in combination. The program instructions recorded on the medium may be those specifically designed and configured for the embodiment, or they may be those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.

[0164] Although the embodiments have been described above with reference to the limited drawings, those skilled in the art can make various modifications and variations from the description above. For example, suitable results can be achieved even if the described techniques are performed in a different order than described, and / or the components of the described system, structure, device, circuit, etc. are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents.

[0165] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims set forth below.

Claims

Claim 1 A liver tumor automatic segmentation and staging device for automatically classifying the stage of hepatocellular carcinoma (HCC), comprising: an image receiving unit that receives an abdominal CT image; an image preprocessing unit that normalizes pixel values, performs linear scaling, and removes backgrounds for the input abdominal CT image; an image augmentation unit that applies a data augmentation technique to the preprocessed image; an image segmentation unit that segments the liver, hepatic vessels, and liver tumors in the augmented image; a mesh generation unit that generates a surface mesh based on the segmented liver, hepatic vessel, and liver tumor regions; a staging analysis unit that analyzes the number, size, and contact with major blood vessels of liver tumors using the generated surface mesh; and a staging classification unit that applies TNM (Tumor, Node, Metastasis) staging criteria and classifies the progression stage of liver tumors based on the analyzed contact status. Claim 2 The liver tumor automatic segmentation and staging device according to claim 1, wherein the image preprocessing unit normalizes the pixel values ​​of the abdominal CT image to between -140 HU and 260 HU, performs linear scaling, and removes the background. Claim 3 In paragraph 2, the image preprocessing unit is characterized by applying an anisotropic diffusion filter to remove noise within the image and performing histogram equalization to enhance contrast, in an automatic liver tumor segmentation and staging device. Claim 4 The liver tumor automatic segmentation and staging device according to claim 1, wherein the image enhancement unit performs at least one of axis flip, rotation, zoom, pixel intensity shift, and affine transformation to improve data learning performance. Claim 5 The liver tumor automatic segmentation and staging device according to claim 1, wherein the image segmentation unit segments the liver, hepatic vessel, and liver tumor, and generates a surface mesh based on the segmentation results. Claim 6 In claim 5, the above image segmentation unit is characterized by extracting the boundaries of the liver, liver blood vessels, and liver tumor through a skip connection that merges the feature map of the encoder stage in the decoder stage, in an automatic liver tumor segmentation and staging device. Claim 7 The liver tumor automatic segmentation and staging device according to claim 1, wherein the mesh generation unit generates surface meshes of the liver, liver blood vessels, and liver tumors by applying a Flying Edges algorithm based on image segmentation results. Claim 8 In claim 1, the liver tumor automatic segmentation and staging classification device is characterized in that the staging analysis unit analyzes the number, size, and contact with major blood vessels of liver tumors based on a surface mesh. Claim 9 In claim 8, the above staging analysis unit is characterized by checking whether the diameter of the blood vessel is 5 mm or more in order to determine whether there is contact between the liver tumor and major blood vessels, in an automatic liver tumor segmentation and staging classification device. Claim 10 The liver tumor automatic segmentation and staging device according to claim 1, wherein the staging unit analyzes the size, number, and vascular invasion of liver tumors by applying TNM staging standards and determines the stage based thereon. Claim 11 A method of operation for an automatic liver tumor segmentation and staging device for automatically classifying the stage of hepatocellular carcinoma (HCC), comprising: receiving an abdominal CT image; a preprocessing step of normalizing pixel values ​​of the input abdominal CT image, performing linear scaling, and removing the background; applying a data augmentation technique to the preprocessed image; segmenting the liver, hepatic vessels, and liver tumors in the augmented image; generating a surface mesh based on the segmented liver, hepatic vessel, and liver tumor regions; analyzing the number, size, and contact with major blood vessels of liver tumors using the generated surface mesh; and applying TNM staging criteria based on the analyzed contact status and classifying the stage of progression of the liver tumor. Claim 12 A method of operation for an automatic liver tumor segmentation and staging device according to claim 11, wherein the preprocessing step comprises normalizing the pixel values ​​of the abdominal CT image to between -140 HU and 260 HU, performing linear scaling, and removing the background. Claim 13 A method of operation for an automatic liver tumor segmentation and staging device according to claim 12, wherein the preprocessing step comprises applying an anisotropic diffusion filter to remove noise from the image and performing histogram equalization to enhance contrast. Claim 14 A method of operation for an automatic liver tumor segmentation and staging classification device, characterized in that, in claim 11, the step of generating a surface mesh of the segmented liver, liver blood vessels, and liver tumor comprises the step of generating a surface mesh by applying a Flying Edges algorithm based on image segmentation results. Claim 15 A method of operation for an automatic liver tumor segmentation and staging classification device according to claim 11, wherein the step of determining whether there is contact between the liver tumor and a major blood vessel includes the step of checking whether the diameter of the blood vessel is 5 mm or more.

Images