Liver 3D printing model construction method, system and equipment based on image data

Patient-specific CT and MRI data are used for 3D reconstruction and printing to create realistic liver and tumor models, addressing the limitations of existing models by enhancing clinical and teaching applications.

CN119068137AActive Publication Date: 2024-12-03ZHEJIANG UNIV

Patent Information

Application Number
CN202410898181.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-05
Publication Date
2024-12-03
Estimated Expiration
2044-07-05

AI Technical Summary

Technical Problem

Existing 3D printed liver models based on normal human imaging data fail to simulate the unique characteristics of liver tumors, such as size, location, morphology, and blood supply, making them inadequate for clinical diagnosis and treatment, as well as medical teaching.

Method used

A method using patient-specific CT and MRI image data for multimodal image segmentation, deep learning image fusion, and 3D reconstruction to create a highly realistic liver and tumor model, incorporating vascular network reconstruction and 3D printing technology.

Benefits of technology

Provides a real, intuitive, and touchable liver model for improved preoperative planning, surgical training, and clinical decision-making, while accurately simulating the spatial relationship between the liver and tumors.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068137A_ABST
    Figure CN119068137A_ABST
Patent Text Reader

Abstract

The invention provides a liver 3D printing model construction method, system and equipment based on image data. The method comprises the following steps: acquiring CT image data and MRI image data of an individual patient; performing multi-modal image segmentation on the CT image data and the MRI image data to obtain a structure segmentation image; fusing the structure segmentation images to obtain a liver and tumor anatomical structure model; performing three-dimensional reconstruction on the liver and tumor anatomical structure model to obtain a target three-dimensional visual model; and performing 3D printing on the target three-dimensional visual model. According to the application, two image modalities of CT and MRI are fused, and more comprehensive and accurate liver anatomical information can be obtained. Clear liver parenchyma and blood vessel images are provided by CT, and more sensitive soft tissue contrast and tumor detection are provided by MRI. The combination of the two can make up for the deficiency of a single image technology, and improve the fidelity and information content of the model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of medical imaging, and in particular to a method, system and device for constructing a 3D printed liver model based on imaging data. Background Art

[0002] Medical imaging data converted into 3D printed models can not only assist doctors in accurate preoperative planning and intraoperative navigation, but can also be used for medical teaching to improve medical students and residents' understanding of liver anatomy and diseases. Studies have used 3D printing technology to make normal liver models for medical teaching and training. These models can display the three-dimensional structural characteristics of the liver, such as anatomical morphology, Couinaud partitions, and vascular distribution and routing. By assembling and disassembling the models, and performing section cutting and other operations, learners can observe and understand the internal structure of the liver in an all-round and multi-level manner, overcoming the limitations of traditional cadaver dissection and two-dimensional atlas teaching to a certain extent.

[0003] However, most of the existing 3D printed liver models are based on normal human imaging data and cannot simulate the situation of liver tumors. Liver tumors have significant individual differences in size, location, morphology, blood supply, etc., and their relationships with surrounding normal liver tissue and blood vessels are also different. The lack of tumor reproduction in 3D printed models makes it difficult to meet the needs of clinical diagnosis and treatment and teaching. On the other hand, the current clinical diagnosis and surgical planning of liver tumors mainly rely on imaging examinations such as CT and MRI. However, these imaging data are all two-dimensional plane images, which make it difficult to intuitively and three-dimensionally display the spatial structural relationship between the liver and tumors, and it is difficult to meet the needs of clinical diagnosis and treatment and teaching. Summary of the invention

[0004] The present application provides a method, system and device for constructing a 3D printed liver model based on imaging data, which can intuitively and three-dimensionally display the spatial structural relationship between the liver and tumors to meet the needs of clinical diagnosis and treatment and teaching.

[0005] The first aspect of the present application provides a method for constructing a 3D printed liver model based on imaging data; A method for constructing a 3D printed liver model based on image data, comprising: Acquire CT image data and MRI image data of individual patients; Perform multimodal image segmentation on CT image data and MRI image data to obtain a structural segmentation image; The structural segmentation images are fused to obtain the anatomical structure model of the liver and tumor; Perform three-dimensional reconstruction of the liver and tumor anatomical structure models to obtain the target three-dimensional visualization model; 3D print the target 3D visualization model.

[0006] Preferably, performing multimodal image segmentation on the CT image data and the MRI image data to obtain a segmented image includes: The CT image data is subjected to threshold segmentation processing according to multiple preset threshold ranges to obtain a threshold structure segmentation image, and the region of interest of the MRI image data is extracted according to the region growing method to obtain an interest structure segmentation image, wherein the structure segmentation image includes an interest structure segmentation image and a threshold structure segmentation image.

[0007] Preferably, before performing multimodal image segmentation on the CT image data and the MRI image data to obtain a structure segmentation image, the process includes: The resolution, pixel size and coordinate system of the CT image data and the MRI image data are adjusted to be consistent.

[0008] Preferably, fusing the structural segmentation images to obtain the anatomical structure model of the liver and tumor includes: The structural segmentation image is input into the deep learning image fusion model to obtain the anatomical structure model of the liver and tumor.

[0009] Preferably, the deep learning image fusion model is a convolutional neural network model or a generative adversarial network model.

[0010] Preferably, the three-dimensional reconstruction of the liver and tumor anatomical structure model to obtain a target three-dimensional visualization model includes: The deep learning-based image segmentation algorithm segments the organ structure of the liver and tumor anatomical structure model; The deep learning-based image feature extraction algorithm extracts the three-dimensional features of organ structures; A target three-dimensional visualization model is generated based on the extracted three-dimensional features of the organ and the three-dimensional reconstruction algorithm.

[0011] Preferably, after the liver and tumor anatomical structure models are three-dimensionally reconstructed to obtain a target three-dimensional visualization model, the following steps are performed: Smooth, denoise and simplify the target 3D visualization model.

[0012] Preferably, after the liver and tumor anatomical structure models are three-dimensionally reconstructed to obtain the target three-dimensional visualization model, the following steps are performed: Convert the target 3D visualization model to STL or OBJ format.

[0013] The present application also provides a system for constructing a 3D printed liver model based on imaging data; A liver 3D printing model construction system based on image data, comprising: An image data acquisition module, used to acquire CT image data and MRI image data of individual patients; An image segmentation module is used to perform multimodal image segmentation on CT image data and MRI image data to obtain a structural segmentation image; Image fusion module, used to fuse the structural segmentation images to obtain the anatomical structure model of the liver and tumor; A three-dimensional reconstruction module is used to perform three-dimensional reconstruction of the liver and tumor anatomical structure models to obtain a target three-dimensional visualization model; The printing module is used to perform 3D printing on the target three-dimensional visualization model.

[0014] In a third aspect of the present application, an electronic device is provided.

[0015] An electronic device includes a processor, a memory and a transceiver, wherein the memory is used to store instructions, the transceiver is used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes a method for constructing a 3D printed liver model based on image data.

[0016] In summary, this application includes the following beneficial technical effects: 1. This application can provide a real, intuitive, and touchable liver model, which is helpful for explaining the condition to patients and their families before and after surgery.

[0017] 2. This application performs zoning markings and simulates surgical operations on the model, which can improve the accuracy of preoperative planning and reduce surgical risks.

[0018] 3. This application can estimate liver function and guide clinical decision-making by calculating the resected and residual liver volumes.

[0019] 4. In this application, the liver-like tissue structure is printed by biological 3D printing, which can be used for in vitro experimental studies such as drug metabolism.

[0020] 5. Vascular network reconstruction technology can display the three-dimensional blood supply of the liver and more accurately determine the relationship between the tumor and surrounding blood vessels.

[0021] 6. This application integrates CT and MRI imaging modalities to obtain more comprehensive and accurate liver anatomical information. CT provides clear images of liver parenchyma and blood vessels, and MRI provides more sensitive soft tissue contrast and tumor detection. The combination of the two can make up for the shortcomings of a single imaging technology and improve the fidelity and information content of the model.

[0022] 7. This application establishes a liver tumor model and uses a laparoscopic system to train laparoscopic ultrasound operations, so that hepatobiliary surgeons can master laparoscopic ultrasound technology and perform simulated tumor ablation under the guidance of laparoscopic ultrasound to shorten the learning curve. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1It is a flowchart of an embodiment of a method for constructing a 3D printed liver model based on image data of the present application; Figure 2 It is CT image data of an embodiment of a method for constructing a 3D printing model of a liver based on image data in the present application; Figure 3 It is MRI image data of an embodiment of a method for constructing a 3D printing model of a liver based on image data in the present application; Figure 4 It is a schematic diagram of a target three-dimensional visualization model from a first perspective of an embodiment of a method for constructing a liver 3D printing model based on image data of the present application; Figure 5 It is a schematic diagram of a target three-dimensional visualization model under a second viewing angle of an embodiment of a method for constructing a liver 3D printing model based on image data of the present application; Figure 6 It is a schematic diagram of a target three-dimensional visualization model under a third perspective of an embodiment of a method for constructing a liver 3D printing model based on image data of the present application; Figure 7 It is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.

[0025] In the description of the embodiments of the present application, words such as "illustrative", "for example" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "illustrative", "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "illustrative", "for example" or "for example" is intended to present related concepts in a concrete way.

[0026] In the description of the embodiments of the present application, the term "and / or" is only a kind of association relationship describing the associated objects, indicating that there may be three kinds of relationships, for example, A and / or B, which can represent: A exists alone, B exists alone, and A and B exist at the same time. In addition, unless otherwise specified, the meaning of the term "multiple" refers to two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Thus, the features defined as "first" and "second" can explicitly or implicitly include one or more of the features. The terms "include", "comprise", "have" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0027] Scenario: In order to overcome the limitations of traditional cadaver dissection and two-dimensional atlas teaching, researchers used 3D printing technology to create a normal liver model for medical teaching and training. Most of the existing 3D printed liver models are based on normal human imaging data and cannot simulate the situation of liver tumors. Liver tumors have significant individual differences in size, location, morphology, blood supply, etc., and their relationship with surrounding normal liver tissue and blood vessels is also different. The lack of realistic reproduction of tumors in 3D printed models makes it difficult to fully meet the needs of clinical diagnosis and treatment and teaching.

[0028] In order to solve this problem, this application uses the patient's CT and MRI image data as the basis, and uses image segmentation, 3D reconstruction and other technologies to accurately extract the 3D structural information of the liver and tumor, and then combines 3D printing technology to produce a highly realistic, multi-material, detachable liver tumor model. This method can not only truly restore the complex structure of the liver and its tumors, but also provide accurate preoperative planning and guidance before surgery.

[0029] Explanation of special terms: Region growing method: an image segmentation technique based on the principle of image homogeneity, that is, similar pixels in an image tend to belong to the same region. The region growing method starts with a set of seed points and gradually adds adjacent pixels to the region represented by these seed points until no more pixels can be added. The basic steps of the region growing method usually include: Selecting seed points: selecting a set of pixels in the image as seeds for the initial region; Defining similarity criteria: determining the criteria for whether a pixel should be added to an existing region, which usually involves the grayscale value, texture, color and other attributes of the pixel; Growing process: starting from the seed point, check the pixels around it. If these pixels meet the similarity criteria, they will be added to the corresponding region and become new seed points; Repeated growth: repeat the above process until all pixels that can be added are added to the corresponding region, or a certain termination condition is reached.

[0030] Mimics: A modular software that can be used in different combinations according to user needs. Mimics software has multiple functions, including but not limited to: Image import and segmentation: supports the import of multiple image formats, and provides a variety of segmentation tools such as grayscale thresholding, regional growing, etc., to help users quickly highlight areas of interest. Image visualization: provides axial, coronal and sagittal views of the original data, as well as three-dimensional views reconstructed from the area of ​​interest, and supports translation, scaling and rotation of three-dimensional views. Image registration and measurement: has image registration, point registration and STL registration functions, as well as point-to-point measurement, contour line and grayscale value measurement, etc.

[0031] The present application provides a method for constructing a 3D printed liver model based on imaging data; Reference Figure 1 A method for constructing a 3D printed liver model based on image data specifically comprises the following steps: S1: Acquire CT image data and MRI image data of individual patients; Specifically including CT image data acquisition and MRI image data acquisition: S11: CT image data acquisition; Reference Figure 1 and Figure 2A 64-row 128-slice spiral CT scanner (such as GE LightSpeed ​​VCT) was used with a scanning voltage of 120 kV, a tube current of 300 mAs, an acquisition frequency of 0.5 s / rot, and a pitch of 1.375:1. 1.0-1.5 mm thin-layer CT images of the patient's upper abdomen were obtained. The scanning range was from the upper edge to the lower edge of the liver. Four phases of images were obtained simultaneously during the scan: the plain phase, the arterial phase (25-30 s after enhancement), the portal venous phase (50-60 s), and the equilibrium phase (100-120 s). The obtained CT image data were stored in the DICOM 3.0 format.

[0032] S12: MRI image data acquisition; Reference Figure 1 and Figure 3 , using a 1.5T or 3.0T magnetic resonance scanner (such as Siemens Skyra), using voxel isotropic three-dimensional rapid acquisition gradient echo sequence (3D VIBE), TR 3.9ms, TE1.4ms, flip angle 9 degrees, scanning layer thickness 2.0-3.0mm, to obtain T1WI dynamic enhanced MRI images, which were collected in the arterial phase, portal venous phase and equilibrium phase after injection of gadopentetate dimeglumine. At the same time, conventional T2WI sequence scanning was performed to obtain anatomical structure images. The obtained images were also stored in DICOM format.

[0033] Among them: personnel can adjust and optimize the data acquisition process according to actual conditions, such as the optimization range of scanning parameters, tube current can be adjusted to 200-400mAs, acquisition frequency 0.4-0.6s / rot, pitch 1.2-1.5:1, to balance image quality and radiation dose; according to the type and size of liver tumors, the scanning layer thickness is appropriately adjusted to 0.5-2.0mm to take into account both resolution and data volume; the time point of the four-phase enhanced scanning can be fine-tuned according to individual cycle time differences to obtain the best vascular imaging effect.

[0034] S2: Adjust the resolution, pixel size and coordinate system of the CT image data and the MRI image data to be consistent; Specifically, the CT image data and MRI image data are imported into medical image post-processing software (such as Materialise Mimics), with the CT image data as a fixed image and the MRI image data as a floating image. Feature points are manually selected for coarse alignment, resampling and other pre-processing, and then the mutual information algorithm is used for fine alignment. Ultimately, the two sets of images have the same resolution and pixel size, and are matched to the same coordinate system.

[0035] S3: performing multimodal image segmentation on the CT image data and the MRI image data to obtain a structure segmentation image; Specifically including steps S31-S32, wherein the structure segmentation image includes an interest structure segmentation image and a threshold structure segmentation image; S31: performing threshold segmentation processing on the CT image data according to a plurality of preset threshold ranges to obtain a threshold structure segmentation image; Specifically, the CT image data is segmented by threshold to obtain a threshold structure segmentation image, and multiple threshold ranges include: liver parenchyma threshold: 50-150HU, tumor threshold: 150-300HU, artery threshold: 250-450HU, and vein threshold: 100-250HU. Among them, the threshold segmentation parameters can be fine-tuned according to the actual image quality and contrast, such as the liver parenchyma and tumor threshold ranges can be adjusted to 40-180HU and 120-350HU respectively.

[0036] S32: extracting the region of interest of the MRI image data according to the region growing method to obtain a segmented image of the structure of interest.

[0037] Specifically, the MRI image data is processed using the region growing method, starting with multiple seed points including tumors and blood vessels, and the region of interest (ROI) is extracted as the structure of interest to segment the image.

[0038] In other embodiments, the blood vessel segmentation may be combined with the region growing method and the threshold method to improve the segmentation accuracy and continuity, and manual editing and correction may be performed when necessary.

[0039] It should be noted that, in this example, after segmenting the interest structure segmentation image and the threshold structure segmentation image, surface smoothing and mesh optimization processing are performed on the liver, tumor, blood vessel areas, etc. in the image to improve the printing quality of the three-dimensional model.

[0040] S4: Fusing the structural segmentation images to obtain the anatomical structure model of the liver and tumor; Specifically, the structure segmentation image is input into the deep learning image fusion model to obtain the liver and tumor anatomical structure model. The deep learning image fusion model is a convolutional neural network model or a generative adversarial network model. The deep learning image fusion model has been trained by personnel to automatically learn the correlation features between the interest structure segmentation image and the threshold structure segmentation image. The training of the deep learning image fusion model is the existing neural network training process, which can also be understood as the deep learning image fusion model is an existing technology and will not be elaborated here. It should be noted that the training process of the deep learning image fusion model can be the input of the interest structure segmentation image and the threshold structure segmentation image into the deep learning image fusion model to generate a fused high-quality image, namely the liver and tumor anatomical structure model.

[0041] It is worth noting that steps S3-S4 also include a vascular network reconstruction step, specifically: Based on the arterial and venous phase data in the CT image data, Mimics reconstructs the vascular network in response to human operation. The hepatic artery is extracted with a threshold of 250-450HU, and the hepatic vein is extracted with a threshold of 100-250HU. Manual editing and correction are performed, and the 1-pixel brush tool is used to process the blood vessels for continuity. The reconstructed hepatic artery, portal vein, and hepatic vein vascular networks are colored to show their three-dimensional running relationship and their positional relationship with the liver and tumor. Finally, they are fused with the liver and tumor anatomical structure models to generate a transparent overall liver vascular network model.

[0042] The specific steps are: 1. Perform blood vessel segmentation on the arteriovenous phase CT image, and select the threshold value according to the above method; 2. Perform morphological processing on the segmentation results, using 1×1×1 structural elements to expand the blood vessels and fill the lumen breaks; 3. The portal vein and hepatic vein are fused and the surface is smoothed to generate a complete liver vascular tree; 4. Align with the liver parenchyma and tumor models, i.e., the liver and tumor anatomical structure models, to obtain a holistic liver model including a fine vascular network.

[0043] Among them, the vascular reconstruction threshold can be adjusted according to the quality of vascular imaging, generally arteries>300HU, veins>150HU. Vascular continuity processing can be combined with morphological algorithms, artificial intelligence and other technologies to improve the degree of automation and efficiency. The positional relationship between the vascular network and the liver and tumor can be presented in a visual way such as color coding and transparent rendering.

[0044] S5: Perform three-dimensional reconstruction of the liver and tumor anatomical structure models to obtain a target three-dimensional visualization model; Specifically, refer to Figure 1 and Figure 4 In Mimics software, three-dimensional reconstruction is performed for liver parenchyma, tumor, hepatic artery, hepatic vein, portal vein and other organ structures in the liver and tumor anatomical structure models; the organ structures of the liver and tumor anatomical structure models can be segmented by deep learning-based image segmentation algorithms; deep learning-based image segmentation algorithms can use U-Net, V-Net, etc. to automatically segment target organs; the segmentation network trained by the deep learning image segmentation algorithm is trained with a large amount of labeled data, and learns the feature representation of organ boundaries and internal structures, so as to quickly and accurately extract the organ region of interest. The segmentation network trained by the deep learning image segmentation algorithm can be an existing trained neural network model.

[0045] In Mimics software, select the region growing method to group the masks of each organ structure and convert the masks into 3D objects. Perform post-processing and optimization on the generated target 3D visualization model, such as smoothing, denoising, and simplification, to make it more suitable for 3D printing requirements. Use editing tools to smooth the 3D model and eliminate burrs and sharp corners. Figure 5 and Figure 6 , adjust the transparency of the 3D model to display the internal structure. Perform spatial registration of the liver parenchyma, tumor, blood vessels, etc. according to the preset anatomical order relationship to obtain a complete target 3D visualization model of the liver.

[0046] Among them, the method of converting the mask into a 3D object includes: volume rendering, maximum density projection, stereo density projection, minimum density projection or surface maximum density projection.

[0047] The segmented liver, tumor, and blood vessel masks are 3D reconstructed using the Marching Cubes algorithm to generate a smooth, defect-free STL format model file; in other embodiments, they can also be converted into an OBJ format model file; In another embodiment, the organ three-dimensional features of the organ structure can be extracted based on a deep learning image feature extraction algorithm; the deep learning image feature extraction algorithm can use a three-dimensional CNN algorithm after automatically learning the high-level semantic features of the organ, and a traditional image feature extraction algorithm, such as a gray level co-occurrence matrix (GLCM), wavelet transform, etc., can be used to replace the deep learning image feature extraction algorithm to extract the organ three-dimensional features of the organ structure; Then, a target 3D visualization model is generated according to the extracted 3D features of the organ and a 3D reconstruction algorithm. The specific 3D reconstruction algorithm includes Marching Cubes or Poisson Surface Reconstruction. The extracted 3D features of the organ are processed by the 3D reconstruction algorithm to generate the target 3D visualization model.

[0048] In other embodiments, a deep learning-based 3D generative model, such as 3D GAN, may also be used to directly generate a target 3D visualization model from the fused image and feature representation.

[0049] After the target 3D visualization model is generated, it is smoothed, denoised and simplified to make it more suitable for 3D printing. The processing technology used can be traditional mesh processing algorithms or 3D model optimization methods based on deep learning.

[0050] In order to facilitate teaching and simulation demonstration, the 3D printed liver model is divided into regions I-VIII according to the Couinaud segmentation system, and each region is labeled with a different color, such as: Zone I: caudate lobe, yellow Area II: posterior superior area of ​​the left lateral lobe, green Area III: left lateral lobe anterior superior area, orange Area Ⅳa: upper area of ​​the left inner lobe, purple Area Ⅳb: Lower area of ​​left inner lobe, brown Zone V: right front lower area, blue Zone VI: right lower back area, indigo Area VII: right upper back area, red Zone VIII: right front upper zone, pink Furthermore, based on the eight segments of Couinaud, segment VIII is further subdivided into two subsegments, VIIIa and VIIIb, which represent the upper and lower branches of the segment VIII vein drainage area, respectively, reflecting the intrahepatic vascular distribution more finely.

[0051] The tumor was located by referring to the CT image data, its relationship with the surrounding blood vessels was marked, and the resection path and range were planned. The CAD measurement tool of Mimics was used to measure the volume of the liver to be resected and the volume of the residual liver, and to evaluate the postoperative liver function reserve.

[0052] Among them, tumor positioning and vascular relationship annotation should refer to imaging diagnosis, and transparent processing or slice analysis can be performed when necessary. Surgical planning should comprehensively consider factors such as tumor size, location, affected liver segment, residual liver volume, etc., and 3D printing can be used to simulate surgical operations.

[0053] S5: 3D printing the target three-dimensional visualization model.

[0054] Specifically, 3D printing preprocessing is required before 3D printing; 3D printing pre-processing: Export the reconstructed liver 3D model in STL format and import it into 3D printing slicing software (such as Cura, Slic3r) for printing preprocessing. Adjust the printing parameters according to the size and morphological characteristics of the model: Print size: Scaled to 1:1 size based on the original model; Printing materials: You can choose PLA, ABS or photosensitive resin, etc., and set different printing temperatures and speeds according to the material characteristics; Layer thickness: 0.1-0.2mm. The smaller the layer thickness, the higher the printing accuracy, but the longer the printing time. Infill rate: 10%-30%. The higher the infill rate, the stronger the model, but the more printing time and material consumption. Support structure: Automatically or manually add support structures according to the overhang and concave conditions of the model to prevent printing deformation; Print direction: According to the structural characteristics of the model, adjust the printing direction to optimize the printing effect and support layout.

[0055] 3D printing is performed after 3D printing preprocessing, specifically: Transfer the data file processed by the slicing software to a 3D printer (such as Stratasys Fortus 450mc). Select the appropriate printing material and load it into the printer, and calibrate the levelness of the printing platform. Start the printing program, and the printer will print layer by layer according to the preset path. After each layer is printed, the platform will drop one layer height until the entire model is printed. During the printing process, adjust the printing parameters as needed and monitor the printer in real time. Spray the release agent appropriately during the printing process, and soak it in the release liquid to dissolve the support material after printing is completed.

[0056] Print size: Scaled to 1:1 size based on the original model; Printing materials: PLA, ABS or photosensitive resin; Layer thickness: 0.1-0.2mm; Filling rate: 10%-30%; Printing speed: 50-80mm / s; Printing temperature: PLA 200-220℃, ABS 230-250℃; Hot bed temperature: PLA 50℃, ABS 80-100℃; Transfer the sliced ​​data file to a 3D printer (such as Stratasys Fortus 450mc) and start printing.

[0057] The printing material can also use flexible materials such as TPU to simulate the touch of the liver.

[0058] (4) Post-printing processing After printing is complete, remove the print from the printing platform and carefully remove the support material. You can use scissors, scalpels and other tools to remove the support, and then use sandpaper to polish the surface of the model until it is smooth and flat. Color the models of different tissue structures, such as brown-yellow for liver parenchyma, red for hepatic artery, blue for hepatic vein and portal vein, and green for tumors to enhance the recognition of the structure. Assemble the liver parenchyma and blood vessels to form a complete liver entity model. Finally, spray transparent protective paint to enhance the strength and durability of the model.

[0059] Among them, a variety of surface treatment technologies can be used, such as grinding, polishing, painting, electroplating, etc., to improve the aesthetics and durability of the model. For delicate vascular structures, dyeing, water transfer and other methods can be used for coloring. The liver model can be assembled using detachable connection methods such as snaps and magnetic suction, which is convenient for repeated use and replacement.

[0060] Among them, it is also necessary to explain: This application also optimizes the preparation and configuration of 3D printing materials, specifically: Preparation of biological 3D printing materials: Gelatin, sodium alginate and fibrinogen were mixed in a mass ratio of 8:1:1 and prepared into a hydrogel ink with a final concentration of 10% using phosphate buffered saline (PBS). Human liver cell line HepG2 was isolated and cultured, added to the hydrogel with a cell density of 107 cells / ml, and stirred evenly at low temperature to obtain a cell-hydrogel mixed ink (Bio-ink) for biological 3D printing.

[0061] Load Bio-ink into the print head of a biological 3D printer (such as Organovo Novogen MMX) and set the printing parameters: Nozzle diameter: 200-500μm Air pressure: 0.1-0.3MPa Printing speed: 5-20mm / s Layer thickness: 100-300μm A layer of CaCl2 solution was spread on the collection plate as a crosslinker, and the cell-hydrogel scaffold was printed layer by layer. After printing, CaCl2 and glutaminase solution were used for crosslinking and curing, and after washing with PBS, it was placed in a 37°C, 5% CO2 incubator and cultured. After 2 weeks, a liver-like tissue structure was formed.

[0062] The specific steps of biological 3D printing are as follows: ① Cell preparation: Isolate and culture HepG2 hepatocytes and adjust the cell density to 107 cells / ml; ②Stent design: Based on the hexagonal structure of liver lobule, a honeycomb stent with a central cavity is designed, with a porosity of >90% and a pore size of 100-500μm; ③Material preparation: Gelatin, sodium alginate and fibrinogen were mixed in a ratio of 8:1:1, dissolved in PBS to prepare a 10% hydrogel, and HepG2 cell suspension was added to prepare Bioink; ④3D printing: A pressure extrusion biological 3D printer was used with a nozzle inner diameter of 400 μm, an air pressure of 0.1 MPa, a printing speed of 10 mm / s, and a layer thickness of 200 μm to print out a cell-scaffold composite structure; ⑤ Post-treatment: cross-link with CaCl2 and glutaminase, wash with PBS, and culture at 37°C, 5% CO2 for 2 weeks to form liver-like tissue; ⑥ Characterization evaluation: Hematoxylin-eosin staining was used to observe the histological morphology, Coomassie brilliant blue and PAS staining were used to evaluate the glycogen and lipid contents, and albumin and urea synthesis tests were used to detect hepatocyte function.

[0063] Among them, the ratio of hydrogel materials can be adjusted as needed, such as increasing the proportion of fibrinogen to increase the strength of the scaffold. The cell density can be optimized according to the tissue type and printing parameters, and the general range is 105-108 cells / ml. Cell culture conditions such as culture medium composition, pH value, temperature, etc. should be optimized according to the cell type.

[0064] Configuration of 3D printing materials: (1) Preparation of hepatocytes: Culture hepatocytes in the logarithmic growth phase and digest and collect the cells; (2) Material preparation: Mix sterilized 4% sodium alginate, 20% gelatin, and 3% fibrinogen in a ratio of 1:1:1 and dissolve for later use; (3) Preparation of cell-loaded hydrogel scaffolds: The hepatocyte suspension and glutamine aminotransferase were mixed at a ratio of 6:1, the cell mixture with TGaes added was evenly mixed with the rewarmed sodium alginate / gelatin / fibrinogen hydrogel system solution, and loaded into a syringe for precooling; (4) 3D bioprinting: The pre-cooled sodium alginate / gelatin / fibrinogen mixture containing the cell suspension is installed into the 3D bioprinter and printed according to the printing route set in the program; (5) Post-printing processing: After printing, 3% CaC12 was used for cross-linking, and then thrombin was added to cross-link fibrinogen, and then cultured in complete culture medium; (6) Removal of sodium alginate saline gel: Use EDTA + sodium citrate to remove the sodium alginate shell and obtain the hepatocytes grown in the 3D in vitro model.

[0065] Among them, bio-3D printing hydrogel materials can select different natural or synthetic polymer materials according to cell types and printing requirements. The mixing ratio of cell suspension and hydrogel can be optimized according to material properties and cell density, and the general cell volume fraction is 10%-50%. Printing parameters such as nozzle diameter, air pressure, printing speed, etc. can be adjusted according to the rheological properties of the material and cell activity. The cross-linking method can select ionic cross-linking, enzymatic cross-linking, photo-cross-linking, etc. according to the material properties, and optimize the cross-linking time and conditions. The shell removal method can select dissolution, enzymatic hydrolysis, etc. to avoid affecting cell activity.

[0066] In addition, this application further designs three 3D printing schemes based on the CT and MRI imaging characteristics of primary liver cancer, metastatic liver cancer and liver hemangioma, respectively simulating their morphology and internal structure to achieve realistic 3D printing models. (1) 3D printing solution for primary liver cancer: l The irregular shape and uneven surface reflect the invasive growth of the tumor.

[0067] The internal material density is uneven, and low-density areas are printed locally to simulate tumor necrosis, cystic degeneration and fatty degeneration.

[0068] l High-density materials are used locally in patchy distribution to simulate intratumoral bleeding.

[0069] lPrint a dense thin layer around the tumor to simulate a pseudocapsule.

[0070] If necessary, print a tree-like pipeline structure inside the lesion to simulate the blood vessels inside the tumor.

[0071] (2) 3D printing solution for metastatic liver cancer: lUse regular shapes such as spheres or ellipses to disperse and print multiple lesions to simulate multiple metastases.

[0072] lUniform low-density material is used inside to reflect the homogeneity of the metastatic tumor.

[0073] l An irregular low-density area is printed in the center, and a high-density ring is printed around it, forming a "bull's eye sign".

[0074] lPrint a fuzzy high-density ring around the lesion to simulate the "halo sign".

[0075] l An extremely low-density area can be printed in the center of some lesions, surrounded by a high-density ring, presenting a "ring target sign".

[0076] (3) 3D printing solution for hepatic hemangioma: l It has a lobed, oval or round appearance with clear borders.

[0077] lThe interior is made of homogeneous low-density material, reflecting the cavernous structure of the hemangioma.

[0078] lIn the center of a lesion with a larger diameter, print a star-shaped or fissure-shaped ultra-low-density area to simulate thrombus, hemorrhage or scar.

[0079] l Enhanced scanning scheme: first print a layer of high-density edge, and then gradually print centripetally until the lesion is filled.

[0080] If necessary, the entire lesion can be printed with extremely high density in the severe T2WI scheme to simulate the "bright light sign".

[0081] It should also be noted that the above-mentioned individual patient CT image data graphs, MRI image data graphs and target 3D visualization models are all from actual patient cases, and the basic information of the cases is as follows: The patient is a 61-year-old female. Seven years ago, she had an abdominal ultrasound at a local hospital, which indicated the possibility of a liver hemangioma, about 2-3 cm in size. She had no discomfort symptoms such as abdominal distension and pain, and did not receive treatment. One week ago, she had an abdominal MRI at a local hospital again, which indicated a liver hemangioma; about 5-7 cm in size, with no obvious pain, no nausea, no belching, no acid reflux, no choking when eating, and other symptoms. The patient had no chills, chills, or fever during the course of the disease. For further diagnosis and treatment, Menzhen planned to admit the patient to the hospital for "liver space-occupying lesions."

[0082] The implementation process of a method for constructing a liver 3D printing model based on image data provided in the present application includes: after obtaining CT image data and MRI image data of an individual patient, performing multimodal image segmentation on the CT image data and the MRI image data to obtain a structural segmentation image; the structural segmentation image includes segmentation images of liver tissue structure and tumor structure; the segmentation of the tumor structure is achieved by the following steps: extracting high-density areas as tumor candidate areas in the CT image by threshold segmentation; highlighting the difference between the tumor area and normal liver tissue by using dynamic contrast enhancement and diffusion-weighted imaging technology in the T1-weighted and T2-weighted images of the MRI; and segmenting the tumor candidate areas extracted from the CT and MRI images. Spatial registration, extracting the precise boundary of the tumor through morphological operations and connectivity analysis; fusing the structural segmentation images to obtain the anatomical structure model of the liver and tumor; using the deep learning algorithm to fuse the liver tissue structure and tumor structure segmentation images to generate a complete anatomical structure model of the liver and tumor; the deep learning algorithm is based on U-Net, fusing the features of CT and MRI modal data in the encoder part, and reconstructing a continuous and smooth three-dimensional structure of the liver and tumor in the decoder part; based on the fused anatomical structure model of the liver and tumor, the geometric features of the liver surface and the tumor surface and the topological features of the duct structure such as blood vessels and bile ducts inside the liver are extracted; using the improved Marching The Cubes algorithm and Dijkstra shortest path algorithm are used to reconstruct the liver and tumor surface meshes and extract the skeleton of the pipeline structure respectively; the reconstructed liver and tumor surface models and pipeline structure models are assembled to obtain high-fidelity three-dimensional visualization models of the liver and tumor; the size, color, transparency, etc. of the visualization model are personalized according to the clinical needs of the doctor; the personalized three-dimensional visualization models of the liver and tumor are converted into three-dimensional printing model data, and appropriate printing materials (such as photosensitive resin, silicone, etc.) and printing process parameters are selected to realize rapid manufacturing of liver and tumor physical models on a 3D printer; the printed models undergo necessary post-processing steps such as cleaning and curing to obtain the final 3D printed models of the liver and liver tumor.

[0083] The present application also provides a method for constructing a 3D printed liver model based on imaging data; A liver 3D printing model construction system based on image data, comprising: An image data acquisition module, used to acquire CT image data and MRI image data of individual patients; An image segmentation module is used to perform multimodal image segmentation on CT image data and MRI image data to obtain a structural segmentation image; Image fusion module, used to fuse the structural segmentation images to obtain the anatomical structure model of the liver and tumor; A three-dimensional reconstruction module is used to perform three-dimensional reconstruction of the liver and tumor anatomical structure models to obtain a target three-dimensional visualization model; The printing module is used to perform 3D printing on the target three-dimensional visualization model.

[0084] Reference Figure 7 , provides a schematic diagram of the structure of an electronic device for an embodiment of the present application. The electronic device may include: at least one processor, at least one network interface, a user interface, a memory, and at least one communication bus.

[0085] The communication bus is used to realize the connection and communication between these components.

[0086] The user interface may include a display screen (Display) and a camera (Camera), and the optional user interface may also include a standard wired interface and a wireless interface.

[0087] The network interface may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0088] Among them, the processor may include one or more processing cores. The processor uses various interfaces and lines to connect various parts of the entire server, and executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory, and calling data stored in the memory. Optionally, the processor can be implemented in at least one hardware form of digital signal processing (DSP), field programmable gate array (FPGA), and programmable logic array (PLA). The processor can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor, but may be implemented separately through a chip.

[0089] Among them, the memory may include a random access memory (RAM) or a read-only memory (Read-Only Memory). Optionally, the memory includes a non-transitory computer-readable storage medium. The memory can be used to store instructions, programs, codes, code sets or instruction sets. The memory may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory may also be optionally at least one storage device located away from the aforementioned processor. The memory as a computer storage medium may include an operating system, a network communication module, a user interface module, and an application program for a method for building a liver 3D printing model based on image data.

[0090] It should be noted that: when the device provided in the above embodiment realizes its function, only the division of the above functional modules is used as an example. In actual application, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0091] In the electronic device, the user interface is mainly used to provide an input interface for the user and obtain data input by the user; and the processor can be used to call an application stored in the memory for a method for constructing a liver 3D printing model based on image data. When executed by one or more processors, the electronic device executes one or more of the methods described in the above embodiments.

[0092] The "unit" and "module" in this specification refer to software and / or hardware that can independently complete a specific function or cooperate with other components to complete a specific function, where the hardware may be, for example, a field programmable gate array (FPGA), an integrated circuit (IC), etc.

[0093] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0094] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0095] In the several embodiments provided in the present application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0096] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0097] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0098] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a memory, including a number of instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned memory includes: U disk, read-only memory (ROM), random access memory (RAM), mobile hard disk, disk or optical disk and other media that can store program codes.

[0099] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable memory, and the memory can include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0100] The above is only an exemplary embodiment of the present disclosure, and the scope of the present disclosure cannot be limited thereto. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the specification and practicing the disclosure here, those skilled in the art will easily think of other embodiments of the present disclosure. This application is intended to cover any modification, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the technical field not recorded in the present disclosure. The description and examples are only regarded as exemplary, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A method for constructing a 3D printed liver model based on imaging data. It is characterized in that include: Acquire CT image data and MRI image data of individual patients; Perform multimodal image segmentation on CT image data and MRI image data to obtain a structural segmentation image; The structural segmentation images are fused to obtain the anatomical structure model of the liver and tumor; Perform three-dimensional reconstruction of the liver and tumor anatomical structure models to obtain the target three-dimensional visualization model; 3D print the target 3D visualization model.

2. A method for constructing a 3D printed liver model based on image data according to claim 1, It is characterized in that Perform multimodal image segmentation on CT image data and MRI image data to obtain segmented images, including: The CT image data is subjected to threshold segmentation processing according to multiple preset threshold ranges to obtain a threshold structure segmentation image, and the region of interest of the MRI image data is extracted according to the region growing method to obtain an interest structure segmentation image, wherein the structure segmentation image includes an interest structure segmentation image and a threshold structure segmentation image.

3. A method for constructing a 3D printed liver model based on image data according to claim 2, It is characterized in that The structure segmentation image includes a segmentation image of the tumor structure, and the segmentation of the tumor structure segmentation image includes: In CT images, high-density areas are extracted as candidate tumor areas through threshold segmentation; Dynamic contrast enhancement and diffusion-weighted imaging techniques are used in T1-weighted and T2-weighted MRI images to highlight the differences between tumor areas and normal liver tissue; The candidate tumor regions extracted from CT and MRI images are spatially registered, and the precise boundaries of the tumor are extracted through morphological operations and connectivity analysis, and the tumor structure segmentation images are obtained by segmentation.

4. A method for constructing a 3D printed liver model based on image data according to claim 3, It is characterized in that The structural segmentation image includes a segmentation image of the liver tissue structure. When the structural segmentation image is fused to obtain the liver and tumor anatomical structure model, it includes: The segmented images of the liver tissue structure and the tumor structure are fused using a deep learning algorithm to generate a complete liver and tumor anatomical structure model, wherein the deep learning algorithm is based on U-Net.

5. The method for constructing a 3D printed liver model based on image data according to claim 1, It is characterized by 3D printing include: By selecting appropriate printing materials and printing process parameters, the target three-dimensional visualization models corresponding to the liver and tumor can be quickly manufactured on a 3D printer, and the printed models can be subjected to cleaning and curing steps to obtain the final 3D printed models of the liver and liver tumor.

6. The method for constructing a 3D printed liver model based on image data according to claim 1, It is characterized in that The liver and tumor anatomical structure models are reconstructed in three dimensions to obtain the target three-dimensional visualization model, including: The deep learning-based image segmentation algorithm segments the organ structure of the liver and tumor anatomical structure model; The deep learning-based image feature extraction algorithm extracts the three-dimensional features of organ structures; A target three-dimensional visualization model is generated based on the extracted three-dimensional features of the organ and the three-dimensional reconstruction algorithm.

7. The method for constructing a 3D printed liver model based on image data according to claim 1, It is characterized in that The liver and tumor anatomical structure models are reconstructed in three dimensions to obtain the target three-dimensional visualization model, including: The encoder part fuses the features of CT and MRI data, and the decoder part reconstructs the continuous and smooth three-dimensional structure of the liver and tumor. Based on the fused liver and tumor anatomical structure models, the geometric features of the liver surface and tumor surface as well as the topological features of the internal liver blood vessels, bile ducts and other pipeline structures are extracted; Assemble the reconstructed liver and tumor surface models and the duct structure model to obtain a high-fidelity 3D visualization model of the liver and tumor; customize the size, color, transparency, etc. of the visualization model according to the doctor's clinical needs; Convert the personalized 3D visualization models of the liver and tumor into 3D printing model data.

8. A method for constructing a 3D printed liver model based on image data according to claim 7, It is characterized in that In extracting the geometric features of the liver surface and the tumor surface and the topological features of the liver's internal blood vessels, bile ducts and other pipeline structures, the Marching Cubes algorithm and the Dijkstra shortest path algorithm are used to respectively achieve the reconstruction of the liver and tumor surface meshes and the skeleton extraction of the pipeline structure.

9. A system for constructing a 3D printed liver model based on the image data of the method according to any one of claims 1 to 8, It is characterized in that include: An image data acquisition module, used to acquire CT image data and MRI image data of individual patients; An image segmentation module is used to perform multimodal image segmentation on CT image data and MRI image data to obtain a structural segmentation image; Image fusion module, used to fuse the structural segmentation images to obtain the anatomical structure model of the liver and tumor; A three-dimensional reconstruction module is used to perform three-dimensional reconstruction of the liver and tumor anatomical structure models to obtain a target three-dimensional visualization model; The printing module is used to perform 3D printing on the target three-dimensional visualization model.

10. Electronic equipment, It is characterized in that It includes a processor, a memory and a transceiver, wherein the memory is used to store instructions, the transceiver is used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes the method according to any one of claims 1 to 8.

Citation Information

Patent Citations

  • Liver three-dimensional reconstruction method based on CTA data and 3D printing model manufactured by same

    CN110216872A

  • Pulmonary nodule image recognition method and device based on multi-modal data fusion

    CN114974518A

  • Adaptive multi-modal three-dimensional brain image fusion method based on unsupervised field

    CN117218046A

  • Liver multi-modal image registration method based on deep learning

    CN117314983A

  • Method for generating a 3D printable model of a patient specific anatomy

    US20210335041A1

Cited By

  • System for evaluating curative effect of treating liver cancer by combining HDRBT with thermal ablation in multi-mode radiomics

    CN120853956A

  • Multimodal radiomics HDRBT combined with thermal ablation therapy for liver cancer efficacy evaluation system

    CN120853956B