Three-dimensional visualization model reconstruction method and system based on iconography

Abstract

The invention relates to the technical field of iconography, in particular to an iconography-based three-dimensional visual model reconstruction method and system, and the method comprises the following steps: carrying out the multi-tissue segmentation in a two-dimensional slice image through employing a Unet-2D algorithm, and obtaining a plurality of tissue segmentation images; voxelization processing is carried out on each tissue segmentation image; distributing a high-contrast color for enhancing the display contrast among the tissues to the three-dimensional volume data of each tissue by combining a multi-objective optimization technology with a Unit-3D algorithm to obtain high-contrast three-dimensional volume data of each tissue; and performing visualization processing on the high-contrast three-dimensional surface model of the liver through a Poisson surface reconstruction technology and a volume rendering technology. According to the method, the three-dimensional reconstruction processing technology is utilized to accurately describe the shape, spatial distribution and the like of the calculus, and through enhanced display processing, when three-dimensional visualization reconstruction is carried out, the hepatic calculus part is highlighted, the calculus is strongly displayed, and the visualization effect is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of imaging processing, and particularly relates to a method and system for reconstructing a three-dimensional visualization model based on imaging. Background Art

[0002] Hepatolithiasis, that is, primary hepatolithiasis, is a common disease in China. Due to its complex lesions, high recurrence rate and often causing serious complications, this disease has become an important cause of death for benign biliary diseases in China. At present, the diagnosis of bile duct stones mainly relies on clinical manifestations and various imaging examinations, and the imaging examinations mainly include B-ultrasound, CT, magnetic resonance, choledochoscope, etc.

[0003] In the prior art, the two-dimensional images obtained by the imaging examination method cannot accurately display the stones in the morphological fusion of liver tissue, intrahepatic and extrahepatic bile duct structures and stones, so it is impossible to intuitively show the distribution and location of the stones to the surgical staff, which is likely to affect the final stone resection effect. Summary of the Invention

[0004] The purpose of the present invention is to provide a method for reconstructing a three-dimensional visualization model based on imaging to solve the technical problem that the prior art cannot accurately display the stones in the morphological fusion of liver tissue, intrahepatic and extrahepatic bile duct structures and stones.

[0005] To solve the above technical problem, the present invention specifically provides the following technical solutions:

[0006] A method for reconstructing a three-dimensional visualization model based on imaging, comprising the following steps:

[0007] Obtain two-dimensional slice images of the liver through CT or MRI;

[0008] In the two-dimensional slice images, use the Unet-2D algorithm for multi-tissue segmentation to obtain multiple tissue segmentation images;

[0009] Perform voxelization processing on each tissue segmentation image to obtain three-dimensional volume data of each tissue;

[0010] Use multi-objective optimization technology combined with the Unet-3D algorithm to assign high-contrast colors for enhancing the display contrast between tissues to the three-dimensional volume data of each tissue to obtain high-contrast three-dimensional volume data of each tissue;

[0011] Reconstruct and generate a high-contrast three-dimensional surface model of the liver by using the Poisson surface reconstruction technology with the high-contrast three-dimensional volume data of each tissue;

[0012] Visualize the high-contrast three-dimensional surface model of the liver through volume rendering technology to obtain a high-contrast visual three-dimensional model of the liver that allows user interaction.

[0013] As a preferred embodiment of the present invention, the method for multi-tissue segmentation using the Unet-2D algorithm includes:

[0014] Use the Unet-2D algorithm to sequentially segment the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and stones in the two-dimensional slice images, and correspondingly obtain the slice segmentation images of the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and stones.

[0015] As a preferred embodiment of the present invention, the voxelization processing method for each tissue segmentation image includes:

[0016] Stack each slice segmentation image of the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and stones respectively to correspondingly obtain the three-dimensional point cloud data of the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and stones \(\{(x si ,y si ,z si )|i∈[1,n s}\), where \(s∈[A,B,C,D,O]\), \(A\) is the identifier of the liver tissue, \(B\) is the identifier of the intrahepatic bile duct tissue, \(C\) is the identifier of the extrahepatic bile duct tissue, \(D\) is the identifier of the blood vessel tissue, \(O\) is the identifier of the stone, \((x si ,y si ,z si )\) is the \(i\)-th point cloud data in the three-dimensional point cloud data of \(s\), \(x si ,y si ,z si are respectively the three-dimensional components of \((x si ,y si ,z si )\) in the three-dimensional coordinate system, and \(n s \) is the total number of point cloud data in the three-dimensional point cloud data of \(s\);

[0017] Assign the gray value attribute in the two-dimensional slice image to each point cloud data in the three-dimensional point cloud data of the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and stones respectively to correspondingly obtain the three-dimensional volume data of the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and stones \(\{V(x si ,y si ,z si )|i∈[1,n s}, where s ∈ [A, B, C, D, O], A is the identifier of liver tissue, B is the identifier of intrahepatic bile duct tissue, C is the identifier of extrahepatic bile duct tissue, D is the identifier of blood vessel tissue, O is the identifier of calculus, and V(x si , y si , z si ) is the i-th voxel in the 3D volume data of s.

[0018] As a preferred embodiment of the present invention, the method for assigning high-contrast colors to the 3D volume data of each tissue includes:

[0019] Set the high-contrast colors assigned to the 3D volume data of liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and calculus to RGB A , RGB B , RGB C , RGB D , RGB O , and the high-contrast 3D volume data of liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and calculus obtained after assigning high-contrast colors is {RGB s (x si , y si , z si )|i ∈ [1, n s}, where s ∈ [A, B, C, D, O], A is the identifier of liver tissue, B is the identifier of intrahepatic bile duct tissue, C is the identifier of extrahepatic bile duct tissue, D is the identifier of blood vessel tissue, O is the identifier of calculus, and RGB s (x si , y si , z si ) is the high-contrast color of the 3D volume data of s;

[0020] Generate a high-contrast 3D surface model M s (RGB) of the liver according to the high-contrast 3D volume data {RGB si (x si , y si )|i ∈ [1, n s}; 3D ;

[0021] Use the Unet-3D algorithm to segment liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and calculus from the high-contrast 3D surface model M 3D (RGB) of the liver to obtain the 3D segmentation results of liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and calculus {(x′ si , y′ si , z′ si)|i ∈ [1, n′ s}, where s ∈ [A, B, C, D, O], A is the identifier of liver tissue, B is the identifier of intrahepatic bile duct tissue, C is the identifier of extrahepatic bile duct tissue, D is the identifier of blood vessel tissue, O is the identifier of calculus, (x′ si , y′ si , z′ si ) is the i-th point cloud data in the 3D segmentation result of s, x′ si , y′ si , z′ si are the three-dimensional components of (x′ si , y′ si , z′ si ) in the three-dimensional coordinate system respectively, n′ s is the total number of point cloud data in the 3D segmentation result of s;

[0022] Use multi-objective optimization technology to construct and solve the objective functions of RGB A , RGB B , RGB C , RGB D , RGB O , RGB

[0023]

[0024] In the formula, g1 and g2 are the first optimization objective and the second optimization objective respectively, min is the minimization operator, {(x Oi , y Oi , z Oi )|i ∈ [1, n O} is the i-th point cloud data in the three-dimensional point cloud data of calculus, x Oi , y Oi , z Oi are the three-dimensional components of (x Oi , y Oi , z Oi ) in the three-dimensional coordinate system respectively, n O is the total number of point cloud data in the three-dimensional point cloud data of calculus, RGB O is the high-contrast color of calculus, RGB s is the high-contrast color of s;

[0025] With the RGB value range as the search space, solve the objective function to obtain the optimal range of RGB A , RGB B , RGB C , RGB D , RGB O ;

[0026] In RGBA , RGB B , RGB C , RGB D , RGB O In the optimal ranges of, the color corresponding to the highest brightness is selected as the high-contrast color assigned to the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and calculus.

[0027] As a preferred embodiment of the present invention, the present invention further includes:

[0028] Using a convolutional neural network to train the mapping relationship between the three-dimensional volume data of each tissue and the high-contrast color of each tissue, to obtain a reinforced display model {RGB s,best |s∈[A,B,C,D,O]} = CNN({(x si ,y si ,z si )|i∈[1,n s}, s∈[A,B,C,D,O]), where CNN is the convolutional neural network, RGB s,best is the optimal solution of the high-contrast color of s, A is the identifier of the liver tissue, B is the identifier of the intrahepatic bile duct tissue, C is the identifier of the extrahepatic bile duct tissue, D is the identifier of the blood vessel tissue, O is the identifier of the calculus, (x si ,y si ,z si ) is the i-th point cloud data in the three-dimensional point cloud data of s, x si ,y si ,z si are the three-dimensional components of (x si ,y si ,z si ) in the three-dimensional coordinate system respectively, and n s is the total number of point cloud data in the three-dimensional point cloud data of s.

[0029] As a preferred embodiment of the present invention, the present invention provides a three-dimensional visualization model reconstruction system based on imaging, which is applied to a three-dimensional visualization model reconstruction method based on imaging. The system includes:

[0030] A data acquisition unit, configured to obtain two-dimensional slice images of the liver through CT or MRI;

[0031] A tissue segmentation unit, configured to perform multi-tissue segmentation on the two-dimensional slice images by using the Unet-2D algorithm to obtain a plurality of tissue segmentation images;

[0032] A voxel processing unit, configured to perform voxelization processing on each tissue segmentation image respectively to obtain the three-dimensional volume data of each tissue;

[0033] A display enhancement unit, which uses multi-objective optimization technology to combine with the Unet-3D algorithm to allocate high-contrast colors for enhancing the display contrast between tissues to the three-dimensional volume data of each tissue, and obtains the high-contrast three-dimensional volume data of each tissue;

[0034] A model reconstruction unit, which uses Poisson surface reconstruction technology to reconstruct and generate a high-contrast three-dimensional surface model of the liver by using the high-contrast three-dimensional volume data of each tissue;

[0035] A visualization unit, which performs visualization processing on the high-contrast three-dimensional surface model of the liver by using volume rendering technology to obtain a high-contrast visual three-dimensional model of the liver that allows user interaction operations.

[0036] As a preferred embodiment of the present invention, the tissue segmentation unit uses the Unet-2D algorithm to sequentially segment the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and stones in the two-dimensional slice images, and correspondingly obtains the slice segmentation images of the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and stones.

[0037] As a preferred embodiment of the present invention, the voxel processing unit's voxelization processing method for each tissue segmentation image includes:

[0038] Stack the slice segmentation images of the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and stones respectively to correspondingly obtain the three-dimensional point cloud data of the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and stones;

[0039] Assign the gray value attribute in the two-dimensional slice image to each point cloud data in the three-dimensional point cloud data of the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and stones respectively to correspondingly obtain the three-dimensional volume data of the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and stones.

[0040] As a preferred embodiment of the present invention, the high-contrast color allocation method of the display enhancement unit for the three-dimensional volume data of each tissue includes:

[0041] Set the high-contrast colors assigned to the three-dimensional volume data of the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and stones to RGB A RGB B RGB C RGB D RGB O After allocating the high-contrast colors, correspondingly obtain the high-contrast three-dimensional volume data of the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and stones;

[0042] Generate a high-contrast three-dimensional surface model M of the liver from high-contrast three-dimensional volume data 3D (RGB);

[0043] Use the Unet-3D algorithm to segment the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and stones from the high-contrast three-dimensional surface model M of the liver 3D (RGB) to obtain the 3D segmentation results of the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and stones;

[0044] Use multi-objective optimization technology to construct an objective function for solving RGB A , RGB B , RGB C , RGB D , RGB O ;

[0045] With the RGB value range as the search space, solve the objective function to obtain the optimal range of RGB A , RGB B , RGB C , RGB D , RGB O ;

[0046] In the optimal range of RGB A , RGB B , RGB C , RGB D , RGB O , select the color corresponding to the highest brightness as the high-contrast color assigned to the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and stones.

[0047] As a preferred embodiment of the present invention, the display enhancement unit also uses a convolutional neural network to train the mapping relationship between the three-dimensional volume data of each tissue and the high-contrast color of each tissue to obtain a strengthened display model {RGB s,best |s∈[A,B,C,D,O]} = CNN({(x si ,y si ,z si )|i∈[1,n s},s∈[A,B,C,D,O]), where CNN is a convolutional neural network, RG s,best is the optimal solution of the high-contrast color of s, A is the identifier of the liver tissue, B is the identifier of the intrahepatic bile duct tissue, C is the identifier of the extrahepatic bile duct tissue, D is the identifier of the blood vessel tissue, O is the identifier of the stone, (x si ,y si ,z si ) is the i-th point cloud data in the three-dimensional point cloud data of s, xsi , y si , z si are respectively the three-dimensional components of (x si , y si , z si ) on a three-dimensional coordinate system, where n s is the total number of point cloud data in the three-dimensional point cloud data of s.

[0048] The present invention has the following beneficial effects compared with the prior art:

[0049] The present invention uses three-dimensional reconstruction processing technology to process CT or MRI data, transforms the original two-dimensional liver image into a three-dimensional model, accurately describes the shape and spatial distribution of stones, and through enhanced display processing, when performing three-dimensional visualization reconstruction, highlights the part of hepatobiliary stones, strongly displays the stones, can intuitively show the distribution and positioning of stones to surgical personnel, and improves the visualization effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only exemplary, and for those of ordinary skill in the art, without creative efforts, other implementation drawings can be obtained according to the provided drawings.

[0051] Figure 1 is a flowchart of a three-dimensional visualization model reconstruction method provided by an embodiment of the present invention;

[0052] Figure 2 is a block diagram of a three-dimensional visualization model reconstruction system provided by an embodiment of the present invention;

[0053] Figure 3 is a schematic diagram of a high-contrast three-dimensional surface model provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0054] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0055] As Figure 1 shown, the present invention provides a three-dimensional visualization model reconstruction method based on imaging, including the following steps:

[0056] Obtain two-dimensional slice images of the liver through CT or MRI;

[0057] In the two-dimensional slice images, use the Unet-2D algorithm for multi-tissue segmentation to obtain multiple tissue segmentation images;

[0058] Perform voxelization processing on each tissue segmentation image respectively to obtain the three-dimensional volume data of each tissue;

[0059] Use multi-objective optimization technology combined with the Unet-3D algorithm to assign high-contrast colors for enhancing the display contrast between tissues to the three-dimensional volume data of each tissue, and obtain the three-dimensional volume data with high contrast of each tissue;

[0060] Reconstruct and generate a three-dimensional surface model of the liver with high contrast by using the Poisson surface reconstruction technology with the three-dimensional volume data with high contrast of each tissue;

[0061] Visualize the three-dimensional surface model of the liver with high contrast through volume rendering technology to obtain a three-dimensional visualization model of the liver with high contrast that allows users to interact (allows users to perform operations such as rotation, scaling, cutting, etc.). The three-dimensional visualization model of the liver with high contrast is conducive to clearly showing the morphology and size of each part of the liver, the three-dimensional anatomy of the intrahepatic and extrahepatic bile ducts, the precise location, size and quantity of stones, the dilation and stenosis sites and degrees of diseased bile ducts, the course of the in-and-out hepatic blood vessels, and even measuring the volume of the liver to be resected and the remaining liver.

[0062] In order to accurately describe the distribution of stones in the liver, the present invention uses three-dimensional reconstruction technology to construct two-dimensional images that are prone to overlapping artifacts into a three-dimensional visualization model, which can clearly display the three-dimensional morphology and mutual relationship of the intrahepatic bile duct tree and blood vessel tree, the size of the stones and their distribution in the bile ducts of each liver segment, whether there are variations in the bile ducts, the degree and scope of bile duct stenosis, blood vessel variations, whether there is atrophy in the liver, etc., to ensure accurate display of the stone distribution.

[0063] The present invention uses the Unet-2D algorithm commonly used in medical image segmentation to segment the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue and stones, so as to identify and segment the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue and stones in the two-dimensional image for three-dimensional reconstruction.

[0064] In order to ensure that the constructed three-dimensional visualization model can enhance the display of stones and facilitate the observation of surgical personnel, the present invention reassigns the colors of the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue and stones in the three-dimensional visualization model through multi-objective optimization technology, replaces the gray values in the original two-dimensional image, and generates a three-dimensional visualization model with high contrast, realizing highlighting the stone distribution in the three-dimensional visualization model and improving the visualization effect.

[0065] The present invention assigns colors to liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and stones through multi-objective optimization to enhance the contrast between tissues and ensure strong display of the stone part. Specifically, there are two aspects of optimization. The first aspect is that the three-dimensional model reconstructed with the high-contrast colors after assignment can obtain the same results as the tissue segmentation based on the gray values of the two-dimensional image when re-segmenting the tissues, thereby ensuring that the assignment of high-contrast colors does not affect the accurate display of the distribution of liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and stones. Therefore, the first optimization goal is that the tissue segmentation results of the three-dimensional model reconstructed with high-contrast colors have the smallest difference from the tissue segmentation results using the gray values of the two-dimensional image, realizing high-precision tissue distribution in the three-dimensional model reconstructed with high-contrast colors;

[0066] The second aspect is that it can ensure that each tissue and stone in the three-dimensional model reconstructed with high-contrast colors can present high contrast, and the closer the position is to the stone distribution, the higher the position contrast, making it easier to distinguish the stone distribution, thereby highlighting the stone distribution position. Therefore, the second optimization goal is that there is a large contrast difference between the colors assigned to liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and stones, and the difference is greater in the part closer to the stone, realizing strong display of the stone in the three-dimensional model reconstructed with high-contrast colors.

[0067] The high-contrast colors of each tissue obtained by solving the above two optimization goals can realize high-precision tissue distribution display in the three-dimensional model reconstructed while achieving strong display of the stone to highlight the stone distribution. The assignment of high-contrast colors is through the optimized selection of objective data, not subjective random setting, which can improve the visualization effect and help the subsequent observation and application of surgical personnel.

[0068] Furthermore, the present invention trains this kind of optimized selection through a neural network, thereby constructing a strengthened display model, which can obtain the mapping relationship between the three-dimensional volume data of each tissue and the high-contrast colors of each tissue, so as to directly obtain the high-contrast colors of each tissue according to the three-dimensional volume data of each tissue, simplify and encapsulate the process of optimizing the color assignment of each tissue, improve efficiency, and facilitate subsequent migration and use, expanding the applicable scenarios of enhanced visualization effect.

[0069] The present invention uses the Unet-2D algorithm commonly used in medical image segmentation to segment liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and stones, thereby identifying and segmenting liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and stones in the two-dimensional image for three-dimensional reconstruction, as follows:

[0070] The method for multi-tissue segmentation using the Unet-2D algorithm includes:

[0071] Using the Unet-2D algorithm to sequentially segment the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and stones in the two-dimensional slice images, and correspondingly obtain the slice segmentation images of the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and stones.

[0072] The voxelization processing method for each tissue segmentation image includes:

[0073] Stack the slice segmentation images of the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and stones respectively to correspondingly obtain the three-dimensional point cloud data of the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and stones {(x si ,y si ,z si )|i∈[1,n s}, where s∈[A,B,C,D,O], A is the identifier of the liver tissue, B is the identifier of the intrahepatic bile duct tissue, C is the identifier of the extrahepatic bile duct tissue, D is the identifier of the blood vessel tissue, O is the identifier of the stone, (x si ,y si ,z si ) is the i-th point cloud data in the three-dimensional point cloud data of s, x si ,y si ,z si are the three-dimensional components of (x si ,y si ,z si ) in the three-dimensional coordinate system respectively, and n s is the total number of point cloud data in the three-dimensional point cloud data of s;

[0074] Assign the gray value attribute in the two-dimensional slice image to each point cloud data in the three-dimensional point cloud data of the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and stones respectively to correspondingly obtain the three-dimensional volume data of the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and stones {V(x si ,y si ,z si )|i∈[1,n s}, where s∈[A,B,C,D,O], A is the identifier of the liver tissue, B is the identifier of the intrahepatic bile duct tissue, C is the identifier of the extrahepatic bile duct tissue, D is the identifier of the blood vessel tissue, O is the identifier of the stone, and V(x si ,y si ,z si ) is the i-th voxel in the three-dimensional volume data of s.

[0075] In order to ensure that the constructed three-dimensional visualization model can enhance the display of calculi and facilitate the observation of surgical personnel, the color of the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and calculi in the three-dimensional visualization model is redistributed through multi-objective optimization technology, replacing the gray values in the original two-dimensional images, generating a three-dimensional visualization model with high contrast, highlighting the distribution of calculi in the three-dimensional visualization model, and improving the visualization effect, as follows:

[0076] The method for allocating high-contrast colors to the three-dimensional volume data of each tissue includes:

[0077] Set the high-contrast colors allocated to the three-dimensional volume data of the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and calculi to RGB A , RGB B , RGB C , RGB D , RGB O , and the high-contrast three-dimensional volume data of the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and calculi obtained after allocating high-contrast colors {RGB s (x si , y si , z si )|i ∈ [1, n s}, where s ∈ [A, B, C, D, O], A is the identifier of the liver tissue, B is the identifier of the intrahepatic bile duct tissue, C is the identifier of the extrahepatic bile duct tissue, D is the identifier of the blood vessel tissue, O is the identifier of the calculi, RGB s (x si , y si , z si ) is the high-contrast color of the three-dimensional volume data of s;

[0078] Generate a high-contrast three-dimensional surface model M of the liver according to the high-contrast three-dimensional volume data {RGB s (x si , y si , z si )|i ∈ [1, n s}; 3D (RGB);

[0079] Use the Unet-3D algorithm to segment the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and calculi from the high-contrast three-dimensional surface model M of the liver 3D (RGB), and obtain the 3D segmentation results of the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and calculi {(x′ si , y′ si , z′ si)|i ∈ [1, n′ s}, where s ∈ [A, B, C, D, O], A is the identifier of liver tissue, B is the identifier of intrahepatic bile duct tissue, C is the identifier of extrahepatic bile duct tissue, D is the identifier of blood vessel tissue, O is the identifier of calculus, (x′ si , y′ si , z′ si ) is the i-th point cloud data in the 3D segmentation result of s, x′ si , y′ si , z′ si are the three-dimensional components of (x′ si , y′ si , z′ si ) in the three-dimensional coordinate system respectively, n′ s is the total number of point cloud data in the 3D segmentation result of s;

[0080] Use multi-objective optimization technology to construct and solve the objective functions of RGB A , RGB B , RGB C , RGB D , RGB O . The objective functions are:

[0081]

[0082] In the formula, g1 and g2 are the first optimization objective and the second optimization objective respectively, min is the minimization operator, {(x Oi , y Oi , z Oi )|i ∈ [1, n O} is the i-th point cloud data in the three-dimensional point cloud data of calculus, x Oi , y Oi , z Oi are the three-dimensional components of (x Oi , y Oi , z Oi ) in the three-dimensional coordinate system respectively, n O is the total number of point cloud data in the three-dimensional point cloud data of calculus, RGB O is the high-contrast color of calculus, RGB s is the high-contrast color of s;

[0083] Take the RGB value range as the search space, solve the objective function to obtain the optimal range of RGB A , RGB B , RGB C , RGB D , RGB O ;

[0084] In RGBA , RGB B , RGB C , RGB D , RGB O In the optimal ranges of , the color corresponding to the highest brightness is selected as the high-contrast color assigned to the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and calculus.

[0085] Specifically, the present invention optimizes two aspects. The first aspect is that when the three-dimensional model reconstructed with the assigned high-contrast color is re-segmented (implemented by the Unet-3D algorithm), the result will be consistent with the tissue segmentation performed on the grayscale values of the two-dimensional image, so as to ensure that the distribution display accuracy of the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and calculus will not be affected by the assignment of the high-contrast color. Therefore, the first optimization goal is that the tissue segmentation result of the three-dimensional model reconstructed with the high-contrast color has the smallest difference from the tissue segmentation result using the grayscale values of the two-dimensional image, and the high-precision distribution of tissues in the three-dimensional model reconstructed with the high-contrast color is achieved.

[0086] Among them, For the first optimization goal, {(x′ si , y′ si , z′ si )|i ∈ [1, n′ s} corresponds to the tissue segmentation results of the three-dimensional model reconstructed with the high-contrast color, and {(x si , y si , z si )|i ∈ [1, n s} corresponds to the tissue segmentation results using the grayscale values of the two-dimensional image. Specifically, the first optimization goal can be broken down into five minimization goals, which are respectively:

[0087] min({(x′ Ai , y′ Ai , z′ Ai )|i ∈ [1, n′ A} ― {(x Ai , y Ai , z Ai )|i ∈ [1, n A}), in terms of RGB A , RGB B , RGB C , RGB D , RGB OThe difference between the liver tissue segmentation results of the 3D model reconstructed by high-contrast color and the liver tissue segmentation results of the 2D image grayscale value is minimized, that is, the influence on the distribution accuracy of liver tissue in the process of high-contrast color allocation to realize enhanced display of stones is minimized;

[0088] min({(x′ Bi ,y′ Bi ,z′ Bi )|i∈[1,n′ B ]}―{(x Bi ,y Bi ,z Bi )|i∈[1,n B ]}), in RGB A , RGB B , RGB C , RGB D , RGB O Minimize the difference between the intrahepatic bile duct tissue segmentation result of the 3D model reconstructed by high-contrast color and the intrahepatic bile duct tissue segmentation result of the 2D image grayscale value, that is, minimize the impact on the distribution accuracy of the intrahepatic bile duct tissue in the process of high-contrast color allocation to achieve enhanced display of stones;

[0089] min({(x′ Ci ,y′ Ci ,z′ Ci )|i∈[1,n′ C ]}―{(x Ci ,y Ci ,z Ci )|i∈[1,n C ]}), in RGB A , RGB B , RGB C , RGB D , RGB O The difference between the extrahepatic bile duct tissue segmentation result of the 3D model reconstructed by high-contrast color and the extrahepatic bile duct tissue segmentation result of the 2D image gray value is minimized, that is, the influence on the distribution accuracy of the extrahepatic bile duct tissue in the process of high-contrast color allocation to realize enhanced display of stones is minimized;

[0090] min({(x′ Di ,y′ Di ,z′ Di )|i∈[1,n′ D ]}―{(x Di ,y Di ,z Di )|i∈[1,n D ]}), in RGBA , RGB B , RGB C , RGB D , RGB O Minimize the difference between the vascular tissue segmentation result of the three-dimensional model reconstructed from high-contrast colors and the vascular tissue segmentation result of the two-dimensional image gray value, that is, minimize the impact on the distribution accuracy of vascular tissue during the process of achieving stone enhancement display through high-contrast color assignment;

[0091] min({(x′ Oi , y′ OI , z′ Oi ) | i ∈ [1, n′ o} ― {(x Oi , y Oi , z Oi ) | i ∈ [1, n O})), with RGB A , RGB B , RGB C , RGB D , RGB O Minimize the difference between the stone segmentation result of the three-dimensional model reconstructed from high-contrast colors and the stone segmentation result of the two-dimensional image gray value, that is, minimize the impact on the distribution accuracy of stone tissue during the process of achieving stone enhancement display through high-contrast color assignment.

[0092] The second aspect is to ensure that each tissue and the stone in the three-dimensional model reconstructed from high-contrast colors can present high contrast, and the closer to the stone distribution position, the higher the position contrast, and it is easier to distinguish the stone distribution, thus highlighting the stone distribution position. Therefore, the second optimization goal is that the contrast differences between the colors assigned to the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, vascular tissue and the stone are large, and the differences are larger in the parts closer to the stone, so as to achieve strong display of the stone in the three-dimensional model reconstructed from high-contrast colors.

[0093] Among them, is the second optimization goal, {(x Oi , y Oi , z Oi ) | i ∈ [1, n O} ― {(x si , y si , z si ) | i ∈ [1, n s} is the distance between the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, vascular tissue and the stone, (RGB O ― RGB s) is the contrast between liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, vascular tissue and stones. The optimization objective expects that when the value of \(\{(x Oi ,y Oi ,z Oi )|i∈[1,n O \}-\{(x si ,y si ,z si )|i∈[1,n s \} is smaller, the value of \((RGB O - RGB s )\) is larger. Specifically, the second optimization objective can be further divided into four minimization objectives, which are respectively:

[0094] In the three-dimensional model reconstructed with high-contrast colors of RGB A , RGB B , RGB C , RGB D , RGB O , the colors assigned to the liver tissue and the stones show high contrast, and the contrast in color assignment between the liver tissue closer to the stones and the stones should be higher. Thus, during the process of achieving enhanced display of stones through high-contrast color assignment, the visual interference of the liver tissue on the display of the stones is minimized;

[0095] In the three-dimensional model reconstructed with high-contrast colors of RGB A , RGB B , RGB C , RGB D , RGB O , the colors assigned to the intrahepatic bile duct tissue and the stones show high contrast, and the contrast in color assignment between the intrahepatic bile duct tissue closer to the stones and the stones should be higher. Thus, during the process of achieving enhanced display of stones through high-contrast color assignment, the visual interference of the intrahepatic bile duct tissue on the display of the stones is minimized;

[0096] In the three-dimensional model reconstructed with high-contrast colors of RGB A , RGB B , RGB C , RGB D , RGB O , the colors assigned to the extrahepatic bile duct tissue and the stones show high contrast, and the contrast in color assignment between the extrahepatic bile duct tissue closer to the stones and the stones should be higher. Thus, during the process of achieving enhanced display of stones through high-contrast color assignment, the visual interference of the extrahepatic bile duct tissue on the display of the stones is minimized;

[0097] In RGB A , RGB B , RGB C , RGB D , RGB O In the three-dimensional model reconstructed with high-contrast colors, the color distribution of vascular tissue and stones shows high contrast, and the contrast in color distribution between vascular tissue closer to the stones and the stones should be higher. Thus, during the process of enhancing the display of stones with high-contrast color distribution, the visual interference of vascular tissue on the display of stones is minimized.

[0098] The high-contrast colors of each tissue obtained by solving the above two optimization objectives can enable the three-dimensional model reconstructed by three-dimensional reconstruction to have a high-precision tissue distribution display while strongly displaying the stones to highlight the stone distribution. The distribution of high-contrast colors is through the optimized selection of objective data, not subjective random settings, which can improve the visualization effect and help the subsequent observation and use by surgical staff.

[0099] Moreover, when RGB A , RGB B , RGB C , RGB D , RGB O When the solution result is a range, the color with the highest brightness will be selected within this range. While ensuring obvious contrast, the detailed textures of each tissue can be more clearly visible, thus improving the readability of the overall three-dimensional model.

[0100] The present invention trains this kind of optimized selection through a neural network, thereby constructing an enhanced display model, which can obtain the mapping relationship between the three-dimensional volume data of each tissue and the high-contrast colors of each tissue, so as to directly obtain the high-contrast colors of each tissue according to the three-dimensional volume data of each tissue, simplify and encapsulate the optimization process of the color distribution of each tissue, improve the efficiency, and facilitate subsequent migration and use, expanding the applicable scenarios of enhanced visualization effect, specifically as follows:

[0101] The present invention further includes: training the mapping relationship between the three-dimensional volume data of each tissue and the high-contrast colors of each tissue by using a convolutional neural network to obtain an enhanced display model {RGB s,best |s∈[A,B,C,D,O]}=CNN({(x si ,y si ,z si )|i∈[1,n s},s∈[A,B,C,D,O]), where CNN is a convolutional neural network, RGB s,bestThe optimal solution for the high-contrast color of s, A is the identifier of liver tissue, B is the identifier of intrahepatic bile duct tissue, C is the identifier of extrahepatic bile duct tissue, D is the identifier of vascular tissue, O is the identifier of a calculus, (x si , y si , z si ) is the i-th point cloud data in the three-dimensional point cloud data of s, x si , y si , z si are respectively the three-dimensional components of (x si , y si , z si ) in the three-dimensional coordinate system, n s is the total number of point cloud data in the three-dimensional point cloud data of s.

[0102] As Figure 2 shown, the present invention provides a three-dimensional visualization model reconstruction system based on imaging, which is applied to a three-dimensional visualization model reconstruction method based on imaging. The system includes:

[0103] A data acquisition unit for obtaining two-dimensional slice images of the liver through CT or MRI;

[0104] A tissue segmentation unit for performing multi-tissue segmentation on the two-dimensional slice images by using the Unet-2D algorithm to obtain multiple tissue segmentation images;

[0105] A voxel processing unit for respectively performing voxelization processing on each tissue segmentation image to obtain three-dimensional volume data of each tissue;

[0106] A display enhancement unit for using multi-objective optimization technology combined with the Unet-3D algorithm to assign high-contrast colors for enhancing the display contrast between tissues to the three-dimensional volume data of each tissue to obtain three-dimensional volume data with high contrast of each tissue;

[0107] A model reconstruction unit for reconstructing and generating a high-contrast three-dimensional surface model of the liver by using the Poisson surface reconstruction technology with the three-dimensional volume data with high contrast of each tissue;

[0108] A visualization unit for visualizing the high-contrast three-dimensional surface model of the liver by using volume rendering technology to obtain a high-contrast visual three-dimensional model of the liver that allows user interaction operations.

[0109] The tissue segmentation unit sequentially segments the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, vascular tissue, and calculus in the two-dimensional slice images by using the Unet-2D algorithm, and correspondingly obtains slice segmentation images of the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, vascular tissue, and calculus.

[0110] The voxel processing method for each tissue segmentation image by the voxel processing unit includes:

[0111] Stack the segmentation images of each slice of the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and stones respectively to obtain the three-dimensional point cloud data of the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and stones;

[0112] Assign the gray value attribute in the two-dimensional slice image to each point cloud data in the three-dimensional point cloud data of the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and stones respectively to obtain the three-dimensional volume data of the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and stones.

[0113] The high-contrast color assignment method for the three-dimensional volume data of each tissue by the display enhancement unit includes:

[0114] Set the high-contrast colors assigned to the three-dimensional volume data of the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and stones to RGB A ,RGB B ,RGB C ,RGB D ,RGB O ,and the high-contrast three-dimensional volume data of the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and stones obtained after assigning high-contrast colors;

[0115] Generate a high-contrast three-dimensional surface model M of the liver based on the high-contrast three-dimensional volume data 3D (RGB);

[0116] Use the Unet-3D algorithm to segment the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and stones from the high-contrast three-dimensional surface model M of the liver 3D (RGB) to obtain the 3D segmentation results of the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and stones;

[0117] Use the multi-objective optimization technique to construct and solve the objective function for RGB A ,RGB B ,RGB C ,RGB D ,RGB O ;

[0118] With the RGB value range as the search space, solve the objective function to obtain RGB A ,RGB B ,RGB C ,RGB D ,RGB OThe optimal range;

[0119] In RGB A , RGB B , RGB C , RGB D , RGB O In the optimal ranges of RGB, the colors corresponding to the highest brightness are selected as the high-contrast colors assigned to the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, blood vessel tissue, and stones.

[0120] The display enhancement unit also uses a convolutional neural network to train the mapping relationship between the three-dimensional volume data of each tissue and the high-contrast color of each tissue, and obtains a strengthened display model {RGB s,best |s∈[A,B,C,D,O]}=CNN({(x si ,y si ,z si )|i∈[1,n s},s∈[A,B,C,D,O]), where CNN is the convolutional neural network, RGB s,best is the optimal solution of the high-contrast color of s, A is the identifier of the liver tissue, B is the identifier of the intrahepatic bile duct tissue, C is the identifier of the extrahepatic bile duct tissue, D is the identifier of the blood vessel tissue, O is the identifier of the stone, (x si ,y si ,z si ) is the i-th point cloud data in the three-dimensional point cloud data of s, x si ,y si ,z si are the three-dimensional components of (x si ,y si ,z si ) in the three-dimensional coordinate system, and n s is the total number of point cloud data in the three-dimensional point cloud data of s.

[0121] The present invention uses three-dimensional reconstruction processing technology to process CT or MRI data, transforms the original two-dimensional liver image into a three-dimensional model, accurately describes the morphology and spatial distribution of stones, etc., can clearly display the three-dimensional morphology and the mutual relationship of the intrahepatic bile duct tree and blood vessel tree, the size of the stones and their distribution in each hepatic segment bile duct, whether there are bile duct variations, the degree and range of bile duct stenosis, blood vessel variations, whether there is liver atrophy, etc. And through enhanced display processing, when performing three-dimensional visualization reconstruction, the part of the hepatobiliary stones is highlighted, and the stones are strongly displayed, which can intuitively show the distribution and positioning of the stones to the surgical staff and improve the visualization effect.

[0122] The above embodiments are only exemplary embodiments of the present application and are not used to limit the present application. The protection scope of the present application is defined by the claims. Those skilled in the art can make various modifications or equivalent replacements within the essence and protection scope of the present application, and such modifications or equivalent replacements should also be regarded as falling within the protection scope of the present application.

Claims

1. A three-dimensional visualization model reconstruction method based on imaging, characterized in that: The following steps are involved: Obtain two-dimensional slice images of the liver through CT or MRI; In the two-dimensional slice image, multi-tissue segmentation is performed using the Unet-2D algorithm to obtain multiple tissue segmentation images; Each tissue segmentation image is voxelized to obtain three-dimensional volume data of each tissue; Using multi-objective optimization technology combined with Unet-3D algorithm, high-contrast colors for enhancing the display contrast between tissues are assigned to the three-dimensional volume data of each tissue, thereby obtaining high-contrast three-dimensional volume data of each tissue; The high-contrast three-dimensional surface model of the liver is reconstructed using the high-contrast three-dimensional volume data of each tissue through Poisson surface reconstruction technology; The high-contrast three-dimensional surface model of the liver is visualized by volume rendering technology to obtain a high-contrast visualized three-dimensional model of the liver that allows user interactive operation.

2. The method for reconstructing a three-dimensional visualization model based on imaging according to claim 1, characterized in that: Methods for multi-tissue segmentation using the Unet-2D algorithm include: The Unet-2D algorithm is used to segment the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, vascular tissue and stones in the two-dimensional slice images in turn, and corresponding slice segmentation images of the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, vascular tissue and stones are obtained.

3. The method for reconstructing a three-dimensional visualization model based on imaging according to claim 2, characterized in that: The voxel processing methods of each tissue segmentation image include: Each slice segmentation image of liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, vascular tissue and calculus is sliced ​​and stacked to obtain the corresponding three-dimensional point cloud data of liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, vascular tissue and calculus {(x si ,y si ,z si )|i∈[1,n s ]}, where s∈[A,B,C,D,O], A is the identifier of liver tissue, B is the identifier of intrahepatic bile duct tissue, C is the identifier of extrahepatic bile duct tissue, D is the identifier of vascular tissue, O is the identifier of stone, (x si ,y si ,z si ) is the i-th point cloud data in the three-dimensional point cloud data of s, x si ,y si ,z si They are (x si ,y si ,z si ) in the three-dimensional coordinate system, n s is the total number of point cloud data in the three-dimensional point cloud data of s; Each point cloud data of the three-dimensional point cloud data of liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, vascular tissue and calculus is assigned the gray value attribute in the two-dimensional slice image, and the three-dimensional volume data {V(x si ,y si ,z si )|i∈[1,n s ]}, where s∈[A,B,C,D,O], A is the identifier of liver tissue, B is the identifier of intrahepatic bile duct tissue, C is the identifier of extrahepatic bile duct tissue, D is the identifier of vascular tissue, O is the identifier of stone, V(x si ,y si ,z si ) is the i-th voxel in the three-dimensional volume data of s.

4. The method for reconstructing a three-dimensional visualization model based on imaging according to claim 3, characterized in that: Methods for assigning high contrast colors to three-dimensional volume data of various tissues include: Set the high contrast color assigned to the 3D volume data of liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, vascular tissue, and stones to RGB A , RGB B , RGB C , RGB D , RGB O , after assigning high contrast colors, the high contrast three-dimensional data of liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, vascular tissue and stones {RGB s (x si ,y si ,z si )|i∈[1,n s ]}, where s∈[A,B,C,D,O], A is the identifier of liver tissue, B is the identifier of intrahepatic bile duct tissue, C is the identifier of extrahepatic bile duct tissue, D is the identifier of vascular tissue, O is the identifier of stone, and RGB s (x si ,y si ,z si ) is the high contrast color of the three-dimensional volume data of s; According to the high contrast 3D volume data {RGB s (x si ,y si ,z si )|i∈[1,n s ]}Generate a high-contrast 3D surface model of the liver 3D (RGB); High contrast three-dimensional surface model of liver using Unet-3D algorithm 3D (RGB) Segment the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, vascular tissue and stones to obtain the 3D segmentation results of the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, vascular tissue and stones {(x′ si ,y′ si ,z′ si ) |i∈[1,n′ s ]}, where s∈[A,B,C,D,O], A is the identifier of liver tissue, B is the identifier of intrahepatic bile duct tissue, C is the identifier of extrahepatic bile duct tissue, D is the identifier of vascular tissue, O is the identifier of stone, (x′ si ,y′ si ,z′ si ) is the i-th point cloud data in the 3D segmentation result of s, x′ si ,y′ si ,z′ si They are (x′ si ,y′ si ,z′ si ) in the three-dimensional coordinate system, n′ s is the total number of point cloud data in the 3D segmentation result of s; Using multi-objective optimization technology to construct a solution for RGB A , RGB B , RGB C , RGB D , RGB O The objective function is: Where g1 and g2 are the first optimization objective and the second optimization objective respectively, min is the minimization operator, {(x Oi ,y Oi ,z Oi )|i∈[1,n O ]} is the i-th point cloud data in the three-dimensional point cloud data of the stone, x Oi ,y Oi ,z Oi They are (x Oi ,y Oi ,Z Oi ) in the three-dimensional coordinate system, n O is the total number of point cloud data in the 3D point cloud data of the stone, RGB O For high contrast colors of stones, RGB s A high contrast color for s; Using the RGB value range as the search space, solve the objective function to obtain RGB A , RGB B , RGB C , RGB D , RGB O The optimal range of In RGB A , RGB B , RGB C , RGB D , RGB O The color corresponding to the highest brightness in the optimal range is selected as the high-contrast color assigned to liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, vascular tissue and stones.

5. The method for reconstructing a three-dimensional visualization model based on imaging according to claim 4, characterized in that: Also includes: The convolutional neural network is used to train the mapping relationship between the three-dimensional volume data of each tissue and the high-contrast color of each tissue to obtain the enhanced display model {RGB s,best |s∈[A,B,C,D,O]}=CNN({(x si ,y si ,z si )|i∈[1,n s ]},s∈[A,B,C,D,o]), where CNN is a convolutional neural network, RGB s,best is the optimal solution of high contrast color of s, A is the identifier of liver tissue, B is the identifier of intrahepatic bile duct tissue, C is the identifier of extrahepatic bile duct tissue, D is the identifier of vascular tissue, O is the identifier of stone, (x si ,y si ,z si ) is the i-th point cloud data in the three-dimensional point cloud data of s, x si ,y si ,z si They are (x si ,y si ,z si ) in the three-dimensional coordinate system, n s is the total number of point cloud data in the three-dimensional point cloud data of s.

6. A three-dimensional visualization model reconstruction system based on imaging, characterized in that: A method for reconstructing a three-dimensional visualization model based on imaging as described in any one of claims 1 to 5, the system comprising: A data acquisition unit, used for acquiring two-dimensional slice images of the liver through CT or MRI; A tissue segmentation unit, used to perform multi-tissue segmentation in the two-dimensional slice image using a Unet-2D algorithm to obtain a plurality of tissue segmentation images; A voxel processing unit, used for voxelizing each tissue segmentation image to obtain three-dimensional volume data of each tissue; A display enhancement unit is used to allocate high-contrast colors for enhancing the display contrast between the tissues to the three-dimensional volume data of each tissue by using a multi-objective optimization technology combined with a Unet-3D algorithm, so as to obtain high-contrast three-dimensional volume data of each tissue; A model reconstruction unit, used for reconstructing a high-contrast three-dimensional surface model of the liver by using the high-contrast three-dimensional volume data of each tissue through Poisson surface reconstruction technology; The visualization unit is used to visualize the high-contrast three-dimensional surface model of the liver through volume rendering technology to obtain a high-contrast visualized three-dimensional model of the liver that allows user interactive operation.

7. The three-dimensional visualization model reconstruction system based on imaging according to claim 6, characterized in that: The tissue segmentation unit uses the Unet-2D algorithm to segment the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, vascular tissue and stones in the two-dimensional slice image in turn, and obtains corresponding slice segmentation images of the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, vascular tissue and stones.

8. The three-dimensional visualization model reconstruction system based on imaging according to claim 7, characterized in that: The voxel processing method of the voxel processing unit for each tissue segmentation image includes: Each slice segmentation image of the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, vascular tissue and calculus is sliced ​​and stacked to obtain corresponding three-dimensional point cloud data of the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, vascular tissue and calculus; Each point cloud data in the three-dimensional point cloud data of liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, vascular tissue and stones is assigned a gray value attribute in the two-dimensional slice image, and the corresponding three-dimensional volume data of liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, vascular tissue and stones are obtained.

9. The three-dimensional visualization model reconstruction system based on imaging according to claim 8, characterized in that: The method for allocating high contrast colors of the three-dimensional volume data of each tissue by the display enhancement unit includes: Set the high contrast color assigned to the 3D volume data of liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, vascular tissue, and stones to RGB A , RGB B , RGB C , RGB D , RGB O , after assigning high-contrast colors, high-contrast three-dimensional volume data of liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, vascular tissue and stones are obtained; Generate a high-contrast 3D surface model M of the liver based on the high-contrast 3D volume data 3D (RGB); High contrast three-dimensional surface model of liver using Unet-3D algorithm 3D (RGB) Segment the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, vascular tissue and stones to obtain the 3D segmentation results of the liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, vascular tissue and stones; Using multi-objective optimization technology to construct a solution for RGB A , RGB B , RGB C , RGB D , RGB O The objective function of Using the RGB value range as the search space, solve the objective function to obtain RGB A , RGB B , RGB C , RGB D , RGB O The optimal range of In RGB A , RGB B , RGB C , RGB D , RGB O The color corresponding to the highest brightness in the optimal range is selected as the high-contrast color assigned to liver tissue, intrahepatic bile duct tissue, extrahepatic bile duct tissue, vascular tissue and stones.

10. The three-dimensional visualization model reconstruction system based on imaging according to claim 9, characterized in that: The display enhancement unit also uses a convolutional neural network to train the mapping relationship between the three-dimensional volume data of each tissue and the high-contrast color of each tissue to obtain an enhanced display model {RGB s,best |s∈[A,B,C,D,O]}=CNN({(x si ,y si ,z si )|i∈[1,n s ]},s∈[A,B,C,D,O]), where CNN is a convolutional neural network, RGB s,best is the optimal solution of high contrast color of s, A is the identifier of liver tissue, B is the identifier of intrahepatic bile duct tissue, C is the identifier of extrahepatic bile duct tissue, D is the identifier of vascular tissue, O is the identifier of stone, (x si ,y si ,z si ) is the i-th point cloud data in the three-dimensional point cloud data of s, x si ,y si ,z si They are (x si ,y si ,x si ) in the three-dimensional coordinate system, n s is the total number of point cloud data in the three-dimensional point cloud data of s.

Images