Liver blood vessel three-dimensional reconstruction method based on deep learning and image processing

Abstract

The invention relates to the technical field of medical image processing and computer-aided diagnosis, in particular to a liver blood vessel three-dimensional reconstruction method based on deep learning and image processing. The method comprises the following steps: 1) constructing a data set containing multiple liver blood vessel CT images for three-dimensional reconstruction; 2) screening out qualified CT images, and ensuring that the reconstruction result is accurate; 3) inputting the screened image into a three-dimensional reconstruction system, and preparing for further processing; 4) preprocessing the CT image by using an image processing technology to highlight the ROI of the liver blood vessel; 5) automatically identifying and segmenting a liver blood vessel structure in the CT image through a semantic segmentation algorithm embedded in the system; 6) performing enhancement processing on the segmented image, and constructing a three-dimensional form of the liver blood vessel from different directions by using a multi-view reconstruction technology; and 7) performing three-dimensional reconstruction on the segmented blood vessel image to generate an accurate liver blood vessel three-dimensional model. Doctors are helped to master the anatomical structure of the human body more clearly, so that the operation safety and the treatment effect are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing and computer-aided diagnosis, and in particular to a method for three-dimensional reconstruction of liver blood vessels based on deep learning and image processing. Background Art

[0002] With the development of medical technology, traditional two-dimensional imaging technology can no longer meet the clinical needs for accurate diagnosis, while three-dimensional reconstruction technology can provide more intuitive images of the liver and its vascular structure. Image data obtained through CT scanning, combined with advanced image processing algorithms and deep learning models, can achieve automatic segmentation and three-dimensional modeling of liver blood vessels, providing important assistance for clinical diagnosis and treatment planning. The development of this technology marks an important step forward in the automation and precision of medical image processing. Summary of the invention

[0003] In view of the limitations of existing medical imaging technology, this paper proposes a new method for three-dimensional reconstruction of liver blood vessels. This method relies on image processing and deep learning technology to achieve accurate three-dimensional modeling of the liver and its vascular system through plain scan CT images. This innovation enables doctors to clearly observe the spatial structure of the liver and blood vessels, providing strong support for the accurate positioning, evaluation and formulation of surgical plans for the disease, while enhancing patients' understanding of the disease, demonstrating its broad prospects and practical value in clinical applications.

[0004] The present invention provides a method for three-dimensional reconstruction of liver blood vessels based on deep learning and image processing, which adopts the following technical solutions:

[0005] A method for three-dimensional reconstruction of liver blood vessels based on deep learning and image processing comprises the following steps:

[0006] 1) Construct a dataset containing multiple liver vascular CT images for three-dimensional reconstruction;

[0007] 2) Screen out CT images of qualified quality to ensure accurate reconstruction results;

[0008] 3) Input the screened image into the 3D reconstruction system for further processing;

[0009] 4) Use image processing technology to preprocess CT images and highlight the ROI of liver vessels;

[0010] 5) Automatically identify and segment the liver vascular structure in CT images through the system's built-in semantic segmentation algorithm;

[0011] 6) Enhance the segmented image and use multi-view reconstruction technology to construct the three-dimensional morphology of liver blood vessels from different directions;

[0012] 7) Perform three-dimensional reconstruction on the segmented vascular image to generate an accurate three-dimensional model of liver blood vessels.

[0013] Preferably, step 4) is to perform image cropping on a specified region of interest (ROI) in the CT plain scan image as part of image preprocessing, namely:

[0014]

[0015] Preferably, in step 4), the region of interest (ROI) of the CT plain scan image is subjected to image resampling.

[0016] Ensuring uniform resolution and coordinate system is part of image preprocessing, namely:

[0017]

[0018] Preferably, the image preprocessing image normalization operation in step 4) performs a normalization operation on the CT scan data according to the region of interest (ROI), which is a part of the image preprocessing, namely:

[0019]

[0020] Preferably, in step 5), a semantic segmentation model automatically marks the liver blood vessels in the CT plain scan image, and a loss function combining Dice loss and cross-entropy loss (CE) is used to optimize the segmentation effect of the liver blood vessels. The formula of the combined loss function can be expressed as:

[0021]

[0022] Substitute (4) (5) into (6), and (0<α<1)

[0023] L=α·L CE +(1-α)·L Dice (6);

[0024] Preferably, the semantic segmentation algorithm embedded in the system in step 5) is a semantic segmentation model for automatically annotating bone vessels in CT plain scan images, which uses the LeakyReLU activation function instead of the traditional ReLU activation function. The model uses the LeakyReLU formula to optimize the segmentation effect of liver vessels, and its formula is expressed as:

[0025]

[0026] Preferably, the semantic segmentation algorithm in step 5) adopts deep learning technology and uses instance normalization instead of the widely used batch normalization as its normalization method. The formula of Instance Normalization is:

[0027]

[0028] Preferably, the enhancement processing performed on the segmented image in step 6) uses a morphological closing operation to optimize the segmentation effect of the liver blood vessels, and the formula is expressed as follows:

[0029] D(I,B)(x)=max{I(y):y∈neighborhood(x,B)}(9);

[0030] E(I,B)(x)=min{I(y):y∈neighborhood(x,B)}(10);

[0031] Substitute (9)(10) into (11)

[0032] closing(I)=D(E(I,B))(11);

[0033] Preferably, in step 7), the three-dimensional reconstruction operation uses volume rendering technology to stack the segmented 2D slice data into a three-dimensional volume data, and uses Gaussian smoothing to interpolate the pixel values ​​between slices to ensure smoothness and accuracy during the reconstruction process. The Gaussian filter interpolation formula is as follows:

[0034]

[0035] Preferably, in step 7), the three-dimensional reconstruction operation uses the MarchingCubes algorithm based on the adaptive threshold to extract the isosurface of the blood vessel from the three-dimensional volume data to generate a three-dimensional model of the blood vessel. This isosurface is generated by interpolating all points in the volume data that exceed the adaptive threshold, and the formula is as follows:

[0036] f(V(x,y,z))=sign(V(x,y,z)-T adaptive (x,y,z))(13);

[0037] Preferably, in step 7), the three-dimensional reconstruction operation uses a set density threshold combined with contrast enhancement technology to remove background and noise and retain only the vascular structure. The mathematical expression of this process is as follows:

[0038]

[0039] Preferably, in step 7), the three-dimensional reconstruction operation includes mapping the density value in the volume data to color and transparency through non-linear color mapping to enhance the visualization effect of the blood vessels. The specific formula is as follows:

[0040]

[0041] The beneficial effects of the present invention are:

[0042] 1. Accurate segmentation and enhanced display: This technology uses cutting-edge semantic segmentation algorithms to accurately identify and divide liver blood vessels in original CT images, and repairs discontinuous parts of blood vessels through specific image processing technology to ensure effective distinction even under low-contrast conditions. Combined with three-dimensional reconstruction technology, this method provides doctors with a clear relative position and morphological view of the lesion area and key anatomical structures, thereby assisting in formulating a more accurate surgical plan.

[0043] 2. Multi-level, multi-dimensional image fusion: This technology invention can perform synthesis processing on multi-layer CT images to create a three-dimensional view containing rich dimensional data. By optimizing the transparency and color mapping of each layer of images, the uniqueness of different anatomical structures can be emphasized, helping doctors to comprehensively examine the lesion area and its relationship with adjacent tissues. This multi-layer image fusion technology can provide more detailed image analysis functions when dealing with complex cases.

[0044] 3. Fast processing and real-time interaction: The image processing technology and 3D reconstruction algorithm in this technology have been specially optimized to quickly process and reconstruct large-scale CT image data. The real-time interactive operation enables doctors to instantly change the viewing angle, manipulate the model rotation or change the display mode, thereby improving the efficiency of image analysis and significantly reducing the time for diagnosis and treatment planning.

[0045] 4. Personalized treatment plan formulation: Using the precise 3D reconstruction and image analysis methods provided by this technology invention, doctors can customize treatment plans based on the specific anatomical characteristics of the patient. Through detailed analysis of the 3D model, doctors can more accurately evaluate the practicality of treatment strategies and select the most appropriate treatment approaches and methods, aiming to improve treatment effectiveness and reduce potential risks. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0047] Figure 1 It is a flow chart of a method for three-dimensional reconstruction of liver blood vessels combining image processing and deep learning of the present invention;

[0048] Figure 2 The plain scan CT liver blood vessel semantic segmentation map of the present invention;

[0049] Figure 3 A schematic diagram of liver blood vessel modeling according to the present invention;

[0050] Figure 4 This is a schematic diagram of modeling after registration of liver blood vessels and CT images according to the present invention; DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] An embodiment of a method for three-dimensional reconstruction of liver blood vessels based on plain scan CT images under a semantic segmentation model of the present invention is as follows: Figure 1 As shown, including:

[0053] See also Figure 1 A method for three-dimensional reconstruction of liver blood vessels combining image processing and deep learning is shown, comprising the following steps:

[0054] 1) Construct a dataset containing multiple liver vascular CT images for three-dimensional reconstruction;

[0055] 2) Screen out CT images of qualified quality to ensure accurate reconstruction results;

[0056] 3) Input the screened image into the 3D reconstruction system for further processing;

[0057] 4) Use image processing technology to preprocess CT images and highlight the ROI of liver vessels;

[0058] 5) Automatically identify and segment the liver vascular structure in CT images through the system's built-in semantic segmentation algorithm;

[0059] 6) Enhance the segmented image and use multi-view reconstruction technology to construct the three-dimensional morphology of liver blood vessels from different directions;

[0060] 7) Perform three-dimensional reconstruction on the segmented vascular image to generate an accurate three-dimensional model of liver blood vessels.

[0061] Furthermore, step 4) performs image cropping on the specified region of interest (ROI) in the CT plain scan image as part of image preprocessing, namely:

[0062]

[0063] Where I'(x,y) is the cropped image. This formula means that if the (x,y) coordinate is inside the cropped area, then I'(x,y) takes the pixel value of I at the corresponding position; otherwise, the pixel value is set to 0 or the background color.

[0064] To ensure the accuracy of the bounding box, functions such as binary_fill_holes are used to close the tiny gaps in the mask.

[0065] Resizing requires that the cropped image can be adjusted to a specific size, and the nearest neighbor interpolation method is used here. The formula is:

[0066]

[0067] Where: w and h are the width and height of the original image, w' and h' are the width and height of the target image, and floor means rounding down to the nearest integer.

[0068] Substitute (2) and (3) into (4) to obtain the nearest neighbor interpolation formula.

[0069]

[0070] Let I(x,y) be the pixel value of the original image at coordinate (x,y), and I'(x',y') be the pixel value of the interpolated image at coordinate (x',y'). The nearest neighbor interpolation method determines the value of I'(x',y') by finding the pixel point (x,y) in the original image that is closest to the target coordinate (x',y').

[0071] Furthermore, in step 4), the region of interest (ROI) of the CT plain scan image is resampled to ensure a uniform resolution and coordinate system, which is part of the image preprocessing. The formula is:

[0072]

[0073] Where original_spacing is the original voxel distance, target_spacing is the target voxel distance, and original_dimension is the original image size

[0074] When resizing an image, you need to choose a suitable interpolation method to calculate the pixel value at the new position. The interpolation method used here is nearest neighbor interpolation.

[0075] Furthermore, in step 4), the image preprocessing image normalization operation is performed to normalize the CT scan data according to the region of interest (ROI), which is a part of the image preprocessing. The formula is:

[0076]

[0077] Where Image(x,y) is the pixel value of the original image at coordinate (x,y), Min(Image) is the minimum value of all pixel values ​​in the image, Max(Image) is the maximum value of all pixel values ​​in the image, and NormalizedImage(x,y) is the pixel value of the standardized image at coordinate (x,y).

[0078] Semantic segmentation: The semantic segmentation model in the 3D reconstruction system is used to automatically annotate the liver blood vessels in CT images.

[0079] Furthermore, in step 5), a semantic segmentation model automatically marks the liver blood vessels in the CT plain scan image, and a loss function combining Dice loss and cross-entropy loss (CE) is used to optimize the segmentation effect of the liver blood vessels. The formula of the combined loss function can be expressed as:

[0080]

[0081] Where X is the predicted segmentation result, Y is the actual segmentation result, |X∩Y| is the size of the intersection of the predicted result and the actual result, and |X| and |Y| are the total areas of the predicted result and the actual result, respectively.

[0082]

[0083] Where M is the number of categories, y o,c is the one-hot encoding of the true label, p o,c is the probability that the model predicts class c.

[0084] Substitute (7)(8) into (9)

[0085] L=α·L CE+(1-α)·L Dice (9);

[0086] Among them, L is the total loss, L CE is the cross entropy loss, which measures the difference between the model’s predicted probability distribution and the true label, L Dice is the Dice loss, which measures the similarity between the predicted segmentation result and the true segmentation result. α is a weight parameter used to balance the two losses, usually between 0 and 1.

[0087] Furthermore, in step 5), the semantic segmentation model uses LeakyReLU as the activation function to replace the traditional ReLU activation function. The formula of LeakyReLU is:

[0088]

[0089] Among them, x is the input value, α is a small positive number (usually between 0.01 and 0.1) used to control the gradient of negative input. When x>0, the function is the same as ReLU and outputs x; when x≤0, the function outputs αx, so that even if the input is negative, the neuron can still receive a certain gradient signal, thus avoiding complete inactivation.

[0090] Furthermore, in step 5), a semantic segmentation model for automatically identifying liver blood vessels in CT plain scan images uses instance normalization instead of the widely used batch normalization as its normalization method. The formula of InstanceNormalization is:

[0091]

[0092] Where x is a single sample of the input feature map, μ is the mean of x, and σ 2 is the standard deviation of x, and ε is a small constant used to prevent division by zero.

[0093] Furthermore, in step 6), the segmented image is enhanced by using a morphological closing operation to optimize the segmentation effect of the liver blood vessels, and the formula is expressed as follows:

[0094] D(I,B)(x)=max{I(y):y∈neighborhood(x,B)}(12);

[0095] Where I is the input 3D image, B is the structural element, x is a pixel position in the image, neighborhood(x,B) represents the neighborhood defined by the structural element B with x as the center, and D(I,B)(x) is the pixel value of the image at position x after the dilation operation, which is the maximum value of all pixel values ​​of the input image B in the defined neighborhood.

[0096] E(I,B)(x)=min{I(y):y∈neighborhood(x,B)}(13);

[0097] Where I is the input 3D image, B is the structural element, x is a pixel position in the image, neighborhood(x,B) represents the neighborhood defined by the structural element B with x as the center, and E(I,B)(x) is the pixel value of the image at position x after the erosion operation, which is the minimum value of all pixel values ​​of the input image B in the defined neighborhood.

[0098] Substitute (12)(13) into (14)

[0099] closing(I)=D(E(I,B))(14);

[0100] Here, the closing operation first applies the erosion operation E to the image I, and then applies the dilation operation D to the eroded result, using the same structure element B.

[0101] Furthermore, in step 7), for the reconstruction of the vascular model, the volume rendering technology is first used to stack the segmented 2D slice data into a three-dimensional volume data, and Gaussian smoothing is used to interpolate the pixel values ​​between slices to ensure smoothness and accuracy during the reconstruction process. The Gaussian filter interpolation formula is as follows:

[0102]

[0103] Where I'(x,y,z) is the interpolated pixel value and σ is the standard deviation of the filter.

[0104] Further, in step 7), the MarchingCubes algorithm based on the adaptive threshold is used to extract the isosurface of the blood vessel from the three-dimensional volume data to generate a three-dimensional model of the blood vessel. This isosurface is generated by interpolating all points in the volume data that exceed the adaptive threshold, and the formula is as follows:

[0105] f(V(x,y,z))=sign(V(x,y,z)-T adaptive (x,y,z)) (16);

[0106] Where: T adaptive (x, y, z) is the threshold that is dynamically adjusted based on local features.

[0107] Furthermore, in step 7), by setting a density threshold and combining contrast enhancement technology, the background and noise are removed and only the vascular structure is retained. The mathematical expression of this process is as follows:

[0108]

[0109] Among them, T vessel is the density threshold of blood vessels.

[0110] Furthermore, in step 7), the density value in the volume data is mapped to color and transparency through nonlinear color mapping to enhance the visualization of blood vessels. The specific formula is as follows:

[0111]

[0112] After executing the above process on the plain scan image, the following Figure 2-4 effect.

[0113] Advantages of the present invention:

[0114] 1) Accurate 3D reconstruction technology: The present invention uses plain scan CT images for 3D reconstruction, which can generate a high-precision 3D model. By carefully preprocessing and optimizing CT image data, the 3D structure of liver blood vessels can be accurately reconstructed, providing doctors with accurate diagnosis and treatment plans.

[0115] 2) Deep learning-assisted automatic labeling: Deep learning technology is used to automatically segment CT images and achieve semi-automatic labeling and model optimization of liver vascular areas. This automated method reduces the complexity and subjectivity of manual operations and improves work efficiency and consistency of results.

[0116] 3) Multi-angle comprehensive analysis: Multi-angle processing of the annotated CT images can comprehensively analyze the three-dimensional structure of liver blood vessels. This multi-angle analysis helps to deeply understand the spatial relationship and structural characteristics of the target area.

[0117] 4) Improve model quality: In the process of building a 3D model, the present invention uses insert technology and optimization algorithms to enhance the quality and smoothness of the model. This process reduces noise and holes on the surface of the model, ensures the continuity and accuracy of the 3D model, and is beneficial to subsequent medical analysis and surgical planning.

[0118] 5) Optimization of complex structures: When dealing with complex structures such as vascular bifurcations, the present invention uses the closed operation principle to perform optimization modeling to truly reflect the biological characteristics of blood vessels and enhance the biological rationality and practicality of the model.

[0119] The above description is a preferred embodiment of the present invention and does not limit the scope of the present invention. Any modification, equivalent replacement or improvement made under the guidance of the spirit and principle of the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the scope defined in the claims.

Claims

1. A method for three-dimensional reconstruction of liver blood vessels based on deep learning and image processing, characterized in that: The following steps are involved: 1) Construct a dataset containing multiple liver vascular CT images for three-dimensional reconstruction; 2) Screen out CT images of qualified quality to ensure accurate reconstruction results; 3) Input the screened image into the 3D reconstruction system for further processing; 4) Use image processing technology to preprocess CT images and highlight the ROI of liver vessels; 5) Automatically identify and segment the liver vascular structure in CT images through the system's built-in semantic segmentation algorithm; 6) Enhance the segmented image and use multi-view reconstruction technology to construct the three-dimensional morphology of liver blood vessels from different directions; 7) Perform three-dimensional reconstruction on the segmented vascular image to generate an accurate three-dimensional model of liver blood vessels.

2. The method for three-dimensional reconstruction of liver blood vessels based on deep learning and image processing according to claim 1, characterized in that: The image preprocessing in step 4) includes image cropping of the specified region of interest (ROI) in the CT plain scan image, namely:

3. The method for three-dimensional reconstruction of liver blood vessels based on deep learning and image processing according to claim 1, characterized in that: Image preprocessing in step 4) includes a resampling operation to ensure a uniform resolution and coordinate system, namely:

4. The method for three-dimensional reconstruction of liver blood vessels based on deep learning and image processing according to claim 1, characterized in that: The image preprocessing in step 4) includes normalization of the CT scan data according to the region of interest (ROI), namely:

5. The method for three-dimensional reconstruction of liver blood vessels based on deep learning and image processing according to claim 1, characterized in that: In step 5), the semantic segmentation model for liver vascular segmentation uses deep learning technology, and the loss function is a combination of Dice loss and cross-entropy loss (CE) to achieve accurate segmentation of liver vessels in CT images. The specific expression of this combined loss function is: Substitute (4) (5) into (6), and (0<α<1) L=α·L CE +(1-a)·L Dice (6)。 6. The method for three-dimensional reconstruction of liver blood vessels combining image processing and deep learning according to claim 1, characterized in that: In step 5), the semantic segmentation algorithm embedded in the system is a semantic segmentation model for automatically annotating bone vessels in CT plain scan images. The LeakyReLU activation function is used to replace the traditional ReLU activation function. The LeakyReLU formula is used to optimize the segmentation effect of liver vessels. The combination function of the LeakyReLU formula is used to segment liver vessels in CT images. The formula of the function can be expressed as:

7. The method for three-dimensional reconstruction of liver blood vessels combining image processing and deep learning according to claim 1, characterized in that: In step 5), the semantic segmentation algorithm uses deep learning technology, and its normalization method uses instance normalization instead of the widely used batch normalization as its normalization method. The formula of Instance Normalization is:

8. The method for three-dimensional reconstruction of liver blood vessels combining image processing and deep learning according to claim 1, characterized in that: The enhancement processing of the segmented image in step 6) includes morphological closing operation image processing to ensure the integrity of the liver blood vessels. At the same time, the image information of different slices in the three-dimensional space is integrated to achieve enhanced quality and smoothness. This process reduces the noise and holes on the model surface and ensures the continuity and accuracy of the 3D model. The formula is: D(I,B)(x)=max{I(y):y∈neighborhood(x,B)}(9); E(I,B)(x)=min{I(y):y∈neighborhood(x,B)}(10); Substitute (9)(10) into (11) closing(I)=D(E(I,B))(11).

9. The method for three-dimensional reconstruction of liver blood vessels combining image processing and deep learning according to claim 1, characterized in that: In step 7), the 3D reconstruction operation uses volume rendering technology to stack the segmented 2D slice data into a 3D volume data, and uses Gaussian smoothing to interpolate the pixel values ​​between slices to ensure smoothness and accuracy during the reconstruction process. The Gaussian filter interpolation formula is as follows: Where I'(x,y,z) is the interpolated pixel value and σ is the standard deviation of the filter.

10. The method for three-dimensional reconstruction of liver blood vessels combining image processing and deep learning according to claim 1, characterized in that: In step 7), the 3D reconstruction operation uses the MarchingCubes algorithm based on the adaptive threshold to extract the isosurface of the blood vessel from the 3D volume data and generate a 3D model of the blood vessel. This isosurface is generated by interpolating all points in the volume data that exceed the adaptive threshold, and the formula is as follows: f(V(x,y,z))=sign(V(x,y,z)-T adaptive (x,y,z)) (13); Where: T adaptive (x, y, z) is the threshold that is dynamically adjusted based on local features.

11. The method for three-dimensional reconstruction of liver blood vessels combining image processing and deep learning according to claim 1, characterized in that: In step 7), the 3D reconstruction operation uses a set density threshold combined with contrast enhancement technology to remove background and noise, leaving only the vascular structure. The mathematical expression of this process is as follows: Among them, T vessel is the density threshold of blood vessels.

12. The method for three-dimensional reconstruction of liver blood vessels combining image processing and deep learning according to claim 1, characterized in that: In step 7), the three-dimensional reconstruction operation includes mapping the density value in the volume data to color and transparency through non-linear color mapping to enhance the visualization effect of the blood vessels. The specific formula is as follows:

Images