Liver blood vessel three-dimensional reconstruction system based on image segmentation

Abstract

The invention relates to the technical field of three-dimensional reconstruction, and provides a hepatic vessel three-dimensional reconstruction system based on image segmentation, which optimizes an encoder and a decoder of a U-Net by using an attention mechanism, a dynamic weighting uncertainty mechanism and an HS-Swish activation function in image segmentation processing. In combination with hierarchical attention enhancement jump connection and multi-scale cavity convolution, the liver image data is precisely processed; in the aspect of three-dimensional reconstruction, a classical Marching Cubes algorithm is improved, interlayer interpolation of an optical flow estimation method is optimized, intersection point calculation precision is improved by adopting a trisection method, and a high-quality three-dimensional reconstruction model is generated; the system effectively solves the problem that a traditional system is insufficient in precision, image segmentation and reconstruction effects are remarkably improved, and reliable support is provided for the field of liver iconography.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional reconstruction, and in particular to a three-dimensional reconstruction system of liver blood vessels based on image segmentation. Background Art

[0002] With the development of science and technology, medical imaging technology is becoming increasingly important in the three-dimensional reconstruction of liver blood vessels; however, the existing three-dimensional reconstruction system of liver blood vessels still has certain shortcomings: first, the traditional three-dimensional reconstruction system of liver blood vessels has weak ability to process noise and artifacts of image data, resulting in low quality of the generated three-dimensional model; at the same time, these systems often have limited ability to extract and fuse multi-scale features, and cannot fully express the complex structural characteristics of liver blood vessels; secondly, the traditional three-dimensional reconstruction system of liver blood vessels has insufficient ability to recognize subtle structures, and only uses a simple two-dimensional slice superposition method to generate a three-dimensional model, ignoring the problems of data continuity and accuracy between slices, resulting in a lack of coherence in the system's detail processing; at the same time, when the traditional system segments the image data, it has low sensitivity to multi-scale features and cannot accurately distinguish the boundaries of different vascular branches, affecting the actual use effect; therefore, there is an urgent need for a three-dimensional reconstruction system of liver blood vessels that can achieve high-precision segmentation and high-quality three-dimensional reconstruction in complex vascular structures. Summary of the invention

[0003] The present invention provides a liver vascular 3D reconstruction system based on image segmentation, which is used to solve the shortcomings of the existing liver vascular 3D reconstruction system in terms of segmentation accuracy, structural detail retention and 3D model generation quality; the system significantly improves the integrity, continuity and accuracy of the reconstruction results through improved image segmentation and 3D reconstruction methods; firstly, in the image segmentation stage, the system adopts the attention mechanism, dynamic weighted uncertainty mechanism and HS-Swish activation function to optimize the feature extraction process of liver image data, and replaces the ordinary skip connection with the hierarchical attention enhanced skip connection to improve the transmission and fusion ability of the multi-scale features of the liver image data; at the same time, through the multi-scale spatial Hole convolution improves the decoding process, enhances the resolution and semantic expression ability, and thus generates more accurate two-dimensional slice data of liver images, providing a more reliable data basis for subsequent three-dimensional reconstruction of liver images; secondly, in the three-dimensional reconstruction stage, the system optimizes the inter-layer interpolation by introducing dynamic weights, nonlinear combinations and regularization terms to improve the continuity and accuracy of the generation of resampled two-dimensional slices of liver images; by replacing linear interpolation with the method of three-partitioning, the accuracy of intersection calculation is significantly improved; combining the above methods, a high-precision three-dimensional vascular model is finally generated; the present invention solves the problem of insufficient accuracy of traditional systems, significantly improves the segmentation and reconstruction effects, and provides reliable support for the field of liver imaging.

[0004] The present invention provides a liver blood vessel three-dimensional reconstruction system based on image segmentation, the system comprises a data acquisition module, a data preprocessing module, an image segmentation module and a three-dimensional reconstruction module;

[0005] A data acquisition module, which collects liver imaging data and voxel spacing information;

[0006] The data preprocessing module removes noise from the liver image data, performs image enhancement and grayscale normalization processing to generate preprocessed liver image data;

[0007] Image segmentation module, establish U-Net, improve U-Net encoder through attention mechanism, dynamic weighted uncertainty mechanism and HS-Swish activation function, obtain improved U-Net encoder, replace ordinary skip connections with layered attention enhanced skip connections, optimize U-Net multi-scale feature transmission and fusion; improve U-Net decoder reconstruction resolution and semantic expression ability through multi-scale hole convolution, obtain improved U-Net decoder, build improved U-Net, use improved U-Net to process pre-processed liver image data, and generate two-dimensional slice annotation information;

[0008] The 3D reconstruction module is based on the Marching Cubes algorithm. It introduces dynamic weights, nonlinear combinations and regularization terms in interlayer interpolation, improves the optical flow estimation method, optimizes the continuity and quality of interlayer slice generation in the Marching Cubes algorithm, replaces linear interpolation with the method of three-partitioning in intersection calculation, improves the accuracy of intersection calculation, and constructs an enhanced MarchingCubes linear algorithm. According to the 2D slice annotation information and voxel spacing information, the enhanced Marching Cubes linear algorithm is used to perform 3D reconstruction and generate a reconstructed 3D model. The spatial structure of liver blood vessels and the spatial understanding of image data are displayed based on the reconstructed 3D model.

[0009] Furthermore, the image segmentation module uses the improved version of U-Net to process the pre-processed liver image data and generate two-dimensional slice annotation information, which specifically includes the following steps:

[0010] Step S1: Encoder feature extraction: The improved U-Net includes an encoder block and a decoder block. The pre-processed liver image data is input into the encoder block. The encoder block extracts the features of the pre-processed liver image data through convolution operations and increases the number of channels. The resolution is reduced layer by layer through the maximum pooling technology to generate multi-scale encoder block feature data. The multi-scale encoder block feature data is fused between the encoder blocks through the improved U-Net encoder to generate high-quality EAF feature data.

[0011] Step S2: Hierarchical attention enhanced jump connection: The high-quality EAF feature data is enhanced by hierarchical attention enhanced jump connection to obtain high-resolution enhanced feature data, and passed to the decoder block;

[0012] Step S3: Decoder attention fusion: The output of the encoder block is used as the decoder feature data, combined with the high-resolution enhanced feature data, and further multi-path feature fusion is performed through the improved U-Net decoder to generate the next layer of fused features, obtain multi-path fused feature data, and generate two-dimensional slice annotation information.

[0013] Furthermore, step S1 specifically includes the following steps:

[0014] Step S11: dividing the adjacent output features of the multi-scale encoder block feature data into high-resolution feature data and low-resolution feature data;

[0015] Step S12: high-resolution feature processing: performing weighted processing on the high-resolution feature data through channel attention and spatial attention, highlighting the information of key channels and spatial regions, and generating a high-resolution weighted feature map;

[0016] Step S13: low-resolution feature processing: performing weighted processing on the low-resolution feature data through channel attention and spatial attention to generate a low-resolution weighted feature map;

[0017] Step S14: Feature fusion: Upsample the low-resolution weighted feature map through transposed convolution to achieve the same resolution as the high-resolution weighted feature map; fuse the low-resolution weighted feature map with the high-resolution weighted feature map through a dynamic weighted uncertainty mechanism, introduce residual connections to retain the feature information of the high-resolution feature data, and generate fused feature data. The formula used is as follows:

[0018] ;

[0019] in, represents the spatial dimension, represents the channel dimension, represents the fusion feature data, represents a high-resolution weighted feature map, represents the low-resolution weighted feature map, represents the transposed convolution operation, and represents the uncertainty weight, , and represents the dynamic fusion weight, represents the encoder layer index, Indicates The original high-resolution feature data of the layer, Residual connection is retained The convolution operation of information, Represents very small constants and prevents division by zero errors;

[0020] Step S15: Fusion feature enhancement processing: Use a 3×3 convolution kernel to perform convolution processing on the fusion feature data to obtain a convolution result; use batch normalization to standardize the convolution result to generate normalized feature data, and then use the HS-Swish activation function to perform nonlinear transformation on the normalized feature data to further enhance the feature expression ability and generate high-quality EAF feature data. The formula used is as follows:

[0021] ;

[0022] in, Represents high-quality EAF feature data, represents the HS-Swish activation function, represents batch normalization, Indicates that a 3×3 convolution kernel is used to perform a convolution operation on the fused feature data;

[0023] ;

[0024] in, Represents the input value, It means to shift the input value 3 units to the right. Represents a variant ReLU function restricted to the range [0, 6].

[0025] Further, step S3 specifically includes the following steps:

[0026] Step S31: performing element-by-element addition of the high-resolution enhancement feature data and the decoder feature data to obtain a preliminary fusion feature;

[0027] Step S32: using the dilated convolution kernels with different dilation rates of the multi-scale dilated convolution to extract the initial fusion feature dilated convolution data with different dilation rates; cascading the dilated convolution data with different dilation rates to form multi-scale feature representation data, performing channel compression, batch normalization and nonlinear activation on the multi-scale feature representation data to obtain low-frequency path feature data; then extracting local detail information from the initial fusion feature through the high-frequency path, capturing the high-frequency spatial information in the initial fusion feature, and obtaining the high-frequency path feature data. The formula used is as follows:

[0028] Multi-scale dilated convolution formula:

[0029] ;

[0030] in, represents the dilation rate of the dilated convolution, The expansion rate is The convolution output of Represents 3×3 dilated convolutions at different expansion rates. represents the preliminary fusion features;

[0031] Step S33: Use the attention weight to control the contribution ratio of the low-frequency path feature data and the high-frequency path feature data, generate the next layer of fusion features, and finally obtain multi-channel fusion feature data.

[0032] Furthermore, the 3D reconstruction module uses the enhanced Marching Cubes linear algorithm to perform 3D reconstruction based on the 2D slice annotation information and voxel spacing information to generate a reconstructed 3D model, which specifically includes the following steps:

[0033] Step B1: voxel spacing resampling: resampling is performed using an interpolation algorithm according to the two-dimensional slice annotation information and the voxel spacing information to obtain resampled two-dimensional slice data;

[0034] Step B2: 2D slice stacking: stack the resampled 2D slice data in order and combine them layer by layer into a 3D voxel matrix to obtain preliminary 3D voxel data;

[0035] Step B3: interlayer interpolation: The intermediate layer optical flow field estimation of the optical flow estimation method is improved by dynamic weights, nonlinear combination terms and regularization terms. The intermediate slice generation in the optical flow estimation method is enhanced by nonlinear combination, edge enhancement and additional control parameters to obtain an improved optical flow estimation method. The improved optical flow estimation method is used to generate intermediate layer slices between the resampled two-dimensional slice data to compensate for the problem of too large interlayer spacing, optimize the continuity of the preliminary three-dimensional voxel data, and generate interlayer interpolated three-dimensional voxel data. The formula used is as follows:

[0036] Forward optical flow field formula:

[0037] ;

[0038] Reverse optical flow field formula:

[0039] ;

[0040] in, Indicates the middle layer number, From the middle layer To the first floor The interpolated optical flow field, Indicates the total number of slices, and represents the interpolation coefficient, Representation middle layer To the last layer Interpolated optical flow field; represents the forward optical flow field from the first layer to the last layer, Represents the reverse optical flow field from the last layer to the first layer; and represents the dynamic weight, represents the nonlinear combination term, represents the regularization strength hyperparameter;

[0041] The formula for generating the intermediate slice is:

[0042] ;

[0043] ;

[0044] in, represents the pixel value of the middle layer slice, represents the bilinear interpolation function, Indicates additional control parameters, represents the first layer slice image, represents the last slice image, represents the fusion term, and represents the gradient, i.e. the edge feature, represents the weight parameter;

[0045] Step B4: voxel state analysis: set an isosurface threshold, traverse each voxel vertex of the interlayer interpolated 3D voxel data, obtain the voxel vertex value, compare the voxel vertex value with the isosurface threshold, generate a voxel state index, mark the valid voxels, and generate a valid voxel set;

[0046] Step B5: Intersection calculation: According to the voxel state index, traverse the edges of the valid voxel set to determine whether the vertices of the edges of the valid voxel set cross the isosurface threshold. For the edges that cross the isosurface threshold, use the three-part method to estimate and calculate the intersection coordinates to obtain the intersection coordinate data;

[0047] Step B6: triangular patch generation: set the voxel state table, search the corresponding triangular patch topology structure from the voxel state table according to the voxel state index, and connect and generate triangular patch data using the intersection coordinate data;

[0048] Step B7: preliminary mesh stitching: stitching all triangular facet data generated in the valid voxel set to form a local three-dimensional surface mesh structure, and obtaining a preliminary three-dimensional surface mesh model;

[0049] Step B8: Optimization processing: Optimize the preliminary three-dimensional surface mesh model through denoising, surface smoothing and hole repairing; and further refine it through a mesh subdivision algorithm to obtain refined three-dimensional mesh data;

[0050] Step B9: Model output: Import the refined 3D mesh data into a 3D visualization tool, output the reconstructed 3D model, and display the spatial structure of liver blood vessels and understand the spatial image data based on the reconstructed 3D model.

[0051] By adopting the above scheme, the beneficial effects achieved by the present invention are as follows:

[0052] In terms of image segmentation, the present invention adopts the means of attention mechanism, dynamic weighted uncertainty mechanism and HS-Swish activation function, which significantly optimizes the system's feature extraction capability for liver image data and realizes the system's accurate capture and transmission of multi-scale features; by replacing traditional jump connections with layered attention enhanced jump connections, the efficiency of multi-scale feature fusion in the system is further improved, and the expression capability of complex liver vascular structures is enhanced; at the same time, the system improves the resolution and semantic expression capability through multi-scale dilated convolution in the decoding stage, making the segmentation results more refined, and effectively solving the problem of insufficient recognition of complex vascular structures by traditional systems;

[0053] In the 3D reconstruction process, the present invention introduces dynamic weights, nonlinear combinations and regularization terms in inter-layer interpolation based on the improved Marching Cubes algorithm, optimizes the optical flow estimation method, and significantly improves the continuity and quality of inter-layer slice generation in the system; by using the trisection method instead of linear interpolation in intersection calculation, the accuracy of intersection calculation of the system is effectively improved, ensuring the detail integrity and structural accuracy of the generated 3D model of liver blood vessels; the above-mentioned improvement measures solve the problems of insufficient continuity and poor detail accuracy between slices in the traditional system, and enhance the reconstruction quality of the system;

[0054] The liver vascular 3D reconstruction system constructed by the present invention not only solves the limitations of traditional systems in liver image analysis, but also significantly improves the accuracy and robustness of the system through precise image segmentation and high-quality 3D reconstruction; the generated 3D liver vascular model has the characteristics of structural continuity, clear details and high resolution, and can be used for accurate liver vascular spatial structure display;

[0055] In addition, the system provides strong support for the spatial understanding of imaging data and can provide high-quality three-dimensional reconstruction models for scientific research, teaching demonstrations, and further analysis of imaging data, effectively promoting the development of the field of liver imaging. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 A schematic diagram of system modules of a liver vascular three-dimensional reconstruction system based on image segmentation proposed by the present invention;

[0057] Figure 2It is a schematic diagram of the structure of the improved version of U-Net in the image segmentation module proposed in Example 2;

[0058] Figure 3 Schematic diagram of the flow of the linear algorithm of the enhanced Marching Cubes in the 3D reconstruction module proposed in Example 8. DETAILED DESCRIPTION

[0059] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0060] Embodiment 1, according to Figure 1 , the present invention provides a three-dimensional reconstruction system of liver blood vessels based on image segmentation, the system includes a data acquisition module, a data preprocessing module, an image segmentation module and a three-dimensional reconstruction module;

[0061] A data acquisition module, which collects liver imaging data and voxel spacing information;

[0062] The data preprocessing module removes noise from the liver image data, performs image enhancement and grayscale normalization processing to generate preprocessed liver image data;

[0063] Image segmentation module, establish U-Net, improve U-Net encoder through attention mechanism, dynamic weighted uncertainty mechanism and HS-Swish activation function, obtain improved U-Net encoder, replace ordinary skip connections with layered attention enhanced skip connections, optimize U-Net multi-scale feature transmission and fusion; improve U-Net decoder reconstruction resolution and semantic expression ability through multi-scale hole convolution, obtain improved U-Net decoder, build improved U-Net, use improved U-Net to process pre-processed liver image data, and generate two-dimensional slice annotation information;

[0064] The 3D reconstruction module is based on the Marching Cubes algorithm. It introduces dynamic weights, nonlinear combinations and regularization terms in interlayer interpolation, improves the optical flow estimation method, optimizes the continuity and quality of interlayer slice generation in the Marching Cubes algorithm, replaces linear interpolation with the method of three-partitioning in intersection calculation, improves the accuracy of intersection calculation, and constructs an enhanced MarchingCubes linear algorithm. According to the 2D slice annotation information and voxel spacing information, the enhanced Marching Cubes linear algorithm is used to perform 3D reconstruction and generate a reconstructed 3D model. The spatial structure of liver blood vessels and the spatial understanding of image data are displayed based on the reconstructed 3D model.

[0065] Embodiment 2, according to Figure 2 Based on the first embodiment, the image segmentation module generates a process of two-dimensional slice annotation information, which specifically includes the following steps:

[0066] Step S1: Encoder feature extraction: The improved U-Net includes an encoder block and a decoder block. The pre-processed liver image data is input into the encoder block. The encoder block extracts the features of the pre-processed liver image data through convolution operations and increases the number of channels. The resolution is reduced layer by layer through the maximum pooling technology to generate multi-scale encoder block feature data. The multi-scale encoder block feature data is fused between the encoder blocks through the improved U-Net encoder to generate high-quality EAF feature data.

[0067] Step S2: Hierarchical attention enhanced jump connection: The high-quality EAF feature data is enhanced by hierarchical attention enhanced jump connection to obtain high-resolution enhanced feature data, and passed to the decoder block;

[0068] Step S3: Decoder attention fusion: The output of the encoder block is used as the decoder feature data, combined with the high-resolution enhanced feature data, and further multi-path feature fusion is performed through the improved U-Net decoder to generate the next layer of fused features, obtain multi-path fused feature data, and generate two-dimensional slice annotation information.

[0069] Embodiment 3, based on embodiment 1, the process of generating two-dimensional slice annotation information by the image segmentation module specifically includes the following steps:

[0070] Step Q1: Encoder feature extraction: The preprocessed liver image data is input into the encoder block, which extracts the features of the preprocessed liver image data and increases the number of channels through convolution operations, reduces the resolution layer by layer through maximum pooling, captures the global and local information of the preprocessed liver image data, and generates multi-scale encoder block feature data; the multi-scale encoder block feature data is fused between encoder blocks through an improved version of the U-Net encoder to generate high-quality EAF feature data;

[0071] Step Q2: Skip connection: Use skip connection to pass high-quality EAF feature data to the decoder block;

[0072] Step Q3: Decoder attention fusion: The output of the encoder block is used as the decoder feature data, combined with the high-quality EAF feature data, and further multi-path feature fusion is performed through the improved U-Net decoder to generate the next layer of fused features, and finally multi-path fused feature data is obtained to generate two-dimensional slice annotation information.

[0073] Embodiment 4, based on embodiment 2, step S1 specifically includes the following steps:

[0074] Step S11: dividing the adjacent output features of the multi-scale encoder block feature data into high-resolution feature data and low-resolution feature data;

[0075] Step S12: high-resolution feature processing: performing weighted processing on the high-resolution feature data through channel attention and spatial attention, highlighting the information of key channels and spatial regions, and generating a high-resolution weighted feature map;

[0076] Step S13: low-resolution feature processing: performing weighted processing on the low-resolution feature data through channel attention and spatial attention to generate a low-resolution weighted feature map;

[0077] Step S14: Feature fusion: Upsample the low-resolution weighted feature map through transposed convolution to achieve the same resolution as the high-resolution weighted feature map; fuse the low-resolution weighted feature map with the high-resolution weighted feature map through a dynamic weighted uncertainty mechanism, introduce residual connections to retain the feature information of the high-resolution feature data, and generate fused feature data. The formula used is as follows:

[0078] ;

[0079] in, represents the spatial dimension, represents the channel dimension, represents the fusion feature data, represents the high-resolution weighted feature map, represents the low-resolution weighted feature map, represents the transposed convolution operation, and represents the uncertainty weight, , and represents the dynamic fusion weight, represents the encoder layer index, Indicates The original high-resolution feature data of the layer, Residual connection is retained The convolution operation of information, Represents very small constants and prevents division by zero errors;

[0080] Step S15: Fusion feature enhancement processing: Use a 3×3 convolution kernel to perform convolution processing on the fusion feature data to obtain a convolution result; use batch normalization to standardize the convolution result to generate normalized feature data, and then use the HS-Swish activation function to perform nonlinear transformation on the normalized feature data to further enhance the feature expression ability and generate high-quality EAF feature data. The formula used is as follows:

[0081] ;

[0082] in, Represents high-quality EAF feature data, represents the HS-Swish activation function, represents batch normalization, Indicates that a 3×3 convolution kernel is used to perform a convolution operation on the fused feature data;

[0083] ;

[0084] in, Represents the input value, It means to shift the input value 3 units to the right. Represents a variant ReLU function restricted to the range [0, 6].

[0085] Embodiment 5, based on embodiment 2, step S1 specifically includes the following steps:

[0086] Step R1: dividing the adjacent output features of the multi-scale encoder block feature data into high-resolution feature data and low-resolution feature data;

[0087] Step R2: High-resolution feature processing: The high-resolution feature data is weighted through channel attention and spatial attention to highlight the information of key channels and spatial regions, and generate a high-resolution weighted feature map;

[0088] Step R3: low-resolution feature processing: low-resolution feature data is weighted through channel attention and spatial attention to generate a low-resolution weighted feature map;

[0089] Step R4: Feature fusion: Upsample the low-resolution weighted feature map through transposed convolution to achieve the same resolution as the high-resolution weighted feature map; fuse the low-resolution weighted feature map with the high-resolution weighted feature map to generate fused feature data. The formula used is as follows:

[0090] ;

[0091] Step R5: Fusion feature enhancement processing: Use a 3×3 convolution kernel to perform convolution processing on the fused feature data to obtain the convolution result; use batch normalization to standardize the convolution result to generate normalized feature data, and then use the ReLU activation function to perform nonlinear transformation on the normalized feature data to further enhance the feature expression ability and generate high-quality EAF feature data.

[0092] Embodiment 6, based on embodiment 4, step S3 specifically includes the following steps:

[0093] Step S31: performing element-by-element addition of the high-resolution enhancement feature data and the decoder feature data to obtain a preliminary fusion feature;

[0094] Step S32: Use the dilated convolution kernels with different dilation rates of the multi-scale dilated convolution to extract the dilated convolution data with different dilation rates in the preliminary fusion features; cascade the dilated convolution data with different dilation rates to form multi-scale feature representation data, perform channel compression, batch normalization and nonlinear activation on the multi-scale feature representation data to obtain low-frequency path feature data; then extract local detail information from the preliminary fusion features through the high-frequency path, capture the high-frequency spatial information in the preliminary fusion features, and obtain high-frequency path feature data. The formula used is as follows:

[0095] Multi-scale dilated convolution formula:

[0096] ;

[0097] in, represents the dilation rate of the dilated convolution, The expansion rate is The convolution output of Represents 3×3 dilated convolutions at different expansion rates. represents the preliminary fusion features;

[0098] Step S33: Use the attention weight to control the contribution ratio of the low-frequency path feature data and the high-frequency path feature data, generate the next layer of fusion features, and obtain multi-channel fusion feature data.

[0099] Embodiment 7, based on embodiment 4, step S3 specifically includes the following steps:

[0100] Step C1: perform element-by-element addition of the high-resolution enhancement feature data and the decoder feature data to obtain a preliminary fusion feature;

[0101] Step C2: Use 3×3 convolution kernels to extract features from the preliminary fusion features to obtain different convolution results; cascade the different convolution results to form multi-scale feature representation data, perform channel compression, batch normalization and nonlinear activation on the multi-scale feature representation data to obtain low-frequency path feature data; then extract local detail information from the preliminary fusion features through high-frequency paths, capture high-frequency spatial information in the features, and obtain high-frequency path feature data;

[0102] Step C3: Use the attention weight to control the contribution ratio of low-frequency path feature data and high-frequency path feature data, generate the next layer of fusion features, and obtain multi-channel fusion feature data.

[0103] Embodiment 8, according to Figure 3 Based on the sixth embodiment, the three-dimensional reconstruction module uses the enhanced Marching Cubes linear algorithm to perform three-dimensional reconstruction according to the two-dimensional slice annotation information and the voxel spacing information, and generates a reconstructed three-dimensional model, which specifically includes the following steps:

[0104] Step B1: voxel spacing resampling: resampling is performed using an interpolation algorithm according to the two-dimensional slice annotation information and the voxel spacing information to obtain resampled two-dimensional slice data;

[0105] Step B2: 2D slice stacking: stack the resampled 2D slice data in order and combine them layer by layer into a 3D voxel matrix to obtain preliminary 3D voxel data;

[0106] Step B3: interlayer interpolation: The intermediate layer optical flow field estimation of the optical flow estimation method is improved by dynamic weights, nonlinear combination terms and regularization terms. The intermediate slice generation in the optical flow estimation method is enhanced by nonlinear combination, edge enhancement and additional control parameters to obtain an improved optical flow estimation method. The improved optical flow estimation method is used to generate intermediate layer slices between the resampled two-dimensional slice data to compensate for the problem of too large interlayer spacing, optimize the continuity of the preliminary three-dimensional voxel data, and generate interlayer interpolated three-dimensional voxel data. The formula used is as follows:

[0107] Forward optical flow field formula:

[0108] ;

[0109] Reverse optical flow field formula:

[0110] ;

[0111] in, Indicates the middle layer number, From the middle layer To the first floor The interpolated optical flow field, Indicates the total number of slices, and represents the interpolation coefficient, Representation middle layer To the last layer Interpolated optical flow field; represents the forward optical flow field from the first layer to the last layer, Represents the reverse optical flow field from the last layer to the first layer; and represents the dynamic weight, represents the nonlinear combination term, represents the regularization strength hyperparameter;

[0112] The formula for generating the intermediate slice is:

[0113] ;

[0114] ;

[0115] in, represents the pixel value of the middle layer slice, represents the bilinear interpolation function, Indicates additional control parameters, represents the first layer slice image, represents the last slice image, represents the fusion term, and represents the gradient, i.e. the edge feature, represents the weight parameter;

[0116] Step B4: voxel state analysis: set an isosurface threshold, traverse each voxel vertex of the interlayer interpolated 3D voxel data, obtain the voxel vertex value, compare the voxel vertex value with the isosurface threshold, generate a voxel state index, mark the valid voxels, and generate a valid voxel set;

[0117] Step B5: Intersection calculation: According to the voxel state index, traverse the edges of the valid voxel set to determine whether the vertices of the edges of the valid voxel set cross the isosurface threshold. For the edges that cross the isosurface threshold, use the three-part method to estimate and calculate the intersection coordinates to obtain the intersection coordinate data;

[0118] Step B6: triangular patch generation: set the voxel state table, search the corresponding triangular patch topology structure from the voxel state table according to the voxel state index, and connect and generate triangular patch data using the intersection coordinate data;

[0119] Step B7: preliminary mesh stitching: stitching all triangular facet data generated in the valid voxel set to form a local three-dimensional surface mesh structure, and obtaining a preliminary three-dimensional surface mesh model;

[0120] Step B8: Optimization processing: Optimize the preliminary three-dimensional surface mesh model through denoising, surface smoothing and hole repairing; and further refine it through a mesh subdivision algorithm to obtain refined three-dimensional mesh data;

[0121] Step B9: Model output: Import the refined 3D mesh data into a 3D visualization tool, output the reconstructed 3D model, and display the spatial structure of liver blood vessels and understand the spatial image data based on the reconstructed 3D model.

[0122] Embodiment 9, based on embodiment 6, the process of generating a reconstructed 3D model by a 3D reconstruction module specifically includes the following steps:

[0123] Step E1: voxel spacing resampling: resampling is performed through an interpolation algorithm according to the two-dimensional slice annotation information and the voxel spacing information to obtain resampled two-dimensional slice data;

[0124] Step E2: 2D slice stacking: stack the resampled 2D slice data in order and combine them layer by layer into a 3D voxel matrix to obtain preliminary 3D voxel data;

[0125] Step E3: voxel state analysis: set an isosurface threshold, traverse each voxel vertex of the preliminary three-dimensional voxel data, obtain the voxel vertex value, compare the voxel vertex value with the isosurface threshold, generate a voxel state index, mark the valid voxels, and generate a valid voxel set;

[0126] Step E4: intersection calculation: according to the voxel state index, traverse the edges of the valid voxel set, determine whether the vertices of the edges of the valid voxel set cross the isosurface threshold, and use linear interpolation to estimate and calculate the intersection coordinates for the edges that cross the isosurface threshold to obtain the intersection coordinate data;

[0127] Step E5: triangular patch generation: set a voxel state table, search for the corresponding triangular patch topology structure from the voxel state table according to the voxel state index, and connect and generate triangular patch data using the intersection coordinate data;

[0128] Step E6: preliminary mesh stitching: stitching all triangular facet data generated in the valid voxel set to form a local three-dimensional surface mesh structure, and obtaining a preliminary three-dimensional surface mesh model;

[0129] Step E7: Optimization processing: Optimize the preliminary three-dimensional surface mesh model through denoising, surface smoothing and hole repairing; and further refine it through a mesh subdivision algorithm to obtain refined three-dimensional mesh data;

[0130] Step E8: Model output: The refined 3D mesh data is imported into a 3D visualization tool, and the reconstructed 3D model is output. The spatial structure of the liver blood vessels and the spatial understanding of the image data are performed based on the reconstructed 3D model.

[0131] The present invention and its embodiments are described above, which is not restrictive. What is shown in the accompanying drawings is only one of the embodiments of the present invention, and the actual structure is not limited to this. In short, if ordinary technicians in this field are inspired by it and do not deviate from the purpose of the invention, they can creatively design structural methods and embodiments similar to the technical solution, which should all fall within the scope of protection of the present invention.

Claims

1. A three-dimensional reconstruction system of liver blood vessels based on image segmentation, comprising a data acquisition module and a data preprocessing module, wherein the data acquisition module acquires liver image data and voxel spacing information; the data preprocessing module preprocesses the liver image data to generate preprocessed liver image data; characterized in that: The system also includes an image segmentation module and a three-dimensional reconstruction module; The image segmentation module establishes a U-Net, improves the U-Net encoder through an attention mechanism, a dynamic weighted uncertainty mechanism, and an HS-Swish activation function to obtain an improved U-Net encoder, optimizes the transmission and fusion of multi-scale features of the U-Net through layered attention enhancement jump connections; improves the reconstruction resolution and semantic expression ability of the U-Net decoder through multi-scale hole convolution to obtain an improved U-Net decoder, constructs an improved U-Net, and uses the improved U-Net to process pre-processed liver image data to generate two-dimensional slice annotation information; The three-dimensional reconstruction module is based on the Marching Cubes algorithm, introduces dynamic weights, nonlinear combinations and regularization terms in inter-layer interpolation, improves the optical flow estimation method, optimizes the continuity and quality of inter-layer slice generation in the Marching Cubes algorithm, replaces linear interpolation with the method of three-partitioning in intersection calculation, improves the accuracy of intersection calculation, and constructs an enhanced MarchingCubes linear algorithm; according to the two-dimensional slice annotation information and voxel spacing information, the enhanced Marching Cubes linear algorithm is used to perform three-dimensional reconstruction to generate a reconstructed three-dimensional model.

2. The system for three-dimensional reconstruction of hepatic blood vessels based on image segmentation according to claim 1, characterized in that: The process of generating two-dimensional slice annotation information by the image segmentation module specifically includes the following steps: Step S1: Encoder feature extraction: The improved U-Net includes an encoder block and a decoder block. The pre-processed liver image data is input into the encoder block. The encoder block extracts the features of the pre-processed liver image data through convolution operations and increases the number of channels. The resolution is reduced layer by layer through the maximum pooling technology to generate multi-scale encoder block feature data. The multi-scale encoder block feature data is fused between the encoder blocks through the improved U-Net encoder to generate high-quality EAF feature data. Step S2: Hierarchical attention enhanced jump connection: The high-quality EAF feature data is enhanced by hierarchical attention enhanced jump connection to obtain high-resolution enhanced feature data, and passed to the decoder block; Step S3: Decoder attention fusion: The output of the encoder block is used as the decoder feature data, combined with the high-resolution enhanced feature data, and multi-path feature fusion is performed through the improved U-Net decoder to generate the next layer of fused features, obtain multi-path fused feature data, and generate two-dimensional slice annotation information.

3. The system for three-dimensional reconstruction of hepatic blood vessels based on image segmentation according to claim 2, characterized in that: The step S1 specifically comprises the following steps: Step S11: dividing the adjacent output features of the multi-scale encoder block feature data into high-resolution feature data and low-resolution feature data; Step S12: high-resolution feature processing: performing weighted processing on the high-resolution feature data through channel attention and spatial attention to generate a high-resolution weighted feature map; Step S13: low-resolution feature processing: performing weighted processing on the low-resolution feature data through channel attention and spatial attention to generate a low-resolution weighted feature map; Step S14: feature fusion: upsampling the low-resolution weighted feature map through transposed convolution to achieve the same resolution as the high-resolution weighted feature map; fusing the low-resolution weighted feature map with the high-resolution weighted feature map through a dynamic weighted uncertainty mechanism, introducing a residual connection to retain the feature information of the high-resolution feature data, and generating fused feature data; Step S15: fusion feature enhancement processing: use a 3×3 convolution kernel to perform convolution processing on the fusion feature data to obtain a convolution result; use batch normalization to standardize the convolution result to generate normalized feature data, and then use the HS-Swish activation function to perform nonlinear transformation on the normalized feature data to generate high-quality EAF feature data.

4. The system for three-dimensional reconstruction of hepatic blood vessels based on image segmentation according to claim 2, characterized in that: The step S3 specifically comprises the following steps: Step S31: performing element-by-element addition of the high-resolution enhancement feature data and the decoder feature data to obtain a preliminary fusion feature; Step S32: extracting the atrous convolution data with different dilation rates from the preliminary fusion features; cascading the atrous convolution data with different dilation rates to form multi-scale feature representation data, performing channel compression, batch normalization and nonlinear activation on the multi-scale feature representation data to obtain low-frequency path feature data; extracting local detail information from the preliminary fusion features through a high-frequency path, capturing high-frequency spatial information in the preliminary fusion features, and obtaining high-frequency path feature data; Step S33: Use the attention weight to control the contribution ratio of the low-frequency path feature data and the high-frequency path feature data, generate the next layer of fusion features, and finally obtain multi-channel fusion feature data.

5. The system for three-dimensional reconstruction of hepatic blood vessels based on image segmentation according to claim 1, characterized in that: The process of generating the reconstructed 3D model by the 3D reconstruction module specifically includes the following steps: Step B1: resampling is performed using an interpolation algorithm according to the two-dimensional slice annotation information and the voxel spacing information to obtain resampled two-dimensional slice data; Step B2: stack the resampled two-dimensional slice data in order and combine them layer by layer into a three-dimensional voxel matrix to obtain preliminary three-dimensional voxel data; Step B3: improving the estimation of the intermediate layer optical flow field of the optical flow estimation method by dynamic weights, nonlinear combination terms and regularization terms, enhancing the generation of intermediate slices in the optical flow estimation method by edge enhancement and additional control parameters, obtaining an improved optical flow estimation method, generating intermediate layer slices between the resampled two-dimensional slice data by using the improved optical flow estimation method, optimizing the continuity of the preliminary three-dimensional voxel data, and generating inter-layer interpolated three-dimensional voxel data; Step B4: setting an isosurface threshold, traversing each voxel vertex of the interlayer interpolated three-dimensional voxel data, obtaining the voxel vertex value, comparing the voxel vertex value with the isosurface threshold, generating a voxel state index, marking valid voxels, and generating a valid voxel set; Step B5: According to the voxel state index, traverse the edges of the valid voxel set to determine whether the vertices of the edges of the valid voxel set cross the isosurface threshold, and use the three-part method to estimate and calculate the intersection coordinates of the edges that cross the isosurface threshold to obtain the intersection coordinate data; Step B6: setting a voxel state table, searching for the corresponding triangular patch topology structure from the voxel state table according to the voxel state index, and connecting and generating triangular patch data using the intersection coordinate data; Step B7: splicing all triangular facet data generated in the valid voxel set to form a local three-dimensional surface mesh structure, and obtaining a preliminary three-dimensional surface mesh model; Step B8: Optimize the preliminary three-dimensional surface mesh model by denoising, surface smoothing and hole repairing; perform refinement processing by mesh subdivision algorithm to obtain refined three-dimensional mesh data; Step B9: Import the refined 3D mesh data into a 3D visualization tool and output the reconstructed 3D model.

Images