Three-dimensional hepatic duct, hepatic artery, portal vein and hepatic vein image segmentation method and system

Abstract

The invention relates to the technical field of image analysis, in particular to a three-dimensional hepatic duct, hepatic artery, portal vein and hepatic vein image segmentation method and system. The method comprises the following steps: acquiring multi-phase medical image data of a liver region; performing region identification on the image data, and extracting hepatic duct, hepatic artery, portal vein and hepatic vein regions; executing structure tracking on the hepatic duct area to obtain a hepatic duct positioning result; carrying out improved contrast extraction on the hepatic artery region to obtain hepatic artery positioning data; performing directional segmentation analysis on the portal vein region to obtain a portal vein positioning result; performing branch angle extraction on the hepatic vein region in combination with the portal vein direction to obtain hepatic vein positioning data; taking the hepatic artery as a reference structure, performing elastic registration on each blood vessel structure, and generating registration data under a unified coordinate; and completing three-dimensional visualization based on a registration result. According to the method, multi-stage image joint segmentation and registration are realized, and the method has the technical advantage of high structural recognition integrity.

Description

Technical Field

[0001] The present invention relates to the technical field of image analysis, and in particular to a method and system for segmenting three-dimensional hepatobiliary, hepatic artery, portal vein and hepatic vein images. Background Art

[0002] With the rapid development of medical imaging equipment, multimodal imaging technologies such as computed tomography (CT) and magnetic resonance imaging (MRI) are increasingly used in clinical diagnosis, especially in the preoperative evaluation of liver diseases, surgical planning of liver cancer / cholangiocarcinoma (especially hilar cholangiocarcinoma), and preoperative simulation of liver transplantation. The three-dimensional visualization and spatial positioning of key vascular structures in the liver (such as the hepatic artery, portal vein, hepatic vein, and bile duct) have put forward higher accuracy and automation requirements. Traditional medical image segmentation methods mostly rely on image processing algorithms based on thresholds, region growing, and active contours. It is difficult to fully handle the characteristics of vascular structures in the image, such as slenderness, branching, multi-directionality, and grayscale proximity. Especially in microvessels, intersections, blurred images, or weak signal areas, problems such as segmentation breaks, positioning offsets, or structural misidentification often occur. Summary of the invention

[0003] In order to solve the above technical problems, the present invention proposes a three-dimensional hepatobiliary, hepatic artery, portal vein and hepatic vein image segmentation method and system to solve at least one of the above technical problems.

[0004] The present application provides a three-dimensional hepatobiliary, hepatic artery, portal vein and hepatic vein image segmentation method, comprising the following steps: Acquiring multi-phase medical image data of a liver region; Performing regional recognition on multi-phase medical image data to obtain hepatobiliary regional data, hepatic artery regional data, portal vein regional data and hepatic vein regional data; Perform structural tracking on the hepatobiliary region data to obtain hepatobiliary location data; perform improved contrast extraction on the hepatic artery region data to obtain hepatic artery location data; perform directional segmentation analysis on the portal vein region data to obtain portal vein location data; perform branch angle extraction on the hepatic vein region data and portal vein location data to obtain hepatic vein location data; Elastic registration is performed based on the hepatobiliary duct positioning data, the hepatic artery positioning data, the portal vein positioning data and the hepatic vein positioning data to obtain elastic registration data; and three-dimensional visualization operations are performed based on the elastic registration data.

[0005] The present invention realizes multi-structure refined recognition based on multi-phase medical image data, and performs differentiated processing in combination with the liver anatomical structure relationship. By performing structural tracking on the hepatobiliary regional data, the connectivity and integrity of the bile duct fine branches can be effectively enhanced; the boundary clarity and positioning accuracy of the hepatic artery can be improved through improved contrast extraction; the structural continuity of the portal vein branches can be enhanced through directional segmentation analysis; the branch angle extraction method is combined with the portal vein direction characteristics to effectively distinguish the hepatic vein from the portal vein and improve the recognition accuracy. Through the elastic registration operation between multiple structures, the unified spatial fusion of various vascular structures is achieved, which is convenient for three-dimensional visualization reconstruction and clinical auxiliary analysis. Compared with the traditional blood vessel segmentation method based on single-phase images or single enhancement processing, the spatial guidance strategy in the present invention is based on anatomical proximity, limits the search area of ​​the target structure, and effectively reduces the false detection rate; the direction modeling part enhances the model's perception of the continuity of the direction of slender blood vessels through main direction gradient calculation and direction field learning; the structure tracking module introduces a path cost function, using signal strength, direction consistency and proximity as constraints to ensure the coherent extraction of tiny branches; the elastic registration strategy uses the hepatic artery as a reference to construct a control point grid to flexibly align the positioning results of multiple structures, while maintaining the topological relationship and improving the accuracy of structural fusion. Overall, the present invention not only improves the extraction integrity and spatial expression clarity of vascular structures, but also ensures the visualization coordination between multiple structures.

[0006] Preferably, the multi-phase medical image data includes arterial phase CT image data, portal venous phase CT image data, equilibrium phase CT image data and T2-weighted MRI image data, and the multi-phase medical image data is subjected to regional recognition to obtain hepatobiliary regional data, hepatic artery regional data, portal vein regional data and hepatic vein regional data, respectively, including: Performing rough segmentation of the liver region on the portal venous phase CT image data to obtain liver region data; Determine the hepatic artery region according to the liver region data and the arterial phase CT image data to obtain the hepatic artery region data; Hepatic vein region data is identified according to liver region data and equilibrium phase CT image data to obtain hepatic vein region data; Directionally guided segmentation is performed based on the liver region data to obtain the portal vein region data; The hepatobiliary region is identified based on the liver region data and the T2-weighted MRI image data to obtain the hepatobiliary region data.

[0007] The present invention introduces arterial phase CT images, portal venous phase CT images, equilibrium phase CT images and T2-weighted MRI images, and combines the rough segmentation of the liver region with structure-specific processing to achieve targeted identification of the hepatic artery, portal vein, hepatic vein and hepatobiliary regions. The hepatic artery region is identified by utilizing the vascular enhancement characteristics in the arterial phase image, and the liver parenchyma is segmented and the portal vein direction is extracted in combination with the portal venous phase image. The hepatic vein is separated based on the venous contrast in the equilibrium phase image, and the hepatobiliary signal is enhanced by using the T2-weighted MRI image to identify the hepatobiliary. Through multi-modal and multi-phase fusion, the structural clarity, regional boundary accuracy and tissue separation ability of image segmentation are improved, effectively overcoming the problems of low recognition accuracy and organ confusion caused by single phase and fuzzy structure in traditional methods, and realizing high-precision anatomical region extraction based on structure, image and strategy, which has significant medical application value and scalability.

[0008] Preferably, performing directionally guided segmentation according to the liver region data to obtain portal vein region data includes: Extract voxel grayscale gradient according to liver region data to obtain voxel grayscale gradient data; Perform main direction clustering on the voxel gray gradient data to obtain main direction clustering data; Perform tensor voting processing based on the main direction clustering data to obtain the blood vessel direction field data; The preset direction-guided segmentation network model is used to segment the vascular direction field data to obtain the portal vein area data, wherein the direction-guided segmentation network model is constructed by using the improved SD-UNet to construct the backbone network, and the attention map or the guided flow is constructed as the output head.

[0009] In the present invention, by extracting voxel grayscale gradients based on liver regional data, and combining main direction clustering with tensor voting mechanisms to construct a vascular direction field, the directional consistency and anatomical coherence of the portal vein branch structure are fully utilized to achieve direction-aware driven segmentation modeling. The improved SD-UNet is used to construct a direction-guided segmentation network model, which integrates the direction field data and combines the attention map or guidance flow mechanism as the output guide, which can effectively improve the responsiveness to thin branches and diversified directions during portal vein identification. Compared with traditional segmentation methods based on grayscale enhancement or static models, the present invention not only has stronger performance in segmentation accuracy and structural continuity, but also significantly enhances the model's utilization efficiency of spatial directional features. It is suitable for the identification of portal vein structures with slender branches and large bending changes. It has higher anatomical consistency and algorithm generalization capabilities, and is suitable for clinical applications and adaptive modeling needs of complex vascular systems.

[0010] Preferably, the step of constructing the preset direction-guided segmentation network model includes the following steps: Obtain historical vascular direction field data and historical annotation data; Perform multi-scale exaggerated convolution parallel processing based on historical vascular direction field data to obtain edge perception layer data; The edge perception layer data is subjected to shallow edge layer processing and high-level feature layer processing to obtain shallow edge layer data and high-level feature layer data respectively, wherein the shallow edge layer processing realizes boundary response enhancement through Sobel / LoG filter or edge channel convolution; Perform residual connection on shallow edge layer data and high-level feature layer data to obtain decoding layer data; The vascular direction network is trained on the decoded layer data using historical annotation data to obtain direction-guided flow data. The vascular direction network training supervises the convolutional network by using the direction consistency loss combined with the Dice loss. The direction-guided flow data is directionally guided to output, and a direction-guided segmentation network model is obtained, wherein the direction-guided output is convolutionally fused with the decoding layer data and the direction confidence is calculated, or the direction-guided attention map is calculated through the direction-guided flow data, and the output result is regulated at the channel level.

[0011] The present invention constructs a direction-guided segmentation network model, integrates historical vascular direction field data and annotation data for network training, and has significant direction perception and structure recognition capabilities. Multi-scale exaggerated convolution is used to achieve structural perception at different scales, and edge channel convolution and Sobel / LoG filtering are used to enhance edge response, taking into account details and semantic features, and then residual connection between the shallow edge layer and the high-level feature layer is used to effectively maintain the clarity of vascular boundaries and the integrity of branch structures. The joint direction consistency loss and Dice loss are used for supervised training, so that the model can accurately fit the real vascular direction characteristics. Further, channel-level regulation is performed by fusing the direction-guided flow with the decoding features, or generating a directional attention map, which significantly enhances the network's response to structural direction information. Compared with traditional image segmentation models, this solution has higher accuracy and robustness in the identification of fine vascular branches, branch angle continuity modeling, and directional consistency maintenance, providing a more efficient and accurate model foundation for the intelligent identification and three-dimensional modeling of detailed vascular systems.

[0012] Preferably, the performing structural tracking on the hepatobiliary region data to obtain the hepatobiliary positioning data includes: Constructing the spatial guidance domain of the hepatobiliary region data to obtain the hepatobiliary spatial guidance domain data; Perform local directional consistency enhancement based on the hepatobiliary spatial guidance domain data to obtain hepatobiliary enhancement data; Performing path tracing according to the hepatobiliary enhancement data to obtain hepatobiliary duct tracing data, wherein the path tracing is to construct a cost function based on signal strength, direction consistency and path proximity to perform minimum cost path tracing; The connectivity of the hepatobiliary duct tracking data is optimized to obtain the hepatobiliary duct positioning data.

[0013] In the present invention, a spatial guidance domain based on the hepatobiliary region is constructed to limit the scope of structural recognition, and the local directional consistency enhancement method is combined to effectively enhance the structural coherence and directional characteristics in the bile duct image. By introducing the cost function of the three factors of signal intensity, directional consistency and path proximity, the minimum cost path tracking operation is performed, and high-precision extraction of the bile duct trunk and fine branches can be achieved. Further combined with the connectivity optimization processing, pseudo-branch structures and broken paths are effectively eliminated, and the continuity and accuracy of the overall tracking are improved. Compared with the traditional method of extracting bile ducts based on thresholds or simple filtering, this method takes into account spatial guidance, directional modeling and path optimization, significantly improving the structural recognition ability in cases of weak signals, complex directions, and small branches. It has stronger clinical adaptability and topological integrity, and provides stable and reliable structural input for three-dimensional bile duct modeling and visualization applications.

[0014] Preferably, performing improved contrast extraction on the hepatic artery region data to obtain hepatic artery positioning data includes: Direction-selective enhanced kernel processing was performed on the hepatic artery area data to obtain the rough positioning data of the first hepatic artery; The edge sensitivity of the hepatic artery area data is adjusted to obtain the rough positioning data of the second hepatic artery; Attention perception enhancement is performed according to the rough positioning data of the first hepatic artery and the rough positioning data of the second hepatic artery, so as to obtain weighted positioning data of the first hepatic artery and weighted positioning data of the second hepatic artery respectively; Confidence voting is performed based on the first hepatic artery weighted positioning data and the second hepatic artery weighted positioning data to obtain hepatic artery fusion positioning data, and centerline extraction is performed based on the hepatic artery fusion positioning data to obtain hepatic artery positioning data.

[0015] In the present invention, by introducing direction-selective enhancement kernel and edge sensitivity regulation processing on the hepatic artery regional data, two structural enhancement pathways are constructed respectively, which can more accurately capture the direction and boundary characteristics of the hepatic artery trunk and its tiny branches. The attention perception mechanism is used to weighted enhance the two types of coarse positioning results respectively, effectively improving the significance and noise suppression ability of the vascular response area. The results of the two enhancement pathways are further fused through the confidence voting mechanism, which strengthens the spatial continuity and positioning consistency of the structure while ensuring the recognition robustness. The centerline information is extracted based on the fusion results to achieve the structured expression of the hepatic artery path. Compared with traditional single-channel enhancement or fixed filtering extraction methods, this method has the advantages of multi-feature fusion, key enhancement, and strong adaptability to small blood vessels. It significantly improves the segmentation and positioning accuracy of the hepatic artery in complex backgrounds and low-contrast images, and is suitable for high-precision vascular modeling and intelligent analysis tasks.

[0016] Preferably, the performing directional segmentation analysis on the portal vein region data to obtain portal vein positioning data includes: Calculating the main directional gradient map of the portal vein area data to obtain the main directional gradient map data; Performing direction-guided segmentation according to the main direction gradient map data to obtain direction-guided segmentation data; Branches are merged and breaks are repaired according to the direction-guided segmentation data to obtain portal vein positioning data.

[0017] In the present invention, the main directional gradient map of the portal vein area data is calculated to accurately capture the local directional characteristics of the blood vessel direction, and combined with the direction-guided segmentation operation, the model can fully perceive the spatial extensibility and direction consistency of the vascular structure during the segmentation process, and effectively improve the recognition and connectivity maintenance capabilities of slender vascular branches. By performing branch merging and break repair on the direction-guided segmentation results, the topological integrity and structural continuity of the segmentation results are enhanced, and stronger structural recognition capabilities are possessed in the intersection area of ​​portal vein branches and weak signal areas. Compared with traditional segmentation methods that rely on grayscale or edge information, this method introduces a directional modeling mechanism, which has higher positioning accuracy and robustness when dealing with curved, slender, and multi-branched venous structures, and can more effectively support subsequent three-dimensional modeling and clinical auxiliary analysis tasks.

[0018] Preferably, extracting the branch angles of the hepatic vein region data and the portal vein positioning data to obtain the hepatic vein positioning data includes: The center line is extracted according to the portal vein positioning data to obtain the portal vein direction map data; Perform skeleton extraction based on the hepatic vein area data to obtain the hepatic vein branch data; Linear fitting is performed on the hepatic vein branch data to obtain the hepatic vein branch fitting data, and the angle of the nearest branch direction is calculated based on the hepatic vein branch fitting data to obtain the branch angle characteristic map data; Narrow neural network classification is performed based on the portal vein direction map data and branch angle feature map data to obtain the hepatic vein positioning data.

[0019] In the present invention, a branch angle discrimination mechanism for hepatic vein identification is constructed by extracting a centerline direction map based on the portal vein positioning result, and performing branch linear fitting and angle feature analysis in combination with the skeleton structure of the hepatic vein region. The angle relationship between the fitted branch direction and the portal vein direction is used to generate a branch angle feature map, and then the branch type is classified and determined by a narrow neural network to achieve accurate positioning of the hepatic vein. The present invention makes full use of the differences in branch direction and spatial orientation between the portal vein and the hepatic vein in the anatomical structure, improves the model's ability to identify and discriminate the two types of venous structures, and has a higher classification accuracy in areas where the structures are adjacent and the signals overlap. Compared with traditional recognition methods based on single image segmentation or grayscale feature extraction, this method introduces a multi-feature intelligent analysis framework that combines structure fitting, angle modeling and lightweight neural networks, and has technical advantages such as strong structural understanding, low computational cost, and high topological recognition accuracy, providing more reliable technical support for the automatic anatomical recognition and three-dimensional modeling of detailed venous systems.

[0020] Preferably, performing elastic registration according to the hepatobiliary duct positioning data, the hepatic artery positioning data, the portal vein positioning data and the hepatic vein positioning data to obtain the elastic registration data includes: Determine the hepatic artery positioning data as reference structure data; According to the reference structure data, the hepatobiliary duct positioning data, the portal vein positioning data and the hepatic vein positioning data are paired between structures to generate inter-structure paired data; The control point network is covered according to the paired data between structures to obtain the elastic registration data.

[0021] In the present invention, by using the hepatic artery positioning data as a reference structure, a unified spatial alignment benchmark between multiple vascular structures is established, and the positioning results of the bile duct, portal vein and hepatic vein are combined to perform structure pairing, which can make full use of the anatomical proximity and spatial distribution rules between the structures to construct an accurate structural correspondence. Based on the results of the pairing between structures, an elastic registration operation is performed using a control point network coverage method, which can implement flexible spatial deformation on multiple structures while maintaining the continuity and topological relationship of the anatomical structures. Compared with traditional rigid registration or methods based on single structure guidance, the present invention effectively improves the spatial coupling degree and overall fusion quality between vascular structures through multi-structure collaborative drive and elastic deformation control point mechanism, especially in medical image scenes where multiple structures intersect, spatial dislocation or deformation is significant, it has higher structural alignment accuracy and three-dimensional modeling adaptability, providing a solid foundation for clinical precision visualization and personalized vascular reconstruction.

[0022] Preferably, a three-dimensional hepatobiliary, hepatic artery, portal vein and hepatic vein image segmentation system is characterized in that it is used to perform the three-dimensional hepatobiliary, hepatic artery, portal vein and hepatic vein image segmentation method as described above, and the three-dimensional hepatobiliary, hepatic artery, portal vein and hepatic vein image segmentation system comprises: A multi-phase image acquisition module, used for acquiring multi-phase medical image data of a liver region; A region recognition module is used to perform region recognition on multi-phase medical image data, and obtain hepatobiliary region data, hepatic artery region data, portal vein region data, and hepatic vein region data respectively; The structural association positioning module is used to perform structural tracking on the hepatic artery area data to obtain the hepatic bile duct positioning data; perform improved contrast extraction on the hepatic artery area data to obtain the hepatic artery positioning data; perform directional segmentation analysis on the portal vein area data to obtain the portal vein positioning data; perform branch angle extraction on the hepatic vein area data and the portal vein positioning data to obtain the hepatic vein positioning data; The three-dimensional structure reconstruction and display module is used to perform elastic registration based on the hepatobiliary duct positioning data, the hepatic artery positioning data, the portal vein positioning data and the hepatic vein positioning data to obtain elastic registration data; and perform three-dimensional visualization operations based on the elastic registration data.

[0023] The beneficial effects of the present invention are as follows: Based on multi-phase medical image data, through modular processing methods with different structures and strategies, it is possible to achieve refined extraction and three-dimensional modeling of multiple key vascular systems in the liver. The bile duct is identified at the path level through structural tracking, the hepatic artery is enhanced with edge clarity and micro-branch positioning capabilities through improved contrast extraction, the portal vein introduces a directional segmentation analysis method to strengthen its structural continuity and branch integrity, and the hepatic vein is combined with the portal vein direction information to distinguish the branch angle, thereby improving the accuracy of structural attribution judgment. Through multi-structure elastic registration and control point network construction based on the hepatic artery, unified spatial alignment and three-dimensional visualization expression of multiple vascular structures are achieved. Compared with the existing processing methods based on a single structure or a unified segmentation model, the present invention can make full use of the anatomical relationship characteristics between different image phases and structures, and has outstanding technical advantages such as accurate structural recognition, high registration robustness, and strong continuity of three-dimensional modeling, which significantly improves the anatomical restoration ability and intelligent level of the clinical diagnosis auxiliary system. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting implementations made with reference to the following drawings: Figure 1 A flowchart showing a method for segmenting three-dimensional hepatic bile duct, hepatic artery, portal vein and hepatic vein images according to an embodiment is shown; Figure 2 A flowchart showing a method for identifying a region of a multi-phase image according to an embodiment of the present invention is shown; Figure 3 A flowchart of the steps of a multi-region structure association positioning method according to an embodiment is shown. DETAILED DESCRIPTION

[0025] The following is a clear and complete description of the technical method of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by technicians in this field without creative work are within the scope of protection of the present invention.

[0026] In addition, the accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor methods and / or microcontroller methods.

[0027] It should be understood that, although the terms "first", "second", etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are used only to distinguish one unit from another unit. For example, without departing from the scope of the exemplary embodiments, the first unit may be referred to as the second unit, and similarly the second unit may be referred to as the first unit. The term "and / or" used herein includes any and all combinations of one or more of the listed associated items.

[0028] See also Figures 1 to 3 The present application provides a three-dimensional hepatobiliary, hepatic artery, portal vein and hepatic vein image segmentation method, comprising the following steps: S1. Acquire multi-phase medical image data of the liver region; In one embodiment, the multi-phase medical image data of the present invention includes a multimodal image sequence from the same subject, specifically including arterial phase CT image data, portal venous phase CT image data, equilibrium phase (delay phase) CT image data and T2-weighted MRI image data. The CT image data is collected from a spiral CT scanning device with 64 layers or more, such as the Revolution CT (256 layers) clinical-grade spiral CT scanner produced by GE. The slice thickness of each layer of the collected image is not more than 1 mm. The scanning protocol is as follows: arterial phase scan: 20 seconds after intravenous injection of iodine contrast agent; portal venous phase scan: 60 seconds after injection; equilibrium phase (delay phase) scan: 180 seconds after injection; scanning parameters include layer thickness 1mm, pitch 1.375, tube voltage 120kVp, and tube current is automatically adjusted. T2-weighted MRI image data is collected using a 3.0T high-field strength MRI scanner, such as the GE SignaPioneer 3.0T system or the Siemens MAGNETOM Skyra 3.0T system. A transverse fast spin echo sequence (FSE) was used, and the parameter settings included: echo time (TE) of 80-100 ms, repetition time (TR) of 4000-6000 ms, slice thickness of no more than 3 mm, and matrix resolution of no less than 256×256, to ensure sufficient soft tissue contrast to identify the hepatobiliary structure.

[0029] S2, performing region recognition on the multi-phase medical image data to obtain hepatobiliary region data, hepatic artery region data, portal vein region data, and hepatic vein region data; In one embodiment, a three-dimensional deep convolutional neural network model based on the U-Net++ structure is used to construct independent semantic segmentation models for key anatomical structures of the liver. The structures involved include the bile duct; hepatic artery; portal vein; and hepatic vein. The U-Net++ model (also known as Nested U-Net) introduces a deep supervision mechanism and dense skip connections. The network input is a three-dimensional cubic image block (3D patch) with a size of Voxels are processed by grayscale normalization and voxel size resampling. The output is a structural probability map, that is, the predicted probability of each voxel in its category. In order to improve the segmentation accuracy and anatomical structure distinction, different anatomical structures are modeled mainly based on their best visualization phase images: the hepatobiliary model is mainly based on delayed phase images; the hepatic artery model is mainly based on arterial phase images; the portal vein is mainly based on portal venous phase images, and the hepatic vein model is mainly based on T2-weighted MRI image data. During the model training process, a weighted combination of Dice coefficient loss and focus loss function is used as the total loss function to take into account the overlap of foreground categories and the focus on difficult-to-separate samples, significantly alleviating the problem of category imbalance.

[0030] S3, perform structural tracking on the hepatobiliary region data to obtain hepatobiliary location data; perform improved contrast extraction on the hepatic artery region data to obtain hepatic artery location data; perform directional segmentation analysis on the portal vein region data to obtain portal vein location data; perform branch angle extraction on the hepatic vein region data and the portal vein location data to obtain hepatic vein location data; In one embodiment, within the first 20 slices of the image center, the operator manually selects the voxel with the largest grayscale value as the starting point of the portal area. This point is usually located at the entrance of the common bile duct or common hepatic duct, and can also be automatically identified according to the anatomical model. Taking the initial point as the starting point, the Fast Marching method is used to construct a forward propagation distance field based on the velocity function to provide overall energy distribution guidance for path search. The minimum cost path search algorithm (such as Dijkstra) is used in the distance field to extract the shortest energy path from the initial point to the end of each branch, and reconstruct the tree centerline structure of the hepatobiliary duct. The starting point is manually selected by the operator from the first 20 slices of the image center as the starting point of the portal area; this area corresponds to the entrance of the common bile duct or common hepatic duct, and can also support automatic identification and substitution according to the anatomical structure. Set growth constraints: neighborhood direction consistency>0.7, calculate the structural consistency between adjacent voxels through the main direction of the Hessian matrix; ensure that the tracking path grows continuously along the direction of the blood vessels / bile ducts, and suppress lateral misdiffusion. The segmentation probability map value is > 0.4; this probability map is derived from the structural probability map output by the aforementioned U-Net++ model; it represents the degree of confidence that the current voxel belongs to the hepatobiliary structure. The output is the centerline data of the three-dimensional tree structure of the hepatobiliary, which is expressed as a spatial curve composed of multiple key points.

[0031] The input image is a normalized arterial phase enhanced CT image. The image has been grayscale normalized and the voxel size has been resampled (unified to 1×1×1mm³). A three-dimensional Frangi filter is applied to the arterial phase image to enhance the local contrast of slender and tubular structures; the filter parameters are set to a multi-scale range. =0.5 to 2.5, with a step size of 0.5; the Frangi filter analyzes the local structure morphology through the eigenvalue of the Hessian matrix and has a good response to linear structures. In the enhanced image, the grayscale mean value in the local window is used. With standard deviation Construct a dynamic threshold; if the gray value of a voxel , it is judged as a foreground voxel; this method can effectively deal with the problems of uneven image brightness and local contrast differences. The above segmentation results are logically ANDed with the liver region mask to retain only the high response area within the liver; the liver region mask is obtained by the above liver segmentation model. Connected domain analysis is performed on the cropped three-dimensional segmentation results; only the top three largest connected regions with a voxel number greater than 500 are retained to remove discrete noise and pseudo-response regions; the output three-dimensional mask represents the main trunk of the hepatic artery and its significant branch structure. The SimpleITK library is used for three-dimensional image filtering and segmentation, and NumPy is combined for numerical calculation and connected domain analysis. The above tools are all open source scientific computing libraries suitable for medical image processing scenarios.

[0032] Directional filtering was performed on the portal venous phase images in enhanced CT images. An adjustable directional filter group was used and the number of directions was set to N=12. This step was used to enhance the linearly distributed vascular structures in the image and obtain the response intensity of each pixel in each direction. For each pixel point, the direction angle corresponding to the maximum value of its filter response in 12 directions was recorded to form the direction vector field of the whole image. The K-Means clustering algorithm (K=3) was used to cluster the direction field and extract the direction with the highest frequency as the dominant direction. Guided by the dominant direction, a 16×16 pixel directional aligned two-dimensional slice window was constructed. Using the trained two-dimensional convolutional neural network (2D CNN), the portal vein / non-portal vein binary classification judgment was performed on the central pixel of each window, so as to realize the recognition and segmentation of the portal vein path. The above recognition results were reconstructed into a three-dimensional voxel structure according to the spatial position to generate the portal vein segmentation mask. At the same time, the path was tracked according to the dominant direction to obtain the directional centerline trajectory of the portal vein. The output consists of two parts: a three-dimensional portal vein voxel mask, which is used to represent the spatial distribution of the portal vein structure; and a central path extracted based on the dominant direction, which is used to represent the main direction of the blood vessel.

[0033] Arterial phase images and portal vein path data were input; the portal vein centerline was obtained by the aforementioned U-Net++ segmentation model and minimum path tracing method; the images used were venous phase images (obtained after 60 to 90 seconds of enhancement delay). Based on the three-dimensional path of the portal vein centerline, all branch nodes with trifurcated or multifurcated structures were identified; the branch points were determined based on the degree of connection of the path nodes ≥ 3, or the direction change exceeded a specific angle threshold. A spherical neighborhood region was constructed at each branch point, with a radius set to 8 voxels, covering all branch structures; the remaining vascular paths and their main direction vectors were extracted from the region. The three-dimensional angle between the direction of the main branch of the portal vein and the directions of other paths in the neighborhood was calculated (based on the vector cosine); if the angle was between 40° and 80°, and the path length was not less than 20 voxels, it was marked as a candidate hepatic vein path. At each branch point Construct a spherical space neighborhood with a radius of 8 voxels around ; Extract all path segments in the neighborhood and calculate their main direction vectors; calculate the three-dimensional angle between the neighborhood path and the trunk direction; if the following conditions are met, it is marked as a candidate hepatic vein path, and the angle range is: . Path length ≥ 20 voxels; the end coordinate point is located at the top of the liver (such as the slice number Or near the exit area of ​​the inferior vena cava (such as within 3 cm of the center coordinates of the superior vena cava segment of the hepatic vein). The candidate path is refined using a three-dimensional ridge extraction algorithm; the precise center path is extracted using a graph optimization method based on minimum cost graph cuts to exclude excess noise or divergent paths. The output is a three-dimensional centerline representation of the main path of the hepatic vein.

[0034] S4. Perform elastic registration based on the hepatobiliary duct positioning data, the hepatic artery positioning data, the portal vein positioning data, and the hepatic vein positioning data to obtain elastic registration data; and perform three-dimensional visualization based on the elastic registration data.

[0035] In one embodiment, for multiple types of vascular structures (including hepatic artery, portal vein, hepatic vein and hepatic bile duct) in liver medical images, elastic registration technology based on B-Spline non-rigid deformation model is used to align each structure to a unified spatial reference coordinate system. The registration process is implemented based on the open source image processing platform Elastix v4.9, specifically including fixing the portal vein structure mask map as the reference coordinate system for spatial registration. The mask maps of three types of structures, hepatic artery, hepatic bile duct and hepatic vein, are used as the targets to be registered. A non-rigid deformation model based on B-Spline is adopted to support local elastic alignment; the grid spacing is set to 32 pixels to control the fineness of the deformed grid; the main metric function is set to mutual information to compare the intensity distribution between grayscale images; the auxiliary metric is the structural similarity index (SSIM) to improve the accuracy of structural alignment; an adaptive stochastic gradient descent optimizer is used for metric calculation; a pyramid registration method is used to set the resolution level to 3 layers, and optimize from low resolution to original resolution layer by layer. Based on Elastix v4.9, it also supports expansion using open source platforms such as Advanced Normalization Tools (ANTs). The output is a deformation mask map of the three structures of hepatic artery, hepatic bile duct, and hepatic vein in a unified spatial coordinate system; at the same time, the corresponding three-dimensional non-rigid deformation field is generated.

[0036] Use VTK to read segmentation mask data; apply Marching Cubes isosurface reconstruction algorithm to each type of structure to generate corresponding triangular mesh (PolyData); assign unique color code to each structure, such as hepatic artery: red; portal vein: blue; hepatic vein: green; hepatic bile duct: yellow; export the reconstructed structure to standard 3D model format (such as STL or OBJ). The front end uses the WebGL-based Three.js library to load and render the model; provide interactive control functions, including 3D rotation and scaling; section browsing; model transparency adjustment; mouse dragging, positioning, and structure selection; auxiliary functions include hovering prompts for structure names; vascular path visualization; distance, angle, and vascular diameter measurement tools. The back end uses the Flask framework to provide model data reading and interface services, the front end uses WebGL to render the model, and the front and back ends are linked through HTTP communication. The function of browsing segmentation results online without client installation is realized, supporting doctors and researchers to access structural models on multiple terminals.

[0037] Preferably, the multi-phase medical image data includes arterial phase CT image data, portal venous phase CT image data, equilibrium phase CT image data and T2-weighted MRI image data, and the multi-phase medical image data is subjected to regional recognition to obtain hepatobiliary regional data, hepatic artery regional data, portal vein regional data and hepatic vein regional data, respectively, including: S21, performing a rough segmentation of the liver region on the portal venous phase CT image data to obtain liver region data; In one embodiment, portal venous phase enhanced CT images are used as input data. This phase is collected about 60 seconds after contrast agent injection, and the contrast between liver parenchyma and surrounding tissue is clearest. The grayscale intensity of the image is limited to [-100,300] Hounsfield units (HU), abnormal low-density and high-density values ​​are removed, and the grayscale contrast of liver parenchyma tissue is enhanced. The image is isotropically voxel resampled to a uniform spatial resolution of , in order to adapt the network input and enhance the consistency of features. A three-dimensional U-Net structure is used for semantic segmentation of the liver area; the network input is a preprocessed standardized CT image, and the output is a probability map; the network training data includes the public Liver Tumor Segmentation Challenge (LiTS) dataset and the clinical manually annotated samples constructed in this project; the label only contains the complete liver area, excluding the intrahepatic blood vessels and bile duct structures, to ensure that the mask boundary is clear. The segmentation results are analyzed by three-dimensional connected domains, and only the area with the largest number of voxels is retained to remove false detection artifacts or non-liver area responses. A morphological closing operation is performed on the mask, using a spherical structural element (radius 2~3 voxels) to fill small holes on the edge and smooth the boundary curve. The output is a three-dimensional binary mask image (mask), which represents the area where the liver is located. The mask is used as a spatial clipping reference for the extraction of blood vessels, bile ducts, and tumor structures to improve the accuracy and computational efficiency of structural positioning.

[0038] S22, determining the hepatic artery region according to the liver region data and the arterial phase CT image data to obtain the hepatic artery region data; In one embodiment, the input is an arterial phase CT image, in which the hepatic artery structure is shown in a high contrast enhancement state. The obtained liver mask is called to perform spatial cropping on the CT image, retaining only the image voxels in the liver area and excluding the interfering structures outside the liver. The Frangi vascular enhancement filter is applied to the cropped image, and the multi-scale strategy is used to Enhances vascular structures with tubular features. Apply a preset threshold to the Frangi filter result , set to the image mean Add 1.5 times the standard deviation , in order to segment the vascular area with strong response. The vascular segmentation results were refined by three-dimensional morphology to extract the vascular centerline path with a width of one pixel. The centerline point closest to the center of the liver portal area was used as the starting point to perform the centerline-based vascular structure tracking and growth process. The extracted vascular network was structurally screened to eliminate false positive vascular segments that were not connected to the liver area, and only the trunk and its vascular branch structures within two levels were retained, that is, the structural path from the trunk to the third-level node was limited.

[0039] S23, identifying the hepatic vein region according to the liver region data and the equilibrium phase CT image data to obtain the hepatic vein region data; In one embodiment, the input equilibrium CT image is collected about 120-180 seconds after the injection of contrast agent; in the equilibrium image, the density contrast between the venous blood vessels and the liver parenchyma is obvious, which is suitable for hepatic vein identification; at the same time, the obtained three-dimensional mask of the liver area (liver area data) is loaded as the basis for spatial cropping. The liver mask is used to crop the original image, retaining only the image voxels in the liver area and excluding interference from non-liver areas. A three-dimensional directional adjustable filter (3DSteerable Filter) or a multi-directional Gabor filter is applied to the cropped image; the filter enhances the structural response with linear and tubular features and highlights the direction of the hepatic vein. The Otsu adaptive threshold segmentation algorithm is applied to the enhanced image to obtain a binary image; the Otsu method automatically selects the optimal segmentation threshold by analyzing the image histogram, which is suitable for bimodal distribution scenarios. All connected regions were analyzed for features, and candidate hepatic vein regions that met the following conditions were retained: the main axis direction was close to the Z axis (vertical axis): that is, the angle between the main direction of the structure and the Z axis was less than 30°; the number of branches was greater than 2: the number of branches was calculated by skeleton extraction and node statistics; the center of the region was close to the top of the liver, that is, the Z coordinate of the center of the region was within 10% of the voxels at the top of the liver mask. The Fast Marching path tracing algorithm was used with the selected top vein region as the starting point; a cost map was constructed, and the vascular response value was used as the inverse function to trace the internal path of the liver below; a continuous path was extracted and the center lines of the main and branch hepatic veins were generated. The hepatic vein region mask was output, a three-dimensional binary image, and the vein region was marked; the structure center path map, the three-dimensional center line of the main hepatic vein and its branches, was expressed as a continuous coordinate point set.

[0040] S24, performing direction-guided segmentation according to the liver region data to obtain portal vein region data; In one embodiment, the input is a portal venous phase CT image and a corresponding liver region mask. In the area defined by the liver mask, the Hessian matrix of each voxel is calculated based on the image intensity value, and the main curvature direction is obtained to construct a voxel-level directional gradient vector field. The direction field is used to represent the main direction of the local vascular structure. A spherical region of interest is defined in the portal region (starting point of the main trunk of the portal vein) as the starting point of path tracking. The region can be automatically generated based on anatomical positioning or doctor annotation. With the starting point as the center, the path tracking operation is performed along the main direction of each voxel in the direction field (such as the maximum curvature direction). The path tracking method adopts a tubular structure tracking algorithm (Tube Tracking), or a Flux Voting method based on directional consistency. During the tracking process, a main trunk path map of the portal vein is gradually established. Using the path map as a guiding skeleton, a local region growth operation is performed in combination with the grayscale features of the surrounding voxels to obtain the overall segmentation result of the portal vein. The following enhancement strategies are used in the region growing process to improve accuracy and robustness: local maximum intensity projection is introduced in the path tracing region to enhance the continuity of small vessels and lateral tributaries; the cosine similarity threshold between the region growing direction and the local main direction is set , to restrict growth to only expand along the main axis of the blood vessel, to prevent erroneous segmentation into non-vascular areas; to construct a preliminary path map to represent the direction of the portal vein trunk. The output includes a 3D voxel mask of the portal vein area; a path map of the portal vein center obtained by tracing; and the structure of the portal vein tributaries, represented as a tree path or a spatial point set.

[0041] S25. Perform hepatobiliary region identification based on the liver region data and the T2-weighted MRI image data to obtain hepatobiliary region data.

[0042] In one embodiment, the hepatobiliary region is identified and extracted based on T2-weighted magnetic resonance imaging images and liver structure data. The input image is a T2-weighted MRI image, because its hepatobiliary structure has good signal contrast under this sequence, which is suitable for bile duct extraction; the T2 MRI image is spatially aligned to the reference coordinate system of the portal venous phase CT image by rigid registration; the liver mask in the CT image is mapped to the MRI image by spatial transformation, and the MRI image voxels of the liver region are cropped accordingly to exclude extrahepatic interfering tissue. Local contrast enhancement and grayscale normalization, including histogram equalization, are applied to the cropped MRI image; histogram matching. A three-dimensional convolutional neural network (such as ResUNet3D) is used for automatic segmentation of the bile duct structure. The network adopts an encoding-decoding structure, which integrates multi-scale features and residual connections, and is suitable for accurate extraction of slender structures such as hepatobiliary ducts. In the model training stage, in view of the problem that the bile duct and gallbladder have similar signals and are easily confused in T2 images, independent labels are set specifically for the bile duct and gallbladder regions to improve the model's discrimination ability and reduce mis-segmentation. Post-processing operations are performed on the segmentation results, including analyzing whether the bile duct structure is connected to the liver portal area to identify effective branches; removing isolated, broken or artifact areas caused by noise interference to improve the anatomical rationality and traceability of the segmented structure. The output is a three-dimensional voxel mask of the hepatobiliary duct to obtain the hepatobiliary regional data.

[0043] Preferably, performing directionally guided segmentation according to the liver region data to obtain portal vein region data includes: Extract voxel grayscale gradient according to liver region data to obtain voxel grayscale gradient data; In one embodiment, a portal venous phase enhanced CT image is used as input data. The image has completed standard preprocessing steps, including grayscale cropping, normalization, and voxel size resampling. The obtained three-dimensional mask of the liver region is used as the processing region, and voxel gradient calculation is performed only within the liver. A three-dimensional Sobel operator or an equivalent first-order differential convolution kernel is used to perform convolution operations in the three spatial dimensions of x, y, and z; the gradient component in each direction is calculated by the following formula: , , , for The rate of change of intensity in the direction, is the grayscale voxel value of the 3D CT image, For calculation The first-order differential convolution kernel of the directional gradient, for The rate of change of intensity in the direction, For calculation The first-order differential convolution kernel of the directional gradient, for The rate of change of intensity in the direction, For calculation The first-order differential convolution kernel of the directional gradient. Calculate the gradient vector of each voxel point and gradient magnitude , . Get the gradient vector map, including the three-dimensional gradient value of each voxel in the liver area , represented as a vector field, and a gradient magnitude map, i.e., a gradient modulus map , characterizing the amplitude of local intensity variation.

[0044] Perform main direction clustering on the voxel gray gradient data to obtain main direction clustering data; In one embodiment, voxel grayscale gradient data is used to select voxel points whose gradient amplitude is greater than the 95% quantile of the whole image for clustering; all direction vectors participating in the clustering will be standardized and converted into unit vectors. The unit vector calculation method is: ,in, For the The unit direction vector of a point, For the The direction vector of a point, For the The magnitude of the direction vector of the point, , For the The direction vector of the point is The weight in direction, For the The direction vector of the point is The weight in direction, For the The direction vector of the point is The spherical K-means algorithm is used to cluster the unit direction vectors. This algorithm is applicable to unit sphere data and can efficiently extract multiple main directions. Its optimization goal is to maximize the consistency of direction, that is, to maximize the sum of the cosine similarities of all direction vectors and the direction of the cluster center to which they belong: , For the The unit direction vector of the voxel, is the cluster center direction vector, the cluster center direction vector to which the point belongs. The dot product represents the cosine value between the two directions. The larger the value, the more consistent the direction. For clustering index, specify Which clustering category it belongs to. Number of clusters It can be set to 4 to 6 according to the actual vascular branch structure, which is enough to cover the trunk and multi-level branch directions, so as to extract The output includes a direction cluster label for each voxel participating in the cluster, indicating the category to which its dominant direction belongs; each direction class represents a potential main direction of a vascular structure.

[0045] Perform tensor voting processing based on the main direction clustering data to obtain the blood vessel direction field data; In one embodiment, each main direction clustering data is initialized as a symmetric second-order tensor ,in is the gradient direction, is the transposed symbol; within the neighborhood window (such as 5×5×5), weighted propagation voting is performed, and each voxel obtains a weighted structure tensor, where the weighted propagation voting is: For point The weighted structure tensor of For point The neighborhood set of is a Gaussian weighted function. The farther the distance, the smaller the weight. For point With neighboring points The Euclidean distance between For point The original structure tensor at , , is the weight of tensor propagation, the closer the point is , the larger the weight, is the natural exponential term, is the distance value, equal to , To control the scale parameter (window scale) of the Gaussian distribution width; perform eigenvalue decomposition on the tensor and extract the main direction: , is the structure tensor, For the eigenvalue decomposition operation, the tensor (or matrix) Perform eigenvalue decomposition, is the principal eigenvalue (largest), is the secondary eigenvalue, is another minor eigenvalue; by the ratio of the main eigenvalue to the total energy , get the direction consistency index; the vascular direction field data includes the main direction vector field, the main direction corresponding to each voxel, which constitutes the vascular direction vector field; the direction consistency map, the direction consistency index of each voxel, characterizes its prominence in the vascular structure.

[0046] The preset direction-guided segmentation network model is used to segment the vascular direction field data to obtain the portal vein area data, wherein the direction-guided segmentation network model is constructed by using the improved SD-UNet to construct the backbone network, and the attention map or the guided flow is constructed as the output head.

[0047] In one embodiment, the encoder part introduces a Dense Block module in each level of convolutional layer to enhance the feature extraction capability; a spatial attention module is inserted in the jump connection to improve the spatial feature alignment effect. The input of the network consists of three channels: Channel 1: portal venous phase CT image; Channel 2: vascular direction field image (the direction vector can be encoded as a three-channel RGB or floating-point tensor); Channel 3: structural consistency score map, which is used to describe the area with prominent directional structure. The network output contains two branches. The main output branch generates a three-dimensional segmentation mask of the portal vein area through a Sigmoid function; the guided output branch generates a directional guidance map, which is used to enhance the directional perception ability or subsequent result visualization during training. During the training process, the loss function is in the following joint form: is the total loss, is the Dice coefficient loss, is the binary cross entropy loss, is the weighting factor of the directional loss, is the cosine similarity loss between direction vectors. The training parameters are set as follows, optimizer: ; Initial learning rate: ; Batch Size: 2; Input patch size: 64×64×64; Number of training epochs: 100-150. The output is a portal vein 3D mask image, i.e. portal vein area data.

[0048] Preferably, the step of constructing the preset direction-guided segmentation network model includes the following steps: Obtain historical vascular direction field data and historical annotation data; In one embodiment, the historical vascular orientation field data is derived from the directional structure extraction results of the existing annotated images; each sample includes the original CT image, the extracted vascular structure tensor orientation field (for example, the main orientation map obtained by tensor voting and eigenvector calculation); the historical annotation data is a manually or semi-automatically annotated portal vein structure area mask (3D voxel label).

[0049] Perform multi-scale exaggerated convolution parallel processing based on historical vascular direction field data to obtain edge perception layer data; In one embodiment, the input feature map is subjected to three-dimensional convolution operations through three parallel paths, and the convolution kernel sizes are: 3×3×3, 5×5×5, 7×7×7, respectively. Each path consists of a three-dimensional convolution layer, a batch normalization layer, and a nonlinear activation function (ReLU), which extracts feature information under different receptive fields. The feature maps output by the three scale paths are spliced ​​in the channel dimension to form a fused multi-scale feature map. Subsequently, a 1×1×1 convolution layer is used to perform channel compression and feature fusion processing to reduce redundancy and maintain structural representation capabilities; In order to enhance the directional perception, a directional weighting mechanism is introduced in the above convolution process, such as obtaining the spatial direction vector of the current voxel point in the sliding window area of ​​each convolution kernel. , the direction comes from the aforementioned direction field calculation module; the response to the neighboring direction position that is consistent (or approximately consistent) with the direction vector is enhanced, for example, the main direction channel is multiplied by the weighting coefficient ; Maintain or suppress the responses in other directions, thereby simulating high-pass excitation behavior and strengthening the response of the image grayscale or structural mutation area. Suppose the current voxel position is , the position to be calculated in the convolution window is , whose directional consistency weight is defined as: , is the direction consistency weight factor, is the directional weighting coefficient (0.5~1.0), To maximize the function, is the main direction vector of the current voxel, is the current voxel (convolution center) position, is any neighborhood voxel position within the convolution window, For point to The unit direction vector of for The model, For point to The modulus of the unit direction vector; the direction-weighted convolution response is expressed as is the direction weighted convolution result, is any neighborhood voxel position within the convolution window, For The convolution window range centered on is the direction consistency weight factor, The convolution kernel is offset The value of The input feature map is The output is a set of edge-aware feature maps that fuse directional features, i.e., edge-aware layer data.

[0050] The edge perception layer data is subjected to shallow edge layer processing and high-level feature layer processing to obtain shallow edge layer data and high-level feature layer data respectively, wherein the shallow edge layer processing realizes boundary response enhancement through Sobel / LoG filter or edge channel convolution; In one embodiment, the input voxel image In three directions Perform the first-order gradient calculation on: ,in For The first-order gradient response in the direction, is the input edge perception layer data, for The first-order gradient convolution kernel in the direction, For The first-order gradient response in the direction, for The first-order gradient convolution kernel in the direction, For The first-order gradient response in the direction, for The first-order gradient convolution kernel in the direction. Calculate the edge response value , is the gradient magnitude map, the gradient magnitude map It will be used as the input of the shallow edge channel to significantly enhance the boundary area. The 3D LoG filter is used to perform second-order differential operations on the image to respond to sharp structural changes, enhance the boundary signals of small blood vessels and weak response areas, and effectively improve the precision of structural detection, thereby constructing an edge guidance channel. (It can be a gradient map, LoG response, etc.); connect it in parallel with the backbone feature as the channel-level attention weight to guide the regional response of the deep feature; focus on the edge area of ​​the structure and obtain the shallow edge layer data.

[0051] In order to obtain the global contextual features of the vascular structure, the 3D version of the ResNet-18 encoder is used as the backbone extraction network: the encoding path contains 4 residual blocks, and each layer uses stride=2 convolution for downsampling to form a multi-level semantic abstraction; each block consists of two sets of convolution + normalization + ReLU + residual connection to improve the robustness of structure recognition; after encoding, the ASPP module is connected: the hollow convolution kernels with expansion rates of 1, 6, 12, and 18 are used in parallel; the receptive field is expanded, the ability to capture large-scale structures is enhanced, and high-level feature layer data is obtained.

[0052] Perform residual connection on shallow edge layer data and high-level feature layer data to obtain decoding layer data; In one embodiment, the shallow edge layer data and the high-level feature layer data are added or concatenated element by element in the channel dimension; to unify the feature dimension after fusion, a 3×3×3 three-dimensional convolution operation is performed on the fused features; an attention mechanism module (such as an SE module or a spatial attention module) is introduced to strengthen the response of significant edge features and improve the fusion ability between edges and global semantics. The fused feature map also has the following characteristics: retaining the edge details of blood vessels in the shallow input, such as the comb-like structure outline of capillaries; integrating the global direction and structural semantics in the high-level output, which helps to maintain the connectivity and complete identification of deep blood vessels. The fused decoding layer feature map is output as the main input feature of the network direction output branch and the structural segmentation branch.

[0053] The vascular direction network is trained on the decoded layer data using historical annotation data to obtain direction-guided flow data. The vascular direction network training supervises the convolutional network by using the direction consistency loss combined with the Dice loss. In one embodiment, a strategy for directional supervision training based on historical annotated data is used to introduce a directional guidance stream output head at the end of the decoder to generate directional vector information corresponding to each voxel as a key input for structure recognition or guided path growth. Two types of supervision information are used: structure mask, such as using standard Dice Loss to optimize the main branch structure mask; directional consistency map, such as using historical expert annotated data to extract the main direction vector of the structure as a directional supervision label (directional vector map or tensor consistency map) to guide the directional branch output. The total loss function consists of structural loss and directional loss, and the weighted combination is as follows: , is the total loss, is the structural consistency loss, is the direction supervision weight, and the recommended value range is 0.3-0.5. is the direction consistency loss, where the structural consistency loss is: ,in represents the predicted structure mask, Represents the true structure mask, represents the number of voxels. By maximizing the overlap area between the predicted result and the true label, It can effectively guide the network to learn the spatial distribution of vascular trunks and thin branches, and characterize the extraction integrity and accuracy of the structural area; the directional consistency loss is the cosine distance loss between directional vectors: ,in is the direction vector predicted by the network, is the label direction vector. In order to generate the structure mask and the direction flow at the same time, a dual output head is designed at the end of the network decoder. The main output head uses the sigmoid activation function to output the structure mask map for binary segmentation; the guided flow output head outputs the directional guided flow map, in which each voxel position contains a unit three-dimensional direction vector.

[0054] The direction-guided flow data is directionally guided to output, and a direction-guided segmentation network model is obtained, wherein the direction-guided output is convolutionally fused with the decoding layer data and the direction confidence is calculated, or the direction-guided attention map is calculated through the direction-guided flow data, and the output result is regulated at the channel level.

[0055] In one embodiment, two methods are selected or combined. Method 1: 3D convolution fusion of the direction guidance flow data G and the decoding layer feature D: encoding the direction guidance flow data G as a direction convolution kernel parameter, which is used to perform local convolution weighting on the decoding layer feature D; the output is a direction confidence map , indicating the consistency of each voxel direction (for example: 0.85 = highly certain that the blood vessel runs in this direction). The direction confidence is used as an enhancement factor for the mask channel to improve the network focusing accuracy. The direction confidence map is used as a weight adjustment factor for the mask map generation module. ,in is the activation function, is a three-dimensional convolution kernel, and its size is preset as , For the vascular direction field data, enhance the response weight of the area with consistent direction and output the segmentation result , thus obtaining the direction-guided segmentation network model.

[0056] Method 2: Direction-guided stream data is input into the directional attention module; the weight map of each channel is output , For the The weight coefficient of the channel indicates the degree to which the channel is consistent with the direction guidance. For directing flow from the direction The function module that calculates the weight of each channel, is the first channel index, is the total number of channels of the feature map, and performs channel-level guidance on the output mask map , It is a multi-channel spatial feature map, that is, direction-guided flow data; it is equivalent to letting the network select a channel with more consistent direction as the main basis for output, and obtaining a feature map after direction-guided attention regulation The characteristic graph after adjustment The segmentation result with enhanced direction is output by 3D convolution to obtain the direction-guided segmentation network model.

[0057] Preferably, the performing structural tracking on the hepatobiliary region data to obtain the hepatobiliary positioning data includes: Constructing the spatial guidance domain of the hepatobiliary region data to obtain the hepatobiliary spatial guidance domain data; In one embodiment, a spatial coordinate tensor is constructed to express the normalized relationship of spatial position. For each voxel, it is normalized according to the image size to: , the resulting three-channel tensor records the spatial coordinates of each position, with the range normalized to . Extract the central region of the spatial guide. According to the mask , calculate its voxel density centroid , the morphological center point extraction method can also be used as a means to enhance stability. The construction radius is The three-dimensional spherical region of interest is used as the starting area for path tracing. Constructing the spatial guidance domain tensor , which is defined as: is the spatial guide domain value, is the distance field weight coefficient, which represents the weight of the distance field from the current voxel to the center point, which is 1.0. is the distance field from the current voxel to the center point, is the edge gradient map weight, which represents the weight of the mask boundary gradient map. is the mask boundary gradient map (extracted using Sobel or Laplacian), which is 0.5. The spatial guidance domain tensor It will be used for directional consistency enhancement processing and path cost function modeling to provide spatial prior constraints for hepatobiliary structure tracking.

[0058] Perform local directional consistency enhancement based on the hepatobiliary spatial guidance domain data to obtain hepatobiliary enhancement data; In one embodiment, for any voxel point in the hepatobiliary spatial guidance domain data , calculate its gradient vector ,in is the original structure tensor, is the image gradient vector (three-dimensional), is the transpose of the image gradient vector (three-dimensional), and then reconstructs the corresponding structure tensor: A Gaussian kernel is used for weighted smoothing in a 5×5×5 three-dimensional neighborhood. , is a smooth tensor, is the three-dimensional Gaussian weight, for The index in the direction, for The index in the direction, for Index in the direction to obtain the smoothed tensor . For smooth tensors Perform eigenvalue decomposition and obtain three sets of eigenvalues If , it means that the neighborhood of the voxel is a slender tubular structure; the corresponding eigenvector represents the local main direction vector. The local direction enhancement value is calculated, and the local direction enhancement function is as follows: ,in Enhanced data for hepatobiliary ducts, is the maximum tensor eigenvalue, Second only to The tensor eigenvalues ​​of , Second to and The tensor eigenvalues ​​of , Provides spatial guidance domain data for the hepatobiliary duct.

[0059] Performing path tracing according to the hepatobiliary enhancement data to obtain hepatobiliary duct tracing data, wherein the path tracing is to construct a cost function based on signal strength, direction consistency and path proximity to perform minimum cost path tracing; In one embodiment, the path , is a set of path points, representing a continuous path, consisting of several voxel points. is the first path point, is the second path point, For the path points, the overall path cost function is defined as: , is the path cost function, which represents the total cost of the “trustworthiness” of the entire path. The smaller the value, the better. is the voxel point order term, is the number of voxel points, is the image intensity weight coefficient, , Enhanced data for hepatobiliary ducts, is the direction change weight coefficient, is the direction change penalty (cosine angle distance), For the path Individual elements, For the path Individual elements, is the distance penalty weight coefficient, is the spatial distance between the path and the guiding area (center of the liver portal), To guide the center of the region (such as the liver hilum); extract the initial point set from the liver hilum center area , as the starting position of path tracing. Based on the Fast Marching algorithm or the Dijkstra shortest path search algorithm, path extension is performed in the three-dimensional image space. The maximum path length is set; the path direction change angle is set to be less than 45°, and if it is greater than 45°, the direction consistency screening is performed to prevent the path from crossing non-target tissues; the output is the central path point set of the hepatobiliary duct, including a three-dimensional path coordinate sequence, representing the trunk and its branch structure.

[0060] The connectivity of the hepatobiliary duct tracking data is optimized to obtain the hepatobiliary duct positioning data.

[0061] In one embodiment, the following attribute analysis is performed on all paths to calculate the length and number of branches of all tracking paths; sub-branches with a length of <20px or not connected to the main trunk are deleted; a distance transformation map of the path point set is generated based on the closed completion of the distance map; a morphological closing operation is performed on path breakpoints with a distance less than 3px; Skeletonize 3D is used to extract the skeleton centerline; BFS is used to merge all branches into a main structure tree to obtain the hepatobiliary duct positioning data.

[0062] Preferably, performing improved contrast extraction on the hepatic artery region data to obtain hepatic artery positioning data includes: Direction-selective enhanced kernel processing was performed on the hepatic artery area data to obtain the rough positioning data of the first hepatic artery; In one embodiment, the input is an enhanced CT image (arterial phase or equilibrium phase) that has been cropped to the liver region. Construct a set of 3D convolution kernels with different spatial orientations , whose direction set is defined as: ,in For Axially is the offset angle, , For direction The voxel response map below, For direction The corresponding convolution kernel is For the input 3D image volume, is a direction in the direction set, is a set of directions; each voxel takes the maximum response value in all directions as the enhancement value: , The maximum response map is the maximum response value of each voxel in all directions, representing the strongest directional enhancement intensity, which is used for hepatic artery enhancement positioning. It is a single direction response graph, the image is in the direction The convolution response value on reflects the strength of the blood vessels or edge structures in that direction. is one of the directions in the set (e.g. +30° along the Z axis), is the maximization function. At the same time, the direction number with the largest response is recorded: , is the optimal response direction index of the voxel, The optimal direction index map function is used to find the direction number of each voxel with the largest response value in all directions. It is a single direction response graph, the image is in the direction The convolution response value on reflects the strength of the blood vessels or edge structures in this direction; the rough positioning data of the first hepatic artery obtained includes .

[0063] The edge sensitivity of the hepatic artery area data is adjusted to obtain the rough positioning data of the second hepatic artery; In one embodiment, a Laplacian of Gaussian (LoG) filter or a Sobel operator combination is applied to the original image to extract edge responses: , It is the edge response map / gradient amplitude map, which reflects the intensity change degree of the image at each voxel point. The larger it is, the more obvious the edge is. For The image gradient component in the direction, For The image gradient component in the direction, For The image gradient component in the direction. Combined with local contrast analysis (local window standard deviation and local mean, the window size is or ) to generate a comparison chart: , It is a local contrast map, which indicates the texture contrast intensity of the local area. The larger the value, the more obvious the regional contrast. is the local standard deviation, and the local area centered on the current voxel (such as or ) reflects the grayscale standard deviation within the range of texture variation. is the local mean, the mean gray value in the same area, used to normalize the contrast intensity, To prevent zero offset, a small constant is used to prevent the denominator from being zero. The edge sensitivity map obtained through the above is: , It is an edge sensitivity map, which fuses the structural response map of edge intensity and local contrast. It is often used to enhance small blood vessels and low-contrast blood vessel areas. is the edge response weight coefficient, which controls the proportion of edge strength in fusion and is set to 0.6. is the edge response diagram, is the contrast weight coefficient, which controls the weight of local contrast in fusion and is set to 0.4. For the local contrast map, set , .

[0064] Attention perception enhancement is performed according to the rough positioning data of the first hepatic artery and the rough positioning data of the second hepatic artery, so as to obtain weighted positioning data of the first hepatic artery and weighted positioning data of the second hepatic artery respectively; In one embodiment, for the rough positioning data of the first hepatic artery and the second hepatic artery rough positioning data , respectively construct channel attention modules: , is the channel attention factor (vector), with a size of , used to adjust the response strength of each channel, is the Sigmoid activation function, with an output range of (0,1). For the input feature map In the spatial dimension Average and output channel-level statistics with dimensions , To compress the fully connected layer, compress the channels to C / r (e.g. r=8), is a nonlinear activation function, To expand the fully connected layer, expand back to the original number of channels ; Use this attention factor to regulate the response graph itself: , , is the weighted response map of the first hepatic artery after adding channel attention, For The calculated channel attention factor, is the rough positioning data of the first hepatic artery, is the weighted response map of the second hepatic artery after adding channel attention, For The calculated channel attention factor, This is the rough positioning data of the second hepatic artery. Add spatial attention: , is a spatial attention map that outputs a weight factor (range [0,1]) for each voxel site. It is a channel fusion convolution layer that outputs a spatial weight map of 1 channel. To concatenate two pooling images in the channel dimension, generate Feature map, To be the channel dimension Do average pooling, To be the channel dimension Do the maximum pooling. The weighted response map that simultaneously adds channel and spatial attention is used as the weighted positioning data of the first hepatic artery. For The calculated channel attention factor, The spatial attention map generated for the first localization map, is the rough positioning data of the first hepatic artery, The weighted response map that simultaneously adds channel and spatial attention is used as the weighted positioning data of the second hepatic artery. For The calculated channel attention factor, The spatial attention map generated for the second localization map, This is the rough positioning data of the second hepatic artery.

[0065] Confidence voting is performed based on the first hepatic artery weighted positioning data and the second hepatic artery weighted positioning data to obtain hepatic artery fusion positioning data, and centerline extraction is performed based on the hepatic artery fusion positioning data to obtain hepatic artery positioning data.

[0066] In one embodiment, , is the fused positioning map (confidence fusion result), is the dynamic normalized weight of the first graph, A weighted localization map of the first hepatic artery (such as an edge perception map or a direction consistency map) is used. is the dynamic normalized weight of the second graph, weighted localization map for the second hepatic artery (e.g., orientation perception map or attention output map); , , is the dynamic normalized weight of the first graph, is the dynamic normalized weight of the second graph, A weighted localization map of the first hepatic artery (such as an edge perception map or a direction consistency map) is used. A weighted localization map of the second hepatic artery (such as a directional perception map or an attention output map) is provided. To prevent small constants from dividing by zero, take values ​​such as ; Fusion graph Set the confidence threshold (such as ), generate a binary mask map: , is a mask image, indicating the credible hepatic artery structure area. is the confidence threshold. Apply the 3D skeletonization method to extract the centerline skeleton, or use the medial axis algorithm in SimpleITK or VTK; further perform path optimization, delete branches with a length of <10px; use the minimum path cost (such as Fast Marching) to complete the interruption points and obtain the hepatic artery positioning data.

[0067] Preferably, the performing directional segmentation analysis on the portal vein region data to obtain portal vein positioning data includes: Calculating the main directional gradient map of the portal vein area data to obtain the main directional gradient map data; In one embodiment, the portal vein region image Perform first-order differentiation in three dimensions (Sobel, Prewitt, or Scharr can be used): , is the gradient vector, the three-dimensional gradient vector of the current voxel, is the portal vein area data, is the voxel in the 3D image Location, is the voxel in the 3D image Location, is the voxel in the 3D image Position. A window around each voxel (e.g. ), calculate the structure tensor , is the local structure tensor, which represents the directional structure covariance of the current voxel in the neighborhood. is the index of the neighborhood index voxel, the index of each voxel coordinate in the local window involved in the calculation, is the neighborhood index voxel, and the coordinates of each voxel in the local window involved in the calculation, is the gradient vector, the three-dimensional gradient vector of the current voxel, is the transpose of the gradient vector, the transpose of the three-dimensional gradient vector of the current voxel, Find the eigendecomposition and select the eigenvector corresponding to the maximum eigenvalue as the main direction vector. Get the main direction gradient map, with one direction vector (unit vector) for each voxel.

[0068] Performing direction-guided segmentation according to the main direction gradient map data to obtain direction-guided segmentation data; In one embodiment, an initial seed point set is set near the portal vein trunk or the liver portal. ; The seed point can be set by anatomical rules, manually selected by experts, or automatically detected by the front module. For each candidate extension point , determine whether it meets the following three conditions in the 26-neighborhood: Voxel point The main direction vector comes from the main direction gradient map, Voxel point The main direction vector comes from the main direction gradient map, is the direction similarity threshold. The larger the value, the stricter the requirement for direction consistency. , Indicates the main direction vector of the point. The larger the threshold, the stricter the direction. ,in is the set grayscale threshold; , is the spatial distance function, is the Euclidean distance, is the maximum expansion distance threshold to prevent jumping or wrong connection. The setting range is 1.5 to 2 px to avoid jumping in the diffusion path. If all the above conditions are met, Add to torrent collection , and continue to expand to its neighborhood until no new points can be added.

[0069] Branches are merged and breaks are repaired according to the direction-guided segmentation data to obtain portal vein positioning data.

[0070] In one embodiment, branch merging includes extracting all connected domains, and for branches with shorter lengths (e.g., <20px), searching in space for: the Euclidean distance between the heads and tails of adjacent branches , is the Euclidean distance threshold (such as 5-10px), which is used to determine whether two branches are spatially adjacent; the main direction vector angle ; If satisfied, use the shortest path to connect (using Fast Marching or shortest graph algorithm); The break repair includes performing a 3D distance transformation D(x, y, z) on the output 3D mask; calculating the skeleton and performing a "bridging" operation on the skeleton breakpoints, finding the connectable area in the distance map, connecting the breakpoints with the minimum cost path, and obtaining the portal vein positioning data.

[0071] Preferably, extracting the branch angles of the hepatic vein region data and the portal vein positioning data to obtain the hepatic vein positioning data includes: The center line is extracted according to the portal vein positioning data to obtain the portal vein direction map data; In one embodiment, based on the portal vein three-dimensional mask image, a three-dimensional skeleton extraction algorithm (such as Medial AxisTransform or skeletonize_3d) is used to extract a skeleton point set, which is expressed as: , is the portal vein skeleton point set, is the first skeleton point, is the second skeleton point, For the Skeleton points, is the total number of skeleton points; for each skeleton point , take a symmetric window in its neighborhood (such as the front and back points), define the neighborhood point set as: , For point The neighborhood set of is the starting point of the neighborhood boundary point, is the center store of the neighborhood point set, is the end point of the neighborhood boundary point; in this neighborhood, a principal direction straight line is fitted using the least squares fitting or principal component analysis (PCA) method. The principal direction vector of the fitting result is recorded as: , For point The main direction vector of The main direction unit vector obtained by fitting the skeleton points in the neighborhood reflects the direction of the skeleton segment where the point is located. The principal direction extraction function obtains the first principal component (maximum variance direction) based on least squares straight line fitting or principal component analysis (PCA); after performing the above operations on all skeleton points, each principal direction vector is assigned to the corresponding voxel position to construct a three-dimensional sparse vector map ,in The portal vein direction map data is obtained, including the portal vein skeleton line point set and the direction vector of each skeleton point.

[0072] Perform skeleton extraction based on the hepatic vein area data to obtain the hepatic vein branch data; In one embodiment, 3D skeleton extraction is performed on the hepatic vein region data (binary mask); all connected regions are retained; all branch line segments are extracted (split by breakpoints); and the starting and ending coordinates of each branch are recorded as a point set sequence.

[0073] Linear fitting is performed on the hepatic vein branch data to obtain the hepatic vein branch fitting data, and the angle of the nearest branch direction is calculated based on the hepatic vein branch fitting data to obtain the branch angle characteristic map data; In one embodiment, for each point set of the hepatic vein branch, a linear least squares fitting or PCA is used to output the main direction vector; the direction vector closest to the branch is searched in the portal vein direction map, such as using the nearest point search method, performing KD tree or spatial indexing, and limiting the maximum radius, such as 10-15px; and the cosine value of the direction angle is calculated: , is the angle between the two vectors (degrees or radians), is the hepatic vein fitting direction vector, is the portal vein reference direction vector, is the modulus of the hepatic vein fitting direction vector, is the modulus of the portal vein reference direction vector; all angles are marked on the hepatic vein branch fitting data to obtain the branch angle characteristic map data.

[0074] Narrow neural network classification is performed based on the portal vein direction map data and branch angle feature map data to obtain the hepatic vein positioning data.

[0075] In one embodiment, narrow neural network classification refers to the use of a shallow neural network structure with a small number of parameters and a low feature dimension to complete the classification task. The width is no more than 16 and the depth is no more than 16 layers. Based on the feature classification of the narrow neural network, hepatic vein positioning data is generated. The classification model used is a lightweight narrow channel neural network (NarrowNet), which is used for voxel-level or branch-level structure recognition in a low-dimensional feature space. The network input features include the three-dimensional spatial position coordinates of each candidate point ; Cosine value of the angle between the portal vein and the , or a direction vector pair The neural network structure includes an input layer that accepts feature vectors of dimension 4; the first hidden layer contains 16 neurons, and the activation function is ReLU; the second hidden layer (Dense2) contains 8 neurons, and the activation function is ReLU; the output layer (Dense3) contains 1 neuron, uses the Sigmoid activation function, and outputs the probability value of the hepatic vein structure. The network training uses the binary cross entropy loss function (BinaryCrossEntropy), with the manually annotated real hepatic vein voxels as positive samples, and non-structural voxels or pseudo-vessels as negative samples. The network output is the classification confidence of each point , and the threshold As the classification standard, the real hepatic vein area is screened out. The output result is the hepatic vein positioning mask map, that is, the hepatic vein positioning data.

[0076] Preferably, performing elastic registration according to the hepatobiliary duct positioning data, the hepatic artery positioning data, the portal vein positioning data and the hepatic vein positioning data to obtain the elastic registration data includes: S31, determining the hepatic artery positioning data as reference structure data; In one embodiment, the hepatic artery has a stable structure, small anatomical differences, and a concentrated direction; the arterial phase image in contrast CT is clear and has a small error; the coordinate system of the hepatic artery centerline or three-dimensional mask is set as the registration reference coordinate system; all target registration structures (hepatobiliary duct, portal vein, hepatic vein) need to be converted to this coordinate system.

[0077] S32, performing inter-structure pairing on the hepatobiliary duct positioning data, the portal vein positioning data, and the hepatic vein positioning data according to the reference structure data to obtain inter-structure pairing data; In one embodiment, matching point pairs are extracted from each structure to provide control constraints for elastic registration, and the centerline point set of the reference structure (hepatic artery) is extracted. ; For each target structure (such as portal vein): extract its centerline point set , calculate the nearest neighbor pairs between point sets: , For the reference point The nearest target point , To find the variable that minimizes a function (for nearest neighbors), is the set of centerline points of any target structure, For the Hepatic artery centerline points, is the centerline point of any target structure; , is a point pair set, representing the set of all valid paired points, is the first point in the centerline point set (reference structure) of the hepatic artery reference structure. Points, is the centerline point set of the portal vein (one of the target structures). Points, is the first point in the centerline point set (reference structure) of the hepatic artery reference structure. Points, is the centerline point set of the hepatobiliary duct (one of the target structures). points; filter the point pairs to obtain the structure pairing data, such as setting the maximum distance threshold (such as <20px) and adding direction consistency (angle <30°).

[0078] In one embodiment, a tree-like map is constructed for the structure; the trunk → first-level branch → second-level branch (recursive structural constraint) is matched to obtain the paired data between structures. In the structural pairing, the trunk pairing is prioritized, and then the nearest neighbor and consistency matching are performed on the first-level branch and the second-level branch in a hierarchical recursive manner to achieve the consistency and topological rationality of the anatomical structure.

[0079] S33, performing control point network coverage according to the inter-structure pairing data to obtain elastic registration data.

[0080] In one embodiment, the image / structure is covered with a grid of control points with a spacing of (or px number), the number of control points is automatically adjusted according to the structure density; the paired point set is used as the constraint driver to guide the elastic deformation of the control point grid, and the deformation field is defined as , represents the voxel The mapping position after deformation. Construct an optimization objective function to achieve deformation field formation: , is the total registration error loss, is a set of structural paired points, representing the key points in the source structure Corresponding points in the target structure A set of paired points is formed, which is used as the registration anchor point. is the position after deformation, point In the current deformation field The mapping position under The spline deformed mesh is interpolated to obtain, is the regularization coefficient, which controls the weight of the deformation smoothing term in the total loss function and is set to , the larger the value, the more it tends to keep the deformation smooth. For the regularization term / deformation smoothness, the smoothness and continuity of the entire deformation field are constrained, using Bending Energy or first / second order derivative penalty terms; the optimization objective function consists of two parts, the paired point reprojection error term (data fidelity) and the smooth regularization term (controlling the local continuity of the registration), and the optimization process is achieved through numerical optimization algorithms such as gradient descent and LBFGS. The output three-dimensional non-rigid deformation field is used for visualization tasks.

[0081] Preferably, a three-dimensional hepatobiliary, hepatic artery, portal vein and hepatic vein image segmentation system is characterized in that it is used to perform the three-dimensional hepatobiliary, hepatic artery, portal vein and hepatic vein image segmentation method as described above, and the three-dimensional hepatobiliary, hepatic artery, portal vein and hepatic vein image segmentation system comprises: A multi-phase image acquisition module, used for acquiring multi-phase medical image data of a liver region; A region recognition module is used to perform region recognition on multi-phase medical image data, and obtain hepatobiliary region data, hepatic artery region data, portal vein region data, and hepatic vein region data respectively; The structural association positioning module is used to perform structural tracking on the hepatic artery area data to obtain the hepatic bile duct positioning data; perform improved contrast extraction on the hepatic artery area data to obtain the hepatic artery positioning data; perform directional segmentation analysis on the portal vein area data to obtain the portal vein positioning data; perform branch angle extraction on the hepatic vein area data and the portal vein positioning data to obtain the hepatic vein positioning data; The three-dimensional structure reconstruction and display module is used to perform elastic registration based on the hepatobiliary duct positioning data, the hepatic artery positioning data, the portal vein positioning data and the hepatic vein positioning data to obtain elastic registration data; and perform three-dimensional visualization operations based on the elastic registration data.

[0082] Therefore, from any point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is limited by the attached application documents rather than the above description, and it is intended that all changes falling within the meaning and scope of equivalent elements of the application documents are included in the present invention.

[0083] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A three-dimensional hepatobiliary, hepatic artery, portal vein and hepatic vein image segmentation method, characterized in that: The following steps are involved: Acquiring multi-phase medical image data of a liver region; Performing regional recognition on multi-phase medical image data to obtain hepatobiliary regional data, hepatic artery regional data, portal vein regional data and hepatic vein regional data; Perform structural tracking on the hepatobiliary region data to obtain hepatobiliary location data; perform improved contrast extraction on the hepatic artery region data to obtain hepatic artery location data; perform directional segmentation analysis on the portal vein region data to obtain portal vein location data; perform branch angle extraction on the hepatic vein region data and portal vein location data to obtain hepatic vein location data; Perform elastic registration according to the hepatobiliary duct positioning data, the hepatic artery positioning data, the portal vein positioning data and the hepatic vein positioning data to obtain elastic registration data; Perform 3D visualization based on elastic registration data.

2. The method according to claim 1, characterized in that The multi-phase medical image data includes arterial phase CT image data, portal venous phase CT image data, equilibrium phase CT image data and T2-weighted MRI image data. The multi-phase medical image data is subjected to regional recognition to obtain hepatobiliary regional data, hepatic artery regional data, portal vein regional data and hepatic vein regional data, respectively, including: Performing rough segmentation of the liver region on the portal venous phase CT image data to obtain liver region data; Determine the hepatic artery region according to the liver region data and the arterial phase CT image data to obtain the hepatic artery region data; Hepatic vein region data is identified according to liver region data and equilibrium phase CT image data to obtain hepatic vein region data; Directionally guided segmentation is performed based on the liver region data to obtain portal vein region data; The hepatobiliary region is identified based on the liver region data and the T2-weighted MRI image data to obtain the hepatobiliary region data.

3. The method according to claim 2, characterized in that The step of performing directionally guided segmentation based on the liver region data to obtain portal vein region data includes: Extract voxel grayscale gradient according to liver region data to obtain voxel grayscale gradient data; Perform main direction clustering on the voxel gray gradient data to obtain main direction clustering data; Perform tensor voting processing based on the main direction clustering data to obtain the blood vessel direction field data; The preset direction-guided segmentation network model is used to segment the vascular direction field data to obtain the portal vein area data, wherein the direction-guided segmentation network model is constructed by using the improved SD-UNet to construct the backbone network, and the attention map or the guided flow is constructed as the output head.

4. The method according to claim 3, characterized in that The steps for constructing the preset direction-guided segmentation network model include the following steps: Obtain historical vascular direction field data and historical annotation data; Perform multi-scale exaggerated convolution parallel processing based on historical vascular direction field data to obtain edge perception layer data; The edge perception layer data is subjected to shallow edge layer processing and high-level feature layer processing to obtain shallow edge layer data and high-level feature layer data respectively, wherein the shallow edge layer processing realizes boundary response enhancement through Sobel / LoG filter or edge channel convolution; Perform residual connection on shallow edge layer data and high-level feature layer data to obtain decoding layer data; The vascular direction network is trained on the decoded layer data using historical annotation data to obtain direction-guided flow data. The vascular direction network training supervises the convolutional network by using the direction consistency loss combined with the Dice loss. The direction-guided flow data is directionally guided to output, and a direction-guided segmentation network model is obtained, wherein the direction-guided output is convolutionally fused with the decoding layer data and the direction confidence is calculated, or the direction-guided attention map is calculated through the direction-guided flow data, and the output result is regulated at the channel level.

5. The method according to claim 1, characterized in that The structural tracking of the hepatobiliary region data to obtain the hepatobiliary positioning data includes: Constructing the spatial guidance domain of the hepatobiliary region data to obtain the hepatobiliary spatial guidance domain data; Perform local directional consistency enhancement based on the hepatobiliary spatial guidance domain data to obtain hepatobiliary enhancement data; Performing path tracing according to the hepatobiliary enhancement data to obtain hepatobiliary duct tracing data, wherein the path tracing is to construct a cost function based on signal strength, direction consistency and path proximity to perform minimum cost path tracing; The connectivity of the hepatobiliary duct tracking data is optimized to obtain the hepatobiliary duct positioning data.

6. The method according to claim 1, characterized in that The improved contrast extraction is performed on the hepatic artery region data to obtain the hepatic artery positioning data, including: Direction-selective enhanced kernel processing was performed on the hepatic artery area data to obtain the rough positioning data of the first hepatic artery; The edge sensitivity of the hepatic artery area data is adjusted to obtain the rough positioning data of the second hepatic artery; Attention perception enhancement is performed according to the rough positioning data of the first hepatic artery and the rough positioning data of the second hepatic artery, so as to obtain weighted positioning data of the first hepatic artery and weighted positioning data of the second hepatic artery respectively; Confidence voting is performed based on the first hepatic artery weighted positioning data and the second hepatic artery weighted positioning data to obtain hepatic artery fusion positioning data, and centerline extraction is performed based on the hepatic artery fusion positioning data to obtain hepatic artery positioning data.

7. The method according to claim 1, characterized in that The directional segmentation analysis of the portal vein area data to obtain the portal vein positioning data includes: Calculating the main directional gradient map of the portal vein area data to obtain the main directional gradient map data; Performing direction-guided segmentation according to the main direction gradient map data to obtain direction-guided segmentation data; Branches are merged and breaks are repaired according to the direction-guided segmentation data to obtain portal vein positioning data.

8. The method according to claim 1, characterized in that: The extracting of branch angles of the hepatic vein region data and the portal vein positioning data to obtain the hepatic vein positioning data includes: The center line is extracted according to the portal vein positioning data to obtain the portal vein direction map data; Perform skeleton extraction based on the hepatic vein area data to obtain the hepatic vein branch data; Linear fitting is performed on the hepatic vein branch data to obtain the hepatic vein branch fitting data, and the angle of the nearest branch direction is calculated based on the hepatic vein branch fitting data to obtain the branch angle characteristic map data; Narrow neural network classification is performed based on the portal vein direction map data and branch angle feature map data to obtain the hepatic vein positioning data.

9. The method according to claim 1, characterized in that: The elastic registration is performed according to the hepatobiliary duct positioning data, the hepatic artery positioning data, the portal vein positioning data and the hepatic vein positioning data to obtain the elastic registration data, including: Determine the hepatic artery positioning data as reference structure data; According to the reference structure data, the hepatobiliary duct positioning data, the portal vein positioning data and the hepatic vein positioning data are paired between structures to generate inter-structure paired data; The control point network is covered according to the paired data between structures to obtain the elastic registration data.

10. A three-dimensional hepatobiliary, hepatic artery, portal vein and hepatic vein image segmentation system, characterized in that: For executing the three-dimensional hepatobiliary, hepatic artery, portal vein and hepatic vein image segmentation method as claimed in claim 1, the three-dimensional hepatobiliary, hepatic artery, portal vein and hepatic vein image segmentation system comprises: A multi-phase image acquisition module, used for acquiring multi-phase medical image data of a liver region; A region recognition module is used to perform region recognition on multi-phase medical image data, and obtain hepatobiliary region data, hepatic artery region data, portal vein region data, and hepatic vein region data respectively; The structural association positioning module is used to perform structural tracking on the hepatic artery area data to obtain the hepatic bile duct positioning data; perform improved contrast extraction on the hepatic artery area data to obtain the hepatic artery positioning data; perform directional segmentation analysis on the portal vein area data to obtain the portal vein positioning data; perform branch angle extraction on the hepatic vein area data and the portal vein positioning data to obtain the hepatic vein positioning data; The three-dimensional structure reconstruction and display module is used to perform elastic registration based on the hepatobiliary duct positioning data, the hepatic artery positioning data, the portal vein positioning data and the hepatic vein positioning data to obtain elastic registration data; and perform three-dimensional visualization operations based on the elastic registration data.

Images