Reconstruction method for three-dimensional images of the liver, gallbladder and pancreas

Abstract

The present invention relates to a method for reconstructing three-dimensional images of the hepatobiliary and pancreatic organs; the technical solution adopted by the present invention includes the following steps: introducing a segmentation network of a U-Net variant for the three organs of the liver, gallbladder, and pancreas, and using a convolutional neural network to segment multimodal images, including separating the liver, bile duct, and pancreas; introducing a boundary enhancement module during the segmentation process; adopting a deep learning-driven non-rigid registration algorithm to spatially align different-modal imaging data; using the anatomical feature points of the organs for registration; according to the registration results, performing multimodal fusion on the liver, gallbladder, and pancreas images, comprehensively utilizing the bone and tissue contrast of CT, the soft tissue resolution of MRI, and the real-time dynamic characteristics of ultrasound to construct fused imaging data; extracting the contours of the hepatobiliary and pancreatic organs from the segmented images to construct a preliminary three-dimensional grid model; adaptively adjusting according to the geometric shapes of different organs to control the distribution of the grid during reconstruction to match the organ morphology.

Description

Technical Field

[0001] The present invention relates to a method for reconstructing three-dimensional images, specifically a method for reconstructing three-dimensional images of the liver, gallbladder, and pancreas. Background Art

[0002] The technology described in Chinese Patent Publication No. CN117529273A mainly uses the combination of ultrasonic images and camera images to reconstruct three-dimensional images. This method depicts blood vessels and blood flow by acquiring Doppler ultrasonic images, combines with the 2D images captured by the camera, and calculates the attitude of the ultrasonic transducer and the 3D coordinates of the ultrasonic image in the world coordinate system by analyzing the position changes of the reference object, thereby realizing three-dimensional reconstruction.

[0003] However, the application of this method in the three-dimensional reconstruction of the liver, gallbladder, and pancreas has many drawbacks, mainly reflected in the following aspects: First, the disclosed technical method relies on the combination of Doppler ultrasound and camera images, and the reconstruction process is mainly aimed at the imaging of blood vessels and blood flow, and the reconstruction ability for solid organs such as the liver, gallbladder, and pancreas is limited. Doppler ultrasound images can well show the blood flow and the course of blood vessels, but the performance of the boundaries and internal structures of solid tissues such as the liver, gallbladder, and pancreas is poor, and the imaging quality depends on the penetration and reflection characteristics of ultrasound. The 2D images captured by the camera lack the ability to image deep structures and are mainly used for the positioning and attitude correction of surface features. Due to the limitations of these two imaging modes, the complex anatomical structures of the liver, gallbladder, and pancreas cannot be accurately restored in practical applications. Especially when facing the complex-shaped bile ducts and pancreatic ducts, the reconstructed three-dimensional images lack detailed expression and accuracy guarantee.

[0004] Secondly, the technical solution of the disclosed technology highly depends on the external reference of the reference object. By tracking the position change of the reference object through a camera to calculate the pose of the ultrasonic transducer, the accuracy of this method is limited by the distribution and recognition accuracy of the reference object. Once the position of the reference object is disturbed or misrecognized, it will lead to deviations in the entire 3D reconstruction. Especially in the hepatobiliary and pancreatic regions, the surrounding structures are complex and tissue movement is frequent, which is easily affected by physiological movements such as breathing and heartbeat, thus affecting the stability of the reference object and further reducing the reconstruction accuracy and reliability of the 3D image. In addition, since this method is based on the surface analysis of camera and ultrasonic images, it lacks in-depth measurement of internal soft tissues and cannot accurately distinguish the subtle differences between diseased tissues and normal tissues, providing insufficient support for the pathological diagnosis of the hepatobiliary and pancreas. Thirdly, the Doppler ultrasonic images of the disclosed technology mainly depict blood flow and vascular structures. The reconstructed 3D images tend to be vascular anatomical images and blood flow distributions, and do not provide high-resolution texture information and detail enhancement for parenchymal tissues, resulting in insufficient recognition ability for subtle lesions such as micro nodules and fibrosis in the hepatobiliary and pancreas. The 3D reconstruction of organs such as the hepatobiliary and pancreas not only needs to present the external shape and vascular network, but also needs to deeply express internal lesions and tissue characteristics, such as the location of tumors inside the liver, the degree of stenosis of bile ducts, and the distribution of fibrosis in the pancreas. These information are extremely crucial for clinical diagnosis and surgical planning, but perform poorly in the current Doppler ultrasonic technology. Thirdly, during the reconstruction process of this method, image reconstruction needs to be achieved through the coordinate transformation and calculation of multiple pixel points in 3D space. Such a reconstruction process often requires complex calculations and a large amount of image data matching, with a large amount of calculation and insufficient real-time performance. Especially when processing high-resolution images, the calculation speed is slow and it is difficult to meet the requirements of real-time clinical applications.

[0005] In addition, since the reconstruction process involves the calibration and integration of multiple data sources, the complexity of the algorithm increases the probability of errors and is prone to data loss or mismatch during the integration process, affecting the accuracy of the reconstruction results. Finally, the reconstruction method of the disclosed technology lacks the processing of enhancing the realism and texture mapping of the reconstructed images. Although it can display blood vessel and blood flow information, in practical applications, the visual effect of the 3D reconstruction images is lacking and cannot provide clinicians with an intuitive and clear display of the internal structure of organs. Summary of the Invention

[0006] The object of the present invention is to provide a method for reconstructing 3D images of the hepatobiliary and pancreas, so as to solve some of the drawbacks pointed out in the background technology.

[0007] The technical solutions adopted by the present invention to solve its above technical problems include the following steps:

[0008] S1. Hepatobiliary and pancreatic image segmentation assisted by deep learning:

[0009] S1.1. Introduce a segmentation network of the U-Net variant for the liver, gallbladder, and pancreas, and use a convolutional neural network to segment multi-modal images, including the separation of the liver, bile duct, and pancreas;

[0010] S1.2. Introduce a boundary enhancement module during the segmentation process;

[0011] S2. Image registration and paired fusion:

[0012] S2.1. Adopt a deep learning-driven non-rigid registration algorithm to spatially align different-modal imaging data; use the anatomical feature points of the organs for registration;

[0013] S2.2. According to the registration results, perform multi-modal fusion on the liver, gallbladder, and pancreas images, and comprehensively utilize the bone and tissue contrast of CT, the soft tissue resolution of MRI, and the real-time dynamic characteristics of ultrasound to construct fused imaging data;

[0014] S3. Adaptive three-dimensional mesh reconstruction:

[0015] S3.1. Extract the contours of the liver, gallbladder, and pancreas from the segmented images to construct a preliminary three-dimensional mesh model; adaptively adjust according to the geometric shapes of different organs to control the distribution of the mesh during reconstruction to match the organ shapes;

[0016] S3.2. Adopt an iterative optimization algorithm to adjust the mesh vertices including the bile duct system and pancreatic duct to fit the actual organ shape;

[0017] S4. Personalized texture mapping and realism enhancement:

[0018] S4.1. Use the fused imaging data to extract the respective texture features of the liver, gallbladder, and pancreas, including tissue density, lesion areas, and blood vessel distributions;

[0019] S4.2. Map the extracted texture features to the surface of the three-dimensional model and dynamically adjust the texture according to individual differences including tumor location and fibrosis degree.

[0020] Furthermore, the non-rigid registration algorithm includes: using a convolutional neural network CNN combined with an attention mechanism and multi-scale feature extraction to identify and locate the organ edges, blood vessel intersections, and lesion area feature points in the images; through an optimized matching algorithm, align the corresponding feature points in different-modal images:

[0021]

[0022] where J match represents the sum of the feature point matching errors, N is the number of feature points; x i and y iThey are the corresponding feature point coordinates in different modality images, representing the position difference; α and β are adjustment coefficients, controlling the weights of different error terms; |·| represents the Euclidean distance, reflecting the distance error between feature points; and They are the gradient information at the feature points, representing the change of local structural features.

[0023] Furthermore, the non-rigid registration algorithm includes: through end-to-end training, directly learning the mapping relationship between images, with the input being unregistered multi-modal images and the output being the aligned images; defining a composite loss function to measure the similarity of the overall image, combining the matching error of anatomical feature points to control the data retention of anatomical structures during the registration process:

[0024]

[0025] where L is the total loss function, measuring the overall quality of registration; λ1 and λ2 are loss weight parameters, used to adjust the balance between pixel-level similarity and feature point matching error; T(x) represents the pixel value of the registered target image, R(x) is the pixel value of the reference image, and (T(x) - R(x)) 2 measures the intensity difference between pixels; Ω is the domain of the image, which is the region of the entire image; J match is the aforementioned feature point matching error part.

[0026] Furthermore, the non-rigid registration algorithm includes: introducing a multi-level pyramid strategy during the registration process, dividing the registration process into multiple resolution levels, and gradually optimizing from low resolution to high resolution; in each pyramid level, generating a preliminary spatial deformation field through deep learning and then performing multiple iterative optimizations to refine the deformation field; to optimize the spatial deformation field, introducing an optimization function to constrain the continuity and smoothness of the deformation:

[0027]

[0028] where is the optimization objective function, measuring the smoothness and structural consistency of the deformation field; φ(x) is the current spatial deformation field function, representing the deformation degree of the image during the current registration process; φ prev (x) is the deformation field of the previous iteration, ensuring the continuity and consistency of the deformation field; κ and ζ are adjustment coefficients, controlling the weights of different error terms; is the gradient magnitude of the deformation field, measuring the local smoothness; ||φ(x) - φ prev (x)|| represents the difference between the current and the previous deformation fields.

[0029] Furthermore, the method for constructing a preliminary three-dimensional mesh model includes: using a deep learning enhanced model to extract boundary data; and converting the extracted contour data into a dense point cloud, where the density of the point cloud is dynamically adjusted according to the complexity of the organ structure to capture the geometric features of each organ:

[0030]

[0031] Among them, E contour represents the energy function for contour extraction, Ω is the domain of the image, representing the range of contour calculation; is the gradient of the image, reflecting the intensity change of the image edge and capturing the sharpness of the boundary; C(x) is the extracted contour function, representing the position of the contour on the image; represents the second derivative of the contour, used to evaluate the smoothness of the contour; α and β are adjustment parameters, controlling the contribution weights of the gradient and smoothness to the overall contour.

[0032] Furthermore, the method for constructing a preliminary three-dimensional mesh model includes: generating a three-dimensional mesh from the dense point cloud data, and converting the point cloud into a triangular mesh using the Delaunay triangulation or Marching Cubes algorithm;

[0033] Among them, during the mesh generation process, the size of the mesh cells is adjusted according to the local density and geometric shape of the point cloud, so as to maintain a high mesh density in the morphologically complex regions including curved bile ducts and pancreatic ducts, while reducing the mesh cell density in the smooth regions including the liver surface; then geometric analysis is performed on the generated mesh to evaluate the shape characteristics of each mesh cell, including curvature, side length, and angle:

[0034]

[0035] Among them, E mesh is the mesh quality evaluation function, used to measure the rationality and detail capture ability of the mesh; κ(x) is the local curvature, describing the degree of bending of the mesh cell surface, especially reflecting the morphological changes in complex regions; L(x) is the side length of the mesh cell, controlling the size and morphological consistency of the mesh cell; θ is the local angle, representing the connection angle between mesh cells, used to detect the smooth transition of the mesh; γ and δ are adjustment parameters, adjusting the influence weights of curvature and side length on the mesh quality.

[0036] Furthermore, the method for constructing a preliminary three-dimensional mesh model includes: adopting an adaptive mesh optimization algorithm to dynamically adjust the mesh cell distribution according to the geometric features of the organ; among them, the adjustment of the mesh density is based on local morphological changes, including increasing the mesh density in high-curvature regions; applying the Laplacian smoothing and deformation remapping adaptive algorithm to ensure that the mesh adjustment retains the shape and boundary features of the original organ:

[0037]

[0038] Among them, E adjust represents the energy function for adaptive adjustment, which measures the rationality of density and morphological changes during the grid adjustment process; λ is the adjustment coefficient that controls the sensitivity of density adjustment; κ thresh is the curvature threshold that defines the trigger condition for high-density adjustment; φ(x) is the grid deformation field function that describes the displacement of the grid during the adjustment process; is the second derivative of the deformation field, which is used to control the smoothness and flatness of the grid.

[0039] Furthermore, the method for constructing the preliminary three-dimensional grid model includes: introducing a deformation control technology based on energy minimization; by optimizing the energy function, maintaining the morphological consistency and boundary smoothness of grid cells during the adjustment process, and controlling the grid deformation in complex regions:

[0040]

[0041] Among them, E deform is the energy function for grid deformation control, which measures the morphological changes of the grid during the adjustment process; μ and ν are adjustment parameters that respectively control the contributions of deformation intensity and boundary smoothness; is the square term of the gradient of the deformation field, which is used to smooth the deformation of the grid; θ edge is the local angle of the grid boundary, which represents the adjustment of the grid at the boundary.

[0042] The three-dimensional image reconstruction method for the hepatobiliary and pancreatic organs of the present invention realizes the high-precision reconstruction of the complex anatomical structures of the hepatobiliary and pancreatic organs through the integration and optimization of multiple innovative technologies, and has the following beneficial effects:

[0043] Through the deep learning-driven non-rigid registration algorithm, combined with the multi-level pyramid strategy and the optimized loss function, the spatial alignment problem between multi-modal images such as CT, MRI, and ultrasound is effectively solved. During the registration process, the anatomical feature points of the organs are used for precise matching, which not only improves the overall registration accuracy but also maintains a high degree of consistency in anatomical structures in details, providing a high-quality data basis for subsequent three-dimensional reconstruction.

[0044] During the construction of the three-dimensional grid model, an adaptive grid optimization algorithm is adopted to dynamically adjust the distribution of grid cells according to the geometric characteristics of the organs, so that a high grid density is maintained in morphologically complex regions such as bile ducts and pancreatic ducts, while the grid density is reduced in smooth regions. This density adjustment method not only improves the expression accuracy of the grid but also optimizes the calculation efficiency, can accurately capture the fine structural features of the hepatobiliary and pancreatic organs, and ensures the authenticity and integrity of the reconstruction results.

[0045] By introducing a deformation control technology based on energy minimization, the morphological changes of the mesh during adjustment are optimized, and the morphological consistency and boundary smoothness of the mesh elements are maintained, especially showing excellent performance in the mesh deformation control of complex regions. This technology effectively prevents the morphological distortion caused by local adjustment, enabling the reconstructed three-dimensional model to truly restore the anatomical structure of the hepatobiliary and pancreas, and providing a more reliable three-dimensional reference for clinical applications. Brief Description of the Drawings

[0046] Figure 1 This is the flowchart of the method for reconstructing three-dimensional images of the hepatobiliary and pancreas of the present invention.

[0047] Figure 2 This is the flowchart for constructing the non-rigid registration algorithm of the present invention.

[0048] Figure 3 This is the flowchart of the method for constructing a preliminary three-dimensional mesh model of the present invention.

[0049] Figure 4 This is the axial CT image reconstructed by the present invention using ASiR-V, ranging from 10% to 100%, with an interval of 10% (A to L), and reconstructed at low (M), medium (N), and high (O) intensity levels using DLIR.

[0050] Figure 5 This is a comparison of 7 image sets reconstructed by the present invention using DLIR and ASiR-V technologies at different intensities.

[0051] Figure 6 This is the present invention for measuring the image noise of the aorta (single ROI) and the liver (three ROIs of the right anterior lobe, right posterior lobe, and left hepatic lobe).

[0052] Figures 7-8 This is the box plot of the present invention depicting the image noise (HU) of 7 image reconstruction technologies (p value < 0.001), where Figure 7 is the average image noise measured at the liver from 3 liver ROIs; Figure 8 is the image noise measured at the aorta. Detailed Description of the Invention

[0053] The following is a detailed description of the specific implementation manner of the present invention in conjunction with the accompanying drawings.

[0054] In conjunction with the attached Figure 1, By adopting deep learning-assisted hepatobiliary and pancreatic image segmentation, the accuracy of image segmentation and the ability to identify complex anatomical structures have been significantly improved. In this method, a U-Net variant segmentation network specifically designed for the liver, gallbladder, and pancreas is first introduced. This network utilizes the powerful feature extraction ability of convolutional neural networks (CNNs) to perform fine-grained segmentation on multimodal images. The U-Net variant structure is optimized and designed to specifically adapt to the morphological characteristics of the liver, gallbladder, and pancreas. It captures multi-scale information from global to local in the image through the symmetric structure of the encoder-decoder and effectively integrates it. In actual operation, the network first performs convolution on the input multimodal images (such as CT and MRI images), extracts different levels of spatial and feature information from them, and then through layer-by-layer decoding and upsampling, restores this information to the same resolution as the input image, finally generating accurate organ segmentation results. This method can accurately separate the liver, bile ducts, and pancreas, solving the problem that traditional segmentation algorithms perform poorly in regions with complex morphology.

[0055] In addition, a boundary enhancement module is introduced during the segmentation process. The design purpose of the boundary enhancement module is to strengthen the ability to identify organ boundaries, especially in those regions with complex morphology and blurred boundaries, such as the junction between the bile duct and the liver or the complex vascular structures around the pancreas. This module usually adds an edge detection branch in the middle layer of the network or combines a specialized loss function to specifically optimize the segmentation accuracy of the boundary region. Through supervised learning, the network assigns higher weights to boundary pixels during the training process, thus generating segmentation results with clearer and more accurate boundaries.

[0056] Through precise spatial alignment and fusion processing of multimodal imaging data using advanced deep learning techniques, the quality and clinical application value of the reconstructed images have been significantly improved. First, a deep learning-driven non-rigid registration algorithm is adopted. This algorithm utilizes the powerful learning ability of the deep learning model to perform non-rigid registration on multimodal images (such as CT, MRI, and ultrasound), enabling high-precision spatial alignment of different modality images. This registration algorithm combines convolutional neural network architectures such as U-Net and can automatically learn the complex spatial deformation relationships in the images, thus achieving automatic registration of the images.

[0057] This process particularly focuses on identifying and matching the anatomical feature points of organs, such as the liver margin, the course of the bile duct, and the characteristic landmark points of the pancreas. These feature points serve as reference points in registration to ensure the spatial consistency of different modality images. By optimizing the loss function of the neural network, the registration results are kept consistent in anatomical structure, thereby improving the registration accuracy. On this basis, according to the registration results, multi-modal fusion of liver, gallbladder, and pancreas images is carried out. This fusion process comprehensively utilizes the advantageous characteristics of each modality image: CT images provide high-contrast bone and tissue information, which helps to clearly display the anatomical structure; MRI images are famous for their high resolution of soft tissues and can delicately present the internal structure and lesion characteristics of the liver, bile duct, and pancreas; ultrasound images have the characteristic of real-time dynamic display and can capture the dynamic movement of the hepatobiliary and pancreatic organs. The multi-modal fusion algorithm integrates the advantages of CT, MRI, and ultrasound images complementarily through weighted averaging, feature fusion, and the feature extraction module of the deep learning network, and constructs fused image data with high resolution and rich details.

[0058] By accurately extracting the contour information of the liver, gallbladder, and pancreas from the segmented images, a preliminary three-dimensional mesh model is constructed, laying a foundation for subsequent refined reconstruction. First, in this process, using the previous segmentation results, the boundary contours of the liver, gallbladder, and pancreas organs are automatically extracted. These contours are further enhanced through edge detection, morphological processing, and deep learning segmentation networks to ensure the smoothness and continuity of their boundaries, especially in morphologically complex regions such as the bile duct and pancreatic duct parts. The extracted contour data is converted into a dense point cloud, and the density of the point cloud is dynamically adjusted according to the geometric characteristics of each organ to better capture the complex morphological features of the liver, gallbladder, and pancreas. Then, based on the point cloud data, the Delaunay triangulation or Marching Cubes algorithm is used to generate a preliminary three-dimensional mesh model. This model maintains a high mesh density in morphologically complex regions (such as curved bile ducts and pancreatic ducts), while reducing the mesh density in relatively smooth regions (such as the liver surface), thus ensuring the balance between the computational efficiency and model accuracy of the reconstruction. On this basis, adaptive adjustment is carried out according to the geometric morphology of different organs. Through adaptive mesh optimization algorithms (such as Laplacian smoothing and deformation remapping), the distribution of the mesh is adjusted to best fit the actual morphology of the organ. The core of the adaptive adjustment lies in automatically increasing or decreasing mesh elements according to local geometric morphological changes, especially near high-curvature regions such as the bile duct and pancreatic duct, increasing the mesh density to accurately represent the complex morphology and ensuring a high degree of coincidence between the mesh and the actual organ structure. To further improve the fitting degree of the mesh model to the actual organ shape, an iterative optimization algorithm is used to refine the adjustment of the mesh vertices, including the optimization of complex morphologies such as the bile duct system and pancreatic duct. By iteratively calculating and adjusting the positions of the mesh vertices, the mesh structure is kept smooth and consistent during the deformation process, and the geometric distortion caused by mesh deformation is minimized. This algorithm usually combines energy minimization techniques. By optimizing the energy function of the deformation field, it ensures that the adjustment of the mesh not only retains the original geometric features but also avoids the problem of mesh distortion caused by excessive adjustment.

[0059] By combining the texture features of imaging data, the tissue details and pathological features of organs are accurately reproduced. First, using multi-modal fusion imaging data, the respective texture features of the liver, gallbladder, and pancreas are extracted from images such as CT, MRI, and ultrasound. These features include important anatomical and pathological information such as tissue density, lesion area, and vascular distribution. Through deep learning and image processing techniques, the gray-scale distribution, texture pattern, and correspondence with actual tissues of each image are carefully analyzed to obtain high-quality texture data with individual characteristics. These texture data can meticulously reflect the internal characteristics of different organs, such as the fine vascular network of the liver, the running pattern of bile ducts, and the degree of fibrosis of pancreatic tissue, ensuring that the texture features can fully express the physiological and pathological states of individuals. After extraction, these personalized texture features are mapped onto the surface of the three-dimensional model, making the three-dimensional model not only have an accurate geometric shape but also be able to present the real texture of the organ.

[0060] This texture mapping process uses UV mapping technology or a deep learning-driven texture synthesis algorithm to accurately fit the texture information of the planar image onto the three-dimensional surface, forming a three-dimensional image model with rich texture and clear details. To further enhance the realism of the model, this method also dynamically adjusts the details of texture mapping according to individual differences, which include individualized pathological features such as the location, size, and shape of tumors, and the degree and distribution of fibrosis. By dynamically adjusting the texture mapping parameters, the characteristics of the lesion area can be highlighted in the three-dimensional model, such as the texture of the tumor, the degree of blurriness of the edge, and the boundary with surrounding tissues, or special texture enhancement can be performed on the fibrotic area, making these pathological features more clearly and prominently presented in the three-dimensional model. This personalized adjustment is achieved by adjusting the density, contrast, and hue of the texture features in three-dimensional space, so that the reconstructed three-dimensional model is more in line with the actual situation of the patient.

[0061] Example 1:

[0062] Combined with the attached Figure 2 In the steps, a 52-year-old male patient went to the hospital due to persistent upper abdominal pain and jaundice. After preliminary examination, the doctor suspected that the patient had a bile duct tumor and pancreatic lesions. To further diagnose and formulate a surgical plan, the hospital decided to perform multi-modal imaging examinations of the liver, gallbladder, and pancreas on the patient, including CT, MRI, and ultrasound, to obtain imaging data in different modalities. These data contain complex information such as the patient's liver, bile ducts, pancreas, and their surrounding vascular structures. However, due to the different imaging mechanisms of each imaging technology, there are significant differences in displacement, deformation, and resolution among the images.

[0063] To accurately reconstruct the three-dimensional model of the hepatobiliary and pancreatic organs, the doctor adopted a deep learning-based non-rigid registration algorithm, especially a method that combines convolutional neural networks (CNNs), attention mechanisms, and multi-scale feature extraction to identify and locate the feature points of organ edges, blood vessel intersections, and lesion areas in the images.

[0064] In actual operation, the registration process first uses the CNN network to process CT and MRI images, extracting the anatomical features of the hepatobiliary and pancreatic organs in each modality image. The attention mechanism is introduced during the training of the CNN network, enabling the network to automatically focus on key regions in the images, such as tumor boundaries, bile duct courses, and peripancreatic blood vessel distributions. Through multi-scale feature extraction, the network can capture feature points at different levels from coarse to fine and locate the key anatomical structures in the images. During this process, a total of N = 150 feature points were extracted, including the edge points of the liver, intersections of bile ducts, and lesion areas around the pancreatic duct, providing the basic data for subsequent registration calculations.

[0065] Next, the feature points in different modality images are aligned by optimizing the matching algorithm, specifically using the optimization function J match to measure the registration error. The value ranges of the adjustment coefficients α and β are set to 0.5 to 2.0 and 1.0 to 3.0 respectively to balance the weights of position differences and gradient information. In actual calculations, α = 1.2 and β = 2.5 are set and substituted into the formula:

[0066]

[0067] Here, |x i -y i | represents the Euclidean distance of the feature point positions in the CT and MRI images. By detecting the position errors of these points in different modality images, the accuracy of position alignment is obtained; represents the gradient difference at the feature point and is used to evaluate the change of local structural features. Especially in complex anatomical regions, such as the junction of lesions and normal tissues, it can reflect the alignment accuracy of the image at the boundary details. After calculation, the average position difference of each feature point is set to 0.8 mm, and the gradient error is 1.1 units. Substituting the above parameters, we get:

[0068]

[0069] This result reflects the total registration error. By analyzing J matchMinimize the parameters of the adjustment network, continuously optimize the registration results between images, so that the final three-dimensional images of the liver, gallbladder, and pancreas achieve high-precision alignment between different modality images. After registration, the doctor obtains a set of high-quality fused images, which can clearly present the true anatomical structure and pathological conditions of the patient's liver, gallbladder, and pancreas. This non-rigid registration algorithm greatly improves the alignment accuracy of multi-modal images and provides reliable data support for three-dimensional reconstruction.

[0070] In the above case, the doctor has obtained the patient's CT, MRI, and ultrasound images, and extracted the key feature points in the multi-modal images through a convolutional neural network (CNN) combined with an attention mechanism. Next, the non-rigid registration algorithm directly learns the mapping relationship between different modality images through an end-to-end deep learning network. Specifically, the input is the unregistered CT and MRI images, and the goal of the network is to align these images so that they have a highly consistent anatomical structure expression in space. During training, the registration network not only learns the global image transformation but also can capture local non-rigid deformations, especially the morphological changes in the complex regions of the liver, gallbladder, and pancreas, such as the curved part of the liver, the course of the bile duct, and the pathological tissues around the pancreas.

[0071] To ensure that the registration results can retain the anatomical information in the images to the greatest extent, the system defines a composite loss function L to measure the overall quality of the registration. This loss function consists of two parts: one part measures the pixel-level similarity, and the other part combines the matching error of the feature points. During the registration process, the pixel-level similarity is used to ensure that the pixel intensity difference between the registered target image T(x) and the reference image R(x) is minimized, that is, the aligned CT and MRI images are as consistent as possible in terms of gray value or intensity. At the same time, the matching error J of the feature points match controls the precise alignment of the anatomical structure and ensures that the key anatomical points (such as blood vessel intersections and lesion areas) are consistent in multi-modal images. To further explain this process, the following formula is used to represent the total loss function:

[0072] L = λ1·∫ Ω (T(x) - R(x)) 2 dx + λ2·J match

[0073] In the formula, L represents the total loss function, which measures the quality of the entire registration process; λ1 and λ2 are the loss weight parameters respectively, controlling the balance between the pixel-level similarity and the feature point matching error. The value range can be set between λ1 = 0.8 and 1.5, and λ2 = 1.0 and 2.0 to ensure a reasonable balance between the two during the registration process. In this case, it is selected that λ1 = 1.2 and λ2 = 1.5. These parameters adjust the focus of the registration, and the weight values determine the different attentions of the network to pixel similarity and anatomical matching.

[0074] In the specific registration process, it is set that the target image T(x) is the MRI image after registration processing, and the reference image R(x) is the CT image. For a given pixel point x, its pixel values T(x) and R(x) are 120 and 115 respectively, indicating the intensity difference of this pixel point in the two images. According to the square formula of pixel difference (T(x) - R(x)) 2 , the intensity difference of this pixel point can be calculated as (120 - 115) 2 = 25. It is set that the range of the image domain Ω is the entire image area, containing 1 million pixel points. Then, the intensity differences of each pixel point are summed up and combined with integration to obtain the total sum of the pixel-level similarity part:

[0075] ∫ Ω (T(x) - R(x)) 2 dx = 25 × 1000000 = 25,000,000

[0076] Next, combining the calculation result 469.8 of the aforementioned feature point matching error J match , the two loss parts can be substituted into the total loss function:

[0077] L = 1.2 × 25,000,000 + 1.5 × 469.8 = 30,000,000 + 704.7 = 30,000,704.7

[0078] By minimizing this total loss function L, the network continuously adjusts the transformation parameters in the registration process to optimize the alignment effect of CT and MRI images. This optimization process ensures the global pixel similarity of the images while maintaining the high-precision alignment of local anatomical features, especially in complex structure areas such as bile ducts and pancreatic ducts.

[0079] Through the previous registration processing, the doctor has initially aligned the patient's CT and MRI images. However, in order to further improve the registration accuracy, especially in capturing details in the complex anatomical areas of the hepatobiliary and pancreatic regions, a multi-level pyramid strategy is adopted. This strategy divides the registration process into multiple resolution levels, gradually optimizing from low resolution to high resolution, and achieving a more refined registration effect through a step-by-step approach.

[0080] In this process, the original image data is first downsampled to generate a set of pyramid levels. Starting from the coarsest image with the lowest resolution, registration is performed at each level, gradually optimizing the deformation field. When processing low-resolution images, the network can quickly capture large-scale global deformations, laying a foundation for subsequent fine optimization at higher resolutions. Specifically, at the first layer with the lowest resolution, the registration network generates an initial spatial deformation field φ(x), which describes the overall deformation between images. Through the deep learning model, the system automatically adjusts this deformation field to achieve a rough alignment. After completing this initial alignment, move to the next level, increase the resolution of the image to a medium level, and use the deformation field of the previous layer as the initial value for multiple iterative optimizations to gradually refine the deformation field and capture more detailed feature changes in the image.

[0081] To further optimize and constrain the smoothness and continuity of the spatial deformation field, an optimization function is introduced at each level to control the smoothness and structural consistency of the deformation field. The optimization function is defined as:

[0082]

[0083] where, is the optimization objective function, measuring the smoothness and structural consistency of the deformation field; φ(x) is the current spatial deformation field function, representing the degree of deformation of the image in the current registration process; φ prev (x) is the deformation field of the previous iteration, ensuring the continuity and consistency of the deformation field during multiple optimizations; κ and ζ are adjustment coefficients, respectively controlling the weights of deformation smoothness and continuity differences. The value range of κ is from 0.3 to 1.0, and the value range of ζ is from 1.0 to 2.5. It is set that in this case, κ = 0.5 and ζ = 1.8, and substitute them into the optimization formula for calculation.

[0084] Set the gradient magnitude of the current deformation field φ(x) to be 0.6, indicating that the local deformation is moderately smooth. At the same time, the difference between the current and the previous iteration's deformation fields ||φ(x) - φ prev (x)|| is 0.4, reflecting the continuity adjustment of the field in this optimization. After substituting these data, the calculation process of the optimization function is as follows:

[0085]

[0086] Set the area corresponding to the domain Ω to be the entire image area, approximately 1 million pixels. Then the sum of the optimization objective function is:

[0087]

[0088] This result reflects the optimization effect of the deformation field at the current level. By minimizing The system optimizes the deformation field layer by layer, so that each iterative adjustment maintains the smoothness and continuity of the image structure and avoids unnatural deformation. After each layer of pyramid optimization, the image gradually returns to its original high resolution, and the optimized deformation fields of all levels are accumulated and superimposed. The final generated deformation field not only captures the global structural consistency, but also finely preserves local details. Through this multi-level pyramid strategy, doctors can obtain highly aligned CT and MRI images, and the three-dimensional structure of the liver, gallbladder and pancreas is finely restored, including the course of the bile duct, the morphology of the pancreatic duct, and the complex structure of the surrounding blood vessels, ensuring the reliability of the image in clinical applications.

[0089] Embodiment 2:

[0090] Combined with Figure 3 In the example of the patient in Example 1, after the doctor accurately registers the liver, gallbladder and pancreas using multimodal images, in order to further achieve 3D reconstruction, a preliminary 3D mesh model needs to be constructed. This step is crucial to the precision of 3D reconstruction, especially for the complex liver, gallbladder and pancreas area, which must capture the true geometric features of the organs to provide reliable 3D model support for subsequent diagnosis and treatment.

[0091] When constructing a 3D mesh model, the deep learning enhancement model is first used to extract the boundary data in the image. Through extensive training, deep learning models, such as the optimized U-Net structure, can automatically identify and extract the precise boundaries of the liver, bile duct, and pancreas in multimodal images, especially for areas with fuzzy boundaries and complex shapes (such as the tortuosity of the bile duct and the small ducts of the pancreas).

[0092] The extracted boundary data is further converted into a dense point cloud, whose density is dynamically adjusted according to the complexity of the organ. Specifically, the generation of the point cloud follows an adaptive density strategy: in areas with drastic morphological changes and high curvature (such as the bile duct junction and near the pancreatic duct), the density of the point cloud will be significantly increased to capture fine geometric features; while in smooth and less changing areas (such as the liver surface), the point cloud density is relatively low to optimize computational efficiency and resource utilization. This density adjustment is achieved by optimizing the energy function E contour The implementation is defined as follows:

[0093]

[0094] Among them, E contour is the energy function of contour extraction, which is used to measure the quality of boundary extraction; Ω is the definition domain of the image, which represents the area where the contour is calculated, including the organ part in the entire image; G is the gradient of the image, representing the intensity change of the image edge, mainly used to capture the sharpness of the boundary. The higher the edge intensity, the more obvious the contour; C(x) is the extracted contour function, representing the position of the contour in the image. is the second derivative of the contour, evaluating the smoothness of the contour to ensure that the boundary is smooth during extraction without excessive irregular changes; α and β are adjustment parameters, respectively controlling the contributions of the image gradient and contour smoothness to the total energy. The value range of α is from 1.0 to 2.0, and the value range of β is from 0.5 to 1.5. These parameters are adjusted according to the actual image characteristics to ensure the accuracy of boundary extraction.

[0095] In this case, the doctor sets α = 1.5 and β = 1.0, and these parameters reflect the balanced requirements for edge intensity and contour smoothness. In the specific calculation, the average value of the image gradient is set to 0.8, indicating moderate edge intensity, and the second derivative of the contour function is 0.6, reflecting that the boundary has good smoothness. Substitute into the above formula for energy calculation:

[0096] E contour = ∫ Ω (1.5·0.8 + 1.0·(0.6) 2 )dx = ∫ Ω (1.2 + 0.36)dx = ∫ Ω 1.56dx

[0097] Set the domain Ω of the image to include the entire organ area, with a total of 500,000 pixel points. Then the total energy of contour extraction is:

[0098] E contour = 1.56×500,000 = 780,000

[0099] This energy value reflects the quality of the extracted boundary. By minimizing this energy function, the system continuously optimizes the parameters of boundary extraction to ensure the accuracy and continuity of the contour. Based on this high-quality boundary data, the generated dense point cloud can accurately describe the complex geometric shape of the hepatobiliary pancreas, providing accurate data support for subsequent three-dimensional mesh construction. In actual operation, after the point cloud is generated, it is converted into a three-dimensional mesh through a triangulation algorithm (such as Delaunay triangulation) to construct a preliminary three-dimensional structure, and this mesh model will become the basis for subsequent refined mesh adjustment and reconstruction.

[0100] The doctor generated a preliminary three-dimensional dataset by extracting boundary data and dense point clouds. To complete the construction of the three-dimensional mesh, the point cloud data was further converted into a triangular mesh. This process used the Delaunay triangulation or Marching Cubes algorithm, which could quickly generate a mesh structure suitable for three-dimensional reconstruction. In specific applications, the point cloud data was converted into triangular meshes through the algorithm, and these mesh elements expressed the three-dimensional structures of the liver, gallbladder, and pancreas in the form of triangular patches. Due to the complexity of organ morphology, such as the tortuous course of the bile duct and the fine structure of the pancreatic duct, in order to accurately express these features, the system needed to dynamically adjust the density of the mesh elements.

[0101] During the mesh generation process, according to the local density of the point cloud and the geometric features of the organ, the size of the mesh elements was automatically optimized and adjusted. Especially in complex morphological regions, such as the curved bile duct and pancreatic duct, the system increased the mesh density to ensure the fine expression of these key structures. Specifically, in these complex regions, the point cloud density was high, the number of generated triangular patches was more, and the cell area was small to improve the accuracy of surface expression; while in smooth regions, such as the liver surface, the mesh density was relatively reduced, and the area of the generated mesh elements was large. This way reduced unnecessary computational amount while ensuring the smoothness and stability of the overall structure.

[0102] To evaluate the rationality and accuracy of the generated three-dimensional mesh, the system performed geometric analysis on the mesh, especially measuring and evaluating the shape features of each mesh element. These features included local curvature, side length, and the connection angle of the mesh elements to ensure that the mesh could not only truly express the morphology of the organ but also maintain good structural consistency in subsequent adjustments and optimizations. To quantify the quality of the mesh, a mesh quality evaluation function E mesh was defined as follows:

[0103]

[0104] where E mesh was used to measure the quality and detail capture ability of the mesh; Ω was the mesh calculation area, covering the entire three-dimensional mesh model; κ(x) was the local curvature, which reflected the degree of bending of the mesh surface at different parts, especially in complex regions such as the bile duct and pancreatic duct, where the curvature value was high, indicating the fineness of the mesh in these regions; L(x) was the side length of the mesh element, controlling the size and morphological consistency of the mesh. Especially in smooth regions, the side length was large to reduce the generation of overly dense meshes; θ was the connection angle between the mesh elements, detecting the smooth transition of the mesh to ensure the smooth connection between each element; γ and δ were adjustment parameters, respectively controlling the influence of curvature and side length on the mesh quality. The value range of γ was from 0.8 to 1.5, and the value range of δ was from 0.5 to 1.2.

[0105] In actual operation, the average value of the local curvature κ(x) is set to 0.7 in complex regions, and the side length change rate is 0.5. These data reflect the performance of the mesh in capturing the complex morphology of the liver, gallbladder, and pancreas. Substitute these parameters into the mesh quality evaluation formula, and set γ = 1.2 and δ = 0.9:

[0106] E mesh = ∫ Ω (1.2·0.7 + 0.9·0.5)dx = ∫ Ω (0.84 + 0.45)dx = ∫ Ω 1.29dx

[0107] If the mesh calculation area contains 800,000 triangular patches, the total mesh quality is:

[0108] E mesh = 1.29×800,000 = 1,032,000

[0109] This energy value indicates the overall quality of the mesh. By continuously adjusting the density, side length, and curvature of the mesh elements to optimize the mesh's detail capture ability, the system can ensure the accuracy and rationality of the 3D mesh in various key regions. In this case, the generated 3D mesh model accurately represents the anatomical features of the patient's liver, gallbladder, and pancreas. In particular, the curvature of the bile duct and the slender structure of the pancreatic duct are highly restored.

[0110] To further optimize the mesh to adapt to the complex anatomical structure of the liver, gallbladder, and pancreas, doctors need to apply an adaptive mesh optimization algorithm to dynamically adjust the distribution of mesh elements to make it more accurately fit the true shape of the organ. This step is particularly crucial, especially in expressing complex morphologies in high-curvature regions such as the curved part of the bile duct and around the pancreatic duct. By increasing the mesh density to refine these structures, while appropriately reducing the mesh density on the smooth liver surface to maintain computational efficiency and resource optimization.

[0111] The core of adaptive mesh optimization lies in dynamically adjusting the distribution of mesh elements according to the geometric characteristics of the organ. Specifically, when implementing, the algorithm uses Laplacian smoothing and deformation remapping techniques. These methods can locally adjust the mesh density without changing the overall structure of the mesh. Laplacian smoothing is used to maintain the smoothness of the mesh and prevent mutations caused by local density changes; while deformation remapping ensures that the mesh still accurately retains the shape and boundary characteristics of the original organ during the adjustment process. The optimization effect is quantitatively evaluated by adaptively adjusting the energy function E adjust as follows:

[0112]

[0113] Among them, E adjust represents an energy function for adaptive adjustment, which is used to measure the rationality of the density and morphological changes of the mesh during the adjustment process; Ω is the computational domain for mesh adjustment, covering the entire three-dimensional mesh model; λ is a regulation coefficient that controls the sensitivity of density adjustment, and its value range is between 0.5 and 2.0; κ thresh is the curvature threshold, which defines the condition for triggering high-density adjustment and is usually set between 0.6 and 1.0 to mark areas with higher curvature; φ(x) is the mesh deformation field function, which describes the displacement changes of the mesh during the adjustment process; is the second derivative of the deformation field, which is used to control the smoothness and flatness of the mesh to ensure that no abrupt deformation occurs during the mesh adjustment.

[0114] In actual operation, the average value of the local curvature κ(x) in the bile duct region is set to 0.9, which is higher than the set curvature threshold κ thresh = 0.7, indicating that the density of the mesh needs to be increased in this area to capture details. The regulation coefficient λ is set to 1.2, indicating that it is more sensitive to curvature changes; at the same time, the second derivative of the deformation field is set to 0.4, reflecting a moderate smoothness of the mesh adjustment. Substitute these data into the optimization formula for calculation:

[0115]

[0116] Set the adjustment domain Ω to cover the entire mesh area, with approximately 600,000 elements, then the total energy is:

[0117] E adjust = (0.787 + 0.4) × 600,000 = 1.187 × 600,000 = 712,200

[0118] This result reflects the overall quality of the current mesh adjustment. By continuously optimizing and minimizing E adjust , the system can finely adjust the mesh elements, enabling the mesh to maintain a high degree of morphological fidelity and structural consistency in regions with complex morphology such as the bile duct and pancreatic duct, while maintaining an appropriate mesh density on the smooth liver surface to ensure the efficiency of calculation and the rational use of resources. After completing the adaptive optimization, the doctor obtains a set of more refined and accurate three-dimensional mesh models. By comparing the meshes before and after adjustment, it can be clearly seen that the mesh density and morphological details of the key structures have been significantly improved, especially the curvature of the bile duct, the complex course of the pancreatic duct, and the smooth transition of the liver surface have all been accurately expressed.

[0119] To further refine the morphology of the grid and ensure the morphological consistency and boundary smoothness of the grid during the overall adjustment, the doctor introduced a deformation control technology based on energy minimization. The core of this technology lies in precisely controlling the morphological adjustment of grid cells by optimizing the energy function. Especially in complex regions, it avoids geometric distortion caused by excessive deformation while ensuring smooth transitions of the grid.

[0120] During the specific operation process, the system introduced a deformation control function E for energy minimization deform , to quantify the morphological changes of the grid during the adjustment. By adjusting the various parameters in the energy function, the deformation of the grid cells is optimized, enabling the grid to not only highly conform to the actual morphology of the organ but also maintain a smooth boundary transition, especially in regions with morphological mutations, such as the edges of the liver, the curved courses of bile ducts, and the complex distributions of pancreatic ducts. The definition of the energy function is as follows:

[0121]

[0122] Among them, E deform represents the energy function for grid deformation control, measuring the morphological changes and consistency of the grid during the adjustment process; Ω is the domain of definition for grid calculation, covering the range of all grid cells; μ and ν are adjustment parameters, controlling the contributions of deformation intensity and boundary smoothness respectively. The value range of μ is from 1.0 to 2.5, and the value range of ν is from 0.5 to 1.5; is the square term of the gradient of the deformation field, used to smooth the deformation of the grid and avoid overly drastic grid deformation in local areas; θ edge is the local angle of the grid boundary, reflecting the adjustment of the grid at the boundary and ensuring a smooth transition of the boundary.

[0123] In practical applications, set the average value of the square of the gradient of the current grid deformation field to be 0.3, indicating that the overall deformation intensity of the grid is moderate. The local angle θ of the boundary edge is 45 degrees, that is, cos(45°) = 0.707, reflecting good smoothness of the grid at the boundary. Select the adjustment parameters μ = 1.8 and ν = 1.0 and substitute them into the formula for calculation:

[0124] E deform = ∫ Ω (1.8·0.3 + 1.0·0.707)dx = ∫ Ω (0.54 + 0.707)dx = ∫ Ω 1.247dx

[0125] Set the grid calculation domain Ω to contain 800,000 grid cells, then the total energy of deformation control is:

[0126] E deform= 1.247 × 800,000 = 997,600

[0127] This energy value represents the overall morphological change of the current grid. By continuously optimizing and minimizing E deform , the system can precisely control the grid adjustment process, enabling the grid to accurately represent the morphological characteristics of the organ in complex regions while maintaining a smooth transition at the boundaries. The optimization results show that the subtle curvature of the bile duct, the multi-level branching of the pancreatic duct, and the smooth connection of the liver surface have been significantly improved. There are no unnatural mutations or distortions between grid cells, ensuring a high degree of authenticity of the reconstructed model.

[0128] This deformation control technique based on energy minimization not only solves the deformation challenges of the grid in complex morphological regions but also effectively prevents morphological distortion caused by local adjustments, providing a more stable and coherent morphological basis for the three-dimensional grid model. In this example, the three-dimensional model generated by the doctor using this technique can clearly display the hepato-biliary-pancreatic anatomical structure of the patient, providing more detailed reference data for preoperative planning and significantly improving the surgical success rate and treatment accuracy. This solution verifies the feasibility and superiority of deformation control based on energy minimization in practical applications and is an essential key step in the three-dimensional image reconstruction of the hepato-biliary-pancreatic system.

[0129] Example 3: Optimized Application of the Three-Dimensional Image Reconstruction Method for the Hepato-Biliary-Pancreatic System Based on ASiR-V and DLIR

[0130] In this example, the applications of ASiR-V (Adaptive Statistical Iterative Reconstruction algorithm) and DLIR (Deep Learning Image Reconstruction algorithm) in the three-dimensional image reconstruction of the hepato-biliary-pancreatic system will be explored. Combining the technical advantages of the present invention, a more optimized three-dimensional reconstruction process will be provided, aiming to improve the visualization quality of the hepato-biliary-pancreatic structure and assist in clinical diagnosis and surgical planning. For a 62-year-old male patient who presented with upper abdominal discomfort and was initially suspected of having hepato-biliary system lesions after preliminary diagnosis, the doctor performed an abdominal CT examination on him and processed the images using different reconstruction algorithms for three-dimensional reconstruction of the hepato-biliary-pancreatic anatomical details.

[0131] 1. Data Acquisition and Processing: The patient underwent an abdominal CT scan to obtain high-resolution imaging data.

[0132] Subsequently, the imaging data was reconstructed using the ASiR-V and DLIR algorithms respectively.

[0133] Combined with the appendix Figure 4As shown, the image reconstruction intensity using the ASiR-V algorithm ranges from 10% to 100% (corresponding to images A to L), while the DLIR algorithm performs reconstructions at low, medium, and high intensity levels (corresponding to M, N, and O in Figure 4). The ASiR-V algorithm effectively reduces the noise in the image by increasing the reconstruction intensity, but the high-intensity reconstruction brings about excessive smoothing of the image, resulting in the loss of some details. Especially at intensities up to 100% ( Figure 4 L), the fine structures of the liver margin, bile ducts, and pancreas appear blurred, affecting the doctor's diagnosis. The DLIR shows better image detail retention and noise reduction effects at different intensities. The low-intensity DLIR ( Figure 4 M) has been able to well balance noise and image clarity, while the medium intensity ( Figure 4 N) further improves the image quality, making the anatomical structures of the liver, bile ducts, and pancreas more obvious; the high-intensity DLIR ( Figure 4 O) has almost no noise and sharp details, providing the best image basis for three-dimensional reconstruction.

[0134] 2. Three-dimensional reconstruction process: Based on the image data reconstructed at high intensity by DLIR, the following optimization steps were carried out using the three-dimensional image reconstruction method for the hepatobiliary and pancreatic regions of the present invention: First, an automatic segmentation of the boundaries of the hepatobiliary and pancreatic regions was performed using a deep learning-assisted boundary detection algorithm to extract the contour data of the liver margin, bile duct course, pancreatic structure, and its lesion areas. Through a multi-modal image fusion strategy (CT and DLIR images), a fine expression of multi-level anatomical structures was achieved, ensuring the integrity and consistency of the data. Then, the segmented multi-modal data was aligned through a non-rigid registration algorithm to ensure the precise overlap of each layer of images in three-dimensional space, providing an accurate basis for the construction of the three-dimensional mesh model.

[0135] 3. Optimized construction of the mesh model: When constructing the preliminary three-dimensional mesh model, a preliminary mesh was generated using the dense point cloud extracted from the segmented data through the Delaunay triangulation algorithm. Considering the complex morphology of the hepatobiliary and pancreatic regions, especially the curved course of the bile ducts and the fine structure of the pancreatic ducts, an adaptive mesh optimization algorithm was applied to dynamically adjust the mesh density according to local morphological changes: in high-curvature regions (such as the bile duct junctions), the mesh density was increased to capture morphological details; in smooth regions (such as the liver surface), the mesh density was reduced to optimize the calculation efficiency. By introducing Laplacian smoothing and deformation remapping techniques, the smoothness and consistency of the mesh elements were adjusted to ensure that the mesh retained the original morphology and boundary features during the adjustment process.

[0136] 4. Deformation control based on energy minimization: To further improve the mesh quality, a deformation control technique based on energy minimization was introduced during the adjustment process. By optimizing the energy function E deform, precisely control the deformation intensity and smoothness of the grid, and prevent grid distortion caused by local adjustment. The adopted energy function is:

[0137]

[0138] In this embodiment, μ = 1.8 and v = 1.0 are set, and the square value of the deformation field gradient is 0.3, and the boundary angle θ edge is 45 degrees The value of the optimized energy function is:

[0139] E deform = ∫ Ω (0.54 + 0.707)dx = 1.247×800,000 = 997,600

[0140] Through the above optimization and reconstruction process, the doctor finally obtained a clear, delicate and extremely low-noise three-dimensional model of the hepatobiliary and pancreatic system, which can clearly display the surface texture of the liver, the subtle curvature of the bile duct and the pathological changes of the pancreas.

[0141] Example 4: A three-dimensional image reconstruction method for the hepatobiliary and pancreatic system optimized based on DLIR and ASiR-V technologies:

[0142] This embodiment combines the image data reconstructed by DLIR (Deep Learning Image Reconstruction Algorithm) and ASiR-V (Adaptive Statistical Iterative Reconstruction Algorithm) at different intensities, and elaborates in detail how to use the three-dimensional image reconstruction method of the present invention to optimize the clinical image quality and improve the effect of diagnosing and treating hepatobiliary and pancreatic diseases. The patient is a 58-year-old female who was admitted to the hospital due to upper abdominal pain. After preliminary imaging examinations, she was suspected of having bile duct stones and pancreatitis. In order to further clarify the scope of the lesion and evaluate the impact of the lesion on the surrounding tissues, the doctor performed an abdominal CT scan on the patient and processed the images using different reconstruction algorithms to perform three-dimensional reconstruction of the anatomical details of the hepatobiliary and pancreatic system.

[0143] The CT scan data of the patient were reconstructed by the DLIR and ASiR-V algorithms at different intensities respectively.

[0144] Appendix Figure 5The comparison of image reconstruction using DLIR (low, medium, and high intensities corresponding to DLIR-L, DLIR-M, and DLIR-H respectively in the figures) and ASiR-V (0%, 10%, 20%, and 30% intensities corresponding to ASiR-V0%, ASiR-V10%, ASiR-V20%, and ASiR-V30% respectively) is shown. The DLIR-reconstructed images exhibit excellent noise suppression effects and detail retention at all intensities. Especially for the images of high-intensity DLIR-H, the structural details of the liver, bile ducts, and pancreas can be clearly shown. While for ASiR-V, the noise is significantly higher at low intensities (0% and 10%). As the intensity increases, the noise decreases, but the image smoothness increases, and some details are lost due to excessive smoothing, especially at 20% and 30% (Figures ASiR-V20% and ASiR-V30%), where the loss of details is more obvious.

[0145] Appendix Figure 6 The measurement of image noise in the aorta and liver regions is shown. Measurements are taken using a single ROI in the aorta and three ROIs at different parts of the liver. The results show that the images reconstructed by DLIR exhibit the lowest noise levels at all intensities. Especially for DLIR-H, the noise values at the aorta and in each region of the liver are significantly lower than those of the ASiR-V images at all levels, which reflects the superiority of DLIR in noise suppression and detail preservation. Compared with ASiR-V, DLIR reconstruction not only provides clearer images but also avoids the loss of details caused by high-intensity smoothing in iterative reconstruction. Especially in the complex hepatobiliary and pancreatic structures, DLIR performs more excellently.

[0146] Appendix Figures 7-8 The noise levels of 7 image reconstruction techniques are depicted in detail by box plots. Figure 7 shows the average image noise measured from 3 liver ROIs, while Figure 8 shows the results of noise measurement at the aorta. It can be seen from the figure that the noise values of DLIR-H are the lowest both in the liver and at the aorta, with the best image quality. In contrast, the noise of ASiR-V is significantly higher at lower intensities, and although the noise is reduced at high intensities, a large amount of detail information is lost during the smoothing process, which is not conducive to the accurate identification of hepatobiliary and pancreatic lesions.

[0147] Based on the above noise measurement and image quality assessment results, in this embodiment, the high-quality CT images reconstructed by DLIR-H are selected as the basis for 3D reconstruction. First, through a deep learning-driven segmentation algorithm, the boundary information of the liver, gallbladder, pancreas, and surrounding blood vessels is extracted to ensure the accuracy and continuity of image segmentation. Subsequently, the multi-modal image registration technology of the present invention is used to precisely align the DLIR images with other imaging modalities (such as MRI and ultrasound) to comprehensively utilize the advantageous characteristics of each modality image, especially the advantages of DLIR in detail preservation and noise control, providing high-precision input data for subsequent 3D modeling.

[0148] During the 3D reconstruction process, through an adaptive mesh optimization algorithm, the grid cell density is dynamically adjusted, so that a high grid density is maintained at complex morphological regions such as bile ducts and pancreatic ducts, while the density is reduced at smooth regions such as the liver surface, optimizing the use of computing resources. To ensure the consistency and boundary smoothness of the mesh during the adjustment process, a deformation control technology based on energy minimization is introduced to optimize the morphological adjustment of the mesh, especially at key lesion sites, ensuring that the 3D model can truly reflect the pathological state of the liver, gallbladder, and pancreas. The finally generated 3D reconstruction model clearly shows the organizational structure of the liver, the course of the bile ducts, and the morphological characteristics of the pancreas, which can help doctors better evaluate the impact of lesions on surrounding tissues in preoperative planning, optimize the selection of surgical paths, and reduce surgical risks. In addition, the advantages of DLIR technology in low noise and high detail preservation make the 3D model show tiny lesions more clearly, providing more intuitive and reliable imaging evidence for clinicians, significantly improving the accuracy of diagnosis and the precision of treatment.

Claims

1. A method for reconstructing three-dimensional images of the liver, gallbladder, and pancreas, characterized in that It includes the following steps: S1. Hepatobiliary and pancreatic image segmentation assisted by deep learning: S1.

1. Introduce a segmentation network of the U-Net variant for the three organs of the liver, gallbladder, and pancreas, and use a convolutional neural network to segment multimodal images, including the separation of the liver, bile duct, and pancreas; S1.

2. Introduce a boundary enhancement module during the segmentation process; S2. Image registration and paired fusion: S2.

1. Adopt a deep learning-driven non-rigid registration algorithm to spatially align image data of different modalities; use the anatomical feature points of the organs for registration; S2.

2. According to the registration results, perform multimodal fusion on the hepatobiliary and pancreatic images, and comprehensively utilize the bone and tissue contrast of CT, the soft tissue resolution of MRI, and the real-time dynamic characteristics of ultrasound to construct fused image data; S3. Adaptive three-dimensional mesh reconstruction: S3.

1. Extract the contours of the liver, gallbladder, and pancreas from the segmented images to construct a preliminary three-dimensional mesh model; adaptively adjust according to the geometric shapes of different organs to control the distribution of the mesh during reconstruction to match the organ shapes; S3.

2. Adopt an iterative optimization algorithm to adjust the mesh vertices including the bile duct system and pancreatic duct to fit the actual organ shape; S4. Personalized texture mapping and realism enhancement: S4.

1. Use the fused image data to extract the respective texture features of the liver, gallbladder, and pancreas, including tissue density, lesion area, and blood vessel distribution; S4.

2. Map the extracted texture features to the surface of the three-dimensional model, and dynamically adjust the texture according to individual differences including tumor location and fibrosis degree; The non-rigid registration algorithm described above includes: using a convolutional neural network CNN combined with an attention mechanism and multi-scale feature extraction to identify and locate the organ edges, blood vessel intersections, and lesion area feature points in the images; through an optimized matching algorithm, align the corresponding feature points in different modality images: Among them, J match represents the sum of the feature point matching errors, and N is the number of feature points; x i and y i are the corresponding feature point coordinates in different modality images, representing the position differences; α and β are adjustment coefficients to control the weights of different error terms; |·| represents the Euclidean distance, reflecting the distance error between feature points; and are the gradient information at the feature points, representing the changes in local structural features; The non-rigid registration algorithm described above includes: through end-to-end training, directly learn the mapping relationship between images, with the input being unregistered multimodal images and the output being the aligned images; define a composite loss function to measure the similarity of the overall image, and combine the matching error of anatomical feature points to control the data retention of anatomical structures during the registration process: Among them, \(L\) is the total loss function, which measures the overall quality of registration; \(\lambda_1\) and \(\lambda_2\) are loss weight parameters used to adjust the balance between pixel-level similarity and feature point matching error; \(T(x)\) represents the pixel value of the registered target image, \(R(x)\) is the pixel value of the reference image, and \((T(x)-R(x))\) 2 measures the intensity difference between pixels; \(\Omega\) is the domain of the image, which is the region of the entire image; \(J\) match is the sum of the feature point matching errors; The non-rigid registration algorithm described above includes: introducing a multi-level pyramid strategy during the registration process, dividing the registration process into multiple resolution levels, and gradually optimizing from low resolution to high resolution; in each layer of the pyramid, generate a preliminary spatial deformation field through deep learning, and then perform multiple iterations of optimization to refine the deformation field; to optimize the spatial deformation field, introduce an optimization function to constrain the continuity and smoothness of the deformation: Among them, is the optimization objective function, which measures the smoothness and structural consistency of the deformation field; φ(x) is the current spatial deformation field function, representing the degree of deformation of the image in the current registration process; φ prev (x) is the deformation field of the previous iteration, ensuring the continuity and consistency of the deformation field; κ and ζ are adjustment coefficients, controlling the weights of different error terms; is the gradient magnitude of the deformation field, measuring local smoothness; ||φ(x) - φ prev (x)|| represents the difference between the current and the previous deformation fields.

2. The reconstruction method of the hepatobiliary and pancreatic three-dimensional image according to claim 1, wherein The method for constructing the preliminary three-dimensional mesh model described above includes: using the boundary data extracted by the deep learning enhancement model; and converting the extracted contour data into a dense point cloud, and the density of the point cloud is dynamically adjusted according to the complexity of the organ structure to capture the geometric features of each organ: Among them, E contour represents the energy function for contour extraction, Ω is the domain of the image, representing the range for contour calculation; is the gradient of the image, reflecting the intensity change of the image edge and capturing the sharpness of the boundary; C(x) is the extracted contour function, indicating the position of the contour on the image; represents the second derivative of the contour, used to evaluate the smoothness of the contour; α and β are adjustment parameters, controlling the contribution weights of the gradient and smoothness to the overall contour.

3. The method for reconstructing a three-dimensional image of the hepatobiliary pancreas according to claim 2, characterized in that The method for constructing the preliminary three-dimensional mesh model described above includes: generating a three-dimensional mesh from the dense point cloud data, and using the Delaunay triangulation or Marching Cubes algorithm to convert the point cloud into a triangular mesh; During the grid generation process, the size of grid cells is adjusted according to the local density and geometric shape of the point cloud, so as to maintain a high grid density in morphologically complex regions including curved bile ducts and pancreatic ducts, while reducing the grid cell density in smooth regions including the liver surface; then geometric analysis is performed on the generated grid to evaluate the shape characteristics of each grid cell, including curvature, side length, and angle: Among them, E mesh is a mesh quality evaluation function, which is used to measure the rationality of the mesh and the ability to capture details; κ(x) is the local curvature, which describes the degree of curvature of the mesh cell surface; L(x) is the side length of the mesh cell, which controls the size and morphological consistency of the mesh cell; θ is the local angle, which represents the connection angle between mesh cells and is used to detect the smooth transition of the mesh; γ and δ are adjustment parameters, which adjust the influence weights of curvature and side length on the mesh quality.

4. The reconstruction method of the hepatobiliary and pancreatic three-dimensional image according to claim 3, wherein The method for constructing a preliminary three-dimensional grid model includes: using an adaptive grid optimization algorithm to dynamically adjust the grid cell distribution according to the geometric characteristics of the organ; among them, the adjustment of grid density is based on local morphological changes, including increasing the grid density in regions with high curvature; applying the Laplacian smoothing and deformation remapping adaptive algorithm to make the grid adjustment retain the shape and boundary characteristics of the original organ: Among them, E adjust represents an energy function for adaptive adjustment, measuring the rationality of density and morphological changes during the grid adjustment process; λ is a regulation coefficient, controlling the sensitivity of density adjustment; κ thresh is a curvature threshold, defining the triggering condition for high-density adjustment; φ(x) is a grid deformation field function, describing the displacement of the grid during the adjustment process; is the second derivative of the deformation field, used to control the smoothness and flatness of the grid.

5. The method for reconstructing a three-dimensional image of the hepatobiliary pancreas according to claim 4, characterized in that The method for constructing a preliminary three-dimensional grid model includes: introducing a deformation control technology based on energy minimization; by optimizing the energy function, maintaining the morphological consistency and boundary smoothness of grid cells during the adjustment process, and controlling the grid deformation in complex regions: Among them, E deform is the energy function for grid deformation control, measuring the morphological changes of the grid during the adjustment process; μ and ν are adjustment parameters, controlling the contributions of deformation intensity and boundary smoothness respectively; is the square term of the gradient of the deformation field, used to smooth the deformation of the grid; θ edge is the local angle of the grid boundary, representing the adjustment of the grid at the boundary.

Images