Reconstruction method of liver, gall and pancreas three-dimensional image

Abstract

The invention relates to a reconstruction method of a liver, gall and pancreas three-dimensional image. According to the technical scheme, the method comprises the following steps: introducing a segmentation network for U-Net variants of the liver, the bile duct and the pancreas, and segmenting a multi-modal image by using a convolutional neural network, including separation of the liver, the bile duct and the pancreas; a boundary enhancement module is introduced in the segmentation process; performing spatial alignment on the image data of different modalities by adopting a deep learning driven non-rigid registration algorithm; carrying out registration by utilizing anatomical feature points of the organs; according to a registration result, performing multi-modal fusion on the liver image, the bile image and the pancreas image, and constructing fusion image data by comprehensively utilizing the bone and tissue contrast ratio of CT, the soft tissue resolution ratio of MRI and the real-time dynamic characteristics of ultrasound; extracting contours of liver, gall and pancreas from the segmented image, and constructing a preliminary three-dimensional mesh model; and according to the geometrical shapes of different organs, adaptive adjustment is carried out, and the distribution of grids during reconstruction is controlled to be matched with the organ shapes.

Description

Technical Field

[0001] The present invention relates to a method for reconstructing a three-dimensional image, in particular to a method for reconstructing a three-dimensional image of the liver, gallbladder and pancreas. Background Art

[0002] The technology described in the Chinese invention patent publication number CN117529273A mainly uses the combination of ultrasound images and camera images to reconstruct three-dimensional images. This method obtains Doppler ultrasound images to depict blood vessels and blood flow, and combines them with 2D images captured by the camera to analyze the position changes of the reference object, calculate the posture of the ultrasound transducer and the 3D coordinates of the ultrasound image in the world coordinate system, thereby achieving three-dimensional reconstruction.

[0003] However, there are many drawbacks in the application of this method in the three-dimensional reconstruction of the liver, gallbladder and pancreas, which are mainly reflected in the following aspects: First, the public technical method relies on the combination of Doppler ultrasound and camera images. The reconstruction process is mainly aimed at the imaging of blood vessels and blood flow, and the reconstruction ability of solid organs such as the liver, gallbladder and pancreas is limited. Doppler ultrasound images can well display the blood flow and blood vessel course, but they are not good at showing the boundaries and internal structures of solid tissues such as the liver, gallbladder and pancreas. The imaging quality depends on the penetration and reflection characteristics of ultrasound. The 2D images captured by the camera lack the ability to image deep structures and are mainly used for the positioning and posture correction of surface features. Due to the limitations of these two imaging modes, it is impossible to accurately restore the complex anatomical structure of the liver, gallbladder and pancreas in practical applications, especially when faced with bile ducts and pancreatic ducts with complex morphology. The reconstructed three-dimensional images lack detail expression and accuracy assurance.

[0004] Secondly, the technical solution of the disclosed technology is highly dependent on the external reference of the reference object. The camera tracks the position change of the reference object to calculate the posture of the ultrasonic transducer. The accuracy of this method is limited by the distribution and recognition accuracy of the reference object. Once the position of the reference object is disturbed or misidentified, it will lead to deviations in the entire three-dimensional reconstruction, especially in the hepatobiliary and pancreatic regions, where the surrounding structures are complex and the tissues move frequently, and are easily affected by physiological movements such as breathing and heartbeat, thereby affecting the stability of the reference object, thereby reducing the reconstruction accuracy and reliability of the three-dimensional image. In addition, since this method is based on surface analysis of cameras and ultrasonic images, it lacks in-depth measurement of internal soft tissues, and cannot accurately distinguish the subtle differences between diseased tissues and normal tissues, and has insufficient support for the pathological diagnosis of the hepatobiliary and pancreatic regions. Thirdly, the Doppler ultrasound images of the disclosed technology mainly depict blood flow and vascular structure, and the reconstructed three-dimensional images tend to be vascular anatomical images and blood flow distribution, and do not provide high-resolution texture information and detail enhancement for substantial tissues, which leads to insufficient recognition of subtle hepatobiliary and pancreatic lesions such as micronodules and fibrosis. The 3D reconstruction of organs such as the liver, gallbladder and pancreas not only needs to present the appearance and vascular network, but also needs to deeply express the internal lesions and tissue characteristics, such as the location of the tumor inside the liver, the degree of stenosis of the bile duct and the distribution of pancreatic fibrosis. This information is extremely critical for clinical diagnosis and surgical planning, but it is not well performed in the current Doppler ultrasound technology. Thirdly, during the reconstruction process, this method needs to achieve image reconstruction through coordinate transformation and calculation of multiple pixel points in three-dimensional space. This reconstruction process often requires complex calculations and a large amount of image data matching, with a large amount of calculation and insufficient real-time performance. Especially when processing high-resolution images, the calculation speed is slow, which is difficult to meet the requirements of real-time clinical applications.

[0005] In addition, since the reconstruction process involves the calibration and integration of multiple data sources, the complexity of the algorithm increases the probability of error, which can easily lead to data loss or mismatch during the integration process, affecting the accuracy of the reconstruction results. Finally, the reconstruction method of the public technology lacks the processing of realism enhancement and texture mapping of the reconstructed image. Although it can display blood vessel and blood flow information, in actual applications, the visual effect of the three-dimensional reconstructed image is lacking and cannot provide clinicians with an intuitive and clear display of the internal structure of the organ. Summary of the invention

[0006] The purpose of the present invention is to provide a method for reconstructing three-dimensional images of the liver, gallbladder and pancreas, thereby solving some of the drawbacks pointed out in the background technology.

[0007] The present invention solves the above-mentioned technical problems by adopting the following technical solution, which includes the following steps:

[0008] S1. Hepatobiliary and pancreatic image segmentation assisted by deep learning:

[0009] S1.1. Introduce a U-Net variant segmentation network for the liver, gallbladder, and pancreas, and use a convolutional neural network to segment multimodal images, including the separation of the liver, bile duct, and pancreas;

[0010] S1.2, introduce boundary enhancement module in the segmentation process;

[0011] S2. Image registration and pairing fusion:

[0012] S2.1. Use a deep learning-driven non-rigid registration algorithm to spatially align image data of different modalities; use the anatomical feature points of the organs for registration;

[0013] S2.2. Based on the registration results, multimodal fusion of liver, gallbladder and pancreas images is performed, and fusion image data is constructed by comprehensively utilizing the bone and tissue contrast of CT, the soft tissue resolution of MRI and the real-time dynamic characteristics of ultrasound;

[0014] S3, Adaptive 3D mesh reconstruction:

[0015] S3.1. Extract the contours of the liver, gallbladder and pancreas from the segmented image and construct a preliminary three-dimensional mesh model; adaptively adjust the mesh distribution according to the geometric morphology of different organs to control the distribution of the mesh during reconstruction to match the organ morphology;

[0016] S3.2, using an iterative optimization algorithm to adjust the mesh vertices including the bile duct system and pancreatic duct to fit the actual organ shape;

[0017] S4, personalized texture mapping and enhanced realism:

[0018] S4.1. Extract the texture features of the liver, gallbladder and pancreas using fused image data, including tissue density, lesion area and vascular distribution;

[0019] S4.2. Map the extracted texture features to the 3D model surface and dynamically adjust the texture based on individual differences including tumor location and degree of fibrosis.

[0020] Furthermore, the non-rigid registration algorithm includes: using a convolutional neural network (CNN) combined with an attention mechanism and multi-scale feature extraction to identify and locate organ edges, vascular intersections, and feature points of lesion areas in images; and aligning corresponding feature points in images of different modalities by optimizing a matching algorithm:

[0021]

[0022] Among them, J match represents the sum of feature point matching errors, N is the number of feature points; x i and iare the coordinates of the corresponding feature points in different modal images, indicating the position difference; α and β are adjustment coefficients, controlling the weights of different error terms; |·| represents the Euclidean distance, reflecting the distance error between feature points; and They are the gradient information at the feature points, indicating the changes in local structural features.

[0023] Furthermore, the non-rigid registration algorithm includes: directly learning the mapping relationship between images through end-to-end training, with the input being an unregistered multimodal image and the output being an aligned image; defining a composite loss function to measure the similarity of the entire image, combining the matching error of anatomical feature points, and controlling the data retention of the anatomical structure during the registration process:

[0024]

[0025] Where L is the total loss function, which measures the overall quality of the registration; λ1 and λ2 are loss weight parameters, which are used to adjust the balance between pixel-level similarity and feature point matching error; T(x) represents the pixel value of the registered target image, R(x) represents the pixel value of the reference image, (T(x)-R(x)) 2 Measures the intensity difference between pixels; Ω is the domain of the image, which is the area of ​​the entire image; J match It is the aforementioned feature point matching error part.

[0026] Furthermore, the non-rigid registration algorithm includes: introducing a multi-level pyramid strategy in the registration process, dividing the registration process into multiple resolution levels, and gradually optimizing from low resolution to high resolution; in each layer of the pyramid, generating a preliminary spatial deformation field through deep learning, and then performing multiple iterative optimizations to refine the deformation field; in order to optimize the spatial deformation field, introducing an optimization function to constrain the continuity and smoothness of the deformation:

[0027]

[0028] in, To optimize the objective function, measure the smoothness and structural consistency of the deformation field; φ(x) is the current spatial deformation field function, which indicates the degree of image deformation during the current registration process; φ prev (x) is the deformation field of the previous iteration, which ensures the continuity and consistency of the deformation field; κ and ζ are adjustment coefficients, which control the weights of different error terms; is the gradient of the deformation field, which measures the local smoothness; ||φ(x)-φ prev (x)|| represents the difference between the current and previous deformation fields.

[0029] Furthermore, the method for constructing a preliminary three-dimensional mesh model includes: using the deep learning enhancement model to extract boundary data; and converting the extracted contour data into a dense point cloud, and the density of the point cloud is dynamically adjusted according to the complexity of the organ structure to capture the geometric characteristics of each organ:

[0030]

[0031] Among them, E contour represents the energy function of contour extraction, Ω is the domain of the image, representing the range of contour calculation; is the gradient of the image, reflecting the intensity change of the edge of the image and capturing the sharpness of the boundary; C(x) is the extracted contour function, indicating the position of the contour on the image; Represents the second-order derivative of the contour, which is used to evaluate the smoothness of the contour; α and β are adjustment parameters that control the contribution weight of the gradient and smoothness to the overall contour.

[0032] Further, the method for constructing a preliminary three-dimensional mesh model includes: generating a three-dimensional mesh from dense point cloud data, converting the point cloud into a triangular mesh using Delaunay triangulation or MarchingCubes algorithm;

[0033] During the mesh generation process, the mesh cell size is adjusted according to the local density and geometry of the point cloud, so that the mesh density is kept high in the morphologically complex areas including the curved bile duct and pancreatic duct, while the mesh cell density is reduced in the smooth areas including the liver surface. The generated mesh is then geometrically analyzed to evaluate the shape characteristics of each mesh cell, including curvature, side length, and angle:

[0034]

[0035] Among them, E mesh is a mesh quality evaluation function, which is used to measure the rationality and detail capture ability of the mesh; κ(x) is the local curvature, which describes the curvature of the mesh unit surface, especially reflecting the morphological changes in complex areas; L(x) is the side length of the mesh unit, which controls the size and morphological consistency of the mesh unit; θ is the local angle, which represents the connection angle between mesh units and is used to detect the smooth transition of the mesh; γ and δ are adjustment parameters, which adjust the influence weight of curvature and side length on mesh quality.

[0036] Furthermore, the method for constructing a preliminary three-dimensional mesh model includes: using an adaptive mesh optimization algorithm to dynamically adjust the distribution of mesh units according to the geometric characteristics of the organ; wherein the mesh density is adjusted according to local morphological changes, including increasing the mesh density in high curvature areas; applying a Laplacian smoothing and deformation remapping adaptive algorithm so that the mesh adjustment retains the shape and boundary characteristics of the original organ:

[0037]

[0038] Among them, E adjust represents the energy function of adaptive adjustment, which measures the rationality of density and morphology changes during grid adjustment; λ is the adjustment coefficient, which controls the sensitivity of density adjustment; κ thresh is the curvature threshold, defining the triggering condition for high-density adjustment; φ(x) is the mesh deformation field function, describing the displacement of the mesh during the adjustment process; is the second-order derivative of the deformation field and is used to control the smoothness and smoothness of the mesh.

[0039] Furthermore, the method for constructing a preliminary three-dimensional mesh model includes: introducing a deformation control technology based on energy minimization; maintaining the morphological consistency and boundary smoothness of mesh units during the adjustment process by optimizing the energy function, and controlling mesh deformation in complex areas:

[0040]

[0041] Among them, E deform is the energy function of mesh deformation control, which measures the morphological change of the mesh during the adjustment process; μ and ν are adjustment parameters, which control the contribution of deformation intensity and boundary smoothness respectively; is the squared gradient term of the deformation field, used to smooth the deformation of the mesh; θ edge is the local angle of the mesh boundary, indicating the adjustment of the mesh at the boundary.

[0042] The hepatobiliary and pancreatic three-dimensional image reconstruction method of the present invention achieves high-precision reconstruction of the complex anatomical structure of the hepatobiliary and pancreatic by integrating and optimizing multiple innovative technologies, and has the following beneficial effects:

[0043] The non-rigid registration algorithm driven by deep learning, combined with a multi-level pyramid strategy and optimized loss function, effectively solves the spatial alignment problem between multi-modal images such as CT, MRI, and ultrasound. In the registration process, the anatomical feature points of the organs are used for precise matching, which not only improves the overall registration accuracy, but also maintains a high degree of consistency in the anatomical structure in details, providing a high-quality data foundation for subsequent 3D reconstruction.

[0044] In the process of constructing the 3D mesh model, an adaptive mesh optimization algorithm is used to dynamically adjust the distribution of mesh units according to the geometric characteristics of the organs, so that high mesh density is maintained in morphologically complex areas such as the bile duct and pancreatic duct, while the mesh density is reduced in smooth areas. This density adjustment method not only improves the expression accuracy of the mesh, but also optimizes the computational efficiency, and can accurately capture the subtle structural features of the liver, gallbladder and pancreas, ensuring the authenticity and integrity of the reconstruction results.

[0045] By introducing deformation control technology based on energy minimization, the morphological changes of the mesh during the adjustment process are optimized, the morphological consistency and boundary smoothness of the mesh units are maintained, and the mesh deformation control in complex areas is excellent. This technology effectively prevents morphological distortion caused by local adjustment, so that the reconstructed three-dimensional model can truly restore the anatomical structure of the liver, gallbladder and pancreas, providing a more reliable three-dimensional reference for clinical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 The figure is a flow chart of the method for reconstructing three-dimensional images of the liver, gallbladder and pancreas of the present invention.

[0047] Figure 2 The figure is a flow chart for constructing the non-rigid registration algorithm of the present invention.

[0048] Figure 3 A flowchart of the method for constructing a preliminary three-dimensional mesh model of the present invention.

[0049] Figure 4 Axial CT images reconstructed using ASiR-V for the present invention range from 10% to 100% with intervals of 10% (A to L) and reconstruction using DLIR for low (M), medium (N) and high (O) intensity levels.

[0050] Figure 5 Seven image sets reconstructed at different intensities using DLIR and ASiR-V techniques for the present invention are compared.

[0051] Figure 6 Image noise of the aorta (single ROI) and liver (three ROIs of right anterior lobe, right posterior lobe and left liver lobe) was measured for the present invention.

[0052] Figure 7-8 The box plot of the present invention depicts the image noise (HU) of 7 image reconstruction techniques (p value < 0.001), where Figure 7 is the average image noise measured at the liver from three liver ROIs; Figure 8 is the image noise measured at the aorta. DETAILED DESCRIPTION

[0053] The specific implementation modes of the present invention will be described in detail below in conjunction with the accompanying drawings.

[0054] Combined with Figure 1, by adopting deep learning-assisted hepatobiliary and pancreatic image segmentation, the accuracy of image segmentation and the ability to recognize complex anatomical structures are significantly improved. In this method, a U-Net variant segmentation network specifically for the liver, gallbladder, and pancreas is first introduced. The network uses the powerful feature extraction ability of convolutional neural network (CNN) to perform fine segmentation processing on multimodal images. The U-Net variant structure is optimized and designed to specifically adapt to the morphological characteristics of the liver, gallbladder, and pancreas. The symmetrical structure of the encoder-decoder captures multi-scale information from the global to the local in the image and effectively integrates it. In actual operation, the network first performs convolution processing on the input multimodal image (such as CT and MRI images) to extract spatial and feature information at different levels, and then restores this information to the same resolution as the input image through layer-by-layer decoding and upsampling, and finally generates accurate organ segmentation results. This method can accurately separate the liver, bile duct, and pancreas, solving the problem that traditional segmentation algorithms perform poorly in complex morphological areas.

[0055] In addition, a boundary enhancement module is introduced in the segmentation process. The boundary enhancement module is designed to enhance the recognition ability of organ boundaries, especially for those areas with complex morphology and fuzzy boundaries, such as the junction of the bile duct and the liver or the complex vascular structure around the pancreas. This module usually optimizes the segmentation accuracy of the boundary area by adding an edge detection branch in the middle layer of the network or combining a special loss function. Through supervised learning, the network gives higher weights to boundary pixels during training, thereby generating segmentation results with clearer and more accurate boundaries.

[0056] Advanced deep learning technology is used to accurately align and fuse multimodal imaging data in space, significantly improving the quality of reconstructed images and their clinical application value. First, a deep learning-driven non-rigid registration algorithm is used. This algorithm uses the powerful learning ability of the deep learning model to perform non-rigid registration on multimodal images (such as CT, MRI, and ultrasound), so that images of different modalities can be aligned in space with high precision. This registration algorithm combines convolutional neural network architectures such as U-Net, which can automatically learn complex spatial deformation relationships in images, thereby achieving automatic registration of images.

[0057] This process pays special attention to the identification and matching of anatomical feature points of organs, such as the edge of the liver, the course of the bile duct and the characteristic landmarks of the pancreas. These feature points are used as reference points in the registration to ensure the spatial consistency of images of different modalities. By optimizing the loss function of the neural network, the registration results are kept consistent in the anatomical structure, thereby improving the registration accuracy. On this basis, according to the registration results, multimodal fusion of liver, gallbladder and pancreas images is performed. This fusion process comprehensively utilizes the advantages of each modality image: CT images provide high-contrast bone and tissue information, which helps to clearly display the anatomical structure; MRI images are known for their high resolution of soft tissues, which can present the internal structure and pathological characteristics of the liver, bile duct and pancreas in detail; ultrasound images have the characteristics of real-time dynamic display, which can capture the dynamic movement of the liver, gallbladder and pancreas. The multimodal fusion algorithm complementarily integrates the advantages of CT, MRI and ultrasound images through weighted averaging, feature fusion and feature extraction modules of deep learning networks to construct fused image data with high resolution and rich details.

[0058] By accurately extracting the contour information of the liver, gallbladder, and pancreas from the segmented images, a preliminary three-dimensional mesh model is constructed, laying the foundation for subsequent refined reconstruction. First, in this process, the boundary contours of the liver, gallbladder, and pancreas are automatically extracted using the previous segmentation results. These contours are further enhanced by edge detection, morphological processing, and deep learning segmentation networks to ensure the smoothness and continuity of their boundaries, especially in areas with complex morphology such as the bile duct and pancreatic duct. The extracted contour data is converted into a dense point cloud, and the density of the point cloud is dynamically adjusted according to the geometric characteristics of each organ to better capture the complex morphological features of the liver, gallbladder, and pancreas. Then, based on the point cloud data, a preliminary three-dimensional mesh model is generated using the Delaunay triangulation or MarchingCubes algorithm. The model maintains a high mesh density in morphologically complex areas (such as the curved bile duct and pancreatic duct) and reduces the mesh density in relatively smooth areas (such as the liver surface), thereby ensuring a balance between the computational efficiency of reconstruction and the accuracy of the model. On this basis, adaptive adjustments are made according to the geometric morphology of different organs. Through adaptive mesh optimization algorithms (such as Laplacian smoothing and deformation remapping), the distribution of the mesh is adjusted to fit the actual morphology of the organ to the greatest extent. The core of adaptive adjustment is to automatically increase or decrease mesh units according to local geometric changes, especially in high curvature areas such as the bile duct and pancreatic duct, increase mesh density to accurately express complex morphology, and ensure that the mesh is highly consistent with the actual organ structure. In order to further improve the fit of the mesh model to the real organ shape, an iterative optimization algorithm is used to refine the mesh vertices, including the optimization of complex morphologies such as the bile duct system and pancreatic duct. The position of the mesh vertices is adjusted through iterative calculation, so that the mesh structure maintains smoothness and consistency during the deformation process, and minimizes the geometric distortion caused by mesh deformation. This algorithm is usually combined with energy minimization technology. By optimizing the energy function of the deformation field, it ensures that the mesh adjustment not only retains the original geometric features, but also avoids the mesh distortion caused by excessive adjustment.

[0059] By combining the texture features of image data, the tissue details and pathological characteristics of the organs can be accurately reproduced. First, the texture features of the liver, gallbladder, and pancreas are extracted from CT, MRI, and ultrasound images using multimodal fusion image data. These features include important anatomical and pathological information such as tissue density, lesion area, and vascular distribution. Through deep learning and image processing technology, the grayscale distribution, texture pattern, and correspondence with the actual tissue of each image are carefully analyzed to obtain high-quality texture data with individual characteristics. These texture data can carefully reflect the intrinsic characteristics of different organs, such as the fine vascular network of the liver, the course pattern of the bile duct, and the degree of fibrosis of the pancreatic tissue, ensuring that the texture features can fully express the physiological and pathological state of the individual. After the extraction is completed, these personalized texture features are mapped to the surface of the 3D model, so that the 3D model not only has an accurate geometric shape, but also can present the real texture of the organ.

[0060] The texture mapping process uses UV mapping technology or deep learning-driven texture synthesis algorithms to accurately fit the texture information of the plane image to the three-dimensional surface, forming a three-dimensional image model with rich texture and clear details. In order to further enhance the realism of the model, the method also dynamically adjusts the details of the texture mapping according to individual differences, including individual pathological features such as the location, size and morphology of the tumor, the degree and distribution of fibrosis, etc. By dynamically adjusting the texture mapping parameters, the characteristics of the lesion area can be highlighted in the three-dimensional model, such as the texture of the tumor, the blurriness of the edge, and the boundary with the surrounding tissue, or the fibrosis area can be specially enhanced to make these pathological features clearer and more prominent in the three-dimensional model. This personalized adjustment is achieved by adjusting the density, contrast and tone of the texture features in three-dimensional space, so that the reconstructed three-dimensional model is more in line with the actual situation of the patient.

[0061] Embodiment 1:

[0062] Combined with Figure 2 Following the steps of the hospital, a 52-year-old male patient went to the hospital for persistent upper abdominal pain and jaundice. After preliminary examination, the doctor suspected that the patient had bile duct tumors and pancreatic lesions. In order to further diagnose and formulate a surgical plan, the hospital decided to perform multimodal imaging examinations of the patient's liver, gallbladder and pancreas, including CT, MRI and ultrasound, to obtain imaging data under different modalities. These data contain complex information such as the patient's liver, bile duct, pancreas and surrounding vascular structures, but due to the different imaging mechanisms of each imaging technology, there are large displacements, deformations and resolution differences between the images.

[0063] In order to accurately reconstruct the three-dimensional model of the liver, gallbladder and pancreas, doctors used a non-rigid registration algorithm based on deep learning, especially a method combining convolutional neural network (CNN), attention mechanism and multi-scale feature extraction to identify and locate the feature points of organ edges, vascular intersections and lesion areas in the image.

[0064] In actual operation, the registration process first uses the CNN network to process CT and MRI images to extract the anatomical features of the liver, gallbladder and pancreas in each modality. The CNN network introduces an attention mechanism during training, which enables the network to automatically focus on key areas in the image, such as tumor boundaries, bile duct courses, and blood vessel distribution around the pancreas. Through multi-scale feature extraction, the network can capture feature points at different levels from coarse to fine, and locate key anatomical structures in the image. In this process, a total of N = 150 feature points were extracted, including the edge points of the liver, the intersection of the bile duct, and the lesion area around the pancreatic duct, providing basic data for subsequent registration calculations.

[0065] Next, the feature points in the images of different modalities are aligned by optimizing the matching algorithm. Specifically, the optimization function J is used. match To measure the registration error. The adjustment coefficients α and β are set to range from 0.5 to 2.0 and 1.0 to 3.0 respectively to balance the weight of position difference and gradient information. In the actual calculation, α = 1.2 and β = 2.5 are set and substituted into the formula:

[0066]

[0067] Here, |x i -y i | represents the Euclidean distance of the feature point positions in CT and MRI images. By detecting the position errors of these points in different modality images, the accuracy of position alignment is obtained; It represents the gradient difference at the feature point, which is used to evaluate the changes in local structural features, especially in complex anatomical areas, such as the junction of lesions and normal tissues, and can reflect the alignment accuracy of the image on the boundary details. After calculation, the average position difference of each feature point is set to 0.8 mm, the gradient error is 1.1 units, and the above parameters are substituted to obtain:

[0068]

[0069] This result reflects the sum of the registration errors. matchBy minimizing the adjustment of the network parameters, the registration results between images are continuously optimized, so that the final three-dimensional images of the liver, gallbladder and pancreas can be aligned with high precision between different modal images. After the registration is completed, the doctor obtains a set of high-quality fused images that can clearly present the true anatomical structure and pathological conditions of the patient's liver, gallbladder and pancreas. This non-rigid registration algorithm greatly improves the alignment accuracy of multi-modal images and provides reliable data support for three-dimensional reconstruction.

[0070] In the above case, the doctor has obtained the patient's CT, MRI and ultrasound images, and extracted the key feature points in the multimodal images through a convolutional neural network (CNN) combined with an attention mechanism. Next, the non-rigid registration algorithm directly learns the mapping relationship between images of different modalities through an end-to-end deep learning network. Specifically, the input is unregistered CT and MRI images, and the goal of the network is to align these images so that they achieve a highly consistent anatomical structure expression in space. During the training process, the registration network not only learns global image transformations, but also captures local non-rigid deformations, especially morphological changes in complex hepatobiliary and pancreatic areas, such as the curved part of the liver, the course of the bile duct, and the diseased tissue around the pancreas.

[0071] In order to ensure that the registration result can retain the anatomical information in the image to the greatest extent, the system defines a composite loss function L to measure the overall quality of the registration. This loss function consists of two parts: one part measures the pixel-level similarity, and the other part combines the matching error of the feature points. During the registration process, pixel-level similarity is used to ensure that the pixel intensity difference between the registered target image T(x) and the reference image R(x) is minimized, that is, the aligned CT and MRI images are as consistent as possible in grayscale value or intensity. At the same time, the matching error J of the feature points match The precise alignment of the anatomical structure is controlled to ensure that key anatomical points (such as vascular intersections and lesion areas) remain consistent in multimodal images. To further explain this process, the following formula is used to express the total loss function:

[0072] L=λ1·∫ Ω (T(x)-R(x)) 2 dx+λ2·J match

[0073] In the formula, L represents the total loss function, which measures the quality of the entire registration process; λ1 and λ2 are loss weight parameters, which control the balance between pixel-level similarity and feature point matching errors. The value range can be set between λ1 = 0.8 and 1.5, and between λ2 = 1.0 and 2.0 to ensure a reasonable balance between the two in the registration process. In this case, λ1 = 1.2 and λ2 = 1.5 are selected. These parameters adjust the emphasis of the registration, and the weight value determines the different attention of the network to pixel similarity and anatomical matching.

[0074] In the specific registration process, the target image T(x) is set to be the MRI image after registration, and the reference image R(x) is set to be the CT image. For a given pixel point x, its pixel values ​​T(x) and R(x) are 120 and 115 respectively, indicating the intensity difference of the pixel point in the two images. According to the square formula of the pixel difference (T(x)-R(x)) 2 , we can calculate the intensity difference of this pixel as (120-115) 2 = 25. Assuming the range of the image definition domain Ω to be the entire image area, containing 1 million pixels, the intensity difference of each pixel is summed up, and combined with the integral, the sum of the pixel-level similarity is obtained:

[0075] ∫ Ω (T(x)-R(x)) 2 dx = 25 × 1,000,000 = 25,000,000

[0076] Next, combined with the aforementioned feature point matching error J match The calculated result is 469.8, and the two loss parts can be substituted into the total loss function:

[0077] L=1.2×25,000,000+1.5×469.8=30,000,000+704.7=30,000,704.7

[0078] By minimizing this total loss function L, the network continuously adjusts the transformation parameters in the registration process to optimize the alignment of CT and MRI images. This optimization process ensures the global pixel similarity of the images while maintaining high-precision alignment of local anatomical features, especially in complex structural areas such as the bile duct and pancreatic duct.

[0079] Through the previous registration process, doctors have preliminarily aligned the patient's CT and MRI images, but in order to further improve the registration accuracy, especially in capturing the details of the complex anatomical areas of the liver, gallbladder and pancreas, a multi-level pyramid strategy was adopted. This strategy divides the registration process into multiple resolution levels, gradually optimizing from low resolution to high resolution, and achieving a more refined registration effect through a layer-by-layer progressive approach.

[0080] In this process, the original image data is first downsampled to generate a set of pyramid levels, starting with the lowest resolution coarse image, and registration is performed at each level to gradually optimize the deformation field. When processing low-resolution images, the network can capture a large range of global deformations relatively quickly, laying the foundation for subsequent fine-grained optimization of higher resolutions. Specifically, in the first layer of the lowest resolution, the registration network generates a preliminary spatial deformation field φ(x) that describes the overall deformation between images. Through the deep learning model, the system automatically adjusts the deformation field to achieve rough alignment. After completing this preliminary alignment, enter the next level, increase the resolution of the image to a medium level, and use the deformation field of the previous layer as the initial value. Perform multiple iterative optimizations to gradually refine the deformation field and capture more detailed feature changes in the image.

[0081] In order to further optimize and constrain the smoothness and continuity of the spatial deformation field, an optimization function is introduced at each level The smoothness and structural consistency of the deformation field are controlled. The optimization function is defined as:

[0082]

[0083] in, To optimize the objective function, measure the smoothness and structural consistency of the deformation field; φ(x) is the current spatial deformation field function, which indicates the degree of image deformation during the current registration process; φ prev (x) is the deformation field of the previous iteration, ensuring that the deformation field maintains continuity and consistency in multiple optimizations; κ and ζ are adjustment coefficients, respectively controlling the weights of deformation smoothness and continuity differences, with κ ranging from 0.3 to 1.0 and ζ ranging from 1.0 to 2.5. In this case, κ=0.5 and ζ=1.8 are set and substituted into the optimization formula for calculation.

[0084] Set the gradient size of the current deformation field φ(x) is 0.6, indicating that the local deformation is moderately smooth, and the difference between the current and previous iterations of the deformation field ||φ(x)-φ prev (x)|| is 0.4, reflecting the continuous adjustment of the midfield in this optimization. After substituting these data, the calculation process of the optimization function is as follows:

[0085]

[0086] Assuming the area corresponding to the definition domain Ω is the entire image area, about 1 million pixels, the sum of the optimization objective function is:

[0087]

[0088] This result reflects the optimization effect of the deformation field at the current level, by minimizing The system optimizes the deformation field layer by layer, so that each iterative adjustment maintains the smoothness and continuity of the image structure and avoids unnatural deformation. After each layer of pyramid optimization, the image gradually returns to its original high resolution, and the optimized deformation fields of all levels are accumulated and superimposed. The final generated deformation field not only captures the global structural consistency, but also finely preserves local details. Through this multi-level pyramid strategy, doctors can obtain highly aligned CT and MRI images, and the three-dimensional structure of the liver, gallbladder and pancreas is finely restored, including the course of the bile duct, the morphology of the pancreatic duct, and the complex structure of the surrounding blood vessels, ensuring the reliability of the image in clinical applications.

[0089] Embodiment 2:

[0090] Combined with Figure 3 In the example of the patient in Example 1, after the doctor accurately registers the liver, gallbladder and pancreas using multimodal images, in order to further achieve 3D reconstruction, a preliminary 3D mesh model needs to be constructed. This step is crucial to the precision of 3D reconstruction, especially for the complex liver, gallbladder and pancreas area, which must capture the true geometric features of the organs to provide reliable 3D model support for subsequent diagnosis and treatment.

[0091] When constructing the 3D mesh model, the deep learning enhanced model is first used to extract the boundary data in the image. Through extensive training, deep learning models, such as the optimized U-Net structure, can automatically identify and extract the precise boundaries of the liver, bile duct, and pancreas in multimodal images, especially for areas with fuzzy boundaries and complex shapes (such as the tortuosity of the bile duct and the small ducts of the pancreas).

[0092] The extracted boundary data is further converted into a dense point cloud, whose density is dynamically adjusted according to the complexity of the organ. Specifically, the generation of the point cloud follows an adaptive density strategy: in areas with drastic morphological changes and high curvature (such as the bile duct junction and near the pancreatic duct), the density of the point cloud will be significantly increased to capture fine geometric features; while in smooth and less changing areas (such as the liver surface), the point cloud density is relatively low to optimize computational efficiency and resource utilization. This density adjustment is achieved by optimizing the energy function E contour The implementation is defined as follows:

[0093]

[0094] Among them, E contour is the energy function of contour extraction, which is used to measure the quality of boundary extraction; Ω is the definition domain of the image, which represents the area where the contour is calculated, including the organ part in the entire image; is the gradient of the image, indicating the intensity change of the edge of the image, and is mainly used to capture the sharpness of the boundary. The higher the edge intensity, the more obvious the contour. C(x) is the extracted contour function, indicating the position of the contour in the image. is the second-order derivative of the contour, which evaluates the smoothness of the contour and ensures that the boundary is smooth during the extraction process without too many irregular changes; α and β are adjustment parameters, which control the contribution of image gradient and contour smoothness to the total energy respectively. The value range of α is 1.0 to 2.0, and the value range of β is 0.5 to 1.5. These parameters are adjusted according to the actual image characteristics to ensure the accuracy of boundary extraction.

[0095] In this case, the doctor set α = 1.5 and β = 1.0. These parameters reflect the balance between edge strength and contour smoothness. The average value of is 0.8, indicating that the edge strength is moderate, and the second-order derivative of the contour function is is 0.6, which reflects that the boundary has good smoothness. Substitute it into the above formula to calculate the energy:

[0096] E contour =∫ Ω (1.5 0.8 + 1.0 (0.6) 2 )dx=∫ Ω (1.2+0.36)dx=∫ Ω 1.56dx

[0097] Assuming that the domain Ω of the image includes the entire organ area, with a total of 500,000 pixels, the total energy of contour extraction is:

[0098] E contour =1.56×500,000=780,000

[0099] This energy value reflects the quality of the extracted boundary. By minimizing the energy function, the system continuously optimizes the parameters of boundary extraction to ensure the accuracy and continuity of the contour. Based on this high-quality boundary data, the generated dense point cloud can accurately describe the complex geometric morphology of the liver, gallbladder and pancreas, providing accurate data support for the subsequent three-dimensional mesh construction. In actual operation, after the point cloud is generated, it is converted into a three-dimensional mesh through a triangulation algorithm (such as Delaunay triangulation) to construct a preliminary three-dimensional structure. This mesh model will become the basis for subsequent refined mesh adjustment and reconstruction.

[0100] Doctors generated a preliminary 3D data set by extracting boundary data and dense point clouds. In order to complete the construction of the 3D mesh, the point cloud data was further converted into a triangular mesh. This process uses Delaunay triangulation or MarchingCubes algorithm to quickly generate a mesh structure suitable for 3D reconstruction. In specific applications, point cloud data is converted into triangular meshes through algorithms. These mesh units express the 3D structure of the liver, gallbladder, and pancreas in the form of triangular faces. Due to the complexity of organ morphology, such as the tortuous course of the bile duct and the fine structure of the pancreatic duct, in order to accurately express these features, the system needs to dynamically adjust the density of the mesh units.

[0101] During the mesh generation process, the size of the mesh unit is automatically optimized and adjusted according to the local density of the point cloud and the geometric characteristics of the organ. Especially in complex morphological areas, such as the curved bile duct and pancreatic duct, the system increases the mesh density to ensure the fine expression of these key structures. Specifically, in these complex areas, the point cloud density is higher, the number of generated triangles is more, and the unit area is smaller to improve the accuracy of the surface expression; while in smooth areas, such as the liver surface, the mesh density is relatively reduced, and the generated mesh unit area is larger. This method reduces unnecessary calculations while ensuring the smoothness and stability of the overall structure.

[0102] In order to evaluate the rationality and accuracy of the generated 3D mesh, the system performs geometric analysis on the mesh, especially measuring and evaluating the shape characteristics of each mesh unit. These characteristics include local curvature, edge length, and connection angle of mesh units to ensure that the mesh can not only truly express the morphology of the organ, but also maintain good structural consistency in subsequent adjustments and optimizations. In order to quantify the quality of the mesh, the mesh quality evaluation function E is defined mesh ,as follows:

[0103]

[0104] Among them, E mesh It is used to measure the quality and detail capture ability of the mesh; Ω is the mesh calculation area, covering the entire three-dimensional mesh model; κ(x) is the local curvature, which reflects the degree of curvature of the mesh surface at different parts, especially in complex areas such as the bile duct and pancreatic duct, where the curvature value is higher, indicating the fineness of the mesh in these areas; L(x) is the side length of the mesh unit, which controls the size and morphological consistency of the mesh, especially in smooth areas where the side length is larger to reduce the generation of overly dense meshes; θ is the connection angle between mesh units, which detects the smooth transition of the mesh and ensures smooth connection between units; γ and δ are adjustment parameters, which respectively control the influence of curvature and side length on mesh quality, with the value range of γ being 0.8 to 1.5 and the value range of δ being 0.5 to 1.2.

[0105] In actual operation, the average value of local curvature κ(x) is set to 0.7 in complex areas, and the edge length change rate =0.5, these data reflect the performance of the mesh in capturing the complex morphology of the liver, gallbladder and pancreas. Substituting these parameters into the mesh quality evaluation formula, set γ = 1.2 and δ = 0.9:

[0106] E mesh =∫ Ω (1.2·0.7+0.9·0.5)dx=∫ Ω (0.84+0.45)dx=∫ Ω 1.29dx

[0107] If the mesh calculation area contains 800,000 triangles, the total mesh quality is:

[0108] E mesh =1.29×800,000=1,032,000

[0109] This energy value indicates the overall quality of the mesh. By continuously adjusting the density, side length and curvature of the mesh units and optimizing the detail capture capability of the mesh, the system can ensure the accuracy and rationality of the 3D mesh in each key area. In this case, the generated 3D mesh model accurately expresses the anatomical features of the patient's liver, gallbladder and pancreas, especially the bends of the bile duct and the delicate structure of the pancreatic duct, which are highly restored.

[0110] In order to further optimize the mesh to adapt to the complex hepatobiliary and pancreatic anatomical structures, doctors need to apply adaptive mesh optimization algorithms to dynamically adjust the distribution of mesh units to more accurately fit the true morphology of the organs. This step is particularly critical, especially in the expression of complex morphologies such as the curved part of the bile duct and around the pancreatic duct in high curvature areas, by increasing the mesh density to refine these structures, while appropriately reducing the mesh density on the smooth liver surface to maintain computational efficiency and resource optimization.

[0111] The core of adaptive mesh optimization is to dynamically adjust the distribution of mesh cells according to the geometric characteristics of the organ. In the specific implementation, the algorithm uses Laplacian smoothing and deformation remapping techniques, which can locally adjust the mesh density without changing the overall structure of the mesh. Laplacian smoothing is used to maintain the smoothness of the mesh and prevent mutations caused by local density changes; while deformation remapping ensures that the mesh still accurately retains the shape and boundary characteristics of the original organ during the adjustment process. The optimization effect is achieved by adaptively adjusting the energy function E adjust To quantify the evaluation, the formula is as follows:

[0112]

[0113] Among them, E adjust represents the energy function of adaptive adjustment, which is used to measure the rationality of the density and morphology changes of the grid during the adjustment process; Ω is the calculation domain of the grid adjustment, covering the entire three-dimensional grid model; λ is the adjustment coefficient, which controls the sensitivity of the density adjustment, and its value range is between 0.5 and 2.0; κ thresh is the curvature threshold, which defines the condition for triggering high-density adjustment, and is usually set to 0.6 to 1.0 to mark the area with higher curvature; φ(x) is the mesh deformation field function, which describes the displacement change of the mesh during the adjustment process; It is the second-order derivative of the deformation field, which is used to control the smoothness and smoothness of the mesh to ensure that the mesh does not produce abrupt deformation during the adjustment process.

[0114] In actual operation, the average value of the local curvature κ(x) in the bile duct area is set to 0.9, which is higher than the set curvature threshold κ thresh = 0.7, indicating that the mesh density needs to be increased in this area to capture details. The adjustment coefficient λ is set to 1.2, indicating that the response to the curvature change is more sensitive; at the same time, the second-order derivative of the deformation field is set is 0.4, which reflects that the smoothness of the grid adjustment is moderate. Substitute these data into the optimization formula for calculation:

[0115]

[0116] Assuming that the adjustment domain Ω includes the entire grid area, which is about 600,000 cells, the total energy is:

[0117] E adjust =(0.787+0.4)×600,000=1.187×600,000=712,200

[0118] This result reflects the overall quality of the current mesh adjustment, which is achieved by continuously optimizing and minimizing E adjust , the system can make fine adjustments to the grid units so that the grid can maintain a high degree of morphological fidelity and structural consistency in areas with complex morphology such as the bile duct and pancreatic duct, while maintaining a moderate grid density on the smooth liver surface to ensure computational efficiency and rational use of resources. After completing the adaptive optimization, the doctor obtained a more refined and accurate three-dimensional grid model. By comparing the grids before and after the adjustment, it can be clearly seen that the grid density and morphological details of the key structures have been significantly improved, especially the curvature of the bile duct, the complex course of the pancreatic duct, and the smooth transition of the liver surface have all been accurately expressed.

[0119] In order to further refine the mesh shape and ensure that the mesh maintains shape consistency and smooth boundaries during overall adjustment, the doctor introduced a deformation control technology based on energy minimization. The core of this technology is to accurately control the shape adjustment of mesh units by optimizing the energy function, especially in complex areas, to avoid geometric distortion caused by excessive deformation, while ensuring smooth transition of the mesh.

[0120] During the specific operation, the system introduces the energy-minimizing deformation control function E deform , to quantify the morphological changes of the mesh during adjustment. By adjusting the various parameters in the energy function, the deformation of the mesh unit is optimized so that the mesh can not only highly match the actual morphology of the organ, but also maintain a smooth boundary transition, especially in areas with morphological mutations, such as the edge of the liver, the bends of the bile duct, and the complex distribution of the pancreatic duct. The definition of the energy function is as follows:

[0121]

[0122] Among them, E deform represents the energy function of mesh deformation control, which measures the morphological change and consistency of the mesh during the adjustment process; Ω is the domain of mesh calculation, covering the range of all mesh units; μ and ν are adjustment parameters, which control the contribution of deformation intensity and boundary smoothness respectively, with the value range of μ being 1.0 to 2.5 and the value range of ν being 0.5 to 1.5; is the squared gradient term of the deformation field, which is used to smooth the deformation of the mesh and avoid excessive deformation of the mesh in the local area; θ edge It is the local angle of the mesh boundary, reflecting the adjustment of the mesh at the boundary to ensure a smooth transition of the boundary.

[0123] In practical applications, set the square gradient value of the current mesh deformation field The average value is 0.3, indicating that the overall deformation strength of the mesh is moderate and the local angle θ of the boundary is edge is 45 degrees, i.e. cos(45°) = 0.707, which reflects that the grid has good smoothness at the boundary. Select adjustment parameters μ = 1.8 and ν = 1.0 and substitute them into the formula for calculation:

[0124] E deform =∫ Ω (1.8·0.3+1.0·0.707)dx=∫ Ω (0.54+0.707)dx=∫ Ω 1.247dx

[0125] Assuming that the grid computation domain Ω contains 800,000 grid elements, the total energy of deformation control is:

[0126] E deform=1.247×800,000=997,600

[0127] This energy value represents the overall shape change of the current grid. deform Through continuous optimization and minimization, the system can accurately control the mesh adjustment process, so that the mesh can accurately express the morphological characteristics of organs in complex areas while maintaining a smooth transition of the boundaries. The optimization results show that the subtle bends of the bile duct, the multi-level branches of the pancreatic duct, and the smooth connection of the liver surface have been significantly improved, and there are no unnatural mutations or distortions between the mesh units, ensuring the high authenticity of the reconstructed model.

[0128] This deformation control technology based on energy minimization not only solves the deformation challenge of the grid in complex morphological areas, but also effectively prevents morphological distortion caused by local adjustments, providing a more stable and coherent morphological basis for the three-dimensional grid model. In this example, the three-dimensional model generated by the doctor through this technology can clearly display the patient's hepatobiliary and pancreatic anatomical structure, providing more detailed reference data for preoperative planning, greatly improving the success rate of surgery and the accuracy of treatment. This solution verifies the feasibility and superiority of deformation control based on energy minimization in practical applications, and is an indispensable key step in the reconstruction of three-dimensional images of the hepatobiliary and pancreatic regions.

[0129] Example 3: Optimized application of hepatobiliary and pancreatic 3D image reconstruction method based on ASiR-V and DLIR

[0130] In this embodiment, the application of ASiR-V (hybrid iterative reconstruction algorithm) and DLIR (deep learning image reconstruction algorithm) in the reconstruction of hepatobiliary and pancreatic three-dimensional images will be explored, and combined with the technical advantages of the present invention, a more optimized three-dimensional reconstruction process is provided, aiming to improve the visualization quality of hepatobiliary and pancreatic structures and assist clinical diagnosis and surgical planning. For a 62-year-old male patient, the patient was treated for upper abdominal discomfort and was initially diagnosed with suspected hepatobiliary system lesions. The doctor performed an abdominal CT scan and used different reconstruction algorithms to process the images in order to reconstruct the anatomical details of the hepatobiliary and pancreatic three-dimensionally.

[0131] 1. Data acquisition and processing: The patient underwent abdominal CT scan to obtain high-resolution imaging data.

[0132] Subsequently, the image data were reconstructed using the ASiR-V and DLIR algorithms, respectively.

[0133] Combined with Figure 4As shown in , the image reconstruction intensity using the ASiR-V algorithm varies from 10% to 100% (corresponding to images A to L), while the DLIR algorithm reconstructs at low, medium, and high intensity levels (corresponding to M, N, and O in image 4). The ASiR-V algorithm effectively reduces the noise in the image by increasing the reconstruction intensity, but the high-intensity reconstruction leads to excessive smoothing of the image, resulting in the loss of some details. Especially at an intensity of up to 100% ( Figure 4 L), the fine structures of the liver edge, bile duct and pancreas appear blurred, affecting the doctor's diagnosis. DLIR shows better image detail preservation and noise reduction at different intensities. Low-intensity DLIR ( Figure 4 M) is already able to balance noise and image clarity well, while medium intensity ( Figure 4 N) further improves image quality, making the anatomical structures of the liver, bile duct, and pancreas more distinct; high-intensity DLIR ( Figure 4 O) Almost noise-free and with sharp details, it provides the best image basis for 3D reconstruction.

[0134] 2. Three-dimensional reconstruction process: Based on the image data reconstructed by DLIR high-intensity, the hepatobiliary and pancreatic three-dimensional image reconstruction method of the present invention is used to carry out the following optimization steps: First, the boundaries of the hepatobiliary and pancreatic are automatically segmented using a deep learning-assisted boundary detection algorithm to extract the contour data of the liver edge, bile duct course, pancreatic structure and its lesion area. Through the multimodal image fusion strategy (CT and DLIR images), the fine expression of multi-level anatomical structures is achieved to ensure the integrity and consistency of the data. Then, the segmented multimodal data is aligned through a non-rigid registration algorithm to ensure the precise overlap of each layer of the image in the three-dimensional space, providing an accurate basis for the construction of a three-dimensional mesh model.

[0135] 3. Optimization and construction of mesh model: When constructing the preliminary three-dimensional mesh model, the dense point cloud extracted from the segmentation data was used to generate the preliminary mesh through the Delaunay triangulation algorithm. Considering the complex morphology of the hepatobiliary and pancreatic regions, especially the tortuous course of the bile duct and the fine structure of the pancreatic duct, an adaptive mesh optimization algorithm was applied to dynamically adjust the mesh density according to local morphological changes: in high curvature areas (such as the bile duct junction), the mesh density was increased to capture morphological details; in smooth areas (such as the liver surface), the mesh density was reduced to optimize computational efficiency. By introducing Laplacian smoothing and deformation remapping technology, the smoothness and consistency of the mesh cells were adjusted to ensure that the mesh retained its original morphology and boundary features during the adjustment process.

[0136] 4. Deformation control based on energy minimization: In order to further improve the mesh quality, a deformation control technology based on energy minimization is introduced during the adjustment process. deform, accurately control the deformation strength and smoothness of the mesh to prevent mesh distortion caused by local adjustments. The energy function used is:

[0137]

[0138] In this embodiment, μ=1.8 and v=1.0, and the square value of the deformation field gradient is is 0.3, the boundary angle θ edge 45 degrees The value of the optimized energy function is:

[0139] E deform =∫ Ω (0.54+0.707)dx=1.247×800,000=997,600

[0140] Through the above-mentioned optimized reconstruction process, doctors finally obtained a clear, delicate and extremely low-noise three-dimensional model of the liver, gallbladder and pancreas, which can clearly show the surface texture of the liver, the subtle bends of the bile duct and the pathological changes of the pancreas.

[0141] Example 4: Liver, gallbladder and pancreas three-dimensional image reconstruction method based on DLIR and ASiR-V technology optimization:

[0142] This embodiment combines the image data reconstructed at different intensities by DLIR (deep learning image reconstruction algorithm) and ASiR-V (hybrid iterative reconstruction algorithm), and elaborates on how to use the hepatobiliary and pancreatic three-dimensional image reconstruction method of the present invention to optimize clinical image quality and improve the diagnosis and treatment planning of hepatobiliary and pancreatic lesions. The patient was a 58-year-old female who was admitted to the hospital due to upper abdominal pain. Preliminary imaging examinations suspected the presence of bile duct stones and pancreatitis. In order to further clarify the scope of the lesion and evaluate the impact of the lesion on the surrounding tissues, the doctor performed an abdominal CT scan on the patient and processed the image using different reconstruction algorithms to perform three-dimensional reconstruction of the anatomical details of the hepatobiliary and pancreatic.

[0143] The patient's CT scan data were reconstructed at different intensities using the DLIR and ASiR-V algorithms, respectively.

[0144] Attached Figure 5The comparison of image reconstruction using DLIR (low, medium and high intensities correspond to Figures DLIR-L, DLIR-M and DLIR-H, respectively) and ASiR-V (0%, 10%, 20% and 30% intensities correspond to ASiR-V0%, ASiR-V10%, ASiR-V20% and ASiR-V30%, respectively) is shown in Figure 3. The DLIR reconstructed images show excellent noise suppression and detail preservation at all intensities, especially the high-intensity DLIR-H images, which can clearly show the structural details of the liver, bile duct and pancreas, while the ASiR-V has significantly larger noise at low intensities (0% and 10%). As the intensity increases, the noise decreases, but the image smoothness increases, and some details are lost due to excessive smoothing, especially at 20% and 30% (Figures ASiR-V20% and ASiR-V30%), the loss of details is more obvious.

[0145] Attached Figure 6 The image noise measurements of the aorta and liver regions are shown in Figure 2. The measurements were performed using a single ROI in the aorta and three ROIs in different parts of the liver. The results showed that the DLIR-reconstructed images showed the lowest noise level at all intensities. In particular, the noise values ​​of DLIR-H in the aorta and various liver regions were significantly lower than those of ASiR-V images at all levels, reflecting the superiority of DLIR in noise suppression and detail preservation. Compared with ASiR-V, DLIR reconstruction not only provides clearer images, but also avoids the loss of details caused by high-intensity smoothing in iterative reconstruction. In particular, DLIR performs better in complex hepatobiliary and pancreatic structures.

[0146] Attached Figure 7-8 The noise levels of seven image reconstruction techniques are detailed in box plots. Figure 7 The average image noise measured from three liver ROIs is shown. Figure 8 The noise measurement results at the aorta are shown in the figure. As can be seen from the figure, DLIR-H has the lowest noise values ​​in the liver and aorta, and has the best image quality; in contrast, ASiR-V has significantly higher noise at lower intensities, and although high intensity reduces noise, it loses a lot of detail information during the smoothing process, which is not conducive to the accurate identification of hepatobiliary and pancreatic lesions.

[0147] Based on the above noise measurement and image quality assessment results, this embodiment uses high-quality CT images reconstructed by DLIR-H as the basis for three-dimensional reconstruction. First, the boundary information of the liver, gallbladder, pancreas and surrounding blood vessels is extracted through a deep learning-driven segmentation algorithm to ensure the accuracy and continuity of image segmentation. Subsequently, the multimodal image registration technology of the present invention is used to accurately align the DLIR image with other imaging modes (such as MRI and ultrasound) to comprehensively utilize the advantages of each modality image, especially the advantages of DLIR in detail retention and noise control, to provide high-precision input data for subsequent three-dimensional modeling.

[0148] During the three-dimensional reconstruction process, the grid cell density is dynamically adjusted through an adaptive grid optimization algorithm, so that high grid density is maintained in complex morphological areas such as the bile duct and pancreatic duct, while the density is reduced in smooth areas such as the liver surface, optimizing the use of computing resources. In order to ensure the consistency and boundary smoothness of the grid during the adjustment process, a deformation control technology based on energy minimization is introduced to optimize the morphological adjustment of the grid, especially in key lesion sites, to ensure that the three-dimensional model can truly reflect the pathological state of the liver, gallbladder and pancreas. The final three-dimensional reconstructed model clearly shows the tissue structure of the liver, the course of the bile duct and the morphological characteristics of the pancreas, which can help doctors better evaluate the impact of lesions on surrounding tissues in preoperative planning, optimize surgical path selection, and reduce surgical risks. In addition, the advantages of DLIR technology in low noise and high detail retention make the three-dimensional model show micro-lesions more clearly, providing clinicians with a more intuitive and reliable imaging basis, significantly improving the accuracy of diagnosis and the precision of treatment.

Claims

1. A method for reconstructing a three-dimensional image of the liver, gallbladder and pancreas, characterized in that The following steps are involved: S1. Hepatobiliary and pancreatic image segmentation assisted by deep learning: S1.

1. Introduce a U-Net variant segmentation network for the liver, gallbladder, and pancreas, and use a convolutional neural network to segment multimodal images, including the separation of the liver, bile duct, and pancreas; S1.2, introduce boundary enhancement module in the segmentation process; S2. Image registration and pairing fusion: S2.

1. Use a deep learning-driven non-rigid registration algorithm to spatially align image data of different modalities; use the anatomical feature points of the organs for registration; S2.

2. Based on the registration results, multimodal fusion of liver, gallbladder and pancreas images is performed, and fusion image data is constructed by comprehensively utilizing the bone and tissue contrast of CT, the soft tissue resolution of MRI and the real-time dynamic characteristics of ultrasound; S3, Adaptive 3D mesh reconstruction: S3.

1. Extract the contours of the liver, gallbladder and pancreas from the segmented image and construct a preliminary three-dimensional mesh model; adaptively adjust the mesh distribution according to the geometric morphology of different organs to control the distribution of the mesh during reconstruction to match the organ morphology; S3.2, using an iterative optimization algorithm to adjust the mesh vertices including the bile duct system and pancreatic duct to fit the actual organ shape; S4, personalized texture mapping and enhanced realism: S4.

1. Extract the texture features of the liver, gallbladder and pancreas using fused image data, including tissue density, lesion area and vascular distribution; S4.

2. Map the extracted texture features to the 3D model surface and dynamically adjust the texture based on individual differences including tumor location and degree of fibrosis.

2. The method for reconstructing a three-dimensional image of the liver, gallbladder and pancreas according to claim 1, characterized in that The non-rigid registration algorithm includes: using a convolutional neural network (CNN) combined with an attention mechanism and multi-scale feature extraction to identify and locate organ edges, vascular intersections, and feature points of lesion areas in images; and aligning corresponding feature points in images of different modalities by optimizing a matching algorithm: Among them, J match represents the sum of feature point matching errors, N is the number of feature points; x i and i are the coordinates of the corresponding feature points in different modal images, indicating the position difference; α and β are adjustment coefficients, controlling the weights of different error terms; |·| represents the Euclidean distance, reflecting the distance error between feature points; and They are the gradient information at the feature points, indicating the changes in local structural features.

3. The method for reconstructing a three-dimensional image of the liver, gallbladder and pancreas according to claim 2, characterized in that The non-rigid registration algorithm includes: directly learning the mapping relationship between images through end-to-end training, with the input being an unregistered multimodal image and the output being an aligned image; defining a composite loss function to measure the similarity of the entire image, combining the matching error of anatomical feature points, and controlling the data retention of the anatomical structure during the registration process.

4. The method for reconstructing a three-dimensional image of the liver, gallbladder and pancreas according to claim 3, characterized in that The non-rigid registration algorithm includes: introducing a multi-level pyramid strategy in the registration process, dividing the registration process into multiple resolution levels, and gradually optimizing from low resolution to high resolution; in each layer of the pyramid, generating a preliminary spatial deformation field through deep learning, and then performing multiple iterative optimizations to refine the deformation field; in order to optimize the spatial deformation field, introducing an optimization function to constrain the continuity and smoothness of the deformation: in, To optimize the objective function, measure the smoothness and structural consistency of the deformation field; φ(x) is the current spatial deformation field function, which indicates the degree of image deformation during the current registration process; φ prev (x) is the deformation field of the previous iteration, which ensures the continuity and consistency of the deformation field; κ and ζ are adjustment coefficients, which control the weights of different error terms; is the gradient of the deformation field, which measures the local smoothness; ||φ(x)-φ prev (x)|| represents the difference between the current and previous deformation fields.

5. The method for reconstructing a three-dimensional image of the liver, gallbladder and pancreas according to any one of claims 1 to 4, characterized in that The method for constructing a preliminary three-dimensional mesh model includes: using a deep learning enhanced model to extract boundary data; and converting the extracted contour data into a dense point cloud, and the density of the point cloud is dynamically adjusted according to the complexity of the organ structure to capture the geometric characteristics of each organ.

6. The method for reconstructing a three-dimensional image of the liver, gallbladder and pancreas according to claim 5, characterized in that The method for constructing a preliminary three-dimensional mesh model includes: generating a three-dimensional mesh from dense point cloud data, and converting the point cloud into a triangular mesh using a Delaunay triangulation or MarchingCubes algorithm; During the mesh generation process, the mesh cell size is adjusted according to the local density and geometry of the point cloud, so that the mesh density is kept high in the morphologically complex areas including the curved bile duct and pancreatic duct, while the mesh cell density is reduced in the smooth areas including the liver surface. The generated mesh is then geometrically analyzed to evaluate the shape characteristics of each mesh cell, including curvature, side length, and angle: Among them, E mesh is a mesh quality evaluation function, which is used to measure the rationality and detail capture ability of the mesh; κ(x) is the local curvature, which describes the curvature of the mesh unit surface, especially reflecting the morphological changes in complex areas; L(x) is the side length of the mesh unit, which controls the size and morphological consistency of the mesh unit; θ is the local angle, which represents the connection angle between mesh units and is used to detect the smooth transition of the mesh; γ and δ are adjustment parameters, which adjust the influence weight of curvature and side length on mesh quality.

7. The method for reconstructing a three-dimensional image of the liver, gallbladder and pancreas according to claim 6, characterized in that The method for constructing a preliminary three-dimensional mesh model includes: using an adaptive mesh optimization algorithm to dynamically adjust the distribution of mesh units according to the geometric characteristics of the organ; wherein the mesh density is adjusted according to local morphological changes, including increasing the mesh density in high curvature areas; and applying a Laplacian smoothing and deformation remapping adaptive algorithm so that the mesh adjustment retains the shape and boundary characteristics of the original organ.

8. The method for reconstructing a three-dimensional image of the liver, gallbladder and pancreas according to claim 7, characterized in that The method for constructing a preliminary three-dimensional mesh model includes: introducing a deformation control technology based on energy minimization; maintaining the morphological consistency and boundary smoothness of mesh units during the adjustment process by optimizing the energy function, and controlling mesh deformation in complex areas: Among them, E deform is the energy function of mesh deformation control, which measures the morphological change of the mesh during the adjustment process; μ and ν are adjustment parameters, which control the contribution of deformation intensity and boundary smoothness respectively; is the squared gradient term of the deformation field, used to smooth the deformation of the mesh; θ edge is the local angle of the mesh boundary, indicating the adjustment of the mesh at the boundary.

Images