3D ultra-high-definition fluorescence medical endoscope vascular image enhancement method, system, device, and storage medium

Through multimodal image registration and depth perception technology, the problem of insufficient depth information acquisition and multi-scale processing in 3D ultra-high-definition fluorescent medical endoscopic vascular image enhancement technology is solved, and the high-quality enhancement and recognition of vascular images is achieved, which improves the improvement of vascular images. Accuracy and safety of minimally invasive surgery.

CN118781172BActive Publication Date: 2024-12-06南京晨新医疗科技有限公司
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411267398.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-11
Publication Date
2024-12-06
Estimated Expiration
2044-09-11

AI Technical Summary

Technical Problem

The existing 3D ultra-high-definition fluorescent medical endoscopic vascular image enhancement technology is insufficient in the acquisition and utilization of depth information, resulting in inconsistent vascular enhancement effects, making it difficult to simultaneously process multi-scale vascular structures, and the image noise and motion artifact suppression effect is not ideal, which affects the impact of Vascular recognition and tracking accuracy.

Method used

Using multimodal image registration and depth perception technology, the vascular structural features are extracted through initial depth estimation map and multi-scale Hessian matrix analysis, combined with the depth-perceived vascular response function and space-time collaborative tracking technology to perform deep adaptive noise reduction and vascular enhancement. , a signal attenuation model is constructed to compensate for signal changes caused by depth, and the original signal is restored through iterative constraint deconvolution algorithm, and finally the vascular skeleton is extracted and the 3D vascular network model is reconstructed by topological maintenance method.

Benefits of technology

Improve the quality and recognizable vascular images, enhance real-time vascular navigation capabilities in minimally invasive surgery, improve the accuracy and safety of the surgery, provide reliable imaging basis for personalized surgical plans, and promote precise medical treatment. develop.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118781172B_ABST
    Figure CN118781172B_ABST
Patent Text Reader

Abstract

The present invention discloses a 3D ultra-high-definition fluorescent medical endoscope vascular image enhancement method and system, device, and storage medium, including the following steps: acquiring and preprocessing the original image sequence; performing multimodal image registration and initial depth estimation; using depth information for adaptive noise reduction and vascular enhancement; building a signal attenuation model and scattering compensation; performing iterative deconvolution and dynamic range optimization; fusing 3D vascular images and depth maps; performing 3D vascular segmentation and skeleton extraction; and reconstructing a 3D vascular network model. The present invention achieves high-quality 3D vascular image enhancement and reconstruction through multi-scale processing of depth perception, spatiotemporal consistency constraints, and multimodal information fusion, improves the accuracy and visualization of vascular identification, and provides more accurate vascular navigation information.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to medical video image processing technology, in particular to a 3D ultra-high definition fluorescent medical endoscope vascular image enhancement method and system, device and storage medium. Background Art

[0002] Minimally invasive surgery, as an important development direction of modern medicine, greatly reduces the risk of trauma and complications for patients. In this field, the application of 3D ultra-high-definition fluorescent medical endoscope technology provides surgeons with the ability to observe fine anatomical structures. Accurately identifying and locating vascular structures is crucial to avoid unexpected bleeding, protect the function of important organs, and ensure surgical safety.

[0003] At present, 3D ultra-high-definition fluorescence medical endoscope vascular image enhancement technology has made significant progress. Traditional methods mainly rely on basic processing techniques such as image filtering and contrast enhancement. With the development of computer vision and machine learning, vascular segmentation and enhancement algorithms based on deep learning have gradually become a research hotspot. These methods can automatically learn the feature representation of blood vessels by training large-scale data sets, thereby improving the accuracy of vascular recognition.

[0004] However, the current 3D ultra-high-definition fluorescence medical endoscope vascular image enhancement technology still faces many challenges. Including: There are deficiencies in the acquisition and utilization of depth information. Existing methods often ignore the impact of image depth on vascular enhancement, resulting in inconsistent vascular enhancement effects at different depths, affecting the overall visualization quality. In addition, the processing of multi-scale vascular structures is still not fine enough. The simultaneous enhancement of micro-vessels and large vessels is a technical difficulty. Existing methods find it difficult to highlight the details of micro-vessels while maintaining the clarity of large vessels. In addition, the suppression of image noise and motion artifacts is not ideal, especially in low illumination conditions. These interference factors seriously affect the accuracy of vascular recognition and tracking. How to effectively integrate fluorescence information, depth information, and white light image information to provide a more comprehensive and accurate representation of the vascular network is still an urgent problem to be solved. Summary of the invention

[0005] The purpose of the invention is to provide a 3D ultra-high-definition fluorescence medical endoscope vascular image enhancement method and system, device, and storage medium to solve the above-mentioned problems existing in the prior art.

[0006] The technical solution, a 3D ultra-high-definition fluorescent medical endoscope vascular image enhancement method, comprises the following steps:

[0007] Step S1, obtaining the original 3D fluorescence endoscope image sequence and the white light image sequence, and performing preliminary quality inspection and preprocessing to remove obvious noise and artifacts; using the mutual information maximization method and the non-rigid registration method to accurately align the 3D fluorescence endoscope image sequence and the white light image sequence to obtain a registered multimodal image; based on the registered multimodal image, using stereo vision technology, generating an initial depth estimation map;

[0008] Step S2, using the initial depth estimation map, perform preliminary depth adaptive denoising on the registered multimodal image; based on the denoised image and the initial depth estimation map, perform the first refinement by the variational optimization method to generate an improved depth map; use the improved depth map to perform the second round of depth adaptive denoising; extract vascular structure features by multi-scale Hessian matrix analysis, and enhance vascular visibility by depth-aware vascular response function; use the dynamic scale integration method to fuse multi-scale responses to obtain a preliminary enhanced vascular image; combine the improved depth map information, use the spatiotemporal collaborative tracking technology to achieve accurate positioning and tracking of moving blood vessels, and obtain a time-consistent vascular enhancement sequence and a final depth map after the second refinement;

[0009] Step S3, constructing a signal attenuation model based on the final depth map to compensate for the signal intensity change caused by depth; modeling and compensating for the scattering effects at different depths through a multi-scale scattering kernel; using a time-consistent vascular enhancement sequence as prior information, and recovering the original signal through an iterative constrained deconvolution algorithm while considering signal attenuation and scattering effects; improving the visual quality and detail visibility of the image through a dynamic range optimization method, and obtaining a final optimized and displayed 3D ultra-high-definition fluorescent medical endoscope vascular image;

[0010] Step S4, fusing the 3D ultra-high-definition fluorescent medical endoscope vascular image and the final depth map to improve the accuracy of vascular recognition; performing 3D vascular segmentation through spatiotemporal consistency constraints and time-consistent vascular enhancement sequences; extracting the vascular skeleton using a topology-preserving method to represent the centerline and branch structure of the blood vessel; reconstructing a complete 3D vascular network model based on the extracted skeleton, radius information and the final depth map; fusing the reconstructed vascular network model with the optimized 3D ultra-high-definition fluorescent medical endoscope vascular image to generate a comprehensive visualization result.

[0011] According to another aspect of the present application, a 3D ultra-high definition fluorescence medical endoscope vascular image enhancement system is also provided, comprising:

[0012] at least one processor; and,

[0013] a memory communicatively connected to at least one of the processors; wherein,

[0014] The memory stores instructions that can be executed by the processor, and the instructions are used to be executed by the processor to implement the 3D ultra-high-definition fluorescence medical endoscope vascular image enhancement method described in any of the above technical solutions.

[0015] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the 3D ultra-high-definition fluorescence medical endoscope vascular image enhancement method described in any one of the above technical solutions are implemented.

[0016] A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the 3D ultra-high-definition fluorescence medical endoscope vascular image enhancement method described in any one of the above technical solutions.

[0017] Beneficial effects: This application first acquires and aligns the original images, highlights the vascular structure through noise reduction and feature extraction, and optimizes the depth information. Then compensates for the signal problems caused by depth to further improve the image quality. Finally, reconstruct the 3D vascular network and merge it with the high-quality 2D image. Through depth-aware multimodal processing, multi-scale vascular enhancement, spatiotemporal consistency optimization and 3D reconstruction technology, not only the quality and recognizability of vascular images are improved, but also strong imaging support is provided for real-time vascular navigation in complex minimally invasive surgery. It improves the accuracy and safety of minimally invasive surgery, provides a reliable imaging basis for the formulation of personalized surgical plans, and thus promotes the development of precision medicine. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a flow chart of the present invention.

[0019] Figure 2 It is a flow chart of step S1 of the present invention.

[0020] Figure 3 It is a flow chart of step S2 of the present invention.

[0021] Figure 4 It is a flow chart of step S3 of the present invention.

[0022] Figure 5 It is a flow chart of step S4 of the present invention. DETAILED DESCRIPTION

[0023] like Figure 1 As shown, the 3D ultra-high-definition fluorescence medical endoscope vascular image enhancement method includes the following steps:

[0024] Step S1, obtaining the original 3D fluorescence endoscope image sequence and the white light image sequence, and performing preliminary quality inspection and preprocessing to remove obvious noise and artifacts; using the mutual information maximization method and the non-rigid registration method to accurately align the 3D fluorescence endoscope image sequence and the white light image sequence to obtain a registered multimodal image; based on the registered multimodal image, using stereo vision technology, generating an initial depth estimation map;

[0025] Step S2, using the initial depth estimation map, perform preliminary depth adaptive denoising on the registered multimodal image; based on the denoised image and the initial depth estimation map, perform the first refinement by the variational optimization method to generate an improved depth map; use the improved depth map to perform the second round of depth adaptive denoising; extract vascular structure features by multi-scale Hessian matrix analysis, and enhance vascular visibility by depth-aware vascular response function; use the dynamic scale integration method to fuse multi-scale responses to obtain a preliminary enhanced vascular image; combine the improved depth map information, use the spatiotemporal collaborative tracking technology to achieve accurate positioning and tracking of moving blood vessels, and obtain a time-consistent vascular enhancement sequence and a final depth map after the second refinement;

[0026] Step S3, constructing a signal attenuation model based on the final depth map to compensate for the signal intensity change caused by depth; modeling and compensating for the scattering effects at different depths through a multi-scale scattering kernel; using a time-consistent vascular enhancement sequence as prior information, and recovering the original signal through an iterative constrained deconvolution algorithm while considering signal attenuation and scattering effects; improving the visual quality and detail visibility of the image through a dynamic range optimization method, and obtaining a final optimized and displayed 3D ultra-high-definition fluorescent medical endoscope vascular image;

[0027] Step S4, fusing the 3D ultra-high-definition fluorescent medical endoscope vascular image and the final depth map to improve the accuracy of vascular recognition; performing 3D vascular segmentation through spatiotemporal consistency constraints and time-consistent vascular enhancement sequences; extracting the vascular skeleton using a topology-preserving method to represent the centerline and branch structure of the blood vessel; reconstructing a complete 3D vascular network model based on the extracted skeleton, radius information and the final depth map; fusing the reconstructed vascular network model with the optimized 3D ultra-high-definition fluorescent medical endoscope vascular image to generate a comprehensive visualization result.

[0028] In this embodiment, the problem of insufficient acquisition and utilization of depth information is solved. By introducing multimodal image registration and initial depth estimation, combined with light field information and multi-view geometric constraints, high-precision depth map construction is achieved. This lays the foundation for subsequent depth adaptive processing and overcomes the defect of existing methods that ignores the impact of image depth on vascular enhancement. The introduction of a depth-adaptive anisotropic diffusion filter and a depth-aware vascular response function keeps the enhancement effect consistent over the entire depth range. This depth adaptive strategy improves the enhancement effect of blood vessels at different depths and improves the overall visualization quality, especially when processing tiny blood vessels in deep tissues.

[0029] This embodiment successfully solves the problem of insufficient precision in processing multi-scale vascular structures. By constructing a multi-scale Hessian matrix and a depth-aware vascular response function, adaptive enhancement of blood vessels of different calibers and depths is achieved. The multi-scale integration strategy further improves the detection capability of micro-vessels while maintaining the integrity of large blood vessels. This method can highlight the details of micro-vessels while maintaining the clarity of large blood vessels, overcoming the limitations of existing methods in dealing with complex vascular networks. Through multimodal fusion and 3D reconstruction, the visibility of micro-vessels is further enhanced while maintaining the topological structure of the overall vascular network, providing comprehensive and accurate vascular navigation information for complex minimally invasive surgery.

[0030] like Figure 2 As shown, according to one aspect of the present application, step S1 is specifically:

[0031] Step S11, acquiring original images from the 3D fluorescence medical endoscope system, including an original 3D fluorescence endoscope image sequence and a corresponding white light image sequence, and preprocessing these original images to obtain a fluorescence image sequence and a white light image sequence after preliminary optimization; wherein the preprocessing includes preliminary quality assessment, pulse noise removal, Gaussian noise suppression, motion artifact removal, and image contrast enhancement;

[0032] Step S12, inputting the preprocessed fluorescence image sequence and the white light image sequence into a preconfigured registration module, performing coarse registration and fine registration, and generating a registered multimodal image set that is precisely aligned with the white light image sequence;

[0033] The coarse registration uses the mutual information maximization method to achieve global rigid transformation and obtain the initial registration parameters; the fine registration uses the non-rigid registration algorithm based on B-spline to construct the optimization objective function and iteratively optimize to obtain the optimal registration transformation applied to the preprocessed fluorescence image sequence;

[0034] Step S13, read the registered multimodal image set, and for each pixel position, calculate the normalized cross-correlation value in its local neighborhood; based on the normalized cross-correlation value, construct a cost volume, and according to the cost volume, find the optimal disparity path at each spatial position through a dynamic programming algorithm, so that the first energy function containing all constraints is minimized; and obtain an initial depth estimation map.

[0035] In this embodiment, accurate alignment of fluorescence images and white light images is achieved through multimodal image registration and initial depth estimation, laying the foundation for subsequent processing. Specifically, the mutual information maximization method is used for coarse registration, which overcomes the intensity difference between different modal images and improves the robustness of registration. Subsequently, a non-rigid registration algorithm based on B-spline is used for fine registration, which effectively compensates for local inconsistencies caused by tissue deformation and respiratory movement. This two-step registration strategy improves the registration accuracy and ensures the accurate correspondence between fluorescence information and anatomical structure. In the initial depth estimation, normalized cross-correlation and dynamic programming algorithms are introduced to construct a cost volume that considers time consistency, which effectively reduces the timing jitter in depth estimation. By integrating multi-view geometric constraints and light field information, the accuracy and stability of depth estimation are improved, especially in challenging areas such as reflective and transparent tissues. This multimodal, multi-constrained depth estimation method provides a reliable foundation for subsequent depth adaptive processing and improves the accuracy and credibility of 3D vascular reconstruction.

[0036] like Figure 3 As shown, according to one aspect of the present application, step S2 is specifically:

[0037] Step S21, using the initial depth estimation map, performing depth adaptive denoising on the registered multimodal image set to obtain a denoised image;

[0038] The adaptive denoising process includes: constructing a deep adaptive anisotropic diffusion filter, performing anisotropic diffusion based on the diffusion coefficient, iteratively calculating using a semi-implicit finite difference format, solving the partial differential equation, and finally obtaining the denoised image;

[0039] Step S22: based on the denoised image and the initial depth estimation map, calling the pre-constructed second energy function, using multimodal information to constrain the depth estimation, using the primal dual interior point method to iteratively solve the optimization problem, minimizing the second energy function, and optimizing the depth map through a variational optimization method to obtain an improved depth map;

[0040] Step S23, constructing a multi-scale Hessian matrix using the improved depth map and the denoised image, calculating the eigenvalues ​​of the Hessian matrix as vascular features, constructing a depth-aware vascular response function based on the eigenvalues, and obtaining a vascular response map; wherein the construction of the vascular response function takes depth information into consideration and adapts to vascular features of different depths through adaptive parameter adjustment;

[0041] Step S24: construct an adaptive weight function based on the vascular response function, calculate the weight based on the optimal scale and scale variance, integrate the multi-scale vascular response through weighted summation, achieve multi-scale vascular integration and enhancement, and obtain a preliminary enhanced vascular image;

[0042] Step S25, combining the improved depth map and the preliminary enhanced vascular image, using the improved optical flow equation to estimate the local motion field, constructing the state vector and observation vector of the extended Kalman filter, realizing vascular tracking through prediction steps and update steps, and using thin plate spline interpolation to perform local non-rigid registration based on the filtering results, finally obtaining a time-consistent vascular enhancement sequence and an optimized final depth map.

[0043] In this embodiment, the quality and recognizability of the vascular image are improved through depth-adaptive noise reduction and vascular enhancement processing. A depth-adaptive anisotropic diffusion filter is used for noise reduction, and the diffusion coefficient is dynamically adjusted according to the depth information, which effectively retains the edge information of blood vessels at different depths and suppresses background noise. Compared with the traditional fixed parameter filter, this method can better adapt to complex endoscopic scenes, maintain the details of deep tissues while reducing the noise in the shadow area. The introduction of multi-scale Hessian matrix analysis and depth-aware vascular response function realizes adaptive enhancement of blood vessels of different calibers and depths. By constructing depth-related adaptive parameters, the enhancement effect is kept consistent throughout the depth range, overcoming the problem of insufficient deep blood vessel enhancement effect of traditional methods. The multi-scale integration strategy further improves the detection capability of micro-vessels while maintaining the integrity of large blood vessels. The introduction of extended Kalman filter and thin plate spline interpolation for spatiotemporal consistency optimization effectively solves the problem of vascular motion caused by heartbeat and breathing, and improves the accuracy and stability of vascular tracking. This vascular enhancement method, which comprehensively considers depth, scale, and timing information, improves the visualization of blood vessels in 3D ultra-high-definition fluorescence endoscopy images and provides a reliable imaging basis for precise intraoperative navigation.

[0044] In addition, this embodiment effectively improves the suppression effect of image noise and motion artifacts, achieves high-quality image denoising, and maintains key structural information. It is particularly suitable for processing endoscopic images under low illumination conditions, and improves the accuracy of blood vessel recognition. The introduction of extended Kalman filter and thin plate spline interpolation for spatiotemporal consistency optimization effectively solves the problem of vascular motion caused by heartbeat and breathing, and improves the accuracy and stability of blood vessel tracking. This processing strategy that comprehensively considers depth, scale, and timing information improves the performance of blood vessel recognition and tracking in dynamic, low-illumination environments.

[0045] like Figure 4 According to one aspect of the present application, step S3 is specifically as follows:

[0046] Step S31: construct a signal attenuation model based on the final depth map, construct an optimization problem, use the alternating direction multiplier method to minimize the difference between the ideal non-attenuated image and the actual image, and introduce a spatial smoothing regularization term to solve the optimization problem, so as to obtain the optimal attenuation coefficient and attenuation model;

[0047] Among them, the signal attenuation model includes local attenuation coefficient and depth information;

[0048] Step S32, constructing a multi-scale anisotropic scattering kernel, calculating a scattering image for each scale, designing an adaptive weight function, and obtaining a scattering compensation image by weighted summation;

[0049] The construction of the scattering kernel takes into account the depth-related anisotropic parameters, the design of the adaptive weight function is based on the depth information, attenuation model and local image features, and the weighted summation process realizes the comprehensive compensation of multi-scale scattering effects.

[0050] Step S33, using the improved Richardson-Lucy algorithm to perform iterative deconvolution to obtain an image after signal compensation and artifact suppression;

[0051] The iterative deconvolution process includes signal attenuation model, scattering compensation and regularization terms, and the original signal is restored and the image quality is improved through multiple iterative optimizations.

[0052] Step S34, calculating the local contrast and local brightness, designing a piecewise linear mapping function and an adaptive gamma correction function, performing dynamic range adjustment and local contrast enhancement on the image after signal compensation, and obtaining a final optimized 3D ultra-high-definition fluorescent medical endoscope vascular image;

[0053] Among them, the parameters of the piecewise linear mapping function are dynamically adjusted according to the local contrast and brightness, and the adaptive gamma correction function takes into account the local image characteristics. Through these processing steps, the dynamic range optimization and visual quality improvement of the image are achieved.

[0054] In this embodiment, the quality and credibility of 3D ultra-high-definition fluorescent medical endoscope vascular images are improved by constructing signal attenuation compensation and scattering effect correction driven by physical models. The adaptive attenuation estimation method based on the final depth map takes into account the scattering and absorption characteristics of tissues, and overcomes the problem of signal intensity variation caused by depth in traditional methods. By introducing cross-validation to determine the optimal regularization strength, the generalization ability of the attenuation model is improved, so that the compensation effect remains stable between different patients and different tissue types. The introduction of multi-scale scattering kernels solves the scattering effect of fluorescence signals in tissues, especially the blurring problem in deep tissues. Through the adjustment of depth-related anisotropic parameters, accurate modeling and compensation of scattering effects at different depths are achieved, and the imaging clarity of deep blood vessels is improved. In the signal recovery stage, an improved Richardson-Lucy algorithm is used for iterative deconvolution, while considering the signal attenuation model, scattering compensation and adaptive regularization terms, effectively balancing signal recovery and noise suppression. The introduced adaptive iteration stop criterion dynamically controls the signal recovery process according to relative error, gradient change and image quality evaluation indicators, avoiding artifacts caused by excessive iteration. Dynamic range optimization is achieved through piecewise linear mapping and adaptive gamma correction, which effectively solves the common local overexposure and underexposure problems in endoscopic images, allowing vascular structures to be clearly presented under different depths and brightness conditions. This comprehensive signal processing method based on physical models not only improves the overall quality of vascular images, but also provides a more reliable data basis for quantitative analysis, which is of great significance for the accurate evaluation of vascular morphology and function.

[0055] like Figure 5 As shown, according to one aspect of the present application, step S4 is specifically:

[0056] Step S41, inputting the final optimized displayed 3D ultra-high-definition fluorescent medical endoscope vascular image and the final depth map into the feature fusion module, applying the Frangi filter to the vascular image to obtain vascular enhancement features, designing an adaptive weight function, and obtaining a fused feature map through weighted fusion;

[0057] Among them, the Frangi filter calculates the vascular related metrics based on the eigenvalues ​​of the Hessian matrix, the adaptive weight function includes local contrast and depth information, and the weighted fusion process is used to effectively combine vascular features and depth information;

[0058] Step S42, using the level set method to perform 3D blood vessel segmentation, constructing a third energy function and introducing a time regularization term and a curvature regularization term, using an alternating direction multiplier method to solve the optimization problem, and obtaining a blood vessel segmentation mask that is consistent in time and space;

[0059] Among them, the level set evolution equation includes the information of the fusion feature map, the time regularization term is used to determine the time coherence of the segmentation result, and the curvature regularization term is used to achieve the smoothness of the segmentation boundary;

[0060] Step S43, applying Euclidean distance transform to the blood vessel segmentation mask to obtain a distance map, using a grayscale skeleton-based method to extract an initial skeleton, detecting and connecting skeleton breakpoints, removing short branches, applying a skeleton thinning algorithm, and finally estimating the blood vessel radius to obtain a blood vessel skeleton and a corresponding radius map;

[0061] The skeleton extraction process takes local maxima into account, breakpoint connection and short branch removal are used to ensure the continuity and representativeness of the skeleton, and skeleton refinement is used to maintain the endpoints unchanged.

[0062] Step S44, extracting the endpoints and bifurcation points of the vascular skeleton, constructing a vascular network graph structure, fitting each vascular segment using 3D Cardinal splines, establishing an adaptive diameter model, applying Laplacian smoothing to ensure a smooth transition at the bifurcation, and using the rotation section method to generate a vascular surface mesh along the spline curve to obtain a vascular network model;

[0063] Among them, the spline fitting takes into account the control points and basis functions, the adaptive pipe diameter model fits the radius change through the least squares method, and the mesh generation process adaptively adjusts the size of the triangle patch according to the local curvature;

[0064] Step S45, fusing the reconstructed vascular network model with the final optimized displayed 3D ultra-high-definition fluorescent medical endoscope vascular image, constructing a multi-layer rendering pipeline, designing an adaptive blending function, applying an edge enhancement filter, and finally applying a high dynamic range tone mapping technique to obtain the final fusion visualization result;

[0065] Among them, the multi-layer rendering pipeline includes a background layer, a network overlay layer and an enhancement layer. The adaptive blending function dynamically adjusts the blending coefficient according to the local contrast and depth information, and the edge enhancement filter is used to highlight the edge of the blood vessels.

[0066] In this embodiment, high-precision and high-fidelity 3D vascular network visualization is achieved through multimodal fusion, 3D vascular segmentation, skeleton extraction and network reconstruction. Frangi filter and adaptive weight function are used for feature fusion, which effectively integrates fluorescence information, depth information and edge features, and improves the recognition accuracy of vascular structure. This multimodal fusion strategy overcomes the limitations of single modality information, especially in the detection of blood vessels in complex backgrounds and low-contrast areas. The level set method is introduced for 3D vascular segmentation, and the segmentation results consistent in time and space are achieved by constructing an energy function containing time regularization terms and curvature regularization terms. This method effectively solves the discontinuity problem of traditional 2D segmentation methods in 3D reconstruction and improves the topological accuracy of vascular networks. In the skeleton extraction stage, Euclidean distance transformation and grayscale skeleton-based methods are used, combined with breakpoint connection and short branch removal strategies to ensure the continuity and representativeness of the extracted skeleton. This improved skeleton extraction method provides a reliable topological structure for subsequent 3D reconstruction, especially when dealing with complex bifurcations and small blood vessels. In the process of 3D vascular network reconstruction, 3D Cardinal splines are used to fit the vascular segments, and an adaptive diameter model is introduced to accurately capture the changes in vascular diameter. The application of Laplacian smoothing ensures a smooth transition at the bifurcation and improves the physiological rationality of the reconstructed model. High-quality 3D vascular network visualization is achieved through a multi-layer rendering pipeline, adaptive blending functions, and HDR tone mapping technology. This comprehensive 3D reconstruction and visualization method not only provides an intuitive and accurate expression of the vascular spatial structure, but also enhances the visibility of tiny blood vessels through deep coding and edge enhancement technology. The entire process fully considers the characteristics of 3D ultra-high-definition fluorescence endoscopes, provides strong imaging support for real-time vascular navigation in complex minimally invasive surgery, and improves the accuracy and safety of surgery.

[0067] According to one aspect of the present application, step S12 is specifically:

[0068] Step S121, constructing a multi-resolution image pyramid, down-sampling the pre-processed fluorescence image sequence and the white light image sequence to obtain image sequences of different resolution levels; wherein a Gaussian filter is used for smoothing, and then secondary sampling is performed to generate a multi-level image sequence from coarse to fine, and the resolution of each level is half of the previous level;

[0069] Step S122: Starting from the coarsest resolution level, a global rigid registration is performed using a mutual information maximization method to obtain initial registration parameters; then, registration is performed step by step to finer resolutions, and each level uses the registration result of the previous level as an initial value, and the registration accuracy is improved through iterative optimization; wherein, at each resolution level, a mutual information objective function is constructed, and the gradient descent method is used for optimization to obtain the optimal rigid transformation parameters of the level, i.e., the rigid registration result;

[0070] Step S123: at the finest resolution level, based on the rigid registration result, construct an adaptive local deformation model for non-rigid registration, and perform registration, specifically:

[0071] The image is divided into overlapping local blocks, and a feature descriptor based on gradient and structural similarity is calculated for each local block;

[0072] Adaptively adjust the deformation freedom and regularization strength of each local block according to the similarity of feature descriptors;

[0073] Construct a global energy function, including data term, smoothing term and consistency constraint term, and use the alternating direction multiplier method to optimize and solve it to obtain the final non-rigid deformation field;

[0074] This deformation field is applied to the preprocessed fluorescence image sequence to generate a set of registered multimodal images that are precisely aligned with the white light image sequence.

[0075] In this embodiment, high-precision multimodal image registration is achieved through a multi-resolution image pyramid and an adaptive local deformation model. A multi-resolution image pyramid is constructed, and a Gaussian filter is used for smoothing and secondary sampling to generate a multi-level image sequence from coarse to fine. This strategy effectively reduces the influence of local minima, improves the convergence speed and robustness of the registration algorithm, and performs well in the case of large deformations and large initial position errors. In the coarse registration stage, the mutual information maximization method is used for global rigid registration, which overcomes the intensity difference between different modal images and provides a reliable initial value for subsequent fine registration. The strategy of registering step by step to a finer resolution ensures the stability and accuracy of the registration process. At the finest resolution level, an adaptive local deformation model is introduced for non-rigid registration, and accurate modeling of local deformation is achieved by calculating feature descriptors based on gradient and structural similarity. The method of adaptively adjusting the deformation degrees of freedom and regularization strength of each local block effectively balances the flexibility and stability of registration, and is particularly suitable for non-rigid changes caused by local tissue deformation and respiratory movement common in endoscopic images. The constructed global energy function contains data terms, smoothing terms and consistency constraints, and is optimized and solved by the alternating direction multiplier method to ensure the global consistency of the registration results. This multi-scale, adaptive registration strategy not only improves the alignment accuracy between fluorescence images and white light images, but also maintains the integrity of local structures, laying a solid foundation for subsequent vascular enhancement and 3D reconstruction.

[0076] According to one aspect of the present application, step S13 is specifically:

[0077] Step S131, read the registered multimodal image set, perform Fourier transform on the image, and analyze the spectral characteristics; construct a 4D light field function by estimating the local light direction; calculate the angle-space correlation in the light field to obtain light field information as a preliminary depth clue;

[0078] Step S132: constructing a depth estimation energy function based on light field information and combined with multi-view geometric constraints;

[0079] Step S133, using the variational method to optimize the constructed energy function, specifically: by discretizing the energy function, constructing a large-scale sparse linear equation system; using the alternating direction multiplier method for iterative solution; using a multi-scale strategy to gradually refine the depth estimation, and obtain a high-precision initial depth estimation map.

[0080] In this embodiment, by combining light field information and multi-view geometric constraints, high-precision and robust initial depth estimation is achieved. The registered multimodal image set is subjected to Fourier transform and spectral analysis, and a 4D light field function is constructed in combination with local light direction estimation. This method makes full use of the implicit depth information in the endoscopic image, especially when dealing with reflective and translucent surfaces, and overcomes the limitations of traditional stereo vision methods in these difficult areas. By calculating the angle-space correlation in the light field, preliminary depth clues are obtained, providing reliable prior information for subsequent fine depth estimation. The depth estimation energy function is constructed based on light field information and multi-view geometric constraints, and image consistency, depth smoothness and multi-view geometric relationships are comprehensively considered. This multi-constrained energy function design effectively reduces the ambiguity in depth estimation and improves the accuracy and reliability of the estimation results. In the optimization process, the variational method is used to optimize the constructed energy function, and a large-scale sparse linear equation system is constructed by discretizing the energy function, and the alternating direction multiplier method is used for iterative solution. This optimization strategy can not only effectively handle large-scale depth estimation problems, but also has a faster convergence speed, meeting the needs of real-time processing. The depth estimation is gradually refined through a multi-scale strategy, which accurately depicts local depth changes while ensuring global consistency, especially when dealing with complex shapes and fine structures. This depth estimation method that combines light field information and multi-view geometric constraints not only improves the depth estimation accuracy in endoscopic scenes, but also enhances the depth perception ability of complex surfaces and tiny structures, providing a high-quality initial depth map for subsequent depth-adaptive vascular enhancement and 3D reconstruction.

[0081] According to one aspect of the present application, step S21 is specifically:

[0082] Step S211: constructing a depth-aware edge detector for the registered multimodal image set based on the initial depth estimation map; calculating the multi-scale gradient of the image through the edge detector, adaptively adjusting the gradient threshold according to the depth information, and obtaining accurate edge map information through non-maximum suppression;

[0083] Step S212: construct a nonlinear filter kernel based on anisotropic diffusion for edge map information; dynamically adjust filter parameters according to local depth and edge strength; adaptively determine the range and strength of the filter to form a structure-preserving adaptive filter;

[0084] Step S213, applying an adaptive filter to the registered multimodal image set for denoising; through an iterative optimization process, effectively suppressing noise while maintaining edge and structural information; adaptively adjusting the filter strength according to the depth information to obtain an image after depth adaptive denoising.

[0085] In this embodiment, high-quality image denoising is achieved while maintaining key structural information through depth-aware edge detection and adaptive nonlinear filtering. A depth-aware edge detector is constructed based on the initial depth estimation map, and multi-scale gradients are calculated and the gradient threshold is adaptively adjusted according to the depth information. This method can accurately identify edge structures of different depths and overcome the limitations of traditional edge detectors in processing complex 3D scenes. The precise edge map obtained by non-maximum suppression provides key guidance information for subsequent structure-preserving filtering. A nonlinear filter kernel based on anisotropic diffusion is constructed for edge information, and the filter parameters are dynamically adjusted according to local depth and edge strength. This adaptive strategy enables the filter to accurately maintain edge and detail information while effectively denoising the smooth area, which is particularly suitable for complex textures and tiny structures commonly found in endoscopic images. By adaptively determining the range and strength of the filter, a structure-preserving adaptive filter is formed, which effectively solves the shortcomings of traditional fixed parameter filters in processing different depths and different feature areas. When applying the adaptive filter for denoising, the noise is effectively suppressed while maintaining edge and structural information through an iterative optimization process. The strategy of adaptively adjusting the filter strength according to the depth information ensures that a consistent denoising effect can be obtained over the entire depth range. This deep adaptive noise reduction method not only improves the overall image quality, but also provides high-quality input data for subsequent vascular enhancement and 3D reconstruction, especially showing significant advantages when dealing with deep tissues and tiny blood vessels. By maintaining key structural information while reducing interference, the clarity and recognizability of 3D ultra-high-definition fluorescence endoscope vascular images are improved, laying the foundation for accurate vascular analysis and navigation.

[0086] According to one aspect of the present application, step S23 is specifically:

[0087] Step S231: construct basic filter templates of different scales and directions based on the image after depth adaptive denoising; dynamically adjust the scale and direction parameters of the filter according to the local depth information; construct a multi-scale and multi-directional blood vessel enhancement filter group, that is, obtain an adaptive filter group;

[0088] Step S232, applying an adaptive filter group to perform a convolution operation on the image, and for each pixel, calculating the maximum response value and its corresponding scale and direction; constructing a blood vessel possibility map according to the response value, scale and direction information, that is, a multi-scale, multi-directional response map;

[0089] Step S233: Based on the vascular possibility map, combined with the original Hessian matrix analysis method, the Hessian matrix and its eigenvalues ​​are calculated, and a fusion function is used to comprehensively consider the filter response and Hessian characteristics to perform feature fusion, and an enhanced vascular feature map is obtained through nonlinear mapping.

[0090] In this embodiment, by constructing a multi-scale, multi-directional blood vessel enhancement filter group with depth perception, high-precision, adaptive blood vessel structure enhancement is achieved. Based on the image after depth adaptive denoising, basic filter templates of different scales and directions are constructed, and the scale and direction parameters of the filter are dynamically adjusted according to the local depth information. This adaptive strategy enables the filter to accurately match the blood vessel structure of different depths and directions, overcoming the limitations of traditional fixed parameter filters in processing complex 3D scenes. In particular, for deep micro-vessels and superficial large blood vessels, it can provide the best enhancement effect at the same time, improving the visibility and continuity of blood vessels. By applying an adaptive filter group to perform convolution operations on the image, and calculating the maximum response value of each pixel and its corresponding scale and direction, a multi-scale, multi-directional blood vessel possibility map is constructed. This method can not only accurately capture the local features of the blood vessel, but also provide the scale and direction information of the blood vessel, providing a rich feature description for subsequent 3D reconstruction. By combining the original Hessian matrix analysis method, the Hessian matrix and its eigenvalues ​​are calculated, and the fusion function is used to comprehensively consider the filter response and Hessian features to perform feature fusion. This multi-feature fusion strategy makes full use of the directional information of the filter response and the shape description ability of the Hessian matrix, and can more accurately distinguish between vascular and non-vascular structures, especially when dealing with complex backgrounds and crossing blood vessels. The enhanced vascular feature map obtained by nonlinear mapping not only improves the contrast and clarity of the blood vessels, but also maintains the continuity and topological structure of the blood vessels. This depth-aware multi-scale vascular enhancement method improves the visualization quality of blood vessels in 3D ultra-high-definition fluorescence endoscopy images, provides high-quality input data for subsequent vascular segmentation and 3D reconstruction, and is of great significance for precise vascular navigation in complex minimally invasive surgery.

[0091] According to one aspect of the present application, step S25 is specifically:

[0092] Step S251: Based on the preliminary enhanced blood vessel image and the improved depth map, a blood vessel motion prediction model is established using the blood vessel position and motion information of the previous frames; the parameters of the prediction model are adjusted according to the depth information to improve the prediction accuracy, a prediction-correction model is formed, and the prediction-correction model is used to generate a predicted blood vessel position and shape, i.e., a prediction result;

[0093] Step S252: read the prediction results according to the preconfigured adaptive sampling strategy, construct a probability distribution map around the predicted blood vessel position, dynamically adjust the sampling density according to the local image features and depth information, and generate a set of candidate blood vessel points after completing the sampling work in the image space;

[0094] Step S253, evaluating and screening the candidate blood vessel points, including: calculating the local feature descriptor of each candidate point; comparing with the result of the prediction model and calculating the matching score; using the Markov random field model to globally optimize the candidate points to obtain the optimal blood vessel tracking result;

[0095] Step S254: Based on the blood vessel tracking result, thin plate spline interpolation is used to perform local non-rigid registration, and finally a time-consistent blood vessel enhancement sequence and an optimized final depth map are obtained.

[0096] In this embodiment, high-precision and stable blood vessel tracking is achieved by combining depth information, motion prediction and adaptive sampling strategy. Based on the preliminary enhanced blood vessel image and improved depth map, a blood vessel motion prediction model is established using the blood vessel position and motion information of the previous frames. By dynamically adjusting the parameters of the prediction model according to the depth information, the prediction accuracy is improved, especially when dealing with complex blood vessel deformation caused by respiratory motion and heartbeat. This prediction-correction model can effectively reduce the tracking error caused by tissue deformation and camera motion, and provide reliable prior information for subsequent accurate sampling. Based on the prediction results, an adaptive sampling strategy is used to construct a probability distribution map around the predicted blood vessel position. By dynamically adjusting the sampling density according to local image features and depth information, key sampling of important areas (such as blood vessel bifurcations and microvessels) is achieved, while unnecessary sampling of background areas is reduced, improving computational efficiency and tracking accuracy. This strategy is particularly suitable for dealing with common problems of uneven illumination and local contrast changes in endoscopic images. When evaluating and screening candidate blood vessel points, the local feature descriptor of each candidate point is calculated, and compared with the results of the prediction model to calculate the matching score. By introducing the Markov random field model for global optimization, the common ambiguity problem in local feature matching is effectively solved, ensuring the spatiotemporal consistency of the tracking results. This global optimization strategy not only improves the accuracy of vascular tracking, but also enhances the robustness to occlusion. Overall, this vascular tracking method that combines depth information, motion prediction, and adaptive sampling improves the accuracy and stability of vascular tracking in 3D ultra-high-definition fluorescence endoscopy images. It can accurately capture subtle changes in blood vessels while maintaining consistency in long-term tracking, providing a reliable foundation for real-time surgical navigation and hemodynamic analysis. This high-precision and stable vascular tracking technology is of great significance for improving the safety and accuracy of minimally invasive surgery, especially when dealing with complex vascular networks and dynamically changing surgical environments.

[0097] In another embodiment of the present application, the S31 is specifically:

[0098] Step S311, using the blood vessel tracking results and the final depth map, constructing an adaptive attenuation estimation method based on a physical model, including: establishing a light propagation theoretical model, taking into account tissue scattering and absorption characteristics; adaptively adjusting model parameters according to depth information and blood vessel position; generating an initial attenuation model;

[0099] Step S312: Based on the initial attenuation model, an iterative optimization algorithm is designed, specifically: constructing an objective function, including a data fitting term and a regularization term; iteratively optimizing model parameters using a gradient descent method; determining the optimal regularization strength through cross-validation to obtain an optimized attenuation model;

[0100] In another embodiment of the present application, an optimized attenuation model is applied to perform adaptive attenuation compensation on the image, and the specific compensation process is: calculating the attenuation compensation factor of each pixel based on the depth information and local image features; performing pixel-level attenuation compensation on the image; and further improving the compensation effect through local contrast enhancement to obtain an attenuation-compensated image.

[0101] In this embodiment, by constructing an adaptive attenuation estimation method based on a physical model, high-precision and personalized signal attenuation compensation is achieved. Using the vascular tracking results and the final depth map, a theoretical model of light propagation is established, taking into account the tissue scattering and absorption characteristics. Compared with the traditional empirical model, this method based on the physical model can more accurately simulate the attenuation behavior of the fluorescence signal at different depths and in different tissues. By adaptively adjusting the model parameters according to the depth information and the position of the blood vessels, the method can adapt to the individual differences of different patients and different tissue types, and improve the accuracy and robustness of the attenuation estimation. The designed iterative optimization algorithm realizes the accurate estimation of the attenuation model parameters by constructing an objective function containing data fitting terms and regularization terms. Iterative optimization using the gradient descent method can effectively handle nonlinear optimization problems and ensure convergence to the global optimal solution. In particular, the introduction of a cross-validation strategy to determine the optimal regularization strength effectively prevents the overfitting problem and improves the generalization ability of the model. This adaptive attenuation estimation method not only improves the imaging quality of blood vessels in deep tissues, but also provides a reliable basis for quantitative analysis, which is of great significance for accurately evaluating vascular morphology and function. By compensating for the signal intensity changes caused by depth, this method enables the vascular structure in the entire field of view to be clearly presented, improving the diagnostic value and surgical navigation capabilities of 3D ultra-high-definition fluorescent medical endoscopes.

[0102] In another embodiment of the present application, the step S33 further includes:

[0103] Step S331, constructing an adaptive regularization framework based on the scattering compensation image and the attenuation compensation result; the construction process is: analyzing the local and global statistical characteristics of the image; designing a spatially varying regularization term, and adaptively adjusting the regularization strength in different regions; constructing an objective function including a data fidelity term and an adaptive regularization term;

[0104] Step S332, calling the iterative optimization algorithm to solve the objective function, specifically, using the split Bregman iteration method to decompose the problem into sub-problems; applying a fast solution algorithm to each sub-problem; and iteratively solving the problem through an alternating optimization strategy to obtain a preliminary recovered signal;

[0105] Step S333, design an adaptive iteration stop criterion to dynamically control the signal recovery process. Specifically, calculate the relative error and gradient change of each iteration; combine image quality evaluation indicators such as structural similarity and peak signal-to-noise ratio to comprehensively evaluate the recovery effect. When the preset stop condition is met, terminate the iteration and output the final restored signal image;

[0106] In another embodiment of the present application, iterative deconvolution is performed by using an improved Richardson-Lucy algorithm.

[0107] In this embodiment, high-quality signal recovery and image reconstruction are achieved by constructing an adaptive regularization framework and an improved iterative deconvolution algorithm. Based on the scattering compensation image and the attenuation compensation result, the constructed adaptive regularization framework fully considers the local and global statistical characteristics of the image. By designing a spatially varying regularization term, the regularization strength is adaptively adjusted in different regions, effectively balancing signal recovery and noise suppression. Compared with the traditional fixed parameter regularization, this method can better adapt to complex endoscopic scenes and effectively suppress background noise while maintaining vascular details. The split Bregman iteration method is used to decompose the problem into sub-problems, and a fast solution algorithm is applied to each sub-problem, thereby improving computational efficiency. By iteratively solving the problem through alternating optimization strategies, large-scale, nonlinear image restoration problems can be effectively handled, and the convergence and stability of the algorithm are guaranteed. The designed adaptive iteration stopping criterion realizes dynamic control of the signal recovery process by comprehensively considering relative error, gradient change and image quality evaluation indicators. This strategy not only avoids artifacts caused by excessive iterations, but also ensures the best quality of the recovery results. Overall, this adaptive regularization and optimized iterative deconvolution method improves the signal quality and spatial resolution of 3D ultra-high-definition fluorescent medical endoscope images, especially when dealing with deep tissues and tiny blood vessels. It provides high-quality input data for subsequent vascular segmentation and 3D reconstruction, which is of great significance for improving the accuracy and safety of minimally invasive surgery.

[0108] In another embodiment of the present application, step S42 may also adopt the following method:

[0109] Step S421: Based on the fused feature map, a multi-feature adaptive threshold segmentation model is constructed and local intensity, gradient and texture features are extracted; depth information is used to perform depth weighting on the features; and dimension reduction is performed through principal component analysis to obtain a comprehensive feature map;

[0110] Step S422: construct an adaptive threshold function for the comprehensive feature map; calculate the global initial threshold using the OTSU algorithm; construct a spatially varying threshold function based on local statistical information and depth data; apply the threshold function to perform preliminary segmentation to obtain a candidate blood vessel region;

[0111] Step S423, perform morphological post-processing and topological optimization on the candidate blood vessel regions; the optimization process is: use conditional morphological operators to remove small isolated areas and fill holes; use blood vessel tracking results as prior information to correct segmentation boundaries; optimize blood vessel topology through skeleton extraction and reconstruction to obtain accurate 3D blood vessel segmentation results.

[0112] In this embodiment, high-precision and robust 3D blood vessel segmentation is achieved through multi-feature adaptive threshold segmentation and topology optimization. The multi-feature adaptive threshold segmentation model constructed based on the fusion feature map extracts local intensity, gradient and texture features and uses depth information for feature weighting, making full use of the rich information of multimodal images. The comprehensive feature map obtained by dimensionality reduction through principal component analysis effectively reduces feature redundancy and improves the computational efficiency of segmentation. The constructed adaptive threshold function combines the global OTSU algorithm and local statistical information, and can adapt to complex backgrounds and uneven lighting conditions. The spatial variation threshold function constructed based on depth data effectively solves the problem of inconsistent segmentation thresholds of blood vessels at different depths, and improves the accuracy and robustness of segmentation. The morphological post-processing and topological optimization of the candidate blood vessel area remove small isolated areas and fill holes through conditional morphological operators, effectively improving the continuity and integrity of the segmentation results. The strategy of using blood vessel tracking results as prior information to correct the segmentation boundary further improves the accuracy of segmentation, especially when dealing with tiny blood vessels and complex bifurcated structures. The method of optimizing the topological structure of blood vessels through skeleton extraction and reconstruction ensures the topological correctness of the segmentation results, providing a reliable foundation for subsequent 3D reconstruction. This 3D vascular segmentation method that comprehensively considers multiple features, depth information, and topological structure not only improves the accuracy and reliability of segmentation, but also maintains the integrity and continuity of the vascular network, which is of great significance for precise vascular navigation and surgical planning in complex minimally invasive surgeries.

[0113] In another embodiment of the present application, step S44 further includes:

[0114] Step S441: Based on the vascular skeleton and radius map, an initial vascular network model is constructed and the endpoints and bifurcation points of the skeleton are extracted to establish a network topology structure; each vascular segment is fitted using a 3D Cardinal spline to obtain a centerline representation; and an initial tubular model is generated in combination with the radius information;

[0115] Step S442, topological optimization and morphological constraints are performed on the initial tubular model. Specifically, topological errors in the network, such as false connections and breaks, are detected and repaired; morphological constraints based on anatomical knowledge, such as branch angles and tube diameter ratios, are adopted; and the direction of the blood vessels is adjusted through local deformation to make it conform to physiological characteristics, thereby obtaining an optimized model;

[0116] Step S443, based on the optimized model, perform refined reconstruction and surface generation, including: using an adaptive subdivision algorithm to increase model details; applying Laplacian smoothing and local detail enhancement to improve surface quality; simulating the elastic properties of the blood vessel wall through a physics-based deformation model to obtain the final 3D vascular network model.

[0117] In this embodiment, through the refined 3D vascular network reconstruction and surface generation, a high-fidelity, physiologically reasonable vascular model is constructed. The initial vascular network model constructed based on the vascular skeleton and radius map establishes a network topology by extracting the endpoints and bifurcation points of the skeleton, laying the foundation for the accurate expression of complex vascular networks. Each vascular segment is fitted using 3D Cardinal splines, and the resulting centerline representation not only maintains the smoothness of the vascular direction, but also accurately captures the bending and twisting of the blood vessels. The initial tubular model generated in combination with the radius information provides a good starting point for subsequent refined reconstruction. The topological optimization and morphological constraints of the initial model effectively solve the topological errors in the network, such as false connections and breaks. The morphological constraints based on anatomical knowledge, such as branch angles and diameter ratios, are used to ensure the physiological rationality of the reconstructed model. The vascular direction is adjusted by local deformation, which further improves the consistency of the model with the actual vascular anatomical structure. In the process of refined reconstruction and surface generation, the adaptive subdivision algorithm is used to increase the model details, which effectively improves the fineness of the vascular surface. The application of Laplacian smoothing and local detail enhancement not only improves the surface quality, but also maintains important anatomical features. By simulating the elastic properties of the vascular wall through a physics-based deformation model, the reconstructed vascular network is closer to the real physiological state. This 3D vascular network reconstruction method that comprehensively considers topological structure, morphological characteristics and physical properties not only provides a highly realistic and physiologically reasonable vascular model, but also provides a reliable imaging basis for intraoperative navigation and surgical planning. It can accurately present the spatial structure and morphological characteristics of complex vascular networks, which is of great significance for improving the accuracy and safety of minimally invasive surgery, especially in the treatment of complex vascular lesions and the formulation of personalized surgical plans, showing significant advantages.

[0118] According to another aspect of the present application, a 3D ultra-high definition fluorescence medical endoscope vascular image enhancement system is also provided, comprising:

[0119] at least one processor; and,

[0120] a memory communicatively connected to at least one of the processors; wherein,

[0121] The memory stores instructions that can be executed by the processor, and the instructions are used to be executed by the processor to implement the 3D ultra-high-definition fluorescence medical endoscope vascular image enhancement method described in any of the above technical solutions.

[0122] In another embodiment of the present application, step S1 is specifically:

[0123] Step S11, obtaining the original 3D fluorescence image sequence F(x, y, z, t) and the corresponding white light image sequence W(x, y, z, t) from the 3D fluorescence medical endoscope system, and performing preprocessing to obtain the preprocessed fluorescence image F_pre(x, y, z, t) and white light image W_pre(x, y, z, t), wherein (x, y, z) represents the spatial coordinates and t represents the time dimension; the preprocessing includes: preliminary quality assessment, pulse noise removal, Gaussian noise suppression, motion artifact removal and image contrast enhancement;

[0124] Step S12, input the preprocessed fluorescence image F_pre(x, y, z, t) and the white light image W_pre(x, y, z, t) into a pre-configured registration module for coarse registration and fine registration, and the multimodal image set I_registered(x, y, z, t) after registration;

[0125] Among them, in the coarse registration stage, the mutual information maximization method is used to perform global rigid transformation to obtain the initial registration parameters;

[0126] In the fine registration stage, a non-rigid registration algorithm based on B-spline is applied to construct the optimization objective function J(T)=λ_1×MI(F_pre(T(x, y, z)), W_pre(x, y, z))+λ_2×SSIM(F_pre(T(x, y, z)), W_pre(x, y, z))+λ_3×R(T), and iteratively optimize the objective function to obtain the optimal registration transformation T(x, y, z, t); where MI represents mutual information, SSIM is the structural similarity index, R(T) is the deformation regularization term, and λ_1, λ_2 and λ_3 are weight coefficients; the optimal registration transformation T(x, y, z, t) is applied to the preprocessed fluorescence image F_pre(x, y, z, t) to generate a registered multimodal image set I_registered(x, y, z, t) that is precisely aligned with the white light image W_pre(x, y, z, t);

[0127] Step S13, using the registered multimodal image set I_registered(x, y, z, t) as input, for each pixel position (x, y, z), calculate the normalized cross correlation value (NCC value) in its local neighborhood, and construct a cost volume C(x, y, z, d), where d is the disparity value; using a dynamic programming algorithm to find the optimal disparity path at each (x, y) position, iteratively optimize, minimize the energy function E(D), and finally obtain an initial depth estimation map D_init(x, y, z, t);

[0128] Wherein, cost volume C(x, y, z, d) = ω1 * NCC(x, y, z, d) + ω2 * exp(-|▽I(x, y, z)| / k_edge) * (1 - exp(-|d - D_prev(x, y, z)| / k_temp)) + ω3 * (1 - exp(-|▽D(x, y, z)| / k_smooth)); where C(x, y, z, d) represents the cost value when the disparity value is d at the spatial position (x, y, z); ω1, ω2, ω3 are weight coefficients used to balance the contribution of each item;

[0129] NCC is used to measure the similarity of image blocks; ▽I(x, y, z) is the image gradient, indicating the local change of the image at (x, y, z); k_edge is a parameter that controls the sensitivity of the image gradient term; D_prev(x, y, z) is the depth estimate of the previous frame; k_temp is a parameter that controls the sensitivity of the temporal consistency term; ▽D(x, y, z) is the depth map gradient, indicating the local change of the depth at (x, y, z); k_smooth is a parameter that controls the sensitivity of the depth smoothing term;

[0130] E(D)=Σ(x,y,z) (C(x,y,z,D(x,y,z))+λ*V(D(x,y,z), d(N(x,y,z)))), where N(x,y,z) represents the neighborhood pixels of (x,y,z), V is the smoothness constraint term, and λ is the weight coefficient; D(x, y, z) represents the depth value at the point (x, y, z) in three-dimensional space.

[0131] In a further embodiment, the energy function may also be: E(D) = λ_lf * E_lightfield(D) +λ_s * E_smooth(D) + λ_c * E_consistency(D, I_multi);

[0132] Where E_lightfield(D) is a data term based on the light field: E_lightfield(D) = Σ_p Σ_q∈Ω(p) ρ(L(x,y,s,t) - L'(x,y,s,t,D_p)), L(x,y,s,t) is the 4D light field function, (x,y) is the spatial coordinate, (s,t) is the angular coordinate. Ω(p) is the neighborhood of pixel p in the light field. L' is the light field reconstructed based on the current depth D_p. ρ is the robustness function.

[0133] E_smooth(D) = Σ_p Σ_q∈N(p) w(D_p, D_q) * ψ(D_p - D_q), where w(D_p,D_q) is the depth-based adaptive weight. E_consistency(D, I_multi) is the multimodal consistency term. E_consistency(D, I_multi) = Σ_p Σ_m φ(I_m(p) - I'_m(p, D_p)), where I_m represents images of different modalities (such as fluorescence and white light), and I'_m is the image reprojected based on the depth D_p.

[0134] In another embodiment of the present application, the process of the dynamic programming algorithm is specifically as follows:

[0135] Construct a 3D cost volume: C(x, y, z, d) = ω1 * NCC(x, y, z, d) + ω2 * exp(-|▽I(x, y, z)| / k_edge) * (1 - exp(-|d - D_prev(x, y, z)| / k_temp)) + ω3 * (1 -exp(-|▽D(x, y, z)| / k_smooth)); where (x, y, z) are spatial coordinates and d is the disparity value.

[0136] 3D path optimization: Convert the 3D problem into an optimization problem of multiple 2D slices. For each yz plane, perform the following steps:

[0137] Initialize the cumulative cost matrix M(y, z, d) = C(x0, y, z, d);

[0138] For each x from x0+1 to xmax: M(y, z, d) = C(x, y, z, d) + min(M(y-1, z, d-1), M(y-1, z, d), M(y-1, z, d+1), M(y, z-1, d-1), M(y, z-1, d), M(y, z-1, d+1));

[0139] Starting from the last column of M, we trace back to obtain the optimal disparity path by finding the minimum cost path for each step.

[0140] Adding smoothness constraints between different yz planes, a Markov Random Field (MRF) model can be used to achieve global consistency.

[0141] Repeat the above steps, using the result of the previous iteration as a priori, until convergence or the preset number of iterations is reached.

[0142] In another embodiment of the present application, step S2 is specifically:

[0143] Step S21, using the initial depth estimation map D_init(x, y, z, t), performing depth adaptive denoising on the registered multimodal image set I_registered(x, y, z, t);

[0144] The adaptive denoising process includes: constructing a deep adaptive anisotropic diffusion filter, performing anisotropic diffusion of the diffusion system, and solving the partial differential equation dI / dt=div(c(x, y, z, t)▽I) by iterative calculation using a semi-implicit finite difference format to obtain the denoised image I_denoised(x, y, z, t);

[0145] Diffusion coefficient c(x, y, z, t) = exp(-(||▽I|| / K(x, y, z, t)) 2 ), where K(x, y, z, t) = k0*(1+α *d_init(x, y, z, t)) is the depth adaptive threshold, and k0 and α are preset constants;

[0146] Step S22, based on the denoised image I_denoised(x, y, z, t) and the initial depth estimate D_init(x, y, z, t), optimizing the depth map by a variational optimization method, including: constructing an energy function E(D), and using multimodal information to constrain the depth estimation, using the primal-dual interior point method to iteratively solve the optimization problem, minimizing the energy function, and obtaining an improved depth map D_improved(x, y, z, t);

[0147] Among them, E(D)=E_data(D)+λ_s*E_smooth(D)+λ_c*E_consistency(D, I_denoised), where E_data is the data term, ensuring that the depth estimation is consistent with the image observation; E_smooth is the smoothing term, maintaining the local continuity of the depth; E_consistency is the consistency term;

[0148] Step S23, using the improved depth map D_improved(x, y, z, t) and the denoised image I_denoised(x, y, z, t), constructing a multi-scale Hessian matrix H_σ(x, y, z, t), calculating the eigenvalues ​​of the Hessian matrix as vascular features, and constructing a depth-aware vascular response function V(x, y, z, t, σ) based on the eigenvalues;

[0149] Among them, H_σ(x, y, z, t)=[d2 I / dx 2 , d 2 I / dxdy,d 2 I / dxdz;d 2 I / dydx,d 2 I / dy 2 , d 2 I / dydz;d 2 I / dzdx,d 2 I / dzdy,d 2 I / dz 2 ], where σ is the Gaussian scale space parameter; the eigenvalues ​​are λ1, λ2, λ3, and |λ1|≤|λ2|≤|λ3|;

[0150] V(x,y,z,t,σ)=(1 - exp(-R_A 2 / 2α(D) 2 ))*exp(-R_B 2 / 2β(D) 2 )*(1-exp(-S 2 / 2γ(D) 2 )), where R_A=|λ2| / |λ3|; R_B =|λ1| / sqrt(|λ2λ3|); S=sqrt(λ1²+λ2²+λ3²); α(D), β(D), γ(D) are depth adaptive parameters, α(D)=α0*exp(-D / D0); β(D)=β0*exp(-D / D0); γ(D)=γ0*exp(-D / D0); D=D_improved(x,y,z,t) is the depth value of the point; D0, α0, β0, γ0 are preset parameters;

[0151] Step S24, based on the vascular response function V(x, y, z, t, σ), a dynamic scale integration method is used to calculate and weighted sum based on an adaptive weight function to perform multi-scale vascular integration and enhancement to obtain a preliminary enhanced vascular image V_enhanced(x, y, z, t);

[0152] Among them, the adaptive weight function w(σ, x, y, z, t) = exp(-(σ-σ_opt(x, y, z, t)) 2 / 2σ_w 2 ), where σ_opt is the optimal scale and σ_w is the scale variance;

[0153] The weighted summed vascular image is V_enhanced(x, y, z, t)=Σ_σw(σ, x, y, z, t)*V(x, y, z, t, σ);

[0154] Step S25: Combine the improved depth map D_improved(x, y, z, t) and the initially enhanced blood vessel image V_enhanced(x, y, z, t) and use the improved optical flow equation ▽Iu+I_t+λ(x, y, z, t)(▽ 2 u)=0 to estimate the local motion field M(x, y, z, t), where λ(x, y, z, t) is the deep adaptive regularization parameter;

[0155] Call the extended Kalman filter for filtering. The state vector X contains the blood vessel position and velocity information, and the observation vector Z contains the features extracted from V_enhanced and M. Through the prediction step X_t - =F_t*X_t-1+B_t*u_t and update step X_t=X_t - +K_t*(Z_t - H_t*X_t - ) realizes blood vessel tracking, where K_t is the Kalman gain;

[0156] Based on the filtering results, thin plate spline interpolation is used for local non-rigid registration to obtain the time-consistent vascular enhancement sequence V_track(x, y, z, t) and the second optimized depth map D_final(x, y, z, t) as vascular positioning and tracking data.

[0157] In another embodiment of the present application, S3 is specifically: step S31, constructing a signal attenuation model A(x, y, z, t)=exp(-μ(x, y, z, t) *d_final(x, y, z, t)) based on the final depth map D_final(x, y, z, t), where μ(x, y, z, t) is a local attenuation coefficient;

[0158] Construct the optimization problem μ_opt=argmin_μ||V_track - A*V_ideal|| 2 +λR(μ), where V_ideal is the ideal unattenuated image and R(μ) is the spatial smoothing regularization term. The optimization problem is solved using the alternating direction multiplier method to obtain the optimal attenuation coefficient μ_opt(x, y, z, t) and attenuation model A(x, y, z, t).

[0159] Step S32: construct a multi-scale anisotropic scattering kernel S_σ(x, y, z)=(1 / 2πσ 2 ) 3 / 2 *exp(-(x 2 +αy 2 +βz 2 ) / 2σ 2 ), where α and β are depth-dependent anisotropy parameters;

[0160] For each scale σ, calculate the scattered image V_scatter_σ(x, y, z, t) = V_track(x, y, z, t) * S_σ(x, y, z);

[0161] Design an adaptive weight function w_σ(x, y, z, t) = f(D_final, A, local_features), where local_features includes local contrast and gradient;

[0162] Obtain the scattered compensation image by weighted summation V_scatter(x, y, z, t) = Σ_σw_σ(x, y, z, t) * V_scatter_σ(x, y, z, t);

[0163] Step S33: Perform iterative deconvolution using an improved Richardson-Lucy algorithm to obtain the image V_compensated(x, y, z, t) after signal compensation and artifact suppression;

[0164] Among them, the iterative formula is V_k+1 = V_k * (A T * (V_track / (A * V_k + V_scatter))) / (1 + λ * div(▽V_k / |▽V_k|)), where V_k is the result of the k-th iteration, A T is the transpose of A, and λ is the regularization parameter;

[0165] Step S34: Calculate the local contrast C(x, y, z, t) and local brightness B(x, y, z, t); Design a piecewise linear mapping function f(x) = {a1x + b1, x < t1; a2x + b2, t1 ≤ x < t2; a3x + b3, x ≥ t2}, where the parameters ai, bi and the threshold ti are dynamically adjusted according to C and B; Apply the piecewise linear mapping function f(x) to perform dynamic range adjustment on V_compensated to obtain V_adjusted; Call the adaptive gamma correction function g(x) = x (γ(x,y,z,t)) , where γ(x, y, z, t) = γ0 * (1 + k1C + k2B), γ0, k1, k2 are preset parameters; Use g(x) to enhance the local contrast of V_adjusted to obtain the final optimized 3D ultra-high-definition fluorescence medical endoscope blood vessel image V_final(x, y, z, t).

[0166] In another embodiment of the present application, step S4 is specifically as follows:

[0167] In step S41, the final optimized displayed 3D ultra-high-definition fluorescence medical endoscope vascular image V_final (x, y, z, t) and the final depth map D_final (x, y, z, t) are input into the feature fusion module;

[0168] Apply the Frangi filter to V_final to obtain the vascular enhancement feature F_v(x, y, z, t)=2R_a 2 R_b 2 (1-exp(-S 2 / (2c 2 )))(1-exp(-2R_a 2 / α 2 )), where R_a, R_b and S are vessel-related metrics calculated based on the eigenvalues ​​of the Hessian matrix. An adaptive weight function ω(x, y, z, t) = sigmoid(λ1C_v + λ2D_final) is designed, where C_v is the local contrast of V_final. The fused feature map is obtained by weighted fusion F_fused(x, y, z, t) = ωF_v + (1-ω)D_final.

[0169] Step S42: Use the level set method to perform 3D blood vessel segmentation. The evolution equation is dφ / dt+F(x, y, z, t)|▽φ|=α▽·(g(|▽I|)▽φ), where φ is the level set function, F is the velocity function based on F_fused, and g is the edge stop function. The time regularization term E_temp(φ_t)=∫(φ_t - φ_t-1 ○t_t-1, t) is introduced. 2 dxdydz, where T_t-1, t is the estimated deformation field from t-1 to t. Add the curvature regularization term E_curv(φ)=∫g(|▽I|)|▽H(φ)|dxdydz, where H is the Heaviside function. Construct the energy function E(φ)=E_data(φ)+λ_tempE_temp(φ)+λ_curvE_curv(φ), use the alternating direction multiplier method (ADMM) to solve the optimization problem, and obtain the spatiotemporally consistent blood vessel segmentation mask M(x, y, z, t).

[0170] Step S43, apply Euclidean distance transform to the blood vessel segmentation mask M(x, y, z, t) to obtain the distance map D_skel(x, y, z, t). Use the grayscale skeleton-based method to extract the initial skeleton S_init(x, y, z, t) = {(x, y, z) | d_skel(x, y, z, t) = local maximum}. Detect the skeleton breakpoints p_break, and if there are breakpoint pairs (p_i, p_j) that satisfy the condition ||p_i - p_j|| < ε, connect them. Remove branches whose length is less than the threshold L_min. Apply the skeleton thinning algorithm S_thin(x, y, z, t) = thin(S_init(x, y, z, t), d_skel(x, y, z, t)), keeping the endpoints unchanged. Estimate the blood vessel radius R(x, y, z, t) = d_skel(x, y, z, t) | S_thin(x, y, z, t) = 1, and obtain the blood vessel skeleton S_thin(x, y, z, t) and the corresponding radius map R(x, y, z, t);

[0171] Step S44, extract the endpoints and bifurcation points of the vascular skeleton S_thin(x, y, z, t), and construct the vascular network graph structure G(V, E), where V is the node set and E is the edge set. For each vascular segment e ∈ E, use 3D Cardinal spline to fit: C_e(s)=Σ_i P_i*F_i,4(s), where P_i is the control point and F_i,4 is the 4th-order basis function. Establish an adaptive diameter model: r_e(s)=r_0+aexp(-bs)+csin(ωs), and its parameters are fitted by the least squares method to the changes of R(x, y, z, t) along the vascular segment. Apply Laplacian smoothing to ensure a smooth transition at the bifurcation. Use the rotated section method to generate a vascular surface mesh along the spline curve, and the size of the triangle patch is adaptively adjusted according to the local curvature to obtain the vascular network model V_network(x, y, z, t);

[0172] Step S45, the reconstructed vascular network model V_network(x, y, z, t) is fused with the final optimized and displayed 3D ultra-high-definition fluorescent medical endoscope vascular image V_final(x, y, z, t) to generate a comprehensive visualization result. Construct a multi-layer rendering pipeline: V_final(x, y, z, t) is used as the background layer; the center line and radius information of V_network(x, y, z, t) is projected onto the image plane as a network overlay layer; according to the depth information of V_network(x, y, z, t), the projection result is deeply encoded and colored as an enhancement layer. Then, an adaptive mixing function B(x, y, z, t) = α(x, y, z, t) * V_final(x, y, z, t) + (1 - α(x, y, z, t)) * P(V_network(x, y, z, t)) is designed, where P is the projection function and α is an adaptive mixing coefficient dynamically adjusted according to local contrast and depth information. Next, an edge enhancement filter E(x, y, z) = B(x, y, z, t) + λ*▽²B(x, y, z, t) is applied, where λ is the enhancement strength and ▽² is the Laplacian operator, to highlight the edge of the blood vessels. High dynamic range (HDR) tone mapping technology is applied to ensure the best visual effect on different display devices, and the final fused visualization result V_fused(x, y, z, t) is obtained.

[0173] In another embodiment of the present application, the image quality assessment index and the accuracy assessment method of blood vessel segmentation and reconstruction are specifically:

[0174] The image quality evaluation indicators are:

[0175] Peak Signal-to-Noise Ratio (PSNR): PSNR = 20 * log10(MAX_I) - 10 * log10(MSE); where MAX_I is the maximum pixel value of the image and MSE is the mean square error.

[0176] Structural Similarity Index (SSIM): SSIM(x, y) = (2μxμy+ c1)(2σxy + c2) / ((μx 2 + μy 2 + c1)(σx 2 + σy 2 + c2)); where μx and μy are the means of x and y, σx and σy are the variances of x and y, and σxy is the covariance of x and y.

[0177] The evaluation index of blood vessel segmentation accuracy is:

[0178] Dice coefficient: Dice = 2|X ∩ Y| / (|X| + |Y|); Jaccard index: Jaccard = |X ∩ Y| / |X ∪ Y|; where X is the predicted segmentation result and Y is the true annotation.

[0179] 3D reconstruction accuracy evaluation: Average surface distance (ASD): ASD = (1 / |S1| * Σmin(d(x, S2)) +1 / |S2| * Σmin(d(y, S1))) / 2; where S1 and S2 are the reconstructed surface and the true surface, respectively, and d(x, S) is the minimum distance from point x to surface S.

[0180] Hausdorff distance: HD = max(max(min(d(x, S2))), max(min(d(y, S1)))); x∈S1, y∈S2.

[0181] According to one aspect of the present application, the multi-scale Hessian filter for depth perception is:

[0182] V(x, y, z, t, σ) = (1 - exp(-Ra 2 / 2α(D) 2 )) * exp(-Rb 2 / 2β(D) 2 ) * (1- exp(-S 2 / 2γ(D) 2 )) Where α(D), β(D), γ(D) are depth-based adaptive parameters: α(D) = α0* (1 + ka * exp(-D / D0)) β(D) = β0 * (1 + kb * exp(-D / D0)) γ(D) = γ0 * (1 +kc * exp(-D / D0))

[0183] Depth-adaptive contrast enhancement: I_enhanced(x, y, z) = I(x, y, z) + λ(D) * (I(x, y, z) - Gσ(D) * I(x, y, z)) where λ(D) = λ0 * (1 + kλ * exp(-D / D0)) and Gσ(D) is a depth-adaptive Gaussian filter.

[0184] According to one aspect of the present application, a multimodal fusion 3D reconstruction method:

[0185] Introduce multimodal feature fusion: F_fused = w1(D) * F_fluorescence + w2(D) * F_white_light + w3(D) * F_depth where wi(D) is an adaptive weight function based on depth.

[0186] Energy function of multimodal consistency constraint: E(S) = Edata(S) + λ1 * Esmooth(S) + λ2 *Econsistency(S, Ifluorescence, Iwhite_light, Idepth) where S is the reconstructed 3D surface, Edata is the data term, Esmooth is the smoothness term, and Econsistency is the multimodal consistency term.

[0187] According to one aspect of the present application, the specific formula of the energy function of step S42 is: E(φ) = Edata(φ) + λtemp * Etemp(φ) + λcurv * Ecurv(φ); wherein: φ is the level set function; Edata(φ) = ∫Ω g(x, y, z) * δ(φ) * |▽φ| dxdydz data term, g(x, y, z) is an edge indication function based on image gradient; Etemp(φ) = ∫Ω (φt - φt-1 ○ Tt-1, t) 2 dxdydz is the time regularization term, Tt-1, where t is the estimated deformation field from t-1 to t; Ecurv(φ) = ∫Ω g(x, y, z) * |▽H(φ)| dxdydz is the curvature regularization term, H(φ) is the Heaviside function; λtemp and λcurv are weight parameters used to balance the contribution of each term.

[0188] The preferred embodiments of the present invention are described in detail above; however, the present invention is not limited to the specific details in the above embodiments. Within the technical concept of the present invention, various equivalent transformations can be made to the technical solutions of the present invention, and these equivalent transformations all belong to the protection scope of the present invention.

Claims

1. A 3D ultra-high definition fluorescence medical endoscope vascular image enhancement method, characterized in that: The steps include: Step S1, obtaining the original 3D fluorescence endoscope image sequence and the white light image sequence, and performing preliminary quality inspection and preprocessing to remove obvious noise and artifacts; using the mutual information maximization method and the non-rigid registration method to accurately align the 3D fluorescence endoscope image sequence and the white light image sequence to obtain a registered multimodal image; based on the registered multimodal image, using stereo vision technology, generating an initial depth estimation map; Step S2: using the initial depth estimation map, perform preliminary depth adaptive denoising on the registered multimodal image; Based on the denoised image and the initial depth estimation map, a first refinement is performed using a variational optimization method to generate an improved depth map. The improved depth map is used to perform a second round of depth adaptive noise reduction. The vascular structure features are extracted through multi-scale Hessian matrix analysis, and the vascular visibility is enhanced through the depth-aware vascular response function. The dynamic scale integration method is used to fuse multi-scale responses to obtain a preliminary enhanced vascular image. Combined with the improved depth map information, the spatiotemporal collaborative tracking technology is used to accurately locate and track the moving blood vessels, and a time-consistent vascular enhancement sequence and a final depth map after a second refinement process are obtained. Step S3, constructing a signal attenuation model based on the final depth map to compensate for the signal intensity change caused by depth; modeling and compensating for the scattering effects at different depths through a multi-scale scattering kernel; using a time-consistent vascular enhancement sequence as prior information, and recovering the original signal through an iterative constrained deconvolution algorithm while considering signal attenuation and scattering effects; improving the visual quality and detail visibility of the image through a dynamic range optimization method, and obtaining a final optimized and displayed 3D ultra-high-definition fluorescent medical endoscope vascular image; Step S4, fusing the 3D ultra-high-definition fluorescent medical endoscope vascular image and the final depth map to improve the accuracy of vascular recognition; performing 3D vascular segmentation through spatiotemporal consistency constraints and time-consistent vascular enhancement sequences; extracting the vascular skeleton using a topology-preserving method to represent the centerline and branch structure of the blood vessel; reconstructing a complete 3D vascular network model based on the extracted skeleton, radius information and the final depth map; fusing the reconstructed vascular network model with the optimized 3D ultra-high-definition fluorescent medical endoscope vascular image to generate a comprehensive visualization result.

2. The 3D ultra-high definition fluorescence medical endoscope vascular image enhancement method according to claim 1, characterized in that: Step S1 is specifically as follows: Step S11, acquiring original images from the 3D fluorescence medical endoscope system, including an original 3D fluorescence endoscope image sequence and a corresponding white light image sequence, and preprocessing the original images to obtain a preprocessed fluorescence image sequence and a white light image sequence that have been preliminarily optimized; wherein the preprocessing includes preliminary quality assessment, pulse noise removal, Gaussian noise suppression, motion artifact removal, and image contrast enhancement; Step S12, inputting the preprocessed fluorescence image sequence and the white light image sequence into a preconfigured registration module, performing coarse registration and fine registration, and generating a registered multimodal image set that is precisely aligned with the white light image sequence; The coarse registration uses the mutual information maximization method to achieve global rigid transformation and obtain the initial registration parameters; the fine registration uses the non-rigid registration algorithm based on B-spline to construct the optimization objective function and iteratively optimize to obtain the optimal registration transformation applied to the preprocessed fluorescence image sequence; Step S13, read the registered multimodal image set, and for each pixel position, calculate the normalized cross-correlation value in its local neighborhood; based on the normalized cross-correlation value, construct a cost volume, and according to the cost volume, find the optimal disparity path at each spatial position through a dynamic programming algorithm, so that the first energy function containing all constraints is minimized; and obtain an initial depth estimation map.

3. The 3D ultra-high definition fluorescence medical endoscope vascular image enhancement method according to claim 2, characterized in that: Step S2 is specifically as follows: Step S21, using the initial depth estimation map, performing depth adaptive denoising on the registered multimodal image set to obtain a denoised image; The adaptive denoising process includes: constructing a deep adaptive anisotropic diffusion filter, performing anisotropic diffusion based on the diffusion coefficient, iteratively calculating using a semi-implicit finite difference format, solving the partial differential equation, and finally obtaining the denoised image; Step S22: based on the denoised image and the initial depth estimation map, calling the pre-constructed second energy function, using multimodal information to constrain the depth estimation, using the primal dual interior point method to iteratively solve the optimization problem, minimizing the second energy function, and optimizing the depth map through a variational optimization method to obtain an improved depth map; Step S23, constructing a multi-scale Hessian matrix using the improved depth map and the denoised image, calculating the eigenvalues ​​of the Hessian matrix as vascular features, constructing a depth-aware vascular response function based on the eigenvalues, and obtaining a vascular response map; wherein the construction of the vascular response function takes depth information into consideration and adapts to vascular features of different depths through adaptive parameter adjustment; Step S24: construct an adaptive weight function based on the vascular response function, calculate the weight based on the optimal scale and scale variance, integrate the multi-scale vascular response through weighted summation, achieve multi-scale vascular integration and enhancement, and obtain a preliminary enhanced vascular image; Step S25, combining the improved depth map and the preliminary enhanced vascular image, using the improved optical flow equation to estimate the local motion field, constructing the state vector and observation vector of the extended Kalman filter, realizing vascular tracking through prediction steps and update steps, and using thin plate spline interpolation to perform local non-rigid registration based on the filtering results, finally obtaining a time-consistent vascular enhancement sequence and an optimized final depth map.

4. The 3D ultra-high definition fluorescence medical endoscope vascular image enhancement method according to claim 3, characterized in that: Step S3 is specifically as follows: Step S31: construct a signal attenuation model based on the final depth map, construct an optimization problem, use the alternating direction multiplier method to minimize the difference between the ideal non-attenuated image and the actual image, and introduce a spatial smoothing regularization term to solve the optimization problem, so as to obtain the optimal attenuation coefficient and attenuation model; Among them, the signal attenuation model includes local attenuation coefficient and depth information; Step S32, constructing a multi-scale anisotropic scattering kernel, calculating a scattering image for each scale, designing an adaptive weight function, and obtaining a scattering compensation image by weighted summation; The construction of the scattering kernel takes into account the depth-related anisotropic parameters, the design of the adaptive weight function is based on the depth information, attenuation model and local image features, and the weighted summation process realizes the comprehensive compensation of multi-scale scattering effects. Step S33, using the improved Richardson-Lucy algorithm to perform iterative deconvolution to obtain an image after signal compensation and artifact suppression; The iterative deconvolution process includes signal attenuation model, scattering compensation and regularization terms, and the original signal is restored and the image quality is improved through multiple iterative optimizations. Step S34, calculate the local contrast and local brightness, design a piecewise linear mapping function and an adaptive gamma correction function, adjust the dynamic range and enhance the local contrast of the image after signal compensation, and obtain the final optimized display of the 3D ultra-high-definition fluorescent medical endoscope vascular image; wherein the parameters of the piecewise linear mapping function are dynamically adjusted according to the local contrast and brightness, and the adaptive gamma correction function takes into account the local image characteristics.

5. The 3D ultra-high definition fluorescence medical endoscope vascular image enhancement method according to claim 4, characterized in that: Step S4 is specifically as follows: Step S41, inputting the final optimized displayed 3D ultra-high-definition fluorescent medical endoscope vascular image and the final depth map into the feature fusion module, applying the Frangi filter to the vascular image to obtain vascular enhancement features, designing an adaptive weight function, and obtaining a fused feature map through weighted fusion; Among them, the Frangi filter calculates the vascular related metrics based on the eigenvalues ​​of the Hessian matrix, the adaptive weight function includes local contrast and depth information, and the weighted fusion process is used to effectively combine vascular features and depth information; Step S42, using the level set method to perform 3D blood vessel segmentation, constructing a third energy function and introducing a time regularization term and a curvature regularization term, using an alternating direction multiplier method to solve the optimization problem, and obtaining a blood vessel segmentation mask that is consistent in time and space; Among them, the evolution equation of the level set includes the information of the fusion feature map, the time regularization term is used to determine the time coherence of the segmentation result, and the curvature regularization term is used to achieve the smoothness of the segmentation boundary; Step S43, applying Euclidean distance transform to the blood vessel segmentation mask to obtain a distance map, using a grayscale skeleton-based method to extract an initial skeleton, detecting and connecting skeleton breakpoints, removing short branches, applying a skeleton thinning algorithm, and finally estimating the blood vessel radius to obtain a blood vessel skeleton and a corresponding radius map; The skeleton extraction process takes local maxima into account, breakpoint connection and short branch removal are used to ensure the continuity and representativeness of the skeleton, and skeleton refinement is used to maintain the endpoints unchanged. Step S44, extracting the endpoints and bifurcation points of the vascular skeleton, constructing a vascular network graph structure, fitting each vascular segment using 3D Cardinal splines, establishing an adaptive diameter model, applying Laplacian smoothing to ensure a smooth transition at the bifurcation, and using the rotation section method to generate a vascular surface mesh along the spline curve to obtain a vascular network model; Among them, the spline fitting takes into account the control points and basis functions, the adaptive pipe diameter model fits the radius change through the least squares method, and the mesh generation process adaptively adjusts the size of the triangle patch according to the local curvature; Step S45, fusing the reconstructed vascular network model with the final optimized displayed 3D ultra-high-definition fluorescent medical endoscope vascular image, constructing a multi-layer rendering pipeline, designing an adaptive blending function, applying an edge enhancement filter, and finally applying a high dynamic range tone mapping technique to obtain a final fusion visualization result, i.e., a comprehensive visualization result; Among them, the multi-layer rendering pipeline includes a background layer, a network overlay layer and an enhancement layer. The adaptive blending function dynamically adjusts the blending coefficient according to the local contrast and depth information, and the edge enhancement filter is used to highlight the edge of the blood vessels.

6. The 3D ultra-high definition fluorescence medical endoscope vascular image enhancement method according to claim 5, characterized in that: Step S12 is specifically as follows: Step S121, constructing a multi-resolution image pyramid, down-sampling the pre-processed fluorescence image sequence and the white light image sequence to obtain image sequences of different resolution levels; wherein a Gaussian filter is used for smoothing, and then secondary sampling is performed to generate a multi-level image sequence from coarse to fine, and the resolution of each level is half of the previous level; Step S122: Starting from the coarsest resolution level, a global rigid registration is performed using a mutual information maximization method to obtain initial registration parameters; then, registration is performed step by step to finer resolutions, and each level uses the registration result of the previous level as an initial value, and the registration accuracy is improved through iterative optimization; wherein, at each resolution level, a mutual information objective function is constructed, and the gradient descent method is used for optimization to obtain the optimal rigid transformation parameters of the level, i.e., the rigid registration result; Step S123: at the finest resolution level, based on the rigid registration result, construct an adaptive local deformation model for non-rigid registration, and perform registration, specifically: The image is divided into overlapping local blocks, and a feature descriptor based on gradient and structural similarity is calculated for each local block; Adaptively adjust the deformation freedom and regularization strength of each local block according to the similarity of feature descriptors; Construct a global energy function, including data term, smoothing term and consistency constraint term, and use the alternating direction multiplier method to optimize and solve it to obtain the final non-rigid deformation field; A nonrigid deformation field is applied to the preprocessed fluorescence image sequence to generate a registered multimodal image set that is precisely aligned with the white light image sequence.

7. The 3D ultra-high definition fluorescence medical endoscope vascular image enhancement method according to claim 5, characterized in that: Step S13 is specifically as follows: Step S131, read the registered multimodal image set, perform Fourier transform on the images in the multimodal image set, and analyze the spectral characteristics; construct a 4D light field function by estimating the local light direction; calculate the angle-space correlation in the light field, and obtain light field information as a preliminary depth clue; Step S132: constructing a depth estimation energy function based on light field information and combined with multi-view geometric constraints; Step S133, using the variational method to optimize the constructed energy function, specifically: by discretizing the energy function, constructing a large-scale sparse linear equation system; using the alternating direction multiplier method for iterative solution; using a multi-scale strategy to gradually refine the depth estimation, and obtain a high-precision initial depth estimation map.

8. The 3D ultra-high definition fluorescence medical endoscope vascular image enhancement method according to claim 5, characterized in that: Step S21 is specifically as follows: Step S211: constructing a depth-aware edge detector for the registered multimodal image set based on the initial depth estimation map; calculating the multi-scale gradient of the image through the edge detector, adaptively adjusting the gradient threshold according to the depth information, and obtaining accurate edge map information through non-maximum suppression; Step S212: construct a nonlinear filter kernel based on anisotropic diffusion for edge map information; dynamically adjust filter parameters according to local depth and edge strength; Adaptively determine the range and strength of the filter to form a structure-preserving adaptive filter; Step S213, applying an adaptive filter to the registered multimodal image set to perform noise reduction processing; Through the iterative optimization process, noise is effectively suppressed while maintaining edge and structural information; the filter strength is adaptively adjusted according to the depth information to obtain a deep adaptive denoised image.

9. The 3D ultra-high definition fluorescence medical endoscope vascular image enhancement method according to claim 5, characterized in that: Step S23 is specifically as follows: Step S231: construct basic filter templates of different scales and directions based on the denoised image; dynamically adjust the scale and direction parameters of the filter according to the local depth information; construct a multi-scale and multi-directional blood vessel enhancement filter group, that is, obtain an adaptive filter group; Step S232, applying an adaptive filter group to perform a convolution operation on the image, and for each pixel, calculating the maximum response value and its corresponding scale and direction; constructing a blood vessel possibility map according to the response value, scale and direction information, that is, a multi-scale, multi-directional response map; Step S233: Based on the vascular possibility map, combined with the original Hessian matrix analysis method, the Hessian matrix and its eigenvalues ​​are calculated, and a fusion function is used to comprehensively consider the filter response and Hessian characteristics to perform feature fusion, and an enhanced vascular feature map is obtained through nonlinear mapping.

10. The 3D ultra-high definition fluorescence medical endoscope vascular image enhancement method according to claim 5, characterized in that: Step S25 is specifically as follows: Step S251: Based on the preliminary enhanced blood vessel image and the improved depth map, a blood vessel motion prediction model is established using the blood vessel position and motion information of the previous predetermined frame; the parameters of the blood vessel motion prediction model are adjusted according to the depth information to improve the prediction accuracy, forming a prediction-correction model, and the prediction-correction model is used to generate a predicted blood vessel position and shape, i.e., a prediction result; Step S252: read the prediction results according to the preconfigured adaptive sampling strategy, construct a probability distribution map around the predicted blood vessel position, dynamically adjust the sampling density according to the local image features and depth information, and generate a set of candidate blood vessel points after completing the sampling work in the image space; Step S253, evaluating and screening the candidate blood vessel points, including: calculating the local feature descriptor of each candidate point; comparing with the result of the prediction model and calculating the matching score; using the Markov random field model to globally optimize the candidate points to obtain the optimal blood vessel tracking result; Step S254: Based on the blood vessel tracking result, thin plate spline interpolation is used to perform local non-rigid registration, and finally a time-consistent blood vessel enhancement sequence and an optimized final depth map are obtained.

11. A 3D ultra-high definition fluorescence medical endoscope vascular image enhancement system, characterized in that: include: at least one processor; as well as, a memory communicatively connected to at least one of the processors; wherein, The memory stores instructions executable by the processor, and the instructions are used to be executed by the processor to implement the 3D ultra-high-definition fluorescence medical endoscope vascular image enhancement method according to any one of claims 1 to 10.

12. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the 3D ultra-high-definition fluorescence medical endoscope vascular image enhancement method described in any one of claims 1 to 10 are implemented.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the 3D ultra-high-definition fluorescence medical endoscope vascular image enhancement method described in any one of claims 1 to 10 are implemented.