Cardiac interventional surgery scene reconstruction method based on 3D ultrasound imaging rendering

Abstract

The present invention discloses a method for reconstructing cardiac interventional surgery scenes based on 3D ultrasound imaging rendering, comprising collecting preoperative computed tomography or magnetic resonance imaging images as prior information; preprocessing a real-time 3D ultrasound image sequence; voxelizing the images in combination with the prior information, converting the image data into a voxel representation in three-dimensional space, and creating an initial set of 3D Gaussian functions; optimizing the parameters of the initial Gaussian functions through an iterative optimization algorithm to minimize the difference between the reconstructed image and the actual image; alternately performing adaptive density control of the 3D Gaussian functions in combination with the shape constraints of the interventional catheter; and rendering tile by tile using a tile-based Gaussian sputtering technique to achieve real-time 3D scene reconstruction and display. The present invention combines prior information and the shape constraints of the interventional catheter, uses 3D Gaussian functions to represent and simulate various structures in the surgical scene, and achieves efficient rendering through the tile-based sputtering technique.

Description

Technical Field

[0001] The present invention relates to the field of robot-assisted surgery scene rendering, and more specifically, to a cardiac interventional surgery scene reconstruction method based on 3D ultrasound imaging rendering. Background Art

[0002] Cardiac interventional surgery is a new technology for diagnosing and treating cardiovascular diseases. It is a diagnostic and treatment method that punctures surface blood vessels, inserts cardiac interventional catheters, and uses specific cardiac interventional catheter operation techniques to diagnose and treat heart diseases. Heart valve replacement is a highly complex cardiovascular interventional surgery designed to treat heart valve diseases such as valvular stenosis or valvular insufficiency. This surgery requires doctors to navigate and operate precisely to ensure the correct installation and function of the valve. Traditionally, doctors have relied on X-ray imaging technology to provide intraoperative visualization assistance. However, this method has several important limitations, including radiation exposure to patients and low imaging contrast for soft tissues.

[0003] Gaussian sputtering is the most effective three-dimensional reconstruction method in recent years. However, the existing reconstruction methods are mainly aimed at general scenarios and have many limitations when applied to special scenarios such as heart valve replacement surgery. For example, the method proposed in the Chinese invention patent with application number 202410417649.7 does not take into account the prior shape information of the heart valve, and it is difficult to meet the surgical requirements for valve reconstruction accuracy. In addition, the heart is not a static organ, and the existing methods lack effective compensation for heart movement, which can easily lead to distortion of the reconstruction results. Moreover, the existing methods also lack specific functions for surgical navigation, such as valve tracking, risk area prompts, etc., which are difficult to meet the actual needs of doctors.

[0004] In recent years, with advances in medical imaging technology, real-time three-dimensional ultrasound (RT3D-US) has gradually become an important tool in cardiac interventional surgery. RT3D-US offers advantages such as radiation-free, real-time imaging, and excellent soft tissue resolution, making it an ideal surgical navigation tool. However, despite its many advantages in cardiac imaging, RT3D-US still faces challenges, such as limited image quality (high noise and low contrast) and a restricted field of view, which restrict its application in high-quality three-dimensional scene reconstruction.

[0005] Therefore, developing a method for reconstructing cardiac interventional surgery scenes based on real-time high-quality rendering of 3D ultrasound imaging is a technical problem that needs to be solved urgently. Summary of the Invention

[0006] To address the aforementioned shortcomings of existing technologies, the present invention provides a method for reconstructing cardiac interventional surgical scenes based on 3D ultrasound imaging rendering. This method utilizes RT3D-US image sequences and employs a series of advanced image processing and 3D reconstruction techniques to achieve high-quality 3D reconstruction and real-time rendering of surgical scenes. The core of this method is the use of 3D Gaussian functions to represent and simulate various structures in the surgical scene, and the efficient rendering of images through tile-based splatting.

[0007] To achieve the above objectives, the present invention provides a method for reconstructing a cardiac interventional surgery scene based on 3D ultrasound imaging rendering, which is characterized by comprising the following steps:

[0008] Using real-time three-dimensional ultrasound imaging equipment to capture dynamic image sequences of the heart and its surrounding structures during cardiac interventional surgery; collecting preoperative computed tomography or magnetic resonance imaging images as prior information;

[0009] Perform preprocessing on the acquired real-time 3D ultrasound image sequence, including noise reduction, contrast adjustment and image enhancement;

[0010] Combined with prior information, the preprocessed image is voxelized, the image data is converted into a voxel representation in three-dimensional space, and an initial set of 3D Gaussian functions is created;

[0011] Each voxel is simulated using a 3D Gaussian function to form a continuous 3D scene representation. The parameters of the initial Gaussian function are optimized through an iterative optimization algorithm to minimize the difference between the reconstructed image and the actual real-time 3D ultrasound image. During the optimization process, adaptive density control of the 3D Gaussian function is performed alternately in conjunction with the shape constraints of the interventional catheter.

[0012] The tile-based Gaussian sputtering technology is used to divide the three-dimensional scene into multiple small tiles, and rendering is performed tile by tile to achieve real-time three-dimensional scene reconstruction and display.

[0013] Furthermore, the real-time 3D ultrasound image sequence is gated using the electrocardiogram (ECG) signal, and images in the same cardiac cycle phase are selected for reconstruction.

[0014] Furthermore, the pretreatment comprises the following steps:

[0015] Anisotropic diffusion filtering: smoothes the image by solving partial differential equations while preserving the edge information of the image as much as possible; the diffusion coefficient $c$ is designed to diffuse quickly in flat areas of the image and slow down near the edges;

[0016] Contrast-limited adaptive histogram equalization: By locally adjusting the histogram of the image, the local contrast of the image is enhanced;

[0017] Morphological operations: Opening (erosion followed by dilation) and closing (dilation followed by erosion) are used to remove small acoustic structures and fill holes in the image.

[0018] Furthermore, the initial set of 3D Gaussian functions is generated based on the preprocessed image and prior information, comprising the following steps:

[0019] Feature point extraction: Use the 3D scale-invariant feature transform (SIFT) algorithm to detect key points in three-dimensional space;

[0020] Feature matching: Using the Random Sampling Consensus (RANSAC) algorithm, feature points from different time frames are matched to obtain a sparse 3D point cloud.

[0021] Initial Gaussian generation: Create initial 3D Gaussian functions centered at the matching 3D points. The initial parameters of these functions are set to reasonable default values ​​for subsequent optimization.

[0022] Furthermore, in the algorithm for optimizing the parameters of the initial Gaussian function, the optimization objective function uses a mixed loss function, which is defined as:

[0023] L=(1-λ)L1+λL SSIM

[0024] Among them, L1 is L1 loss, L SSIM is the structural similarity (SSIM) loss, and λ is the balancing factor;

[0025] Guided by prior information, for each 3D Gaussian function, its position, covariance matrix, opacity, and color are optimized. The Gaussian distribution of the interventional catheter area is constrained by using a cylindrical model. Gradient descent optimization is performed using the Adam optimizer, and the learning rate is gradually adjusted in combination with a learning rate decay strategy to ensure the stability and convergence of the optimization process.

[0026] Furthermore, the covariance matrix Σ is optimized by decomposing it into rotation and scaling matrices, decomposing Σ into the rotation matrix R and the scaling matrix S, Σ = |RSS T R T |, where T is the matrix transpose, optimizing the quaternion representing R and the scaling vector representing S; the opacity α is limited to the range [0, 1) by the sigmoid function; the color is optimized by gradually increasing the spherical harmonics (SH) coefficients from low to high order.

[0027] Furthermore, the density control of 3D Gaussian is performed alternately during the optimization process, including adding more Gaussian functions in complex areas, reducing the number of Gaussian functions in relatively smooth areas, or adjusting the distribution of Gaussian functions by splitting and merging.

[0028] Further, adding more Gaussian functions includes copying existing Gaussian functions and moving them along the gradient direction in the area with larger gradient; reducing the number of Gaussian functions includes removing Gaussian functions with opacity α less than the threshold ε. α Gaussian; Gaussian splitting is to split the larger Gaussian into two smaller Gaussians, and the position of the new Gaussian is obtained by sampling the PDF of the original Gaussian; regular resetting of the Gaussian is to set the α of all Gaussians to a value close to 0 every N iterations, and retain the necessary Gaussians through optimization.

[0029] Furthermore, the tile-based Gaussian sputtering technique includes the following steps:

[0030] Divide the image into multiple tiles;

[0031] For each tile, filter out the 3D Gaussians that intersect with it;

[0032] Use the GPU Radix sorting algorithm to sort the Gaussians based on depth and tile ID;

[0033] Rendering using the alpha blending model:

[0034]

[0035] Among them, C is the final color of the pixel, n is the number of Gaussians that affect the pixel, T i is the cumulative transparency, α i is opacity, c i is the color, σ i is the density, δ i is the Gaussian thickness;

[0036] Backward propagation: traverse the Gaussian from back to front, and calculate the gradient using the opacity values ​​accumulated in the forward propagation.

[0037] Furthermore, for heart valve replacement surgery, during the 3D Gaussian function optimization process, the optimization weight of the Gaussian function of the valve region is increased.

[0038] Compared with the prior art, the present invention has the following technical effects:

[0039] (1) This paper develops a 3D reconstruction initialization and optimization strategy tailored to the characteristics of RT3D-US images, overcoming the limitations of traditional methods. Doctors can obtain high-quality, high-resolution 3D images of cardiac scenes during surgery, which not only helps improve surgical accuracy and safety but also provides more intuitive surgical navigation and decision support. Furthermore, the method's real-time rendering capability ensures that doctors can instantly see changes during surgery and adjust their operational strategies in a timely manner.

[0040] (2) The present invention takes into account the dynamic characteristics of cardiac motion and designs an algorithm for adaptively adjusting the density of the Gaussian function, thereby improving the accuracy and efficiency of scene reconstruction to achieve the positioning and tracking of the heart valve.

[0041] (3) By introducing prior information and interventional catheter shape constraints, the present invention can realize functions such as interventional catheter tracking and specific area enhancement to meet the specific needs of cardiac interventional surgery.

[0042] (4) This invention is the first in the field of cardiac interventional surgery to use 3D Gaussian sputtering technology to perform high-quality, real-time scene reconstruction of RT3D-US. It is also suitable for other minimally invasive surgeries guided by ultrasound (such as liver tumor ablation, prostate resuscitation, etc.), and has broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is a flow chart of a method for reconstructing a cardiac interventional surgery scene in one embodiment of the present invention;

[0044] Figure 2 2 is a framework diagram of a three-dimensional Gaussian function initialization and optimization algorithm in one embodiment of the present invention. DETAILED DESCRIPTION

[0045] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.

[0046] In the following detailed description, many specific details are set forth to provide a more thorough understanding of the present invention. However, it is apparent to those skilled in the art that well-known algorithms and models (such as SIFT, RANSAC, and tile-based GPURadix sorting algorithm) are not shown in detail to avoid obscuring the main purpose of the present invention.

[0047] In addition, the order of execution of actions, steps, etc. in the devices and methods shown in the claims, specifications and drawings can be implemented in any order as long as there is no special explicit limitation on the order and the output of the previous processing is not used in the subsequent processing.

[0048] The present invention first uses real-time three-dimensional ultrasound imaging equipment to capture dynamic image sequences of the heart and its surrounding structures during heart valve replacement surgery. These image sequences contain detailed information about the valve, interventional catheter, and other related cross-sectional structures during surgery. At the same time, preoperative computed tomography or magnetic resonance imaging images are collected as prior information. Figure 1 The cardiac intervention surgery scene reconstruction method based on 3D ultrasound imaging rendering includes the following steps:

[0049] Step S1, Image Preprocessing: A series of preprocessing operations are performed on the acquired RT3D-US images, including but not limited to noise reduction, contrast adjustment, and image enhancement. These steps aim to improve image quality and provide clearer input for subsequent 3D Gaussian generation and optimization. As a concrete example, we use a series of preprocessing steps:

[0050] Anisotropic diffusion filtering: This filtering method smoothes the image by solving partial differential equations while preserving the edge information of the image as much as possible. The diffusion coefficient $c$ is designed so that the diffusion speed is fast in the flat areas of the image and slows down near the edges, thus achieving the dual purpose of noise reduction and edge preservation.

[0051] Contrast Limited Adaptive Histogram Equalization (CLAHE): By locally adjusting the histogram of an image, CLAHE enhances the local contrast of the image, making details that were originally compressed in dark or bright areas more visible.

[0052] Morphological operations: Opening (erosion followed by dilation) and closing (dilation followed by erosion) are used to remove small acoustic structures and fill holes in the image. These operations can further smooth the image and improve the results of subsequent processing.

[0053] Step S2, initial 3D Gaussian generation: Create an initial set of 3D Gaussian functions based on the preprocessed image and prior information. Specifically, the anatomical structure information in the prior information is used to guide the position and size distribution of the initial Gaussian function, so that the initial Gaussian function better matches the heart structure. The parameters of these Gaussian functions (position, covariance matrix, opacity and color) will serve as the starting point for subsequent optimization. As a specific example that can be implemented, see Figure 2 , the initial set of 3D Gaussian functions is generated based on the preprocessed image:

[0054] Feature point extraction: Use the 3D Scale-Invariant Feature Transform (SIFT) algorithm to detect key points in three-dimensional space. These points are crucial for subsequent feature matching. SIFT is a feature extraction algorithm in computer vision that extracts feature points from images that are invariant to scale, rotation, and brightness. It detects and describes feature points by finding local extreme points in spaces of different scales.

[0055] Feature matching: Using the Random Sample Consensus (RANSAC) algorithm, feature points from different time frames are matched to generate a sparse 3D point cloud. RANSAC is a robust statistical algorithm for estimating model parameters. It fits the model by randomly selecting a subset of data points, then calculates the fit of the remaining data points to the model, ultimately selecting the model with the best fit.

[0056] Initial Gaussian generation: Create initial 3D Gaussian functions centered on the matching 3D points. The initial parameters of these functions are set to reasonable default values ​​in preparation for subsequent optimization.

[0057] Step S3, 3D Gaussian optimization: The parameters of the initial Gaussian function are adjusted through an iterative optimization algorithm to more accurately represent the structures in the surgical scene. This step is critical because it directly affects the accuracy of the reconstructed scene and the quality of the rendering. As a specific example, in the algorithm for optimizing the parameters of the initial Gaussian function, the optimization objective function uses a hybrid loss function defined as:

[0058] L=(1-λ)L1+λL SSIM

[0059] Among them, L1 is L1 loss, L SSIM is the structural similarity (SSIM) loss, λ is the balancing factor;

[0060] Guided by prior information, for each 3D Gaussian function, its position, covariance matrix, opacity, and color are optimized. The Gaussian distribution of the interventional catheter area is constrained by using a cylindrical model. Gradient descent optimization is performed using the Adam optimizer, and the learning rate is gradually adjusted in combination with a learning rate decay strategy to ensure the stability and convergence of the optimization process.

[0061] Furthermore, the covariance matrix Σ is optimized by decomposing it into a rotation matrix R and a scaling matrix S, Σ = |RSS T R T |, where T represents the transpose of the matrix, the quaternion representing R and the scaling vector representing S are optimized; the opacity α is limited to the range [0, 1) by the sigmoid function; the color is optimized by gradually increasing the spherical harmonics (SH) coefficients from low to high order.

[0062] The density control of 3D Gaussian is performed alternately during the optimization process, including adding more Gaussian functions in complex areas, reducing the number of Gaussian functions in relatively smooth areas, or adjusting the distribution of Gaussian functions by splitting and merging.

[0063] Adding more Gaussian functions involves duplicating existing Gaussian functions and moving them along the gradient in the area with larger gradient; reducing the number of Gaussian functions involves removing Gaussian functions with opacity α less than the threshold ε. α Gaussian; Gaussian splitting is to split the larger Gaussian into two smaller Gaussians, and the position of the new Gaussian is obtained by sampling the PDF of the original Gaussian; regular resetting of the Gaussian is to set the α of all Gaussians to a value close to 0 every N iterations, and retain the necessary Gaussians through optimization.

[0064] Step S4: Using tile-based Gaussian sputtering technology, the 3D scene is divided into multiple small tiles, and each tile is rendered one by one to achieve real-time 3D scene reconstruction and display. This technology efficiently renders high-resolution images while supporting real-time updates, which is crucial for surgical environments that require instant feedback. As a specific example, the tile-based Gaussian sputtering technology includes the following steps:

[0065] Divide the image into multiple tiles;

[0066] For each tile, filter out the 3D Gaussians that intersect with it;

[0067] Use the GPU Radix sorting algorithm to sort Gaussian functions based on depth and tile ID. This algorithm is used to sort Gaussian functions in 3D scenes for efficient rendering. It does this by dividing the scene into multiple small tiles and then using GPU parallel computing within each tile to achieve the sorting.

[0068] Rendering using the alpha blending model:

[0069]

[0070] Among them, C is the final color of the pixel, n is the number of Gaussians that affect the pixel, T i is the cumulative transparency, α i is opacity, c i is the color, σ i is the density, δ i is the Gaussian thickness;

[0071] Backward propagation: traverse the Gaussian from back to front, and calculate the gradient using the opacity values ​​accumulated in the forward propagation.

[0072] The following are some specific examples based on the above reconstruction method, showing how to apply the method in the reconstruction of heart valve replacement scenarios.

[0073] Example 1: Standard Heart Valve Replacement Reconstruction

[0074] Data preparation: Preoperative CT scan images and intraoperative real-time three-dimensional ultrasound (RT3D-US) image sequences were collected.

[0075] Preprocessing: Anisotropic diffusion filtering, CLAHE enhancement and morphological operations are performed on RT3D-US images to improve image quality.

[0076] 3D Gaussian generation: Use the SIFT algorithm to extract key points, and use the RANSAC algorithm to match feature points from different time frames to generate a sparse 3D point cloud. Create an initial 3D Gaussian function with the matched 3D points as the center.

[0077] 3D Gaussian Optimization: Use the Adam optimizer, combined with L1 loss and SSIM loss, to optimize the parameters (position, covariance matrix, opacity, and color) of each 3D Gaussian. The Adam optimizer is a gradient-based optimization algorithm used to minimize the loss function. It accelerates convergence by dynamically adjusting the learning rate and momentum.

[0078] Adaptive density control: During the optimization process, Gaussian addition, splitting and deletion are performed alternately to control the distribution of Gaussian density.

[0079] Fast differentiable rendering: Uses a tile-based GPU Radix sorting algorithm to quickly render 3D scenes.

[0080] Accurately locating and tracking the position of the valve is crucial in valve replacement surgery. Traditional ultrasound images are noisy and low-quality, making them incapable of identifying and tracking the valve. This embodiment utilizes a deep learning algorithm based on reconstructed, real-time 3D ultrasound images to identify and track the valve within the ultrasound image. This positional information is then incorporated into the scene reconstruction process, aiding doctors in making clearer diagnoses and surgical treatments.

[0081] Example 2: Reconstruction combined with ECG signal gating

[0082] Data preparation: Preoperative CT scan images, intraoperative RT3D-US image sequences and electrocardiogram (ECG) signals were collected.

[0083] Preprocessing: Preprocess the RT3D-US image as in Example 1.

[0084] ECG signal gating: Based on the ECG signal, RT3D-US images of the same cardiac cycle phase are selected for reconstruction to reduce the influence of cardiac motion.

[0085] 3D Gaussian generation and optimization: Similar to Example 1, but using the gated image for 3D Gaussian generation and optimization.

[0086] Adaptive density control and fast differentiable rendering: similar to Example 1.

[0087] In addition to using preoperative CT or MRI images as prior information, this embodiment also introduces a more refined 3D valve model. This model can be a general valve model or customized based on the patient's preoperative imaging. We precisely register the prior valve model with the real-time ultrasound image to ensure that the model accurately guides scene reconstruction.

[0088] The motion patterns of the heart can provide important constraint information for scene reconstruction. This embodiment uses electrocardiogram (ECG) signals to obtain periodic information of heart motion, establish an accurate heart motion model, and adjust it according to the patient's actual situation. Combined with the prior model of heart motion, dynamic compensation is performed for scene reconstruction.

[0089] Example 3: Reconstruction combining prior information on the shape of an interventional catheter

[0090] Data preparation: Preoperative CT scan images, intraoperative RT3D-US image sequences, and interventional catheter shape information were collected.

[0091] Preprocessing: Preprocess the RT3D-US image as in Example 1.

[0092] Interventional Catheter Shape Constraint: During the 3D Gaussian optimization process, a cylindrical model is used to constrain the Gaussian distribution of the interventional catheter region, improving the reconstruction accuracy of the interventional catheter shape. Specifically, by fitting a cylindrical model to constrain the catheter's shape and position, the catheter's visualization in the reconstructed scene is improved.

[0093] 3D Gaussian generation and optimization: Similar to Example 1, but with the addition of interventional catheter shape constraints.

[0094] Adaptive density control and fast differentiable rendering: similar to Example 1.

[0095] Example 4: Multimodal Fusion Reconstruction

[0096] Data preparation: Preoperative CT scan images, intraoperative RT3D-US image series, and preoperative MRI images were collected.

[0097] Preprocessing: Preprocess the RT3D-US image as in Example 1.

[0098] Multimodal fusion: Preoperative CT and MRI images are used as prior information to guide the initialization and optimization of 3D Gaussian and improve the accuracy of the reconstructed scene.

[0099] 3D Gaussian generation and optimization: Similar to Example 1, but with the addition of a multimodal fusion step.

[0100] Adaptive density control and fast differentiable rendering: similar to Example 1.

[0101] The key technical indicators of the reconstructed scene obtained in the above embodiment include:

[0102] Spatial resolution: ≤0.5mm

[0103] Time resolution: ≥25fps

[0104] End-to-end latency: ≤100ms

[0105] Rendering quality (PSNR): ≥30dB

[0106] To meet the special needs of heart valve replacement, we have made the following improvements to the basic 3D Gaussian sputtering (3DGS) method:

[0107] Interventional Catheter Tracking: During the 3D Gaussian optimization process, we incorporate prior information about the catheter's shape. By constraining the Gaussian distribution of the catheter region using a cylindrical model, we can more accurately capture the catheter's shape and position. This step helps improve the visualization of the catheter in the reconstructed scene.

[0108] Valve region enhancement: To improve valve reconstruction accuracy, we assign a higher optimization weight to the Gaussian function in the valve region. This results in a greater adjustment of the Gaussian function in the valve region during optimization to better fit the valve structure. This step helps improve valve visualization in the reconstructed scene.

[0109] Multimodal Fusion: To further improve the accuracy of the reconstructed scene, we use preoperative computed tomography (CT) or magnetic resonance imaging (MRI) images as prior information to guide the initialization and optimization of the 3D Gaussian. By fusing information from multiple imaging modalities, we can more accurately reconstruct the heart valve replacement scene.

[0110] As a preferred technical solution, cardiac motion compensation can be performed by using electrocardiogram (ECG) signals to gate the real-time 3D ultrasound (RT3D-US) image sequence. By selecting images from the same cardiac cycle phase for reconstruction, the effects of cardiac motion on the reconstruction results can be reduced. This step helps improve the accuracy of the reconstructed scene.

[0111] If the above-mentioned cardiac interventional surgery scene reconstruction method based on 3D ultrasound imaging rendering is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Therefore, the essence of this technical solution or the part that contributes to the prior art or the part of this technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling an electronic device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0112] In summary, the present invention provides a method for reconstructing cardiac interventional surgery scenes based on 3D ultrasound imaging rendering, including collecting preoperative computed tomography or magnetic resonance imaging images as prior information; preprocessing real-time 3D ultrasound image sequences; voxelizing the images in combination with the prior information, converting the image data into voxel representations in three-dimensional space, and creating an initial set of 3D Gaussian functions; optimizing the parameters of the initial Gaussian functions through an iterative optimization algorithm to minimize the difference between the reconstructed image and the actual image; combining the shape constraints of the interventional catheter, alternatingly performing adaptive density control of the 3D Gaussian function; and using tile-based Gaussian sputtering technology to render tile by tile to achieve real-time 3D scene reconstruction and display. The present invention combines prior information and the shape constraints of the interventional catheter, uses 3D Gaussian functions to represent and simulate various structures in the surgical scene, and achieves efficient rendering through tile-based sputtering technology.

[0113] Those skilled in the art should understand that they can implement variations by combining the prior art with the above embodiments, which will not be described in detail here. Such variations do not affect the essence of the present invention and will not be described in detail here.

[0114] The above describes the preferred embodiments of the present invention. It should be understood that the present invention is not limited to the above-mentioned specific embodiments, and the devices and structures that are not described in detail should be understood to be implemented in a common manner in the art; any technician familiar with the art can use the above-mentioned disclosed methods and technical contents to make many possible changes and modifications to the technical solutions of the present invention without departing from the scope of the technical solutions of the present invention, or modify them into equivalent embodiments of equivalent changes, which does not affect the essential content of the present invention. Therefore, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention that do not depart from the content of the technical solutions of the present invention are still within the scope of protection of the technical solutions of the present invention.

[0115] Those skilled in the art will appreciate that the units, i.e., algorithm steps, of the various examples described in conjunction with this embodiment can be implemented using electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

Claims

1. A cardiac interventional surgery scene reconstruction method based on 3D ultrasound imaging rendering, characterized in that: The following steps are involved: Using real-time three-dimensional ultrasound imaging equipment to capture dynamic image sequences of the heart and its surrounding structures during cardiac interventional surgery; collecting preoperative computed tomography or magnetic resonance imaging images as prior information; Perform preprocessing on the acquired real-time 3D ultrasound image sequence, including noise reduction, contrast adjustment and image enhancement; Combined with prior information, the preprocessed image is voxelized, the image data is converted into a voxel representation in three-dimensional space, and an initial set of 3D Gaussian functions is created; Each voxel is simulated using a 3D Gaussian function to form a continuous 3D scene representation. The parameters of the initial Gaussian function are optimized through an iterative optimization algorithm to minimize the difference between the reconstructed image and the actual real-time 3D ultrasound image. During the optimization process, adaptive density control of the 3D Gaussian function is performed alternately in conjunction with the shape constraints of the interventional catheter. Using tile-based Gaussian sputtering technology, the three-dimensional scene is divided into multiple small tiles, and each tile is rendered one by one to achieve real-time three-dimensional scene reconstruction and display; The tile-based Gaussian sputtering technique includes the following steps: Divide the image into multiple tiles; For each tile, filter out the 3D Gaussians that intersect with it; Use the GPU Radix sorting algorithm to sort the Gaussians based on depth and tile ID; Rendering using the alpha blending model: ， Among them, C is the final color of the pixel, n is the number of Gaussians that affect the pixel, T i is the cumulative transparency, α i is opacity, c i is the color, σ i is the density, δ i is the Gaussian thickness; Backward propagation: traverse the Gaussian from back to front, and calculate the gradient using the opacity values ​​accumulated in the forward propagation.

2. The cardiac interventional surgery scene reconstruction method based on 3D ultrasound imaging rendering according to claim 1, characterized in that: The real-time 3D ultrasound image sequence is gated using the electrocardiogram (ECG) signal, and images in the same cardiac cycle phase are selected for reconstruction.

3. The cardiac interventional surgery scene reconstruction method based on 3D ultrasound imaging rendering according to claim 1 or 2, characterized in that: The pretreatment comprises the following steps: Anisotropic diffusion filtering: smoothes the image by solving partial differential equations while preserving the edge information of the image as much as possible; the diffusion coefficient c is designed to diffuse quickly in flat areas of the image and slow down near the edges; Contrast-limited adaptive histogram equalization: By locally adjusting the histogram of the image, the local contrast of the image is enhanced; Morphological operations: Opening (erosion followed by dilation) and closing (dilation followed by erosion) are used to remove small acoustic structures and fill holes in the image.

4. The cardiac interventional surgery scene reconstruction method based on 3D ultrasound imaging rendering according to claim 1 or 2, characterized in that: The initial set of 3D Gaussian functions is generated based on the preprocessed image and prior information, including the following steps: Feature point extraction: Use the 3D scale-invariant feature transform (SIFT) algorithm to detect key points in three-dimensional space; Feature matching: Using the Random Sampling Consensus (RANSAC) algorithm, feature points from different time frames are matched to obtain a sparse 3D point cloud. Initial Gaussian generation: Create initial 3D Gaussian functions centered at the matching 3D points. The initial parameters of these functions are set to reasonable default values ​​for subsequent optimization.

5. The cardiac interventional surgery scene reconstruction method based on 3D ultrasound imaging rendering according to claim 4, characterized in that: In the algorithm for optimizing the parameters of the initial Gaussian function, the optimization objective function uses a mixed loss function, which is defined as: ， in, is the L1 loss, is the structural similarity (SSIM) loss, λ is the balancing factor; Guided by prior information, for each 3D Gaussian function, its position, covariance matrix, opacity, and color are optimized. The Gaussian distribution of the interventional catheter area is constrained by using a cylindrical model. Gradient descent optimization is performed using the Adam optimizer, and the learning rate is gradually adjusted in combination with a learning rate decay strategy to ensure the stability and convergence of the optimization process.

6. The cardiac interventional surgery scene reconstruction method based on 3D ultrasound imaging rendering according to claim 5, characterized in that: The covariance matrix Σ is optimized by decomposing it into rotation and scaling matrices, decomposing Σ into rotation matrix R and scaling matrix S, Σ = Where T represents the transpose of the matrix, the quaternion representing R and the scaling vector representing S are optimized; the opacity α is limited to the range [0, 1) by the sigmoid function; the color is optimized by gradually increasing the spherical harmonics (SH) coefficients from low order to high order.

7. The cardiac interventional surgery scene reconstruction method based on 3D ultrasound imaging rendering according to claim 5, characterized in that: The density control of 3D Gaussian is performed alternately during the optimization process, including adding more Gaussian functions in complex areas, reducing the number of Gaussian functions in relatively smooth areas, or adjusting the distribution of Gaussian functions by splitting and merging.

8. The cardiac interventional surgery scene reconstruction method based on 3D ultrasound imaging rendering according to claim 7, characterized in that: Adding more Gaussian functions involves copying existing Gaussian functions and moving them along the gradient direction in areas with larger gradients; reducing the number of Gaussian functions involves removing Gaussian functions with opacity α less than a threshold. Gaussian; Gaussian splitting is to split the larger Gaussian into two smaller Gaussians, and the position of the new Gaussian is obtained by sampling the PDF of the original Gaussian; regular resetting of the Gaussian is to set the α of all Gaussians to a value close to 0 every N iterations, and retain the necessary Gaussians through optimization.

9. The cardiac interventional surgery scene reconstruction method based on 3D ultrasound imaging rendering according to claim 1 or 2, characterized in that: For heart valve replacement surgery, during the 3D Gaussian function optimization process, the optimization weight of the Gaussian function in the valve region is increased.

Images