Three-dimensional reconstruction method, system and device for vascular intervention operation area

Abstract

The application discloses a blood vessel intervention operation area three-dimensional reconstruction method, system and device, acquires an intravascular ultrasound image, an image processing system acquires an input frame; image preprocessing is carried out on the input frame; the interest point of the preprocessed image is extracted; the pose of the intravascular ultrasound sensor is calculated, and the space registration of the interest point is carried out; augmented reality 3D modeling is carried out based on the feature points; image rendering based on deep learning is carried out; subsequent input frames are added, the above process is repeated, and a real-time continuous image video stream is formed; and the real-time video stream is played through a display. Compared with the prior art, the application can provide three-dimensional visualization information of a blood vessel intervention catheter head operation area for a doctor, thereby assisting the doctor to better implement the blood vessel intervention operation. The length of the intravascular ultrasound is used to obtain the depth information of the image, so that the lack of spatial scale information of the two-dimensional image obtained from the intravascular ultrasound is compensated.

Description

Technical Field

[0001] This invention belongs to the field of three-dimensional reconstruction of vascular interventional surgical areas, specifically relating to a method, system, and device for three-dimensional reconstruction of vascular interventional surgical areas. Background Technology

[0002] Interventional vascular surgery refers to the diagnosis and treatment of vascular lesions by inserting instruments such as guidewires and catheters into the body through puncture of blood vessels. It is a type of minimally invasive procedure. It has advantages such as precision, short operation time, minimal surgical trauma, short postoperative recovery time, and less pain for patients, and is gradually becoming one of the main methods for treating cardiovascular diseases.

[0003] Intravenous ultrasound (IVUS) is a medical imaging technique that combines non-invasive ultrasound technology with invasive catheter technology, using a special catheter with an ultrasound probe attached to its end. IVUS uses ultrasound principles to detect the structure inside blood vessels, the vessel walls, and surrounding tissues, and can be used to guide interventional vascular treatments, making it a valuable tool for cardiovascular interventional surgeons.

[0004] Augmented Reality (AR) is a technology that can cleverly integrate virtual information with the real world. Based on the augmented reality endoscopic projection method, through three-dimensional reconstruction and spatial coordinate transformation, an endoscopic projection view is generated in a virtual preoperative three-dimensional model. The accurate structural information of the three-dimensional model is supplemented with the real details of the endoscopic image, providing doctors with more intuitive and realistic surgical information.

[0005] Most existing vascular interventional surgical robots require doctors to remotely view the angiography information displayed on a monitor to determine the position and status of the catheter, such as the Siemens CorPath GRX system.

[0006] Traditional medical imaging equipment (such as CT, X-ray, IVUS, etc.) obtains two-dimensional image sequences, which are sampled at intervals from one angle according to certain rules, resulting in incomplete information. During surgery, doctors usually need to compare a large number of images, consuming a lot of time and energy. Moreover, determining the position of the catheter in the images also relies on the doctor's spatial imagination of the two-dimensional images and their experience in interpreting images, which prolongs the operation time and limits the surgical outcome.

[0007] Therefore, providing doctors with visual information about the surgical area at the tip of the vascular interventional catheter to assist them in performing vascular interventional surgeries is an urgent problem to be solved and can also improve the effectiveness of vascular interventional surgery robots. Summary of the Invention

[0008] The technical problem to be solved by the present invention is to provide a three-dimensional reconstruction method for the vascular interventional surgical area, which solves the problem that the two-dimensional images are not intuitive for doctors to see during surgery in the prior art.

[0009] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0010] The three-dimensional reconstruction method for the vascular interventional surgical area includes the following steps:

[0011] Step 1: Acquire intravascular ultrasound images as input frames;

[0012] Step 2: Perform image preprocessing on the input frame to extract interest points;

[0013] Step 3: Estimate the pose of the intravascular ultrasound sensor and perform spatial registration on the points of interest to obtain the three-dimensional spatial coordinates of the points of interest;

[0014] Step 4: Perform 3D modeling based on the three-dimensional spatial coordinates, and render the modeled image;

[0015] Step 5: Continuously acquire input frames and execute steps 2 to 4 in sequence to form a real-time continuous image and video stream.

[0016] The specific process of image preprocessing is as follows:

[0017] First, the input frame is read; then, the input frame is converted to grayscale; next, median filtering is performed on the input frame to remove isolated noise points; finally, Frost filtering is applied to the image, adjusting the filter's effect on image processing by changing the corresponding parameter values.

[0018] The specific process of the median filtering operation is as follows:

[0019] Using a two-dimensional sliding template with a certain structure, the pixels within the template are sorted according to their pixel values ​​to generate a monotonically increasing or decreasing two-dimensional data sequence. The gray value of each pixel is set to the median of the gray values ​​of all pixels within a certain neighborhood window of that point.

[0020] The formula for the Frost filter is as follows:

[0021]

[0022] Where (x, y) are the coordinates of the pixel to be denoised, i and j represent the offset of (x, y) within a certain window size, and m(x+i, y+j) is the weighted value of the pixel value, calculated using the following formula:

[0023]

[0024] Where K1 is the parameter used for filtering; K is a normalization constant. This is the squared value of the coefficient of variation of the window in the image domain;

[0025] The method for determining the points of interest is as follows:

[0026] First, select any pixel from the image and set its grayscale value to l. p ;

[0027] Secondly, set a suitable threshold t;

[0028] Then, a discretized Bresenham circle is defined with the pixel as the center, and several pixels on the boundary of this circle are obtained;

[0029] Finally, select n consecutive pixels on the boundary of this circle and determine their pixel values. If the pixel values ​​of these consecutive pixels are all greater than l, then... p +t is larger, or both are greater than l p If -t is small, then the selected pixel is a point of interest.

[0030] The specific process for estimating the pose of the intravascular ultrasound sensor is as follows:

[0031] The IMU sensor on the tip of the interventional catheter is used to sense the position and orientation information of the catheter tip. The IMU sensor is placed behind the intravascular ultrasound sensor at the tip of the catheter, and the position and orientation of the intravascular ultrasound sensor is obtained by kinematic calculation of the corresponding information.

[0032] The method for determining the three-dimensional spatial coordinates of the point of interest in step 3 is as follows:

[0033] The space is divided into several individual elements, processed one by one, and the voxels intersecting with the isosurfaces are found. The intersection points with the isosurfaces are calculated using linear interpolation. The order of the isosurface patches is determined by the normal vectors. Finally, these patches are connected to complete the reconstruction of the 3D surface. The specific steps are as follows: several slices of the image are read into memory; some slices are scanned according to voxels; the index of the cube is calculated by comparing the vertex values ​​of the voxels with the obtained isosurface values; the edge list is looked up from the lookup table using the index; the exact position of the vertex of the triangular patch is calculated by linear interpolation using the gray value of each edge vertex; the unit normal of each voxel vertex is calculated and interpolated to each vertex of the triangular patch; the vertices and normals of the triangular patch are output.

[0034] To further address the issue of facilitating the observation of intravascular conditions during interventional vascular surgery, this invention also provides a three-dimensional reconstruction system and device for the interventional vascular surgery area, the specific technical solution of which is as follows:

[0035] The 3D reconstruction system for the vascular interventional surgical area includes a pose perception module, a kinematics calculation module, an intravascular ultrasound imaging module, an image preprocessing module, a 3D modeling and rendering module, and an external display device. The pose perception module senses the posture information of the intravascular ultrasound sensor and obtains the sensor's pose through kinematics calculation. The intravascular ultrasound imaging module and the image preprocessing module acquire 2D information from the image, which is then integrated in the 3D modeling and rendering module to achieve real-time reconstruction of the surgical area, and the result is output through the display device.

[0036] A vascular interventional surgical device includes an interventional surgical catheter, a catheter driving part, a physician operating device, and a three-dimensional reconstruction system; wherein, the interventional surgical catheter is equipped with an intravascular ultrasound imaging sensor to acquire intravascular ultrasound images; the three-dimensional reconstruction system processes the received intravascular ultrasound images using the method described in any one of claims 1 to 7 to acquire three-dimensional reconstructed images, and displays the corresponding three-dimensional reconstructed images through a display device.

[0037] The 3D reconstruction system runs on a host computer with storage media.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] 1. Provides real-time 3D modeling images of the catheter tip for vascular interventional surgeons / operators of vascular interventional surgical robots. Compared with traditional 2D images, 3D modeling images can effectively reduce the burden of image reading and spatial imagination for surgeons, effectively assist surgeons in decision-making and surgical operations, and effectively improve the accuracy and effectiveness of interventional catheter surgery.

[0040] 2. The FAST algorithm was used in the point of interest extraction. This algorithm is characterized by its high speed and efficiency, which can effectively improve the system's operating efficiency, shorten the image processing time, and reduce the spatial scale and delay between the actual position of the interventional catheter and the position displayed on the monitor.

[0041] 3. During spatial registration, the depth information of the image is obtained by using the length of the vascular interventional catheter advancement, which effectively makes up for the lack of spatial scale information in the two-dimensional images obtained by intravascular interventional ultrasound.

[0042] 4. The acquisition of visual information within blood vessels utilizes intravascular ultrasound technology, avoiding the radiation hazards to doctors and patients associated with traditional X-ray angiography. It also shortens the angiography imaging time, thereby reducing the operation time and improving the efficiency and effectiveness of the operation. Furthermore, it allows for a higher degree of integration of vascular interventional surgical instruments, which is beneficial for the lightweighting and miniaturization of surgical instruments.

[0043] 5. Augmented reality technology was used to reconstruct a three-dimensional image of the surgical site at the tip of the interventional catheter, improving interactivity with surgeons. When applied to the field of surgical robots, it can enhance human-computer interaction. Attached Figure Description

[0044] Figure 1 This is a schematic diagram of the system flow of the present invention.

[0045] Figure 2 This is a schematic diagram of the image preprocessing module of the present invention.

[0046] Figure 3 This is a schematic diagram of the three-dimensional reconstruction module of the present invention.

[0047] Figure 4 This is a schematic diagram of the system architecture of the present invention. Detailed Implementation

[0048] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present application.

[0049] The three-dimensional reconstruction method for the vascular interventional surgical area includes the following steps:

[0050] Step 1: Acquire intravascular ultrasound images as input frames;

[0051] Step 2: Perform image preprocessing on the input frame to extract interest points;

[0052] Step 3: Estimate the pose of the intravascular ultrasound sensor and perform spatial registration on the points of interest to obtain the three-dimensional spatial coordinates of the points of interest;

[0053] Step 4: Perform 3D modeling based on the three-dimensional spatial coordinates, and render the modeled image;

[0054] Step 5: Continuously acquire input frames and execute steps 2 to 4 in sequence to form a real-time continuous image and video stream.

[0055] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0056] Specific Implementation Example 1, such as Figure 1 As shown,

[0057] This embodiment details the system architecture and processing flow of the present invention; for specific flow details, please refer to... Figure 1 In vascular interventional surgery, the intravascular ultrasound sensor at the catheter tip acquires intravascular ultrasound images. The image processing system acquires the input frame and performs image processing. Converting the input image to grayscale simplifies the matrix, reduces data volume, and thus improves processing speed. Next, the processed image undergoes filtering. First, median filtering is performed to remove isolated noise points in the ultrasound image caused by interference from surrounding signals. Then, Frost filtering is performed to remove speckle noise caused by the coherence characteristics of the ultrasound imaging system. After filtering, the system uses the FAST (features from accelerated segment test) algorithm to detect points of interest. This algorithm detects the pixel values ​​around a candidate point of interest based on the grayscale values ​​of the surrounding image. If a candidate point has a sufficiently large difference in grayscale value from a candidate point within its neighborhood, then that point is considered a point of interest.

[0058] The specific steps are as follows:

[0059] The first step is to select a pixel P from the image. We will then determine if it is a point of interest. We first set its grayscale value to l. p ;

[0060] The second step is to set a suitable threshold t;

[0061] The third step is to consider a discretized Bresenham circle with a radius of 3 pixels centered on the pixel, and 16 pixels on the boundary of the circle.

[0062] Fourth step: If there are n consecutive pixels on this circle of size 16 pixels, their pixel values ​​are all greater than l. p +t is larger, or both are larger than l. p If -t is small, then it is a point of interest. Usually, n=9 yields the best results.

[0063] The pose of an intravascular ultrasound sensor is estimated using a motion model, specifically the PnP algorithm. Based on the camera model, the relationship between spatial point positions and pixel positions can be established. Considering n three-dimensional spatial points P and their projections p, the camera pose R,t is calculated. Its Lie group representation is T. Assume the coordinates P of a certain spatial point... i =[X i ,Y i Z i ] Τ Its projected pixel coordinates are L i =[u i ,vi ] Τ The following formula can be obtained:

[0064]

[0065] Thus, we achieved the three-dimensional coordinate transformation of the catheter tip, and spatial modeling can be performed next.

[0066] This embodiment uses the moving cubes method for spatial modeling. The basic idea of ​​the moving cubes (MC) algorithm is to divide the space into individual elements, process them one by one, find the volume elements that intersect with the isosurfaces, calculate the intersection points with the isosurfaces using linear interpolation, determine the order of the isosurface patches using normal vectors, and finally connect these patches to complete the reconstruction of the 3D surface. The specific steps are as follows:

[0067] Read four slices of the image into memory; scan two slices according to voxels; calculate the index of the cube by comparing the eight vertex values ​​of the voxels with the obtained isosurface values; use the index to look up the edge list from the lookup table; use the gray value of each edge vertex to calculate the exact position of the triangle facet vertex by linear interpolation; calculate the unit normal of each voxel vertex and interpolate the normal to each vertex of the triangle facet; output the vertices and normals of the triangle facet.

[0068] Next, a deep learning model is used to model the intravascular region surrounding the catheter tip. Suitable neural network models include 3D RU-net and NeRF. The dataset used is a pre-labeled dataset of vascular anatomy images. This allows for the creation of a realistic 3D interactive rendering of the vascular anatomy.

[0069] Subsequent input frames from the intravascular ultrasound sensor undergo the same image processing and modeling process, and are synthesized into a continuous video stream based on time sequence, which is then displayed to the medical staff performing the surgery.

[0070] Specific embodiment two, such as Figure 2 As shown,

[0071] This embodiment illustrates the preprocessing steps for intravascular ultrasound images.

[0072] First, the image processing system reads the input frame;

[0073] Secondly, grayscale conversion of the input frame can effectively reduce the amount of image data and improve the efficiency of computation.

[0074] Then, median filtering is performed on the input frame to remove isolated noise points. Median filtering is a non-linear smoothing technique that sets the gray value of each pixel to the median of the gray values ​​of all pixels within a certain neighborhood window.

[0075] The specific process of the median filtering operation is as follows:

[0076] Using a two-dimensional sliding template with a certain structure, the pixels within the template are sorted according to their pixel values ​​to generate a monotonically increasing or decreasing two-dimensional data sequence. The gray value of each pixel is set to the median of the gray values ​​of all pixels within a certain neighborhood window of that point.

[0077] The formula for the Frost filter is as follows:

[0078]

[0079] Where (x, y) are the coordinates of the pixel to be denoised, i and j represent the offset of (x, y) within a certain window size, and m(x+i, y+j) is the weighted value of the pixel value, which decreases as the distance increases. Therefore, the estimated value of a pixel in the image is a weighted average of all pixel values ​​within a certain window in the noisy image. The weight m(x+i, y+j) is calculated as follows:

[0080] m(x+i,y+j)=K2αexp[-α|t|)

[0081] in,

[0082] It is a constant, and This is called the coefficient of variation of the window in the image domain, and K2 is a normalization constant.

[0083] We can derive the filtering formula for the Frost filter:

[0084]

[0085] Where K1 is the parameter used for filtering; K is a normalization constant. This is the squared value of the coefficient of variation of the window in the image domain.

[0086] We can adjust the filter's effect on image processing by changing the value of the parameter K1 in the expression. The larger K1 is, the better the image smoothing effect, and vice versa, the more edge information is preserved.

[0087] Thus, we completed the preprocessing of intravascular ultrasound images.

[0088] Specific embodiment three, such as Figure 3 As shown.

[0089] This embodiment describes the 3D reconstruction module workflow. First, feature points are extracted from the preprocessed image using the FAST algorithm. Combined with the calculated catheter tip pose information, a 3D spatial model is created using the moving cubes method. Then, the established spatial model is imported into a pre-calibrated vascular anatomy image dataset, and deep learning algorithms are used for coloring and texture rendering. Suitable neural network models include 3D RU-net and NeRF. This achieves 3D reconstruction and rendering of the surgical area of ​​the catheter tip, and such reconstruction accurately reflects the actual state of the catheter within the body.

[0090] The specific process for estimating the pose of the intravascular ultrasound sensor is as follows:

[0091] The IMU sensor on the tip of the interventional catheter is used to sense the position and orientation information of the catheter tip. The IMU sensor is placed behind the intravascular ultrasound sensor at the tip of the catheter, and the position and orientation of the intravascular ultrasound sensor is obtained by kinematic calculation of the corresponding information.

[0092] The method for determining the three-dimensional spatial coordinates of a point of interest is as follows:

[0093] The space is divided into several individual elements, processed one by one, and the voxels intersecting with the isosurfaces are found. The intersection points with the isosurfaces are calculated using linear interpolation. The order of the isosurface patches is determined by the normal vectors. Finally, these patches are connected to complete the reconstruction of the 3D surface. The specific steps are as follows: several slices of the image are read into memory; some slices are scanned according to voxels; the index of the cube is calculated by comparing the vertex values ​​of the voxels with the obtained isosurface values; the edge list is looked up from the lookup table using the index; the exact position of the vertex of the triangular patch is calculated by linear interpolation using the gray value of each edge vertex; the unit normal of each voxel vertex is calculated and interpolated to each vertex of the triangular patch; the vertices and normals of the triangular patch are output.

[0094] To further address the issue of facilitating the observation of intravascular conditions during interventional vascular surgery, this invention also provides a three-dimensional reconstruction system and device for the interventional vascular surgery area, the specific technical solution of which is as follows:

[0095] The 3D reconstruction system for the vascular interventional surgical area includes a pose perception module, a kinematics calculation module, an intravascular ultrasound imaging module, an image preprocessing module, a 3D modeling and rendering module, and an external display device. The pose perception module senses the posture information of the intravascular ultrasound sensor and obtains the sensor's pose through kinematics calculation. The intravascular ultrasound imaging module and the image preprocessing module acquire 2D information from the image, which is then integrated in the 3D modeling and rendering module to achieve real-time reconstruction of the surgical area, and the result is output through the display device.

[0096] The vascular interventional surgical device includes an interventional surgical catheter, a catheter driving part, a physician operating device, and the three-dimensional reconstruction system; wherein, the interventional surgical catheter is equipped with an intravascular ultrasound imaging sensor to acquire intravascular ultrasound images; the three-dimensional reconstruction system processes the received intravascular ultrasound images using the method described above to acquire three-dimensional reconstructed images, and displays the corresponding three-dimensional reconstructed images through a display device.

[0097] The 3D reconstruction system runs on a host computer with storage media.

[0098] Specific embodiment four, such as Figure 4 As shown.

[0099] This embodiment illustrates the system architecture of the present invention. The entire system mainly consists of six modules: a pose perception module, a kinematics calculation module, an intravascular ultrasound imaging module, an image preprocessing module, a 3D modeling and rendering module, and an external display device. Specifically, the pose perception module senses the posture information of the intravascular ultrasound sensor and obtains the pose of the sensor through kinematics calculation. The intravascular ultrasound imaging module and the image preprocessing module acquire two-dimensional information from the image, which is then integrated in the 3D modeling and rendering module to achieve real-time reconstruction of the surgical area, and the result is output through the display device.

[0100] Both embodiments three and four involve running a program that includes this method, comprising the following steps:

[0101] S1. Acquire intravascular ultrasound images; the image processing system acquires the input frame.

[0102] S2. Perform image preprocessing on the input frame;

[0103] S3. Extract interest points from the preprocessed image;

[0104] S4. Calculate the pose of the intravascular ultrasound sensor and perform spatial registration of the points of interest.

[0105] S5. Augmented Reality 3D Modeling Based on Feature Points;

[0106] S6. Perform deep learning-based image rendering;

[0107] S7. Add subsequent input frames and repeat the above process to form a real-time continuous image and video stream;

[0108] S8. Play a live video stream via a monitor.

[0109] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0110] In the above embodiments, the image preprocessing includes converting to grayscale; using median filtering to remove isolated noise points in the ultrasound image; using Frost filtering to remove speckle noise; the point of interest extraction uses the FAST algorithm; the spatial registration uses a camera model, combined with the length of the vascular interventional catheter advancement to obtain the image's depth information, thereby providing accurate three-dimensional spatial coordinates for the points of interest; the 3D modeling uses the moving cubes method; the rendering uses a pre-calibrated vascular anatomy image dataset, and the deep learning model can be 3D RU-net, NeRF, etc.; the augmented reality processing includes the following method: using a PnP kinematic model to solve the pose of the intravascular ultrasound sensor at the catheter tip. In addition to the intravascular ultrasound sensor, the catheter tip also has a miniature six-axis inertial sensor installed to obtain acceleration information in the corresponding orientation, assisting in correcting the kinematic model solution. The corresponding kinematic information is used to realize the coordinate mapping from the two-dimensional image to three-dimensional space using the camera model.

[0111] Compared with existing technologies, this invention can provide doctors with three-dimensional visualization information of the surgical area at the tip of the vascular interventional catheter, thereby assisting doctors in performing vascular interventional surgeries more effectively. The depth information of the image is obtained by utilizing the length of catheter advancement, thus compensating for the lack of spatial scale information in two-dimensional images obtained from intravascular ultrasound.

[0112] It should be noted that the terms "comprising" and "having" and any variations thereof in the specification, claims and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such process, method, product or device.

[0113] The preferred embodiments of the present invention have been described in detail above. Those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

[0114] It should be understood that this solution is not limited to the specific embodiments described above. Devices and structures not described in detail herein should be understood as being implemented in a manner common to the art. Any person skilled in the art can make many possible variations and modifications to this solution, or modify it into equivalent embodiments, without departing from the scope of this solution, using the methods and techniques disclosed above. This does not affect the substantive content of this solution. Therefore, any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of this solution, without departing from its scope, still fall within the protection scope of this solution.

Claims

1. A three-dimensional reconstruction method for vascular interventional surgical areas, characterized in that: Includes the following steps: Step 1: Acquire intravascular ultrasound images as input frames; Step 2: Perform image preprocessing on the input frame to extract interest points; the method for determining the interest points is as follows: First, select any pixel from the image and set its grayscale value to [value missing]. ; Secondly, set an appropriate threshold. ; Then, a discretized Bresenham circle is defined with the pixel as the center, and several pixels on the boundary of this circle are obtained; Finally, select n consecutive pixels on the boundary of this circle and judge the pixel values ​​of these consecutive pixels. If the pixel values ​​of these consecutive pixels are all greater than 1, then the selection is considered complete. Bigger, or all of them are bigger If the pixel is small, then the selected pixel is a point of interest; Step 3: Estimate the pose of the intravascular ultrasound sensor and perform spatial registration on the points of interest to obtain the three-dimensional spatial coordinates of the points of interest; The space is divided into several individual elements, processed one by one, and the voxels that intersect with the isosurface are found. The intersection points with the isosurface are calculated using linear interpolation. The order of the isosurface patches is determined by the normal vector. Finally, these patches are connected to complete the reconstruction of the 3D surface. The specific steps are as follows: several slices of the image are read into memory; some slices are scanned according to voxels; the index of the cube is calculated by comparing the vertex values ​​of the voxels with the obtained isosurface values. Use an index to look up the edge list from the lookup table; use the grayscale value of each edge vertex to calculate the exact position of the triangle facet vertex using linear interpolation; calculate the unit normal of each volume vertex and interpolate the normal to each vertex of the triangle facet; output the vertices and normals of the triangle facet. Step 4: Perform 3D modeling based on the three-dimensional spatial coordinates, and render the modeled image; Step 5: Continuously acquire input frames and execute steps 2 to 4 in sequence to form a real-time continuous image and video stream.

2. The three-dimensional reconstruction method for the vascular interventional surgical area according to claim 1, characterized in that: The specific process of image preprocessing is as follows: First, the input frame is read; then, the input frame is converted to grayscale; next, median filtering is performed on the input frame to remove isolated noise points; finally, Frost filtering is applied to the image, adjusting the filter's effect on image processing by changing the corresponding parameter values.

3. The three-dimensional reconstruction method for the vascular interventional surgical area according to claim 2, characterized in that: The specific process of the median filtering operation is as follows: Using a two-dimensional sliding template, the pixels within the template are sorted according to their pixel values ​​to generate a monotonically increasing or decreasing two-dimensional data sequence. The gray value of each pixel is set to the median gray value of all pixels within a certain neighborhood window of that point.

4. The three-dimensional reconstruction method for the vascular interventional surgical area according to claim 3, characterized in that: The formula for the Frost filter is as follows: in, Here are the coordinates of the pixels that need to be denoised. and Indicates within a window of a certain size offset, The weighted value of the pixel values ​​is calculated using the following formula: in, These are the parameters used for filtering; It is a normalization constant; This is the squared value of the coefficient of variation of the window in the image domain; .

5. The method for three-dimensional reconstruction of the vascular interventional surgical area according to claim 1, characterized in that: The specific process for estimating the pose of the intravascular ultrasound sensor is as follows: The IMU sensor on the tip of the interventional catheter is used to sense the position and orientation information of the catheter tip. The IMU sensor is placed behind the intravascular ultrasound sensor at the tip of the catheter, and the position and orientation of the intravascular ultrasound sensor is obtained by kinematic calculation of the corresponding information.

6. A system for implementing the three-dimensional reconstruction method for the vascular interventional surgical area according to any one of claims 1 to 5, characterized in that: It includes a pose perception module, a kinematics calculation module, an intravascular ultrasound imaging module, an image preprocessing module, a 3D modeling and rendering module, and an external display device. The pose perception module senses the posture information of the intravascular ultrasound sensor and obtains the pose of the intravascular ultrasound sensor through kinematics calculation. The intravascular ultrasound imaging module and the image preprocessing module can acquire two-dimensional information from the image, integrate it in the 3D modeling and rendering module to realize the reconstruction of the surgical area, and output it through the display device.

7. A vascular interventional surgical device, characterized in that: The invention includes an interventional surgical catheter, a catheter drive, a physician operating device, and the three-dimensional reconstruction system as described in claim 6; wherein the interventional surgical catheter is equipped with an intravascular ultrasound imaging sensor to acquire intravascular ultrasound images; the three-dimensional reconstruction system processes the received intravascular ultrasound images using the method described in any one of claims 1 to 5 to acquire three-dimensional reconstructed images, and displays the corresponding three-dimensional reconstructed images through a display device.

8. The vascular interventional surgical device according to claim 7, characterized in that: The 3D reconstruction system runs on a host computer with storage media.

Images