Percutaneous puncture real-time path planning system with fusion of ultrasound and CT (Computed Tomography) images

Abstract

The invention relates to an ultrasonic and CT image fusion percutaneous puncture real-time path planning system, and relates to the technical field of medical image navigation, and the method comprises the steps: extracting skeleton multi-level features in a preoperative CT image and skeleton interface and acoustic features in an intraoperative real-time ultrasonic image, and carrying out the unmarked real-time registration and fusion through a multi-level matching engine; and calculating and displaying a real-time puncture path according to the path length, the obstacle avoidance distance and the tissue penetrability cost in the fused image space on the basis of a dynamic confidence evaluation result. According to the method, high-precision image fusion without in-vitro marking, adaptive guidance of dynamic change of tissues in an operation and path planning of fusion biomechanical safety consideration are realized.

Description

Technical Field

[0001] This invention relates to the field of medical image navigation technology, and more specifically, to a real-time percutaneous puncture path planning system that integrates ultrasound and CT images. Background Technology

[0002] In percutaneous puncture procedures, such as nerve blocks, biopsies, or ablation treatments, image guidance is crucial for improving surgical precision and safety. Currently, clinical practice mainly relies on single-modal imaging such as ultrasound, computed tomography (CT), or magnetic resonance imaging (MRI) for guidance. Ultrasound imaging is real-time, but its image resolution is limited, making it difficult to clearly display deep and complex anatomical structures; while CT or MRI can provide high-resolution static anatomical images, they cannot reflect the dynamic interaction between the puncture needle and tissue in real time. This forces the surgeon to perform multimodal information fusion and path planning in their mind, which is highly dependent on personal experience and spatial imagination, easily leading to inaccurate puncture path planning, increased number of operations, and a long learning curve.

[0003] To overcome the limitations of single-modality imaging, existing technologies attempt to fuse preoperative high-resolution images with intraoperative real-time images. For example, Chinese invention patent CN120918797A discloses a dynamic obstacle avoidance navigation method for nerve puncture based on multimodal image fusion. This method fuses preoperative static images with intraoperative ultrasound and uses artificial intelligence to segment key structures for 3D reconstruction and path planning. However, this method relies on positioning markers pasted on the patient's skin to achieve coordinate system unification during image fusion. This method is invasive, and the markers may shift or fall off due to patient movement, sweating, or disinfection, affecting registration accuracy and stability. Furthermore, while its path planning considers obstacle avoidance, it lacks a quantitative assessment of tissue permeability, an important biomechanical factor.

[0004] For example, Chinese invention patent CN110025379A discloses a real-time navigation system and method for fusing ultrasound and CT images. This system utilizes optical positioning sensors and tracers to spatially track CT equipment, surgical instruments, and ultrasound probes, achieving image fusion and instrument navigation. The shortcomings of this technology lie in its reliance on external optical or electromagnetic positioning systems, resulting in complex system construction, high costs, and positioning accuracy susceptible to interference from the operating room environment. Furthermore, its image registration and navigation logic are relatively fixed, lacking adaptive optimization capabilities based on surgical process data.

[0005] In summary, existing technologies suffer from several problems, including reliance on in vitro markers or external positioning devices, insufficient adaptability to intraoperative tissue dynamic deformation, lack of self-learning and optimization capabilities, and insufficient consideration of tissue biomechanical characteristics in path planning. Summary of the Invention

[0006] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a real-time percutaneous puncture path planning system that integrates ultrasound and CT images, thereby addressing the system complexity and stability issues caused by reliance on external markers or positioning devices in the prior art.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a real-time percutaneous puncture path planning system based on ultrasound and CT image fusion, comprising: Image acquisition module: used to acquire the patient's preoperative CT three-dimensional volume images and intraoperative real-time two-dimensional ultrasound image sequences, providing raw multimodal data sources for subsequent processing; Feature extraction module: connected to the image acquisition module, used to extract the macroscopic three-dimensional geometric model and mesoscopic attribute feature map of the skeleton from the CT three-dimensional volume image, and extract the skeleton interface contour features and acoustic attribute features from the real-time two-dimensional ultrasound image sequence, so as to transform the image information of different modalities into a unified, computable and matchable feature expression. Registration and fusion module: connected to the feature extraction module, which has a built-in multi-level matching engine; the multi-level matching engine performs first-level matching and second-level matching; The first-level matching performs fast initial registration based on geometric contours between the skeletal interface contour features and the macroscopic three-dimensional geometric model, outputs the first real-time pose, and establishes a rough spatial correspondence. The second-level matching performs fine-grained matching and optimization of the acoustic attribute features and the mesoscopic attribute feature map based on physical attributes, according to the first real-time pose, and outputs a second real-time pose with higher accuracy, and corrects the registration error caused by tissue deformation. The registration and fusion module, based on the second real-time pose, dynamically fuses the real-time two-dimensional ultrasound image sequence with the CT three-dimensional volume image in space to generate a composite navigation view that contains both high-resolution static anatomical details and real-time dynamic information. The path planning module, connected to the registration and fusion module, is used to calculate the optimal puncture path from a specified needle insertion point on the real-time ultrasound image plane to a preset target point in the CT three-dimensional space based on the second real-time pose in the fused image space, and to overlay the puncture path on the real-time two-dimensional ultrasound image sequence in real time, thereby providing the surgeon with intuitive and spatially accurate needle insertion trajectory guidance.

[0008] Furthermore, the mesoscopic attribute feature map is an acoustic impedance distribution map or elastic modulus distribution map generated by inverting the gray value distribution of the CT three-dimensional volume image based on a specific physical model. This process establishes a mapping relationship between the density information of the CT image and the acoustic or mechanical properties of the tissue, providing a compatible data basis for subsequent quantitative comparison with ultrasound acoustic features.

[0009] Furthermore, the second-level matching employs a feature fusion method based on an attention mechanism. This method dynamically assigns a weight coefficient to each feature element in the acoustic attribute feature and the mesoscopic attribute feature map. The weight coefficient is automatically calculated based on the spatial proximity of the feature element, the importance of the feature dimension, and the current matching confidence. The final matching result is determined by the overall similarity of all feature elements after weighted fusion according to their weights. This mechanism enables the matching process to adaptively focus on more discriminative feature regions and suppress interference from noise or regions with high uncertainty, thereby enhancing the stability of the matching and its tolerance to local feature changes.

[0010] Furthermore, it also includes a confidence assessment module, which is connected to the registration and fusion module; the input of the confidence assessment module is the geometric residual of the first-level matching, the similarity score of the second-level matching, and the time series stability measure of the second real-time pose across multiple consecutive frames; The confidence assessment module integrates the aforementioned multi-source evidence and outputs a comprehensive confidence score. This score is used to objectively quantify the overall reliability of the current frame image registration result, serving as a key indicator for the system to judge the availability of navigation information and make subsequent decisions.

[0011] Furthermore, the path planning module controls the display mode and content of the puncture path guidance information in a tiered manner based on the overall confidence score: When the overall confidence score is greater than or equal to the first threshold, the registration is determined to be reliable, and the system displays the calculated optimal puncture path in its entirety. When the overall confidence score is less than the first threshold but greater than or equal to the second threshold, it is determined that the registration accuracy is limited. While displaying the path, the system simultaneously outputs an accuracy warning sign in a prominent visual or auditory form. When the overall confidence score is less than the second threshold, the registration is determined to be unreliable. The system actively stops displaying path guidance and triggers a clear relocation operation prompt to prevent providing misleading navigation under low-quality registration.

[0012] Furthermore, it also includes a strategy adjustment module, which connects the confidence assessment module and the registration fusion module; The strategy adjustment module continuously analyzes the short-term and long-term trends of the comprehensive confidence score, and dynamically adjusts the key operating parameters of the second-level matching in the multi-level matching engine based on these trends, including the iteration termination condition, convergence threshold, and whether deformation compensation is enabled. This feedback adjustment mechanism enables the matching algorithm to self-optimize based on the actual intraoperative image quality and tissue motion state, thereby achieving an adaptive balance between computational resources, real-time performance, and registration accuracy.

[0013] Furthermore, the operation of the path planning module in calculating the puncture path includes: S1, Receive the coordinates of the two-dimensional needle insertion point interactively defined by the surgeon on the current frame of the real-time two-dimensional ultrasound image sequence; S2, based on the coordinate transformation relationship defined by the second real-time pose, accurately map the coordinates of the two-dimensional needle insertion point to the global coordinate system of the CT three-dimensional volumetric image to obtain the corresponding three-dimensional space needle insertion point coordinates, and complete the coordinate dimensionality transformation from the two-dimensional image plane to the three-dimensional anatomical space. S3, In the CT three-dimensional space, calculate the straight line segment connecting the coordinates of the three-dimensional needle insertion point and the coordinates of the preset target point, as the geometric basis of the theoretical puncture path; S4. The three-dimensional spatial straight path is projected in reverse onto the imaging plane of the real-time two-dimensional ultrasound image sequence according to the current imaging geometry, generating a corresponding two-dimensional guide line. This guide line is then superimposed and displayed in the ultrasound video stream in real time, so as to intuitively present the three-dimensional spatial planning results in the two-dimensional operation field of view.

[0014] Furthermore, the path planning module also integrates a real-time obstacle avoidance detection function: based on a three-dimensional spatial model of one or more key danger zones segmented and defined in the CT three-dimensional volumetric image before surgery; During path calculation or simulated needle insertion, the shortest spatial distance between the theoretical puncture path or real-time needle tip position and the surface of the three-dimensional model of each danger zone is calculated in real time. When any of the shortest distances is detected to be less than the preset safe distance threshold, the system immediately triggers visual warnings such as brightening and flashing in the corresponding risk section of the superimposed two-dimensional guide line and / or within the projection range of the dangerous area in the real-time ultrasonic image, and may be accompanied by graded sound alarms, thereby achieving proactive and real-time early warning of potential collision risks.

[0015] Furthermore, when the feature extraction module extracts the acoustic attribute features from the real-time two-dimensional ultrasound image sequence, it prioritizes processing the raw radio frequency signal acquired by the ultrasound equipment. Within the tissue depth range determined by the bone interface contour features, the radio frequency signal is analyzed, its average energy is calculated to reflect the interface reflection intensity, its frequency spectrum center is calculated to reflect the tissue's response characteristics to the sound wave frequency, and its attenuation coefficient is calculated to reflect the sound wave propagation loss in the tissue. These acoustic physical parameters, directly derived from the raw signals, are more stable in characterizing tissue properties than the grayscale values ​​of conventional B-ultrasound images, creating conditions for robust matching with acoustic property features derived from CT.

[0016] The technical effects and advantages of this invention are as follows: First, regarding markerless real-time localization and image fusion, this invention constructs a multi-level matching engine by extracting the macroscopic geometric model and mesoscopic acoustic properties of the skeleton from CT images and simultaneously extracting the corresponding skeleton interface contours and acoustic properties from real-time ultrasound images. This engine first performs initial registration based on the geometric contours, then performs matching using acoustic properties, and can be compensated for by incorporating estimated local tissue deformation fields. This process utilizes the patient's own skeleton as a stable anatomical reference, eliminating reliance on any external markers or external optical / electromagnetic tracking devices. This simplifies the system composition, reduces the risk of registration failure due to marker displacement or external signal interference, and helps improve the system's clinical applicability and robustness.

[0017] Regarding dynamic confidence assessment and system adaptation: During the registration process, this invention integrates multi-source information such as matching quality, geometric consistency, temporal smoothness, and image quality to calculate a comprehensive confidence score in real time. This score not only provides the surgeon with intuitive navigation status prompts (such as normal guidance, accuracy warning, or paused guidance) but also serves as a key input to the strategy adjustment module. Based on the historical trend of the confidence score, the strategy adjustment module dynamically adjusts parameters in the matching phase (such as convergence threshold) and whether to enable deformation compensation. This adaptive adjustment mechanism based on real-time performance feedback enables the system to better adapt to differences in patient anatomy, image quality fluctuations, and intraoperative tissue dynamics, enhancing the system's environmental adaptability and stability.

[0018] Regarding safe path planning that incorporates biomechanical properties, the path planning module of this invention, when calculating the optimal puncture path, considers not only the geometric path length and distance to hazardous structures, but also introduces a tissue permeability cost assessment based on CT Henle unit values. Through a preset cost function, the puncture difficulty of tissues of different densities (such as fat, muscle, and bone) traversed by the path is quantified, and this quantified cost, along with the geometric obstacle avoidance cost, is incorporated into the total path cost for optimization. This method ensures that the final planned path, while maintaining geometric safety, further favors channels through low-resistance tissues, providing surgeons with path suggestions that integrate anatomical safety and operational feasibility. This helps improve the prediction of operational safety during preoperative planning and intraoperative guidance. Attached Figure Description

[0019] Figure 1 This is the main flowchart of the label-free real-time fusion and guidance system of the present invention; Figure 2 This is a flowchart of the two-level matching engine of the present invention; Figure 3 This is a flowchart of the safe optimal path planning and real-time obstacle avoidance process of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Example 1 As attached Figures 1 to 3 The implementation details of the real-time percutaneous puncture path planning system, which integrates ultrasound and CT images, are as follows: The system provided in this embodiment can be physically implemented by a medical workstation integrating a high-performance graphics processor and a central processing unit, which connects to a CT scanner and an ultrasound diagnostic instrument via a data interface. The system's logical architecture includes four sequentially connected modules: an image acquisition module, a feature extraction module, a registration and fusion module, and a path planning module, with each module executing sequentially according to the data flow.

[0022] During system operation, the image acquisition module first acquires the patient's preoperative CT 3D volumetric image and intraoperative real-time ultrasound image sequence. Then, the feature extraction module processes these two types of images in parallel, extracting the geometric model and acoustic property feature map of the skeleton from the CT image, extracting real-time skeleton interface contour features and acoustic property features from the ultrasound image, and evaluating the quality of the ultrasound image.

[0023] Then, the registration and fusion module uses the extracted features to calculate the pose of the ultrasound image plane relative to the CT coordinate system through a matching engine with two levels, spatially fuses the ultrasound image and CT image, and evaluates the confidence of this registration.

[0024] Finally, within the fused image space, the path planning module calculates and dynamically displays a puncture path from the needle insertion point on the ultrasound image to the target point in the CT space, taking into account the predefined obstacle avoidance area and real-time pose. The display status of this path is controlled by the registration confidence.

[0025] I. Specific implementation of the image acquisition module: This module is responsible for acquiring raw data from external imaging equipment to provide input for subsequent processing.

[0026] 1. Preoperative CT three-dimensional volumetric image (denoted as...) Acquisition of ) Before surgery, the patient undergoes a CT scan targeting the surgical area (such as the lumbar spine, chest, or abdomen). The scan parameters are typically set as follows: tube voltage 120kV, tube current 250mAs, slice thickness 0.625mm or 1mm, and reconstruction matrix 512×512.

[0027] The system acquires complete CT data directly from the hospital's image archiving and communication system or CT scanner through medical digital imaging and communication protocols. . It is a three-dimensional array, each element of which Indicates spatial location The X-ray attenuation coefficient at a given location is expressed in Heinz units. The typical spatial resolution is 0.5mm×0.5mm×1mm.

[0028] 2. Intraoperative real-time two-dimensional ultrasound imaging sequence (denoted as...) Acquisition of ) During the procedure, the ultrasound probe is placed on the target area on the patient's body surface. The system is compatible with linear or convex array probes, with a center frequency range of 5-12MHz to accommodate imaging needs at different depths.

[0029] The system captures the B-mode image sequence output by the ultrasound equipment in real time at a rate of 30 frames per second via a video capture card or the digital video interface provided by the ultrasound equipment. ,in For frame number ( Each frame. It is a two-dimensional grayscale image matrix, typically with a resolution of 640×480 pixels or 800×600 pixels. Optionally, the system can also synchronously receive the raw radio frequency signal data stream from ultrasound. For more precise acoustic feature analysis.

[0030] II. Specific implementation of the feature extraction module: This module obtains... and The process involves extracting various features for registration and evaluating the quality of ultrasound images. The process is divided into two parts: CT feature extraction and ultrasound feature extraction.

[0031] 1. CT Feature Extraction: Macroscopic three-dimensional geometric model (denoted as Extraction of ) Input: CT volumetric images .

[0032] Process: To Threshold segmentation is applied. Since bones appear as high Henlein unit values ​​in CT images, a global bone threshold is set. In this embodiment, HU. (will) All of the above satisfy The voxels are labeled as candidate bone points. Then, a 3D connected component analysis algorithm is used to extract the maximum connected region of the target area (such as a single vertebra or rib segment) from the candidate points. Finally, the moving cube algorithm is applied to this region to generate a surface mesh model composed of triangular facets. The model Used for subsequent first-level matching.

[0033] Mesoscopic acoustic property characteristic diagram (denoted as) Generation of ) Input: Images of the same CT volume .

[0034] Process: To match acoustic properties with ultrasound features, a feature map reflecting the acoustic characteristics of the tissue needs to be generated from CT images. This embodiment constructs a three-dimensional feature map. Its size and The same, but each voxel stores a three-dimensional feature vector. The three features are designed as follows: 1. :by The average value of all voxel Heinz units in a local cubic region centered at a point with sides of 3 voxels reflects the local tissue density.

[0035] 2. The standard deviation of the Heinz unit values ​​in the aforementioned local area reflects the local tissue homogeneity.

[0036] 3. : Through a preset conversion function The calculated estimated acoustic impedance value. (Function) Based on the empirical relationship between the Heinz unit value of CT and acoustic impedance, it is established as a piecewise linear function: like ,but ; like ,but ; like ,but .

[0037] The unit is MRayl.

[0038] The three-dimensional feature map It will be compared with ultrasonic acoustic features in the second level of matching.

[0039] 2. Ultrasonic Feature Extraction and Image Quality Assessment: Skeletal interface contour features (denoted as Extraction and quality assessment of ) Input: Current frame ultrasound image (or alternatively, ).

[0040] process: 1. Edge detection: First, for... Applying a Gaussian filter (standard deviation) The pixels are smoothed to suppress noise, and then an initial edge map is obtained using the Canny edge detector (low threshold = 30, high threshold = 100). .

[0041] 2. Contour Filtering: From Extract all continuous edge chains. Based on the characteristic that skeletal interfaces typically exhibit hyperechoic, continuous, and relatively long edges in ultrasound images, calculate the average gradient amplitude of each edge chain. and length Retain satisfaction (This embodiment) )and (This embodiment) The longest edge chain (pixels) is used as the candidate skeletal contour. .

[0042] 3. Quality Score To evaluate the quality of candidate contours, two metrics are calculated: sharpness and clarity. (defined as) (Average gradient magnitude of all pixels) and continuity (defined as) The total length divided by the straight-line distance between its first and last pixels; the closer this ratio is to 1, the better the continuity. Normalizing to the [0, 1] interval yields Final quality score Calculated by the following formula: ; in and It is the weighting coefficient (in this embodiment, we take...) This is used to balance the importance of clarity and continuity.

[0043] Output: If Greater than or equal to the preset quality threshold (This embodiment) ), then As a formal skeletal interface contour feature Output the data and continue with acoustic attribute feature extraction. Otherwise, it is determined to be a low-quality frame, further feature extraction for the current frame is paused, and the system will attempt to use features from historical qualified frames or wait for the next frame.

[0044] Acoustic properties (denoted as) Extraction of ) Input: High-quality current frame ultrasound data (preferably the original radio frequency signal) If it cannot be used, then use (replacement) and the corresponding outline .

[0045] Process: along The indicated depth line, in (or Take a width of (the corresponding pixel row) (This embodiment) A signal window (with 1 sampling point) is defined. The three acoustic parameters of the signal within this window are calculated: 1. Average Energy: This reflects the signal strength.

[0046] 2. Center frequency: ,in yes The power spectrum reflects the main frequency components of the signal.

[0047] 3. Bandwidth attenuation slope: Through the The signal is obtained by linear fitting within the effective frequency band near the center frequency of the ultrasound probe (for example, if the center frequency of the probe is 7.5 MHz, then the 5-10 MHz band is analyzed), reflecting the signal attenuation characteristics with frequency.

[0048] Output: Combine these three parameters into a feature vector. This refers to the acoustic properties of the current frame.

[0049] Local tissue deformation field (denoted as) Estimate: Input: Multiple consecutive high-quality ultrasound images. When the system skips some frames due to poor image quality, it uses the two most recent valid ultrasound images that are adjacent in time and both passed the quality assessment. and and their outlines and .

[0050] Procedure: To estimate soft tissue deformation caused by breathing or probe pressure, this embodiment calculates the deformation field based on two consecutive frames. A strip-shaped region of interest is defined around the perimeter. The Farneback dense optical flow method is used to calculate the optical flow within this region. arrive A dense displacement vector field (unit: pixels) is generated. This vector field is then subjected to median filtering (window size 3×3 pixels) to remove noise, resulting in a smooth two-dimensional displacement field.

[0051] Finally, the ultrasound images are converted into a two-dimensional displacement field with physical scale (unit: mm) using spatial calibration parameters (the number of millimeters per pixel, which is automatically provided by the ultrasound equipment based on the current probe model and imaging depth, or obtained by the system after calibration using a standard phantom before surgery). , It describes the deformation of local soft tissue.

[0052] III. Specific Implementation of the Registration and Fusion Module: This module performs image registration, fusion, and confidence assessment, and its core is a multi-level matching engine.

[0053] 1. Multi-level matching engine: First-level matching (initial registration): Input: Ultrasonic bone contour (Two-dimensional pixel set), CT skeletal geometry model (Set of vertices of a 3D triangular mesh).

[0054] Process: This level uses an iterative nearest-point algorithm based on projection for initial registration. First, the 3D model... The set of vertices is used as the target point cloud. The two-dimensional contour is... The pixel coordinates are obtained through an assumed average depth. (This embodiment takes) The source point cloud is a 3D point cloud back-projected from the nominal focal length of the ultrasound probe (mm, set according to the standard focal length). The goal of the algorithm is to find a pose of the ultrasound imaging plane in CT space. (in It is a 3x3 rotation matrix. (is a 3x1 translation vector), such that... After being projected onto this plane, its projected profile is the same as... The best match. Specifically, the following steps are performed iteratively until convergence (convergence condition: mean square error change is less than 0.1 pixels or the number of iterations reaches 50): 1. Projection: Using the current... target point cloud Projected onto a two-dimensional plane.

[0055] 2. Corresponding point lookup: For each point in the 3D source point cloud, find the corresponding point in the 3D target point cloud (model). Find the point with the closest Euclidean distance in the vertex set and establish the corresponding point pair.

[0056] 3. Transformation Solution: Using singular value decomposition, find a rigid body transformation that minimizes the sum of squared distances from the source point cloud to its corresponding points, and update the solution. .

[0057] Output: Optimized pose That is, the first real-time pose and the final residual of this matching. (Mean square distance between matching point pairs, unit: pixels) 2 ).

[0058] Second-level matching (refined matching): Input: Ultrasonic acoustic features CT acoustic property feature map First real-time pose and (optionally) local deformation field .

[0059] Process: This level is in Based on the provided initial alignment, fine-tuning is performed using acoustic properties to compensate for possible local deformations and improve accuracy.

[0060] 1. Feature Sampling and Deformation Compensation: Assuming the extraction of ultrasonic features... At that time, the corresponding two-dimensional pixel coordinate set in the ultrasound image is .

[0061] Based on the first real-time pose and ultrasound imaging geometric model, Back-projecting to CT 3D space yields the corresponding set of 3D spatial locations. .exist Mid-sampling position The three-dimensional feature vectors at the location are used to obtain a set of CT feature vectors. .

[0062] Before generating instructions, the strategy adjustment module checks the output of the feature extraction module. Is it valid (not empty and the quality flag is true)? If the instruction requires "Enable deformation compensation" and If it works, use it first. For sampling location Perform physical offset compensation to obtain X', then from The features at position X' are sampled to obtain the compensated features. If the instruction requires but If invalid, the second-level matching engine will automatically downgrade to the "disable deformation compensation" mode and use it directly. .

[0063] 2. Feature fusion based on attention mechanism: computation and (or The corresponding feature vector in each dimension ( The absolute differences (corresponding to energy, frequency, and slope respectively) .

[0064] Define attention weights ,in It is a preset scale parameter (in this embodiment, it is taken as...). This weight reflects the importance of each feature dimension in the matching process; the smaller the difference, the greater the weight.

[0065] 3. Similarity Calculation and Optimization: Calculate the weighted similarity score. : ; in It is the first The preset maximum range of feature differences (in this embodiment) ), used for normalization.

[0066] Through fine-tuning To maximize The fine-tuning is performed using the Gauss-Newton iteration method, with the position search range set to ±5mm, the angle search range to ±3°, and the iteration step size to 0.1mm and 0.1°, respectively.

[0067] The convergence condition for the iteration is the weighted similarity score. The change is less than the threshold The threshold Dynamically set by the strategy adjustment module (default) ).

[0068] If the Gauss-Newton iteration method fails to converge within the preset search range (i.e.) If the second-level matching fails to improve effectively, or if the pose change after convergence exceeds the safety threshold, the second-level matching is considered a failure, and the system will maintain [the following condition]. As the current output pose, and the overall confidence score will be used. Forced to be lower than The value triggers the "Relocate" prompt.

[0069] Output: Optimized pose This refers to the second real-time pose, and it also outputs the similarity score for this match. .

[0070] 2. Confidence Assessment Module: Input: The final residual of the first-level matching Similarity score of second-level matching Current frame image quality score And recently Frame (in this embodiment, the frame is taken) )of sequence.

[0071] Process: This module integrates information from multiple sources to assess the reliability of this registration.

[0072] 1. Calculate the fractions of each item: All fractions are dimensionless scalars after their respective normalization processes, with a range of [0, 1].

[0073] Match Quality Score: (Use similarity scores directly).

[0074] Geometric consistency score: ,in It is the preset maximum acceptable residual (in this embodiment) Pixels 2 The smaller the residual, the higher the score for this item.

[0075] Time smoothness score: Calculate the most recent frame Standard deviation of translation and rotation components and .

[0076] ,in and It is a preset scale (in this embodiment) mm, (°), the smoother the pose change, the higher this score.

[0077] Image quality score: .

[0078] 2. Overall confidence score Calculation: ; in These are the weighting coefficients, and This embodiment takes It emphasizes the importance of matching quality and geometric consistency. The range of values ​​is [0, 1], and the higher the value, the higher the confidence level.

[0079] in, , , , , These parameters are system preset values ​​obtained through offline analysis and optimization of a large amount of historical surgical data. In practical applications, they can be fine-tuned according to different models of imaging equipment.

[0080] Output: Overall confidence score .

[0081] 3. Strategy Adjustment Module: Input: Real-time updated overall confidence score And its historical sequence.

[0082] Process: This module is based on Based on historical trends, the strategy parameters of the second-level matching are dynamically adjusted to balance accuracy and robustness.

[0083] Maintain a length of (This embodiment) )of Value buffer, calculation buffer mean The last 5 The slope of the value's decline ,as well as variance of values .

[0084] Decision logic and instruction output: like (This embodiment) )or (This embodiment) If the system is unstable, a command is sent to the second-level matching engine to relax the convergence threshold (i.e.,...). Change the value from the default 0.01 to 0.05 and force deformation compensation to be enabled.

[0085] like (This embodiment) )and (This embodiment) If the state is stable, then send the instruction to tighten the convergence threshold (i.e.,...). Set it to 0.005), and skip deformation compensation.

[0086] like In Between but (This embodiment) If the confidence level fluctuates drastically, then send the instruction: enable deformation compensation and appropriately relax the convergence threshold (i.e.,...) Set to 0.03).

[0087] After receiving the above instructions, the second-level matching engine dynamically adjusts its internal parameters. The status of the deformation compensation function being enabled.

[0088] 4. Image fusion: In obtaining Then, the system utilizes Defined spatial transformation relationships, real-time transformation of CT 3D data The corresponding cross-sections (such as multi-planar reconstructed views) are rendered as semi-transparent images and precisely overlaid onto real-time ultrasound images. The display integrates both CT anatomical information and real-time ultrasound dynamic information, providing the surgeon with a view that simultaneously includes high-resolution CT anatomical information and real-time ultrasound dynamic information.

[0089] IV. Specific Implementation of the Path Planning Module: This module performs real-time, safe puncture path planning based on fused images and intuitively guides the surgeon through the planning results. The system uses a "status retention timer." When historical qualified frame results are used continuously for more than [a certain period]... If no new qualified frame is obtained within 5 seconds (e.g., 5 seconds), the system will automatically adjust the overall confidence level. Reset to 0 and force clear historical pose data, prompting that a new calibration is required to avoid using invalid historical data after significant patient movement.

[0090] 1. User interaction and coordinate mapping: The surgeon selects a "designated needle entry point" on the skin by clicking on the ultrasound image portion of the real-time fused image using a mouse or touchscreen. The coordinates of this point are: (pixel coordinates).

[0091] The system is based on the second real-time pose. Through back projection calculation, Mapping to the CT 3D space yields the corresponding coordinates of the "3D needle insertion point". (Unit: mm)

[0092] "Preset target" By the doctor in the preoperative CT 3D image Pre-select and save interactively.

[0093] 2. Obstacle avoidance zone definition: During the preoperative preparation phase, doctors or systems rely on CT images. Using threshold segmentation and region growing algorithms, key structures that need to be avoided, such as large blood vessels, nerve roots, and pleura, are automatically or semi-automatically segmented. Each structure is represented as a three-dimensional surface model. The collection of these models serves as the three-dimensional model of the obstacle avoidance area as defined in this embodiment.

[0094] 3. Cost assessment of organizational permeability: In CT images In puncture, tissue density directly affects the difficulty and risk of needle penetration. A tissue permeability cost function is defined. ,in Values ​​are in Henle units. Thresholds are set based on typical CT value ranges for human soft tissue and bone. HU (distinguishing between easily penetrable soft tissue) and HU (bone threshold) can be fine-tuned by doctors through the system interface based on histogram analysis of individual patient CT images. The function definition is as follows: ,if (This represents substances like fat and liquid that can easily penetrate tissues at the lowest cost.) ,if (Represents muscles, fascia, and other tissues with moderate resistance; moderate cost) ,if (Representing the skeleton, considered an impenetrable barrier, at an extremely high cost) This function assesses the penetration cost based on a preset range of Henlein unit values ​​and quantifies it into a calculable value.

[0095] 4. Optimal path calculation: Input: 3D needle entry point Preset target CT images Obstacle avoidance model Cost function .

[0096] process: S1: Receive And calculated .

[0097] S2: In and Define a three-dimensional cylindrical "path search space" between points, with its axis being the line connecting two points and its radius being... The value is set according to the uncertainty of the target area, usually 10%-20% of the distance between two points, and the default value in this embodiment is 15mm.

[0098] S3: Discretizes the search space into a 3D mesh graph with a default mesh resolution of 1mm to balance computational accuracy and speed. Each node in the 3D mesh is connected to its 26 neighbors (i.e., adjacent nodes in the front-back, left-right, top-bottom, and all diagonal directions). Movement between nodes costs the Euclidean distance between their centers. This connection method allows paths to travel along more natural curves in 3D space. For each node in the graph... Calculate the three costs: 1. Basic path cost: Defined as starting from the origin To the node The actual path length (in mm) along the grid edge, with shorter paths encouraged.

[0099] 2. Obstacle avoidance costs: ,in It is a node Up to the nearest obstacle avoidance model The shortest distance on the surface (unit: mm). This is the gain coefficient for the obstacle avoidance cost term, used to adjust the sensitivity of the distance's influence on the cost (in this embodiment). ), It is a very small constant (in this embodiment) This is used to avoid the denominator being zero. The closer to the danger zone, the higher this cost.

[0100] 3. Cost of organizational penetration: ,in It is a node The CT value in Henle unit at this location. That is, for nodes The assessment results of the cost of tissue permeability guide the path to prioritize passing through easily permeable tissues and avoid bone.

[0101] S4: Node Total cost The calculation is as follows: ; in, The weighting coefficients for each cost item (in this embodiment, we take...) This is used to balance path length, security, and difficulty of tissue penetration. This process integrates organizational permeability costs into the total path cost. The A* search algorithm is used to find pathways from... arrive And the path that minimizes the total cost, the heuristic function of the A* algorithm. Define as the current node To the target point The Euclidean distance is used to ensure the optimality and efficiency of the search. The path found is the optimal 3D path. It consists of a series of three-dimensional spatial points.

[0102] Output: Optimal 3D path .

[0103] 5. Path display and status control: S5: Will Based on current pose Projected onto the ultrasound image plane, this yields a two-dimensional "guide line". And displayed in real time with highlighted colors. The above provides the surgeon with intuitive guidance on the direction of needle insertion.

[0104] To ensure that guidance information is displayed only when registration is reliable, the system presets two confidence thresholds. and (This embodiment) , ), and based on real-time confidence level Control the display status: like The system displays normally. The lines are solid green.

[0105] like The system still displays However, the lines turned into yellow dashed lines, and a "Limited Precision" icon warning appeared in the corner of the screen.

[0106] like The system stopped displaying. The screen displays a prominent message in the center: "Low registration confidence, please adjust the probe."

[0107] 6. Real-time obstacle avoidance detection: During simulated or actual needle insertion (if the puncture needle is equipped with a positioning sensor), the system acquires the current three-dimensional position of the needle tip in real time. .

[0108] calculate arrive Distance to the nearest point And all obstacle avoidance models shortest distance (Unit: mm)

[0109] Set a preset safe distance (This embodiment) mm), serving as a risk warning boundary.

[0110] like Then in The line segment corresponding to the danger position will switch to flashing red and emit an intermittent "beep beep" alarm sound at a frequency of 800Hz through the system audio interface to remind the operator of the risk of collision.

[0111] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A real-time percutaneous puncture path planning system based on ultrasound and CT image fusion, characterized in that, include: Image acquisition module: used to acquire preoperative CT three-dimensional volume images of patients and intraoperative real-time two-dimensional ultrasound image sequences; Feature extraction module: connected to the image acquisition module, used to extract the macroscopic three-dimensional geometric model and mesoscopic attribute feature map of the skeleton from the CT three-dimensional volume image, and to extract the skeleton interface contour features and acoustic attribute features from the real-time two-dimensional ultrasound image sequence; Registration and fusion module: connected to the feature extraction module, which has a built-in multi-level matching engine; the multi-level matching engine performs first-level matching and second-level matching; The first-level matching performs initial registration with the macroscopic three-dimensional geometric model based on the skeletal interface contour features, and outputs the first real-time pose. The second-level matching performs a fine-grained matching of the acoustic attribute features and the mesoscopic attribute feature map based on the first real-time pose, and outputs the optimized second real-time pose; The registration and fusion module spatially fuses the real-time two-dimensional ultrasound image sequence with the CT three-dimensional volumetric image based on the second real-time pose. The path planning module, connected to the registration and fusion module, is used to calculate the puncture path from a specified needle insertion point on the real-time ultrasound image plane to a preset target point in the CT three-dimensional space in the fused image space, and to overlay the puncture path on the real-time two-dimensional ultrasound image sequence.

2. The real-time percutaneous puncture path planning system based on ultrasound and CT image fusion according to claim 1, characterized in that, The mesoscopic attribute feature map is an acoustic impedance distribution map or an elastic modulus distribution map generated by physical model inversion of the gray value distribution of the CT three-dimensional volume image.

3. The real-time percutaneous puncture path planning system based on ultrasound and CT image fusion according to claim 1, characterized in that, The second-level matching employs a feature fusion method based on an attention mechanism. This method assigns a dynamic weight to each feature element in the acoustic attribute feature and the mesoscopic attribute feature map. The dynamic weight is calculated based on the spatial coordinates and feature dimension of the feature element, and the matching result is determined by the weighted feature similarity.

4. The real-time percutaneous puncture path planning system based on ultrasound and CT image fusion according to claim 1, characterized in that, It also includes a confidence assessment module, which is connected to the registration and fusion module; The input to the confidence evaluation module is the residual of the first-level matching, the similarity score of the second-level matching, and the change in the second real-time pose between consecutive frames. The output of the confidence assessment module is a comprehensive confidence score.

5. The real-time percutaneous puncture path planning system based on ultrasound and CT image fusion according to claim 4, characterized in that, The path planning module controls the display status of the puncture path based on the comprehensive confidence score: When the overall confidence score is greater than or equal to the first threshold, the complete puncture path is displayed; When the overall confidence score is less than the first threshold and greater than or equal to the second threshold, the puncture path is displayed and an accuracy warning indicator is displayed simultaneously. when When the overall confidence score is less than the second threshold, the puncture path is stopped from being displayed and a repositioning prompt is generated.

6. The real-time percutaneous puncture path planning system based on ultrasound and CT image fusion according to claim 4 or 5, characterized in that, It also includes a strategy adjustment module, which connects the confidence assessment module and the registration fusion module; the strategy adjustment module adjusts the number of iterations or the convergence threshold of the second-level matching in the multi-level matching engine according to the historical trend of the comprehensive confidence score.

7. The real-time percutaneous puncture path planning system based on ultrasound and CT image fusion according to claim 1, characterized in that, The path planning module calculates the puncture path, including the following operations: S1, receiving the coordinates of the two-dimensional needle insertion point defined on the current frame of the real-time two-dimensional ultrasound image sequence; S2, based on the second real-time pose, the coordinates of the two-dimensional needle insertion point are mapped to the coordinate system of the CT three-dimensional volumetric image to obtain the coordinates of the three-dimensional needle insertion point; S3, calculate the three-dimensional straight path between the three-dimensional needle insertion point coordinates and the preset target point; S4, the three-dimensional straight path is projected in reverse onto the imaging plane of the real-time two-dimensional ultrasound image sequence to generate two-dimensional guide lines for superimposed display.

8. The real-time percutaneous puncture path planning system based on ultrasound and CT image fusion according to claim 7, characterized in that, The path planning module also performs obstacle avoidance detection: The three-dimensional spatial extent of at least one hazardous area is predefined in the CT three-dimensional volumetric image; In step S3, the shortest distance between the three-dimensional straight path and the spatial range of each danger zone is calculated in real time; When any of the distances is less than the preset safe distance, a high-brightness warning is issued for the corresponding segment of the two-dimensional guide line or the projection area of ​​the danger zone on the ultrasound image.

9. The real-time percutaneous puncture path planning system based on ultrasound and CT image fusion according to claim 1, characterized in that, When the feature extraction module extracts the acoustic attribute features from the real-time two-dimensional ultrasound image sequence, it specifically calculates the average energy, frequency spectrum center, and attenuation coefficient of the signal from the original radio frequency signal of the ultrasound device within the depth range corresponding to the bone interface contour features.

Images