Radio frequency ablation navigation device based on multi-modal image fusion

Abstract

The invention discloses a radiofrequency ablation navigation device based on multi-modal image fusion. The device comprises a respiration sensor, an image reconstruction server, an interaction terminal and a processor. The respiration sensor is worn on a target object and used for collecting respiration data of the target object. The image reconstruction server is used for receiving the multi-modal image of the target object and reconstructing the multi-modal image into a three-dimensional model; the three-dimensional model comprises an organ geometric model, nerve distribution and a blood vessel network. And the interaction terminal is connected with the image reconstruction server and is used for marking a starting point and an ending point for the three-dimensional model. The processor is connected with the respiration sensor, the image reconstruction server and the interaction terminal; and the processor generates a predicted path output according to the respiration data and the three-dimensional model marking the starting point and the ending point. The method has the advantages that respiratory movement errors are accurately compensated, the nerve mistaken injury risk is reduced, and the operation safety is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of radiofrequency ablation navigation, and particularly to a radiofrequency ablation navigation device based on multimodal image fusion. Background Art

[0002] Existing traditional radiofrequency ablation navigation systems are mainly based on static three-dimensional reconstruction technology of medical images. By constructing an organ model of the target area through pre-operative CT or MRI images, they assist surgeons in planning ablation paths and improve surgeons' spatial cognitive ability of complex anatomical structures.

[0003] However, the traditional system faces the following problems in practical applications: The respiratory motion compensation mechanism of the existing system is imperfect. The system usually relies on respiratory gating technology for simple respiratory phase synchronization, but fails to establish an accurate mapping relationship between different phases (such as end-inspiration and end-expiration) within the respiratory cycle and the dynamic displacement of the organ. Especially when dealing with organs near the diaphragm (such as the liver and kidneys), the target drift error caused by respiration increases, easily resulting in positioning deviation of surgical instruments and increasing the risk of accidental injury to blood vessels or nerves.

[0004] In view of the above problems, there is an urgent need for a radiofrequency ablation navigation device that can deeply fuse multimodal image features and dynamically compensate for the influence of respiratory motion. Summary of the Invention

[0005] Aiming at the problem that the existing radiofrequency ablation navigation device does not fully consider the dynamic offset of the target position caused by respiration, the present invention provides a radiofrequency ablation navigation device, system and method based on multimodal image fusion.

[0006] To achieve the above object, the technical solution of the present invention is as follows:

[0007] According to one aspect of the present application, the present application discloses a radiofrequency ablation navigation device based on multimodal image fusion, including a respiratory sensor, an image reconstruction server, an interaction terminal and a processor.

[0008] A breathing sensor is worn on a target object to collect the breathing data of the target object. An image reconstruction server is used to receive the multi-modal images of the target object and reconstruct them into a three-dimensional model; the three-dimensional model includes an organ geometry model, nerve distribution, and blood vessel network. An interaction terminal is connected to the image reconstruction server and is used to mark the starting point and the ending point on the three-dimensional model. A processor is connected to the breathing sensor, the image reconstruction server, and the interaction terminal; the processor is used for: obtaining the breathing data and the three-dimensional model marked with the starting point and the ending point; generating a virtual sectional image of the target object through a multi-scale convolutional neural network for the three-dimensional model marked with the starting point and the ending point; the virtual sectional image includes a navigation path from the starting point to the ending point; constructing the breathing data and the multi-modal images into a phase curve representing the mapping relationship between the breathing state and the target point, and obtaining a first time region and a second time region through the phase curve; the first time region is the valley value region within the breathing cycle, and the second time region is the peak value region within the breathing cycle; calculating the displacement difference between the target point in the second time region and the ending point in the virtual sectional image; determining whether the displacement difference is less than a preset safety threshold, if so, determining that the navigation path does not need to be adjusted; otherwise, re-planning the navigation path according to the target point in the first time region and determining whether it passes through a preset important nerve region, if so, iteratively re-planning the navigation path until the important nerve region is avoided; and outputting the navigation path avoiding the important nerve region as a predicted path.

[0009] According to another aspect of the present application, the present application also discloses a radiofrequency ablation navigation system based on multi-modal image fusion, including a data acquisition module, a data processing module, a displacement difference calculation module, a path judgment module, and a result output module.

[0010] The data acquisition module is used to obtain the multi-modal images, breathing data, and the three-dimensional model marked with the starting point and the ending point of the target object. The data processing module is used to generate the virtual sectional image of the target object through a multi-scale convolutional neural network for the three-dimensional model marked with the starting point and the ending point; the virtual sectional image includes a navigation path from the starting point to the ending point; and is also used to construct the breathing data and the multi-modal images into a phase curve representing the mapping relationship between the breathing state and the target point, and obtaining a first time region and a second time region through the phase curve; the first time region is the valley value region within the breathing cycle, and the second time region is the peak value region within the breathing cycle. The displacement difference calculation module is used to calculate the displacement difference between the target point in the second time region and the ending point in the virtual sectional image. The path judgment module is used to determine whether the displacement difference is less than a preset safety threshold, if so, determining that the navigation path does not need to be adjusted; otherwise, re-planning the navigation path according to the target point in the first time region and determining whether it passes through a preset important nerve region, if so, iteratively re-planning the navigation path until the important nerve region is avoided. The result output module is used to output the navigation path avoiding the important nerve region as a predicted path.

[0011] According to another aspect of the present application, the present application also discloses a radiofrequency ablation navigation method based on multimodal image fusion, including the following steps:

[0012] Obtain the multimodal images, respiratory data, and a three-dimensional model marking the starting point and the ending point of a target object; generate a virtual sectional image of the target object from the three-dimensional model marking the starting point and the ending point through a multi-scale convolutional neural network; the virtual sectional image includes a navigation path from the starting point to the ending point; construct a phase curve representing the mapping relationship between the respiratory state and the target points from the respiratory data and the multimodal images, and obtain a first time region and a second time region through the phase curve; wherein, the first time region is the valley value region within the respiratory cycle, and the second time region is the peak value region within the respiratory cycle; calculate the displacement difference between the target points in the second time region and the ending point in the virtual sectional image; determine whether the displacement difference is less than a preset safety threshold, if so, determine that the navigation path does not need to be adjusted; otherwise, re-plan the navigation path according to the target points in the first time region and determine whether it crosses a preset important nerve region, if so, iteratively re-plan the navigation path until avoiding the important nerve region; and output the navigation path avoiding the important nerve region as a predicted path.

[0013] Compared with the prior art, the beneficial effects of the present invention are:

[0014] The present application collects respiratory data in real time through a respiratory sensor, reconstructs a three-dimensional model in combination with multimodal images and constructs a phase curve, dynamically calculates the displacement difference of target points within the respiratory cycle, automatically adjusts the navigation path to avoid important nerve regions, and has the advantages of accurately compensating for respiratory motion errors, reducing the risk of accidental nerve injury, and improving operation safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The disclosure of the present invention will be described with reference to the accompanying drawings. It should be understood that the drawings are only for illustrative purposes and are not intended to limit the protection scope of the present invention. In the drawings, the same reference numerals are used to refer to the same components. Among them:

[0016] Figure 1 is a schematic diagram of a radiofrequency ablation navigation device based on multimodal image fusion introduced in the present invention;

[0017] Figure 2 is based on Figure 1 the working logic flowchart of the processor;

[0018] Figure 3 is based on Figure 1 the logic flowchart of re-planning the navigation path;

[0019] Figure 4Logic flowchart for the processor to perform safety warnings based on respiratory data;

[0020] Figure 5 For the Figure 1 Schematic diagram of the connection between the processor and the virtual reality device and the virtual reality server;

[0021] Figure 6 For the Figure 1 Schematic diagram of the phase curve;

[0022] Figure 7 Schematic diagram of a radiofrequency ablation navigation system based on multimodal image fusion;

[0023] Figure 8 Schematic diagram of a radiofrequency ablation navigation method based on multimodal image fusion. Detailed implementation manners

[0024] It is easy to understand that according to the technical solution of the present invention, without changing the essence of the present invention, those of ordinary skill in the art can propose various structural ways and implementation ways that can be mutually replaced. Therefore, the following detailed implementation manners and the accompanying drawings are only exemplary descriptions of the technical solution of the present invention, and should not be regarded as all of the present invention or as a limitation or restriction on the technical solution of the present invention.

[0025] Application Overview

[0026] In the prior art, traditional radiofrequency ablation navigation systems rely on static three-dimensional reconstruction technology to construct organ models through preoperative CT or MRI images to assist in path planning. However, the existing systems have the problem of imperfect respiratory motion compensation mechanisms. The respiratory gating technology only achieves simple respiratory phase synchronization and fails to establish an accurate mapping relationship between different time phases within the respiratory cycle and the dynamic displacement of organs. Especially in the surgical scenarios of organs near the diaphragm, the respiratory motion causes a significant increase in the target drift error, and the positioning deviation of surgical instruments may lead to the risk of accidental injury to blood vessels or nerves. For example, in liver radiofrequency ablation surgery, due to the failure to predict the organ displacement difference between the end of exhalation and the end of inhalation, the actual position of the ablation needle may deviate from the planned path by more than 2 millimeters.

[0027] To solve the above problems, the inventors first observed that the organ displacement within the respiratory cycle exhibits regular fluctuation characteristics. For example, the organ position is relatively stable at the end of exhalation, while significant displacement occurs at the end of inhalation following diaphragmatic movement. Based on this, they considered how to dynamically associate the respiratory phase with the organ position and establish a mapping model between the respiratory state and the target point drift. Further, it was found that multi-modal images can provide anatomical structure information at different time points. If synchronized with the respiratory waveform, the phase-displacement relationship can be extracted through deep learning. Thus, the core solution idea was formed: real-time acquisition of respiratory data through a respiratory sensor, combined with multi-modal image reconstruction of a three-dimensional model, construction of a phase curve to divide the stable and active periods of organ displacement, verification of the path offset in the active period and re-planning of the path based on the stable period, and the introduction of a neural region avoidance mechanism to ensure safety.

[0028] After introducing the basic concept of the present invention, the embodiments of the present invention will be specifically introduced below with reference to the accompanying drawings.

[0029] Exemplary Device

[0030] As Figure 1 and Figure 2 shown, a radiofrequency ablation navigation device based on multi-modal image fusion is introduced, including a radiofrequency ablation navigation device with a respiratory sensor, an image reconstruction server, an interactive terminal, and a processor. The respiratory sensor is worn on the target object to collect respiratory data. The image reconstruction server receives multi-modal images and reconstructs a three-dimensional model including an organ geometric model, nerve distribution, and vascular network. The interactive terminal is connected to the image reconstruction server and is used to mark the starting and ending points of the three-dimensional model. The processor is connected to the respiratory sensor, the image reconstruction server, and the interactive terminal, and the processor performs the following steps: obtaining respiratory data and the three-dimensional model with marked starting and ending points; generating a virtual cross-sectional image including a navigation path through a multi-scale convolutional neural network; constructing a phase curve of the respiratory data and the multi-modal images to divide the first time region and the second time region within the respiratory cycle; calculating the displacement difference between the target point and the ending point in the second time region; judging the path adjustment requirement according to the displacement difference, and iteratively avoiding important nerve regions when re-planning the path, and finally outputting a predicted path.

[0031] Among them, the multi-scale convolutional neural network refers to a deep learning model composed of a combination of convolutional layers with different receptive fields. The specific operation of generating a virtual cross-sectional image including a navigation path from the three-dimensional model with marked starting and ending points through the multi-scale convolutional neural network is as follows:

[0032] 1. Uniformly grid and normalize the three-dimensional model so that the neural network can process data in a unified format, that is, discretize the continuous model into a 3D voxel grid and integrate geometric, nerve, and vascular data into multi-channel voxel data.

[0033] 2. Coordinate fuse the processed three-dimensional tensor with channel information and the starting point , the ending point coordinates, then extract local structures through small-scale convolutions (such as 3×3×3) and perceive long-distance paths through large-scale convolutions (such as 7×7×7). According to the positions of the starting point and the ending point, construct a Gaussian path guidance weight map to make the network focus on learning the path area, and generate real path cross-sectional images through supervised learning.

[0034] 3. Employ a neural network composed of multiple layers of 3D convolutions, skip connections, and attention mechanisms , and obtain a sequence of virtual cross-sectional images through function mapping: ; where represents the sequence of virtual cross-sectional images generated by the network, , represent the two-dimensional pixel coordinates of each cross-sectional view, ∈[0,L] is the progress position along the path.

[0035] 4. Construct the position of each cross-section in the three-dimensional voxel: ; where represents the main direction of the path. The slice pixels ( ) are mapped to three-dimensional space positions: ; where , construct an orthogonal basis to form the cross-sectional plane; then read the predicted values from the network output.

[0036] 5. The final output of the virtual cross-sectional path image sequence is .

[0037] The phase curve refers to the continuous function of the organ displacement changing with the respiratory phase during the respiratory cycle, which can be specifically generated by fitting the discrete data points in the phase-displacement mapping table and is used to characterize the dynamic relationship between the respiratory state and the position of the target point. The first time region refers to the continuous time period with an amplitude lower than the preset threshold in the respiratory waveform, which can be specifically determined by detecting the local minimum region of the respiratory waveform and is used to select the time window with relatively stable organ positions. The virtual cross-sectional image refers to a three-dimensional visualization model that fuses multi-modal image features, which can be specifically realized by combining volume rendering technology with transparency adjustment and is used to display the three-dimensional navigation path from the starting point to the ending point. The important nerve region refers to the three-dimensional space region marked by the anatomical database and expanded with a buffer zone, which can be specifically generated by registering the standard nerve grid model to the three-dimensional model of the target object and is used to define the avoidance region for path planning.

[0038] Specifically, the respiratory sensor continuously collects chest and abdominal movement data and transmits it to the processor for waveform analysis. The image reconstruction server registers and fuses multi-modal images such as CT and MRI to construct a three-dimensional anatomical model containing blood vessel and nerve distributions. The operator marks the starting and ending points of the ablation path on the model surface through the interactive terminal. The processor inputs the marked three-dimensional model into a multi-scale convolutional neural network to generate a virtual image that can penetrate multiple layers of tissue, where the navigation path is displayed as a highlighted line. At the same time, the processor aligns the respiratory data with the time stamps of the multi-modal images, extracts the centroid coordinates of the organs corresponding to each respiratory phase, and fits to obtain a phase curve. According to the curve, the valley value area with relatively small organ displacement during the respiratory cycle is defined as the first time area, and the peak value area with relatively large displacement is defined as the second time area. Calculate the Euclidean distance between the actual target point and the planned end point in the second time area. If it exceeds the safety threshold, re-plan the path based on the stable position in the first time area. During the re-planning process, the processor detects whether the new path crosses the buffer zone of important nerve areas. If a crossing occurs, avoidance nodes are inserted and the path curvature is optimized until the minimum safety distance requirement is met. The finally output predicted path not only adapts to organ displacement caused by breathing but also avoids the risk of nerve injury.

[0039] Compared with the prior art, traditional devices use static respiratory gating to trigger image acquisition only at specific phases and cannot dynamically track organ displacement changes. In this solution, a continuous mapping relationship between the respiratory state and organ position is established through the phase curve for displacement compensation throughout the entire respiratory cycle. The prior art relies on a single-modal image reconstruction model and it is difficult to accurately display the spatial distributions of nerves and blood vessels. This solution fuses multi-modal image features, combines a standard anatomical database to label important nerve areas, and enables the path planning to have a fine avoidance ability. Traditional path planning algorithms do not consider the cumulative dynamic error caused by breathing. This solution verifies the path offset in the active displacement stage, re-plans based on the stable stage, and iteratively optimizes to reduce the positioning error.

[0040] To facilitate the understanding of the above embodiments, the following will take a specific application scenario of the above embodiments as an example for illustration:

[0041] The background data of the target object is a 65-year-old male, weighing 72 kg, with a height of 175 cm, an isolated solid nodule in the lower right lobe of the lung, with a diameter of about 2.3 cm, located near the confluence of the inferior pulmonary vein and the bronchus, about 3.5 cm from the pleura. The accompanying symptom is chronic obstructive pulmonary disease (COPD).

[0042] Plan: Percutaneous radiofrequency ablation is to be performed, and it is necessary to accurately avoid the pulmonary artery branches and adjacent vagus nerve fibers during the operation.

[0043] As Figure 1As shown, the target object is equipped with a chest belt sensor (sampling frequency 100Hz) to record respiratory movements in real time. The image reconstruction server fuses three imaging modalities: CT, MRI, and PET. CT is used to clarify the tumor morphology and lung structure; MRI is used for nerve distribution modeling; PET is used for localizing metabolically active areas to assist in identifying the tumor boundary. After the image reconstruction server establishes a three-dimensional model, the operator can mark the starting point (skin puncture point) and the ending point (tumor center) on the three-dimensional model through an interactive terminal. Among them, the reconstructed three-dimensional model includes pulmonary anatomical structures (lung segments, bronchi, blood vessels, pleura) and important nerve distribution areas (such as the vagus nerve and sympathetic nerve chain).

[0044] The marked three-dimensional model is uploaded to the processor and receives respiratory data collected by the respiratory sensor. The respiratory data includes:

[0045] Peak inspiratory phases (0.0s, 3.5s, 7.1s…);

[0046] Peak expiratory phases (1.8s, 5.2s, 9.0s…).

[0047] The average respiratory cycle is calculated: 3.6 seconds

[0048] The cycle is divided into:

[0049] The first time region (stable period): from the end of exhalation to the beginning of inhalation (for path execution);

[0050] The second time region (active period): the period of rapid lung tissue movement (for predicting deviations).

[0051] The maximum displacement difference between the target point and the ending point in the second time region is calculated: Δ = 4.3mm, but the allowable path error threshold is set at 3mm, and it is determined that path correction is required. A multi-scale convolutional neural network is used to generate a new path map to avoid the following high-risk regions in the path: the lower right lobe bronchial artery (2.1mm away from the original path), the vagus nerve bundle region (2.8mm proximal).

[0052] The final feasible path is output: length: 5.8cm, puncture angle: 28° relative to the chest wall, maximum error prediction: 1.9mm (meeting the requirements of stable puncture).

[0053] Through the above technical solutions, the present application realizes the dynamic correlation compensation of respiratory movements and organ displacements, and solves the problem of target drift caused by imperfect respiratory compensation mechanisms in traditional devices. Through multi-modal image fusion and nerve region avoidance mechanisms, important anatomical structures are actively avoided during the path planning stage, reducing the risk of nerve injury during surgical operations. The virtual dissection image provides three-dimensional visual navigation of multiple layers of tissues, helping the surgeon accurately judge the spatial relationship between the instrument and the surrounding tissues.

[0054] As Figure 3 shown, the present application further proposes that after determining that the navigation path crosses an important nerve area, avoidance nodes are inserted around the intersection point of the navigation path and the edge of the important nerve area, and then the gradient descent method is used to optimize the curvature of the navigation path; the distances between the optimized navigation path and the important nerve area are sorted in ascending order, and then the Top1 distance is taken as the minimum distance; it is judged whether the minimum distance is greater than or equal to a preset distance threshold. If so, the curvature of the navigation path is re-optimized and re-judged until the minimum distance is less than the distance threshold or the preset number of iterations (such as 50 times) is reached, and then the iteration stops and an abnormal warning is issued; otherwise, it is judged whether the increase in the total length of the optimized navigation path is greater than a preset tolerance (such as 15%). If so, an abnormal warning is issued; otherwise, the optimized navigation path is output.

[0055] Among them, the avoidance node refers to the geometric coordinate points inserted around the intersection point of the navigation path and the nerve area. Specifically, the path segmentation algorithm can be used to generate a buffer area around the intersection point and select the coordinate points far from the nerve as new nodes, so that the path bypasses the dangerous area. Optimizing the curvature of the navigation path by the gradient descent method means calculating the change rate of the path curvature and adjusting the node coordinates along the direction of curvature reduction. Specifically, the numerical difference method can be used to calculate the curvature gradient, and the node positions are iteratively adjusted step by step to make the path smooth. Taking the Top1 distance as the minimum distance after sorting the distances in ascending order means sorting the Euclidean distances between each segment of the path and the edge of the nerve area and extracting the minimum value. Specifically, a spatial geometry calculation tool can be used to traverse the distances between the path nodes and the grid vertices of the nerve area. For the distance threshold, a basic threshold is set according to the physiological characteristics (such as diameter, functional importance) of the target nerve, and then it is corrected according to the specificity of the target object.

[0056] Specifically, when it is detected that the path crosses the nerve area, the path direction is changed by inserting avoidance nodes around the intersection point, and then the gradient descent method is used to optimize the path curvature, reducing the risk of instrument operation caused by path mutation. The minimum distance between the optimized path and the nerve area is accurately identified by sorting in ascending order. If this distance does not reach the safety threshold, the iterative optimization process is triggered to re-adjust the curvature until the condition is met or the maximum number of iterations is reached. If the increase in the total length of the path exceeds the tolerance, a warning is issued to avoid overly extending the path and affecting the surgical efficiency. For example, when the increase in the length of the optimized path exceeds the tolerance, the processor stops automatic adjustment and prompts for manual intervention, thus achieving a balance between safety and operation efficiency.

[0057] Compared with the prior art, traditional path planning methods usually only perform single path adjustment based on static obstacle avoidance thresholds, and cannot dynamically optimize the curvature and iteratively correct the minimum safety distance. The prior art lacks a precise avoidance mechanism for nerve regions. After path adjustment, there may still be local high-risk contact points, and the rationality of the path length cannot be effectively controlled. For example, traditional methods may generate paths using an avoidance area with a fixed radius, resulting in sudden changes in path curvature or overly long detour distances.

[0058] Through the above technical solutions, this application solves the problem that the navigation path may accidentally injure important nerve regions, and realizes the smooth optimization of path curvature and the dynamic control of the safety distance. The iterative adjustment mechanism ensures that the minimum distance between the path and the nerve region meets the safety threshold, and at the same time, the length increase judgment is used to avoid a decrease in path efficiency caused by excessive adjustment. The abnormal warning mechanism provides a clear decision-making basis for manual intervention, forming a complete path optimization closed loop.

[0059] This application further proposes the following specific steps for setting important nerve regions: establish a three-dimensional polygon mesh model based on the nerve distribution coordinates in the standard anatomical database, label the risk levels according to the importance of nerve functions, and mark them in different colors on the three-dimensional polygon mesh model; screen out the nerve distribution regions with levels greater than the preset level value, and expand them by a preset distance to generate a safety buffer zone that prohibits crossing, forming a standard important nerve region; obtain the three-dimensional model of the target object marked with the starting point and the ending point, and divide the target area according to the starting point and the ending point on the three-dimensional model; align the features of the three-dimensional polygon mesh model and the three-dimensional model, and map the standard nerve region of the three-dimensional polygon mesh model onto the target region of the three-dimensional model, and after correction, form the important nerve region of the three-dimensional model of the target person.

[0060] Among them, the three-dimensional polygon mesh model refers to a nerve distribution model with a topological structure established based on standard anatomical data. Specifically, it can be generated by extracting nerve coordinates using a medical image segmentation algorithm and then splicing triangular patches. Its function is to provide a standardized nerve distribution reference for different patients. The risk level refers to the clinical injury risk level divided according to the importance of nerve functions. Specifically, it can be marked using the nerve injury consequence grading standard in the clinical guidelines. Its function is to provide a quantitative basis for subsequent screening of key avoidance regions. Expanding the preset distance refers to the operation parameter for geometric expansion of high-risk nerve regions. Specifically, it can use the Euclidean distance transformation algorithm to extend the set value outward. Its function is to construct a dynamic safety buffer zone to cope with the organ displacement error caused by respiratory movement. Feature alignment refers to the process of spatially registering the standard nerve grid and the patient's three-dimensional model. Specifically, it can be realized using landmark matching or surface registration algorithms. Its function is to map the standard anatomical structure into the patient's individualized model.

[0061] Specifically, first, a three-dimensional grid model with nerve coordinates is established through a standard anatomical database, and the nerve regions are color-coded according to clinical risk grading. Subsequently, high-risk regions are screened out, and a safety buffer zone is generated by distance expansion to form a standard avoidance region. In the patient's three-dimensional model, the target operation region is determined according to the starting and ending points of the surgical path, and the standard nerve grid is spatially aligned with the patient model through a feature registration algorithm, so that the standard avoidance region is accurately mapped onto the patient's anatomical structure, and finally an individualized important nerve region is generated. This process compensates for the respiratory motion error through dynamic expansion of the buffer zone and solves the problem of individual anatomical differences of patients by using feature alignment, ensuring that key nerve structures can be accurately identified and avoided during navigation path planning.

[0062] Compared with the prior art, traditional methods usually use a safety buffer with a fixed distance or rely on the operator's experience to manually label nerve regions, resulting in problems of over-conservatism or omission of high-risk regions. The prior art lacks a compensation mechanism for the dynamic change of nerve position under the influence of respiratory motion and cannot achieve accurate mapping of the standard anatomical structure and the patient's individual model. This solution screens and expands the dynamic buffer through risk grading, avoiding unnecessary path extension while ensuring safety. The feature alignment technology is used to achieve automatic adaptation of the standard nerve distribution and the patient's image, significantly improving the recognition accuracy of important nerve regions.

[0063] Through the above technical solution, this application effectively solves the problem of inaccurate recognition of important nerve regions in radiofrequency ablation path planning. Through the design of a dynamically expanded safety buffer zone, it can adaptively compensate for the organ displacement caused by respiratory motion and avoid misjudgment of nerves caused by organ movement. The feature alignment mechanism based on the standard anatomical database ensures the consistency of nerve region mapping among different patients and reduces the error caused by individual anatomical variations. The risk level annotation provides an intuitive basis for the operator to evaluate the nerve injury risk, enabling the navigation path planning to achieve the optimal path selection on the premise of ensuring safety.

[0064] This application further proposes the specific steps of constructing a phase curve representing the mapping relationship between respiratory data and multi-modal images for the respiratory state and target points, including: dividing the respiratory data according to the respiratory phase, associating each frame of multi-modal image with a unique respiratory phase based on timestamp alignment; using the multi-modal image at the end-expiratory phase as the reference coordinate system and aligning the multi-modal images at other phases to the reference coordinate system; extracting the three-dimensional centroid coordinates of the target region of each frame of multi-modal image through deep learning; dividing a respiratory cycle into N phase intervals according to the amplitude of the respiratory waveform, statistically calculating all the three-dimensional centroid coordinates within each phase interval and averaging to obtain a phase-displacement mapping table, which can be specifically referred to Table 1; fitting the phase-displacement mapping table to obtain a phase curve, as Figure 6 shown.

[0065] Table 1: Phase-Displacement Mapping Table

[0066] Phase ϕ Displacement d (mm) 0.0 0.0 0.1 2.3 0.2 4.5 0.3 6.2 0.4 7.1 0.5 7.4 0.6 6.9 0.7 5.3 0.8 3.0 0.9 1.0 1.0 0.0

[0067] Among them, the respiratory phase division refers to dividing the respiratory cycle into multiple time intervals, which can be specifically achieved by detecting the peaks and valleys of the respiratory waveform. Its role is to discretize the dynamic respiratory process into analyzable stages. The timestamp alignment refers to matching the time axes of the image data and the respiratory data using a synchronization signal, which can be specifically implemented by using a hardware trigger or a software interpolation algorithm to ensure the strict correspondence between the image frames and the respiratory phases. The reference coordinate system refers to using the image at the end-expiration moment as the spatial reference, which can be specifically achieved by using a rigid registration or an affine transformation algorithm. By using the characteristic that the organ position is relatively stable at this phase, the registration accuracy is improved. The extraction of the three-dimensional centroid coordinates refers to calculating the spatial center coordinates of the target area, which can be specifically implemented by using a convolutional neural network combined with a region segmentation technique to quantify the displacement of the organ position with the change of respiration. The phase interval division refers to equally dividing or dividing proportionally according to the respiratory amplitude, which can be specifically implemented by using a threshold method or a clustering algorithm to statistically analyze the displacement law of the target points in different respiratory stages. The phase curve fitting refers to generating a continuous function curve to describe the relationship between the displacement and the respiratory phase, which can be specifically implemented by using a polynomial regression or a spline interpolation algorithm to provide a mathematical model for dynamic navigation.

[0068] Specifically, this technical solution establishes the mapping relationship between the respiratory state and the organ displacement through the following steps: First, the waveform data collected by the respiratory sensor is divided into different phases, and at the same time, the multi-modal images are strictly bound to the respiratory phases through timestamps to eliminate the errors caused by data asynchrony; then, the image at the end-expiration phase is selected as the spatial reference, and all other phase images are aligned to this coordinate system through a registration algorithm to form a unified spatial reference system; subsequently, a deep learning model is used to automatically identify the target area and calculate its three-dimensional centroid coordinates to avoid the subjective errors of manual annotation; further, the entire respiratory cycle is divided into several intervals according to the amplitude, and the average value of the centroid coordinates in each interval is statistically analyzed to generate a discrete phase-displacement mapping table; finally, the discrete data is converted into a continuous function through curve fitting to establish a quantitative relationship model between the respiratory phase and the organ displacement.

[0069] Compared with the existing technologies, the traditional respiratory gating technology only triggers image acquisition through the respiratory phase and does not establish a dynamic mapping relationship between the organ displacement and the phase, resulting in limited compensation accuracy. This solution realizes the accurate modeling of the organ displacement during the respiratory cycle by introducing methods such as timestamp synchronization, reference coordinate system registration, deep learning feature extraction, and phase interval statistics, and overcomes the problem of target drift caused by the dynamic changes of the organ in the traditional technology.

[0070] Through the above technical solution, the present application can convert the dynamic association between the respiratory state and organ displacement into a quantifiable phase curve, providing a basis for dynamic compensation for real-time navigation, effectively reducing the target positioning error caused by respiratory movement, and improving the accuracy of radiofrequency ablation path planning. This solution solves the problem of imperfect respiratory movement compensation mechanism in the prior art through the deep fusion of multi-modal images and respiratory data. Especially when dealing with organs near the diaphragm, it significantly reduces the risk of accidental injury to blood vessels or nerves.

[0071] The present application further proposes a technical solution for generating a stable window through real-time respiratory data acquisition and cycle prediction, including acquiring real-time respiratory data of a target object, and predicting the first time region of the next stage through cycle detection of the real-time respiratory data and outputting it as a stable window.

[0072] Among them, the real-time respiratory data refers to the physiological signals generated by continuously collecting the respiratory movement of the target object through a respiratory sensor. Specifically, a piezoelectric sensor or an acceleration sensor can be used to achieve this, which is used to reflect the dynamic displacement characteristics of organs during the respiratory cycle. Cycle detection refers to the periodic analysis of the respiratory waveform to predict the future respiratory phase. Specifically, an autoregressive model or a fast Fourier transform algorithm can be used to achieve this. By extracting the frequency and amplitude characteristics of the respiratory waveform, different respiratory phase intervals are divided. The first time region refers to the stable stage with the smallest organ displacement during the respiratory cycle. Specifically, it can be determined through the valley region of the phase curve. During this stage, the organ position is relatively fixed, which is suitable for surgical operations.

[0073] Specifically, the real-time respiratory data is continuously collected and input into the cycle detection module. By analyzing the periodic characteristics of the respiratory waveform, the respiratory phase change of the next stage is predicted. When it is detected that the respiratory cycle enters a relatively flat waveform region, based on the phase curve fitting result of historical data, the first time region of the next stage is predicted in advance. This stable window corresponds to the valley region in the respiratory cycle, where the organ displacement amplitude is the smallest, and it can be used as an ideal time window for operating surgical instruments. By dynamically predicting the stable window, it can provide accurate timing reference for the operator, so as to execute navigation path planning or instrument positioning in the stage with the least influence of respiratory movement, and reduce the target drift error caused by respiration.

[0074] Compared with the prior art, traditional respiratory gating techniques usually rely on fixed time intervals or manual triggers for respiratory phase synchronization, and cannot dynamically adjust the prediction window according to the real-time respiratory state. However, this solution can actively predict the stable window of the next stage through a real-time data-driven cycle detection algorithm, realizing the dynamic matching of respiratory phase and surgical operation, and avoiding the prediction deviation caused by the change of respiratory rhythm in the traditional method.

[0075] Through the above technical solution, the present application solves the problem that the traditional navigation system cannot predict the respiratory stable window in real time, and realizes the precise selection of the surgical operation timing. By dynamically predicting the first time region and outputting the stable window, the interference of respiratory movement on organ positioning is effectively reduced, and the instrument positioning error caused by misjudgment of the respiratory phase is reduced, thereby improving the reliability of the navigation path planning and the surgical safety.

[0076] As Figure 6 shown, the present application further proposes a technical solution for detecting the current respiratory phase by acquiring the respiratory data of the target object in real time and triggering a safety warning signal during the high-risk period of the respiratory cycle.

[0077] Among them, the real-time respiratory data refers to the respiratory waveform signal continuously collected by the sensor worn on the target object, which can be specifically implemented by a piezoelectric sensor or an acceleration sensor for dynamically tracking the respiratory movement state. Among them, the periodic detection refers to extracting the periodic characteristics from the respiratory data to determine the current respiratory phase, which can be specifically implemented by a sliding window Fourier transform or a peak detection algorithm to decompose the respiratory waveform into different phase intervals. Among them, the second time region refers to the peak region with the largest organ displacement amplitude during the respiratory cycle, which can be specifically determined by the extreme value interval of the displacement change in the phase curve and is used to characterize the high-risk period.

[0078] Specifically, the respiratory sensor continuously collects the real-time respiratory signal of the target object and transmits the data to the processor. The processor performs spectral analysis on the respiratory signal through a sliding window Fourier transform, extracts the main frequency component during the respiratory cycle, and determines the interval where the current respiratory phase is located. When it is detected that the current phase enters the second time region, the processor triggers the safety warning module, for example, notifies the operator to pause the instrument operation through a sound alarm or a visual prompt signal. At this stage, since the organ displacement caused by respiration reaches the peak, continuing the operation may cause the navigation path to deviate from the expected position. The warning signal forces the operator to wait until the respiratory cycle enters the low-risk trough region, thereby reducing the risk of accidental injury.

[0079] Compared with the prior art, the traditional respiratory gating technology can only synchronize image acquisition at a preset fixed phase, and cannot track the dynamic respiratory phase in real time and associate the high-risk period. This solution actively intervenes in the operation process by dynamically analyzing the respiratory cycle and judging the triggering conditions of the high-risk period in real time, avoiding the organ drift caused by respiration from exceeding the safe range.

[0080] Through the above technical solution, the present application realizes the linkage between the real-time monitoring of the respiratory phase and the risk warning, effectively reduces the positioning deviation of the surgical instrument under the influence of respiratory movement, and reduces the risk of accidental injury to nerves or blood vessels caused by organ displacement.

[0081] Please refer to Figure 5, the present application further proposes a radiofrequency ablation navigation device based on multi-modal image fusion, including a virtual reality device and a virtual reality server. The virtual reality device is worn on the operator, and the virtual reality server is connected to the virtual reality device and the processor, and is used to display the three-dimensional model with the predicted path output by the processor through the virtual reality device.

[0082] The purpose of adding the virtual reality device and the virtual reality server is to serve operators with insufficient experience. For some operators, they have solid basic operation experience, but their individual differential emergency response ability for different target objects is weak. In actual operation, sudden situations may affect the exertion of their normal level. Therefore, through virtual reality, pre-rehearsal can be carried out in advance. While enhancing operation experience, it can also pre-rehearse sudden situations, improve the operator's emergency response ability, and enhance the operator's confidence at the same time.

[0083] Among them, the virtual reality device refers to a hardware unit that generates a three-dimensional space visualization interface through a head-mounted display device or a stereoscopic projection device. Specifically, it can be implemented by using augmented reality glasses equipped with a binocular display module. Its function is to provide an immersive observation environment for the operator and solve the problem that traditional two-dimensional screens cannot present spatial dynamic relationships. The virtual reality server refers to an intermediate processing module used to receive three-dimensional model data and convert it into a format that can be parsed by the virtual reality device. Specifically, it can be implemented by using a distributed graphics rendering server. Its function is to overlay and display the navigation path and the three-dimensional model of the respiratory dynamic change in real time through a coordinate mapping and data synchronization mechanism, ensuring the spatial consistency between the virtual scene and the actual anatomical structure.

[0084] Specifically, by wearing the virtual reality device, the operator can directly observe the three-dimensional model generated by the processor and the stereoscopic projection of the predicted path. After receiving the three-dimensional model output by the processor, the virtual reality server divides it into multiple rendering layers and superimposes the navigation path on the surface of the organ geometry model in the form of a highlighted line. When the processor detects a respiratory emergency state, it simulates the organ displacement and path offset through a fuzzy logic algorithm to generate a sequence of changes in the three-dimensional model in the state of respiratory arrest or rhythm disorder. The virtual reality server sends these change sequences to the virtual reality device in the form of dynamic frames, enabling the operator to experience extreme scenarios such as instrument positioning deviation and blood vessel compression in the virtual environment, and simulating instrument operation through an interactive handle to complete targeted training on avoiding nerves and path correction.

[0085] Compared with the prior art, traditional navigation systems rely on flat displays to present static 3D models. Operators need to construct spatial cognition through 2D views, which is prone to perspective deviation. In contrast, this solution uses virtual reality devices to construct a holographic three-dimensional scene, enabling operators to observe the real-time relationship between the dynamic displacement of organs and the navigation path from the first-person perspective. At the same time, the virtual reality server simulates complex anatomical changes in the emergency state of breathing, providing training conditions close to the real surgical environment for operators, and solving the technical defect that traditional systems cannot pre-enact unexpected situations.

[0086] Through the above technical solution, this application enables operators to conduct training on navigation path operations during the breathing cycle change and emergency state in a virtual environment. By repeatedly simulating the process of instrument movement and nerve avoidance, the operation proficiency is improved. At the same time, by superimposing the breathing dynamic model and the navigation path in real time, the path planning error caused by the breathing phase switch is reduced, and the risk of misoperation due to lack of experience in actual surgery is minimized.

[0087] This application further proposes to simulate a 3D model in the emergency state of breathing through a fuzzy logic algorithm using breathing data and 3D models marking the starting point and the ending point, and send the 3D model in the emergency state of breathing to the virtual reality server as reference data for emergency situation simulation by the virtual reality server. The emergency state of breathing includes respiratory arrest, tachypnea, and arrhythmia.

[0088] Among them, the fuzzy logic algorithm refers to an algorithm for processing non-linear data based on fuzzy set theory. Specifically, it can be implemented through steps such as input variable fuzzification, rule base reasoning, and defuzzification, and is used to process the non-deterministic characteristics of breathing data and simulate the dynamic displacement of organs in abnormal breathing states. Among them, the 3D model in the emergency state of breathing refers to an organ morphological change model of the target object generated by a processor. Specifically, it can be realized by dynamically associating multi-modal imaging data with breathing parameters, and is used to reflect the three-dimensional spatial offset of organs during respiratory arrest or arrhythmia. Among them, the virtual reality server refers to a computing device for storing and transmitting 3D model data. Specifically, it can be implemented through a distributed rendering architecture and is used to transmit the emergency state model to the virtual reality device in real time.

[0089] Specifically, after obtaining the respiratory data and the three-dimensional model, the respiratory rate and amplitude parameters are fuzzified through a fuzzy logic algorithm to establish a non-linear mapping relationship between the abnormal breathing pattern and the organ displacement. For example, when respiratory arrest is detected, the algorithm infers the continuous drift trajectory of the organ in the static state based on historical data and generates the corresponding three-dimensional model; when rapid breathing is detected, the algorithm calculates the three-dimensional displacement of the high-frequency tremor of the organ according to the mutation characteristics of the respiratory waveform. The generated emergency state model is transmitted to the virtual reality device of the operator through the virtual reality server, forming a virtual surgical scene containing the dynamic offset of the organ for the operator to practice the path adjustment operation in the emergency state.

[0090] The specific calculation process of simulating the three-dimensional model through the fuzzy logic algorithm is as follows:

[0091] I. Input parameter fuzzification

[0092] A similar function is established for all inputs using the triangular membership function, and the membership degree of each input under different semantic labels (such as "low", "medium", "high") is obtained respectively. The membership degree is:

[0093] ;

[0094] Among them, represents the membership degree value of the variable belonging to the semantic label of "low", ranging between [0, 1], represents the respiratory rate, represents the left endpoint of the "low" membership function, represents the vertex of the "low" membership function, represents the right endpoint of the "low" membership function.

[0095] II. Fuzzy rule inference

[0096] Through machine learning training or manual design of the rule base, the activation degree of each rule is inferred using the Mamdani type, and the activation results of each rule are combined to obtain the output fuzzy set

[0097] III. Output defuzzification

[0098] Each direction is calculated independently. For example, the centroid method is used to calculate the output as the three-dimensional offset vector of the organ .

[0099] IV. Three-dimensional model construction

[0100] Update the original three-dimensional model grid coordinates using the offset vector: ; Among them, represents the updated model vertex coordinates, Represents the vertex coordinates of the original 3D model, and constructs a sub-region offset weight function according to the organ movement region , and then obtains the final model: .

[0101] Compared with the prior art, the traditional navigation system can only handle the organ displacement compensation under normal breathing conditions, but cannot generate an organ dynamic model during abnormal breathing. This solution associates the abnormal breathing pattern with the organ displacement characteristics through a fuzzy logic algorithm, constructs a 3D model under extreme conditions such as respiratory arrest and rapid breathing, and fills the gap in the emergency scenario data in virtual reality training.

[0102] Through the above technical solution, this application solves the problem that the traditional system cannot generate an organ dynamic model in a respiratory emergency state, provides organ displacement data during abnormal breathing for virtual reality devices, enables operators to familiarize themselves with the navigation path adjustment method in the case of respiratory arrest or rhythm disorder in a simulated environment, and reduces the risk of misoperation caused by sudden abnormal breathing during actual surgery.

[0103] In summary, the main idea of this embodiment is to first generate a predicted path based on the pre-collected data, provide a data basis for the operator to familiarize himself with the relevant characteristics of the target object before actual operation, and combine virtual reality technology to pre-play for the operator in advance, increasing the operator's operation proficiency and improving his emergency response ability, and increasing the success rate of actual operation. And during actual operation, it can provide data support for the operator. By combining respiratory sensor data with a multi-scale convolutional neural network, a virtual sectional image with blood vessel / nerve distribution is generated, and the displacement of the target point is dynamically compensated by the respiratory phase curve, reducing the puncture deviation caused by respiratory movement; through the iterative path planning algorithm, after detecting that the displacement exceeds the limit during the respiratory peak period, the path is automatically re-planned and important nerve areas are avoided, reducing the risk of nerve injury and massive bleeding caused by the predicted path.

[0104] Exemplary System

[0105] As Figure 7 shown, this application further proposes a radiofrequency ablation navigation system based on multi-modal image fusion, including a data acquisition module, a data processing module, a displacement difference calculation module, a path judgment module, and a result output module. The data acquisition module is used to obtain multi-modal images, respiratory data, and a 3D model marking the starting point and the ending point of the target object. The data processing module is used to generate a virtual sectional image from the 3D model through a multi-scale convolutional neural network, and construct a phase curve from the respiratory data and the multi-modal images to divide the valley region and the peak region within the respiratory cycle. The displacement difference calculation module is used to calculate the displacement difference between the target point and the ending point in the peak region. The path judgment module is used to trigger path re-planning according to the displacement difference threshold, and re-plan the path in the valley region to avoid important nerve regions. The result output module is used to output the final predicted path.

[0106] Among them, multimodal images refer to image data obtained by different medical imaging devices, such as CT, MRI or ultrasound images. Specifically, an image reconstruction server can be used to perform three-dimensional reconstruction of multimodal data to form a comprehensive model including organ geometry model, nerve distribution and vascular network, providing an anatomical structure basis for dynamic path planning. Respiratory data refers to the respiratory waveform signal collected by the respiratory sensor. Specifically, a chest and abdominal motion sensor or an airflow sensor can be used to obtain the organ displacement characteristics of different phases in the respiratory cycle, providing time series data support for constructing the phase curve. The phase curve refers to the mapping relationship model between the respiratory state and the displacement of the target point. Specifically, a deep learning algorithm can be used to fit the three-dimensional centroid coordinates of the multimodal images in the respiratory cycle to quantify the correlation between the respiratory phase and the organ displacement, providing a basis for dividing the stable window. Virtual sectional images refer to the three-dimensional visualization path generated by a multi-scale convolutional neural network. Specifically, convolution kernels of different scales can be used to extract the organ edge, vascular direction and nerve distribution characteristics to generate a three-dimensional sectional view containing the navigation path to assist the operator in path planning. The displacement difference refers to the deviation between the actual position of the target point in the respiratory peak area and the end point in the virtual image. Specifically, the Euclidean distance calculation method can be used to evaluate the degree of path deviation caused by respiratory movement and provide trigger conditions for path replanning. The important nerve area refers to the preset anatomical area that is prohibited from crossing. Specifically, a three-dimensional polygonal mesh model can be established based on the standard anatomical database, and a safe area can be generated by expanding the buffer zone to reduce the risk of nerve injury during surgery.

[0107] Specifically, the data acquisition module first obtains the target object's respiratory data, multimodal images, and a three-dimensional model with marked start and end points. The data processing module uses a multi-scale convolutional neural network to extract features from the three-dimensional model and generates a virtual cross-sectional image containing the navigation path. At the same time, by analyzing the time alignment relationship between the respiratory data and the multimodal image, a phase curve is constructed to divide the valley area and peak area of the respiratory cycle. The displacement difference calculation module calculates the deviation value between the actual displacement of the target point and the planned end point in the respiratory peak area. When the displacement difference exceeds the safety threshold, the path judgment module replans the path in the respiratory valley area and iteratively optimizes based on the mapping relationship of important neural areas until a safe path that avoids the nerves is generated. The result output module finally outputs the optimized path as the prediction result.

[0108] In some specific embodiments, the construction of the phase curve can be achieved through respiratory phase division and three-dimensional centroid coordinate statistics. For example, a respiratory cycle can be divided into multiple phase intervals, and the average coordinates of the centroid of the target region within each interval are calculated, and then a phase-displacement curve is fitted. The generation of virtual cross-sectional images can adopt a cascaded convolutional structure. For example, the organ contour is first extracted through a large-scale convolutional kernel, and then the blood vessel branches are refined through a small-scale convolutional kernel, and finally a three-dimensional view including the navigation path is generated by fusion. The iterative optimization of path replanning can adjust the path curvature through the gradient descent method and insert avoidance nodes at the edges of important nerve regions. For example, buffer nodes are set around the intersection of the path and the nerve region, and the path direction is gradually optimized until the safety distance threshold is met.

[0109] Compared with the prior art, traditional radiofrequency ablation navigation systems rely on static three-dimensional models and simple respiratory gating techniques, and only perform rough compensation through respiratory phase synchronization, unable to establish a dynamic mapping relationship of organ displacement within the respiratory cycle. In this solution, the correlation between the respiratory state and the target point displacement is quantified through the phase curve, the displacement deviation is evaluated in real time in the respiratory peak region, and the path is dynamically adjusted in the trough region. Combining multi-modal image fusion and nerve region mapping, the collaborative optimization of path planning and respiratory compensation is achieved. In addition, the prior art lacks an avoidance mechanism for important nerve regions, while this solution ensures that the navigation path always avoids high-risk nerve regions during the dynamic adjustment process by iteratively optimizing the path curvature and safety distance threshold.

[0110] Through the above technical solutions, this application can solve the problem of organ displacement mapping deviation caused by inaccurate respiratory motion compensation in traditional systems. By constructing a dynamic association model between the respiratory phase and the target point displacement, a stable window is divided within the respiratory cycle for path adjustment, effectively reducing the positioning error caused by respiratory motion. At the same time, combining the mapping of important nerve regions and the path iterative optimization mechanism, the risk of nerve injury caused by path replanning is avoided. This solution can realize the dynamic generation and real-time correction of high-precision navigation paths in radiofrequency ablation surgeries in regions significantly affected by respiratory motion such as the liver and kidneys.

[0111] Exemplary Method

[0112] Such as Figure 8The present application further proposes a radiofrequency ablation navigation method based on multimodal image fusion, including the following steps: obtaining multimodal images, respiratory data, and a three-dimensional model marking the starting point and the ending point of a target object; generating a virtual sectional image of the target object from the three-dimensional model marking the starting point and the ending point through a multi-scale convolutional neural network; constructing a phase curve representing the mapping relationship between the respiratory state and the target point from the respiratory data and the multimodal images, and obtaining a first time region and a second time region from the phase curve; calculating the displacement difference between the target point and the ending point in the virtual sectional image in the second time region; determining whether the displacement difference is less than a preset safety threshold; deciding whether to re-plan the navigation path according to the determination result and determining whether to cross an important nerve region; and outputting the navigation path avoiding the important nerve region as a predicted path.

[0113] Among them, the multimodal images refer to medical image data with different tissue characteristic information obtained through CT, MRI, and ultrasound imaging technologies. Specifically, an image registration algorithm can be used to achieve multi-source data fusion for constructing a three-dimensional model including the geometric shape of an organ and the distribution of nerves and blood vessels. The respiratory data refers to the chest and abdomen movement signals collected by piezoelectric or inertial sensors. Specifically, a time-frequency analysis method can be used to extract the characteristics of the respiratory cycle for establishing the correlation between the dynamic displacement of an organ and the respiratory phase. The phase curve refers to a continuous function curve generated by fitting the relationship between the average displacement of an organ within the respiratory cycle and time. Specifically, a cubic spline interpolation algorithm can be used to achieve data fitting for dividing the stable phase and the violently displaced phase of an organ in the respiratory cycle. The displacement difference refers to the three-dimensional Euclidean distance between the coordinates of the target point and the coordinates of the ending point in the virtual sectional image in the second time region. Specifically, a coordinate transformation algorithm can be used to calculate the spatial position difference for evaluating the deviation at the end of the navigation path caused by respiratory movement.

[0114] Specifically, after generating a three-dimensional model through multimodal image fusion, this method realizes the closed-loop optimization of the navigation path in combination with the respiratory phase dynamic division mechanism. The multi-scale convolutional neural network performs hierarchical feature extraction on the three-dimensional model to generate a virtual sectional image including the distribution of blood vessels and nerves to determine the initial navigation path. During the construction of the respiratory phase curve, the image frames are associated with the respiratory phase based on timestamp alignment, and a phase-displacement mapping table is generated by statistically calculating the average displacement of the organ centroid in each phase interval, and then a continuous curve reflecting the dynamic changes of the organ is obtained by fitting. When calculating the displacement difference in the second time region, if the deviation exceeds the safety threshold, the path re-planning mechanism is triggered. The first time region with a smaller displacement is preferentially selected to re-determine the target point, and the path curvature is optimized by the gradient descent method to avoid important nerve regions. During the path iterative optimization process, avoidance nodes are inserted and the minimum distance between the path and the nerve region is calculated until the avoidance condition is met or the iteration termination threshold is reached.

[0115] Compared with the prior art, the existing radiofrequency ablation navigation system relies on static image reconstruction synchronized with respiratory gating, and can only achieve a rough matching of the respiratory phase, unable to establish a dynamic mapping relationship between organ displacement and respiratory state. This method generates a dynamic three-dimensional model through multi-modal image fusion, divides the stable region of organ displacement by combining phase curves, and actively compensates for respiratory motion errors during the path planning stage. The prior art lacks a closed-loop judgment mechanism for avoiding nerve regions, while this method real-time detects the overlapping state of the path and important nerve regions during the path replanning process, and actively avoids by optimizing curvature and adjusting avoidance nodes.

[0116] Through the above technical solutions, this application solves the problem of cumulative target drift error caused by insufficient respiratory motion compensation in traditional radiofrequency ablation navigation, and reduces the end-point positioning deviation caused by excessive respiratory amplitude through the dynamic phase division and path replanning mechanism. At the same time, through the iterative optimization algorithm for avoiding nerve regions, the risk of surgical instruments accidentally touching important nerve structures due to organ displacement is effectively avoided, and the safety of the navigation path is improved.

[0117] The technical scope of the present invention is not limited to the content described above. Those skilled in the art can make various deformations and modifications to the above embodiments without departing from the technical idea of the present invention, and these deformations and modifications should all fall within the protection scope of the present invention.

Claims

1. A radiofrequency ablation navigation device based on multimodal image fusion, characterized in that Comprising: A respiration sensor, which is worn on a target object and is used to collect respiration data of the target object; An image reconstruction server, which is used to receive multi-modal images of the target object and reconstruct them into a three-dimensional model; the three-dimensional model includes an organ geometry model, nerve distribution, and blood vessel network; An interaction terminal, which is connected to the image reconstruction server and is used to mark a starting point and an ending point on the three-dimensional model; A processor, which is connected to the respiration sensor, the image reconstruction server, and the interaction terminal; the processor is used for: Obtaining the respiration data and the three-dimensional model marked with the starting point and the ending point; Generating a virtual dissection image of the target object from the three-dimensional model marked with the starting point and the ending point through a multi-scale convolutional neural network; the virtual dissection image contains a navigation path from the starting point to the ending point; Constructing the respiration data and the multi-modal images into a phase curve representing the mapping relationship between the respiration state and the target points, and obtaining a first time region and a second time region through the phase curve; the first time region is the valley value region within the respiration cycle, and the second time region is the peak value region within the respiration cycle; Calculating the displacement difference between the target point in the second time region and the ending point in the virtual dissection image; Judging whether the displacement difference is less than a preset safety threshold. If so, it is determined that the navigation path does not need to be adjusted; Otherwise, re-plan the navigation path according to the target point in the first time region and judge whether it crosses a preset important nerve region. If so, iteratively re-plan the navigation path until the important nerve region is avoided; And Outputting the navigation path that avoids the important nerve region as a predicted path.

2. The radiofrequency ablation navigation device based on multimodal image fusion according to claim 1, characterized in that, The specific steps of iteratively re-planning the navigation path until the important nerve region is avoided are as follows: After determining that the navigation path crosses the important nerve region, insert avoidance nodes around the intersection point of the navigation path and the edge of the important nerve region, and then optimize the curvature of the navigation path using the gradient descent method; Arrange the distances between the optimized navigation path and the important nerve region in ascending order, and then take the Top1 distance as the minimum distance; Judge whether the minimum distance is greater than or equal to a preset distance threshold. If so, re-optimize the curvature of the navigation path and re-judge until the minimum distance is less than the distance threshold, or stop iterating and issue an exception warning when the preset iteration times are reached; Otherwise, judge whether the increase in the total length of the optimized navigation path is greater than a preset tolerance. If so, issue an exception warning; Otherwise, output the optimized navigation path.

3. The radiofrequency ablation navigation device based on multimodal image fusion according to claim 2, characterized in that, The specific steps of setting the important nerve region are as follows; Establish a three-dimensional polygon mesh model according to the nerve distribution coordinates in the standard anatomical database, mark the risk level according to the importance of nerve functions, and mark it in different colors on the three-dimensional polygon mesh model; Screen out the nerve distribution regions with a level greater than a preset level value, and expand them by a preset distance to generate a safety buffer zone that is prohibited from being crossed, forming a standard important nerve region; Obtain the three-dimensional model of the target object marked with the starting point and the ending point, and divide the target region according to the starting point and the ending point on the three-dimensional model; Align the three-dimensional polygon mesh model with the three-dimensional model features, and map the standard nerve region of the three-dimensional polygon mesh model onto the target region of the three-dimensional model. After correction, form the important nerve region of the three-dimensional model of the target person.

4. The radiofrequency ablation navigation device based on multimodal image fusion according to claim 1, wherein The specific steps of constructing the breathing data and the multimodal images into a phase curve representing the mapping relationship between the breathing state and the target points are as follows: Divide the breathing data according to the breathing phase, and associate each frame of multimodal image with a unique breathing phase based on timestamp alignment; Using the multimodal image at the end-expiration phase as the reference coordinate system, align the multimodal images of other phases to the reference coordinate system; Extract the three-dimensional centroid coordinates of the target region of each frame of multimodal image through deep learning; Divide a breathing cycle into N phase intervals according to the breathing waveform amplitude, and then statistically calculate all the three-dimensional centroid coordinates within each phase interval and calculate the average to obtain the phase-displacement mapping table; Fit the phase-displacement mapping table to obtain a phase curve representing the mapping relationship between the breathing state and the target points.

5. The radiofrequency ablation navigation device based on multimodal image fusion according to claim 1, characterized in that, The processor further includes: Obtain the real-time breathing data of the target object; Predict the next-stage first time region through cycle detection of the real-time breathing data and output it as a stable window.

6. The radiofrequency ablation navigation device based on multimodal image fusion according to claim 1, characterized in that, The processor further includes: Obtain the real-time breathing data of the target object; Obtain the current breathing phase through cycle detection of the real-time breathing data; Judge whether the current breathing phase enters the second time region. If so, issue a safety warning.

7. The radiofrequency ablation navigation device based on multimodal image fusion according to claim 1, characterized in that, The radiofrequency ablation navigation device based on multimodal image fusion further includes: A virtual reality device, which is worn on the operator; A virtual reality server, which is connected to the virtual reality device and the processor, and is used to display the three-dimensional model with the predicted path output by the processor through the virtual reality device.

8. The radiofrequency ablation navigation device based on multimodal image fusion according to claim 7, wherein The processor is further used to simulate a three-dimensional model in a breathing emergency state through the breathing data and the three-dimensional model marked with the starting point and the ending point by using a fuzzy logic algorithm, and send the three-dimensional model in the breathing emergency state to the virtual reality server for use as the emergency situation simulation reference data of the virtual reality server; the breathing emergency states include respiratory arrest, tachypnea, and arrhythmia.

9. A radiofrequency ablation navigation system based on multimodal image fusion, characterized in that, It includes: A data acquisition module, which is used to obtain the multimodal images, breathing data of the target object, and the three-dimensional model marked with the starting point and the ending point; A data processing module, which is used to generate a virtual sectional image of the target object through a multi-scale convolutional neural network for the three-dimensional model marked with the starting point and the ending point; the virtual sectional image includes a navigation path from the starting point to the ending point; It is also used to construct the breathing data and the multimodal images into a phase curve representing the mapping relationship between the breathing state and the target points, and obtain a first time region and a second time region through the phase curve; the first time region is the valley value region within the breathing cycle, and the second time region is the peak value region within the breathing cycle; A displacement difference calculation module, which is used to calculate the displacement difference between the target point in the second time region and the ending point in the virtual sectional image; A path judgment module, which is used to judge whether the displacement difference is less than a preset safety threshold. If so, it is determined that the navigation path does not need to be adjusted; Otherwise, the navigation path is re-planned according to the target point in the first time region, and it is judged whether it crosses a preset important nerve region. If so, the navigation path is iteratively re-planned until the important nerve region is avoided; A result output module, which is used to output the navigation path that avoids the important nerve region as a predicted path.

10. A radiofrequency ablation navigation method based on multimodal image fusion, characterized in that, It includes the following steps: Obtain the multi-modal image, respiratory data of the target object, and the three-dimensional model marking the starting point and the ending point; Generate a virtual sectional image of the target object from the three-dimensional model marking the starting point and the ending point through a multi-scale convolutional neural network; the virtual sectional image includes a navigation path from the starting point to the ending point; Construct the respiratory data and the multi-modal image into a phase curve representing the mapping relationship between the respiratory state and the target point, and obtain a first time region and a second time region through the phase curve; wherein, the first time region is the valley region in the respiratory cycle, and the second time region is the peak region in the respiratory cycle; Calculate the displacement difference between the target point in the second time region and the ending point in the virtual sectional image; Judge whether the displacement difference is less than a preset safety threshold. If so, it is determined that the navigation path does not need to be adjusted; Otherwise, re-plan the navigation path according to the target point in the first time region and judge whether it crosses a preset important nerve region. If so, iteratively re-plan the navigation path until the important nerve region is avoided; and Output the navigation path that avoids the important nerve region as a predicted path.

Images