Real-time image registration and path planning system for bronchoscopic surgical navigation

Abstract

The present invention discloses a real-time image registration and path planning system for bronchoscopic surgical navigation, which relates to the technical field of image registration. The system is used to solve the problem that in minimally invasive thoracic and abdominal interventional surgery, organ movement and deformation caused by patient breathing will have a significant impact on the registration accuracy of the navigation system, and the spatial registration of preoperative images and intraoperative patient anatomical structures often causes errors due to mismatch of changes in respiratory phase. The system integrates multi-posture data, comprehensively evaluates the degree of respiratory interference through dual indicators, and uses respiratory cycle-driven deformation prediction to solve the problem that traditional static deformation fields cannot match dynamic breathing. The system also corrects displacement errors through tidal volume observations, and uses registration residuals to inversely optimize B-spline control point weights, so that the deformation field can adapt to the differences in respiratory amplitude and frequency of different patients. The registration error output by the filter is used as a teacher signal for online distillation learning, accurately capturing local residual deformation and chronic changes that are difficult to describe with traditional global models.

Description

Technical Field

[0001] The present invention relates to the technical field of image registration, and more particularly to a real-time image registration and path planning system in bronchoscopic surgery navigation. Background Art

[0002] The real-time image registration and path planning system used in bronchoscopic surgical navigation integrates medical image processing, computer vision, and robotics technologies to improve surgical precision and safety. Real-time image registration aligns medical images taken at different times and angles, ensuring that physicians can accurately visualize the patient's internal structures. It typically relies on techniques such as image feature extraction and matching, and non-rigid registration to accommodate airway deformation. Path planning technology provides optimal trajectory for bronchoscopic surgery, avoiding anatomical structures and reducing surgical risks.

[0003] The existing technology has the following deficiencies:

[0004] During minimally invasive thoracic and abdominal interventional procedures, organ movement and deformation caused by the patient's breathing can significantly affect the registration accuracy of the navigation system. Spatial registration between preoperative images and the patient's anatomical structures during surgery often results in errors due to mismatches in respiratory phase. Traditional methods often assume a constant patient breathing pattern and utilize fixed respiratory gating or breath-holding techniques to reduce the effects of motion. However, once the actual breathing pattern changes, the previously calibrated registration relationship may become invalid, resulting in a sharp increase in intraoperative navigation errors.

[0005] In view of the above problems, the present invention proposes a solution. Summary of the Invention

[0006] In order to overcome the above-mentioned defects of the prior art, an embodiment of the present invention provides a real-time image registration and path planning system for bronchoscopic surgery navigation to solve the problems raised in the above-mentioned background technology.

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] The real-time image registration and path planning system for bronchoscopic surgical navigation includes: an impact assessment module, a reference deformation generation module, a residual deformation learning module, and a fusion and closed-loop correction module, with signal connections between each module;

[0009] The impact assessment module obtains tidal volume and respiratory cycle, monitors the reprojection positioning deviation of surgical instruments, and uses the Pearson correlation coefficient algorithm to calculate the respiratory-registration coupling coefficient index based on the tidal volume fluctuation and the reprojection positioning deviation of surgical instruments. The trigger events of the surgical instrument reprojection positioning deviation exceeding the preset safety threshold are counted and combined with the respiratory cycle to obtain the reprojection deviation trigger frequency index. When any index exceeds the corresponding preset threshold, it is determined that respiratory interference has interfered with registration, and the navigation mode is switched from conventional mode to adaptive mode.

[0010] The reference deformation generation module obtains the respiratory phase angle and phase change rate, and combines the tidal volume with multiple sets of B-spline control points to encapsulate them into the extended Kalman filter state vector. The respiratory cycle advances the respiratory phase prediction, the tidal volume observation value corrects the respiratory displacement error, and the registration reprojection positioning deviation is reversely optimized to continuously output the reference deformation field that changes with respiratory fluctuations.

[0011] The residual deformation learning module intercepts the respiratory waveform and electromagnetic posture sequence, uses the registration error generated by the filter as the teacher signal for online distillation, and predicts the local residual deformation field through a multi-layer time series transformation network;

[0012] The fusion and closed-loop correction module adds the baseline deformation field and the local residual deformation field to generate a total deformation field; if the deformation difference of the total deformation field for two consecutive frames exceeds the preset deformation threshold, the dynamic window smoothing path is triggered only for the area exceeding the preset deformation threshold, and the reprojection positioning deviation of the surgical instrument is continuously monitored. When the reprojection deviation trigger frequency index exceeds the preset threshold for several consecutive frames, the adaptive mode is first downgraded to a pure electromagnetic positioning mode, and then restored to the adaptive mode after additional image correction and update.

[0013] In a preferred embodiment, the method comprises the following steps:

[0014] The tidal volume fluctuation is recorded as the respiratory signal change x(t) and the reprojection positioning deviation of the surgical instrument is recorded as the navigation registration error e(t). The Pearson correlation coefficient between the respiratory signal change and the navigation registration error is calculated and defined as the respiratory-registration coupling coefficient index.

[0015] Monitor the reprojection positioning deviation of surgical instruments. When it exceeds the preset safety threshold, it is defined as a trigger event. The reprojection deviation trigger frequency index is measured by combining the number of trigger events and the respiratory cycle.

[0016] In a preferred embodiment, the method comprises the following steps:

[0017] The respiratory phase angle , phase change rate , tidal volume V and N B-spline control point offsets are encapsulated as an extended Kalman filter state vector, expressed as: ; where k is the current moment, is the respiratory phase angle at the current moment k, is the respiratory phase change rate at the current moment k, is the tidal volume at the current moment k, is the deformation parameter of the i-th B-spline control point at the current time k, and N is the total number of B-spline control points;

[0018] The state transition model of the extended Kalman filter algorithm is used to predict, update and output a continuous reference deformation field that changes with the respiratory phase.

[0019] In a preferred embodiment, the method comprises the following steps:

[0020] The state transition model prediction steps of the extended Kalman filter algorithm are as follows;

[0021] The respiratory phase angle advances according to the phase change rate: ; where k is the time step index of the current moment in the extended Kalman filter algorithm, is the time step;

[0022] Tidal volume is based on a sinusoidal correlation, using the maximum tidal volume of the previous respiratory cycle. Predicted tidal volume;

[0023] The B-spline control point parameters adopt a constant model: ;in, is the deformation parameter of the i-th B-spline control point at the current moment k, is the process noise;

[0024] And update and output the reference deformation field based on the predicted value.

[0025] In a preferred embodiment, the method comprises the following steps:

[0026] The data of a time window of length T is used as input, including the respiratory waveform signal recorded as The tip posture sequence of the instrument obtained by electromagnetic tracking is denoted as , the input sequence set of the neural network is obtained by combining the respiratory waveform signal and the instrument tip posture sequence ;

[0027] Predict local residual deformation through a multi-layer temporal transformation network;

[0028] The registration error generated by the state transition model of the extended Kalman filter algorithm is recorded as , expressed as ;in, The position of the surgical instrument predicted by EKF based on the current deformation field is the tip posture of the instrument obtained by actual electromagnetic tracking, is the registration error output by the extended Kalman filter;

[0029] And combined with the input sequence set of the neural network The most recent sequence in the input is used for prediction ;

[0030] Calculating losses ;

[0031] Gradient descent optimizes the parameters of the multi-layer time series transformation network, distills the residual deformation not processed by the extended Kalman filter algorithm, and outputs the local residual deformation field.

[0032] In a preferred embodiment, the method comprises the following steps:

[0033] Add the reference deformation field and the local residual deformation field to generate the total deformation field;

[0034] If the deformation difference between two consecutive frames of the total deformation field exceeds the preset deformation threshold, the dynamic window smoothing path is triggered for the affected area;

[0035] Keep the paths of unchanged areas unchanged;

[0036] Replan the path in bronchoscopic surgical navigation for the mutation area, ensuring that the starting point connects to the old trajectory and the end point reaches the target under the new deformation;

[0037] Control the path curvature and speed changes in bronchoscopic surgical navigation within a safe range.

[0038] In a preferred embodiment, the method comprises the following steps:

[0039] Continuously monitor the reprojection positioning deviation of surgical instruments. If it exceeds the limit continuously, switch to pure electromagnetic positioning mode;

[0040] New CT images were acquired and registered with the preoperative images, and the system was switched back to adaptive mode.

[0041] The technical effects and advantages of the real-time image registration and path planning system for bronchoscopic surgery navigation of the present invention are as follows:

[0042] It integrates physiological signals such as tidal volume, airway pressure, and respiratory flow with electromagnetic tip posture data, and comprehensively evaluates the degree of respiratory interference through dual indicators: the respiratory-registration coupling coefficient index and the reprojection deviation trigger frequency index. Compared with traditional methods that rely solely on electromagnetic tracking or a single physiological signal, it avoids misjudgment due to the failure of a single signal, and uses respiratory cycle-driven deformation prediction to solve the problem that traditional static deformation fields cannot match dynamic breathing. It also corrects displacement errors through tidal volume observations and uses registration residuals to inversely optimize the weights of B-spline control points, so that the deformation field can adapt to the differences in respiratory amplitude and frequency of different patients, significantly improving the individualized accuracy of deformation modeling. Using the registration error output by the filter as the teacher signal for online distillation learning, it can accurately capture local residual deformations and chronic changes that are difficult to describe with traditional global models. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is a flow chart of the real-time image registration and path planning system in bronchoscopic surgery navigation of the present invention.

[0044] Figure 2 This is a module diagram of the real-time image registration and path planning system in bronchoscopic surgery navigation of the present invention. DETAILED DESCRIPTION

[0045] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0046] Example: See Figure 1-Figure 2 As shown, the present invention discloses a real-time image registration and path planning system for bronchoscopic surgical navigation, including: an influence assessment module, a reference deformation generation module, a residual deformation learning module, and a fusion and closed-loop correction module, and the signals between the modules are connected.

[0047] The impact assessment module obtains tidal volume and respiratory cycle, monitors the reprojection positioning deviation of surgical instruments, and uses the Pearson correlation coefficient algorithm to calculate the respiratory-registration coupling coefficient index based on the tidal volume fluctuation and the reprojection positioning deviation of surgical instruments. The trigger events of the surgical instrument reprojection positioning deviation exceeding the preset safety threshold are counted and combined with the respiratory cycle to obtain the reprojection deviation trigger frequency index. When any index exceeds the corresponding preset threshold, it is determined that respiratory interference has interfered with registration, and the navigation mode is switched from conventional mode to adaptive mode.

[0048] The reference deformation generation module obtains the respiratory phase angle and phase change rate, and combines the tidal volume with multiple sets of B-spline control points to encapsulate them into the extended Kalman filter state vector. The respiratory cycle advances the respiratory phase prediction, the tidal volume observation value corrects the respiratory displacement error, and the registration reprojection positioning deviation is reversely optimized to continuously output the reference deformation field that changes with respiratory fluctuations.

[0049] The residual deformation learning module intercepts the respiratory waveform and electromagnetic posture sequence, uses the registration error generated by the filter as the teacher signal for online distillation, and predicts the local residual deformation field through a multi-layer time series transformation network;

[0050] The fusion and closed-loop correction module adds the baseline deformation field and the local residual deformation field to generate a total deformation field; if the deformation difference of the total deformation field for two consecutive frames exceeds the preset deformation threshold, the dynamic window smoothing path is triggered only for the area exceeding the preset deformation threshold, and the reprojection positioning deviation of the surgical instrument is continuously monitored. When the reprojection deviation trigger frequency index exceeds the preset threshold for several consecutive frames, the adaptive mode is first downgraded to a pure electromagnetic positioning mode, and then restored to the adaptive mode after additional image correction and update.

[0051] In the impact assessment module, tidal volume and respiratory cycle are obtained, and the reprojection positioning deviation of surgical instruments is monitored. The respiratory-registration coupling coefficient index is calculated using the Pearson correlation coefficient algorithm based on the tidal volume fluctuation and the reprojection positioning deviation of surgical instruments. The trigger events of the reprojection positioning deviation of surgical instruments exceeding the preset safety threshold are counted and combined with the respiratory cycle to obtain the reprojection deviation trigger frequency index. When any index exceeds the corresponding preset threshold, it is determined that respiratory interference has interfered with registration, and the navigation mode is switched from conventional mode to adaptive mode. The specific contents include:

[0052] Collects a variety of respiratory-related physiological parameters and navigation sensor data, including the patient's tidal volume and respiratory cycle for each breath, and monitors the reprojection positioning deviation of surgical instruments. By analyzing these data, two key indicators are calculated: the respiration-registration coupling coefficient and the reprojection deviation trigger frequency.

[0053] Within a time window, the tidal volume fluctuation is calculated and recorded as the respiratory signal change x(t) and the positioning deviation of the surgical instrument in the image space is recorded as the navigation registration error e(t). The correlation coefficient K between the respiratory signal change and the navigation registration error and the respiratory-registration coupling coefficient index are defined as the Pearson correlation coefficient between the two. Its value range is -1 to 1, with larger absolute values ​​indicating a more significant impact of respiratory changes on the registration error. When K exceeds the preset threshold, it indicates that the registration error is significantly coupled with respiration. A positive correlation indicates a trend of increasing error during inhalation and decreasing error during exhalation, while a negative correlation indicates the opposite. When respiratory motion significantly affects registration, the value will deviate significantly from zero.

[0054] Monitor the deviation between the reprojected position of surgical instruments on intraoperative images and the preoperative planned path. A trigger event is defined as any deviation exceeding a preset safety threshold. If trigger events occur too frequently, it means that the current registration is no longer able to accurately reflect the patient's anatomical movements.

[0055] The reprojection error triggering frequency is measured by combining the number of triggering events and the respiratory cycle. When the triggering frequency exceeds the preset threshold, it is considered that the registration error is triggered too frequently by respiratory drive;

[0056] The two aforementioned metrics automatically assess the degree to which the current respiratory condition interferes with navigation registration. When either metric exceeds its threshold, the system determines that respiration is significantly affecting registration. The current respiratory motion is significantly impacting registration accuracy. Without compensation, the surgical instrument may deviate from its intended path, risking accidental injury to healthy tissue. The system then switches to adaptive mode, preparing to initiate the next step of the baseline deformation generation process to dynamically address displacement caused by respiration. If the metric remains within the threshold, no additional compensation is deemed necessary, and the current registration scheme is maintained to avoid unnecessary computational overhead.

[0057] In the reference deformation generation module, the respiratory phase angle and phase change rate are obtained and combined with the tidal volume and multiple sets of B-spline control points to encapsulate them into the extended Kalman filter state vector. The respiratory phase prediction is advanced by the respiratory cycle, the tidal volume observation value corrects the respiratory displacement error, and the B-spline control point weights are reversely optimized based on the registration reprojection positioning deviation to continuously output the reference deformation field that changes with respiratory fluctuations. The specific contents include:

[0058] The extended Kalman filter uses prior knowledge of respiratory periodicity and real-time ventilator data to predict and correct lung deformation, thereby outputting a baseline deformation model that changes with the respiratory phase. The state vector x of the extended Kalman filter is defined, which integrates the phase information of the respiratory motion and the deformation field parameters:

[0059] Respiratory phase angle and phase change rate : Use an angle to represent the position of the current respiratory cycle, the respiratory phase angle It is defined as 0 at the end of exhalation and 0 at the end of inspiration. , then return to the end of exhalation To complete one cycle, the phase change rate It reflects the respiratory rate, and incorporating it into the state can capture short-term changes in respiratory rate;

[0060] The tidal volume V represents the tidal volume corresponding to the progress of the current respiratory cycle. V is considered a function that changes with the phase and is expressed as a percentage of the cumulative ventilation volume of the current breath. Incorporating V into the state helps the filter adjust the deformation amplitude according to the actual ventilation changes. If the patient inhales deeply in this cycle, that is, the tidal volume is high, the displacement amplitude of each point in the lung may increase accordingly.

[0061] Select several control points to generate a smooth deformation field using a cubic B-spline control point function. The offset of the control points is used as part of the state vector to describe the shape of the lung baseline deformation field. N control points are selected, each with three coordinate offset components. These offset values ​​or related parameters are then incorporated into the state vector.

[0062] The advantage of B-spline control points is that they can approximate complex deformations with fewer parameters, and the change of control points will affect the deformation of their neighborhood, meeting the continuous and smooth requirements of organ deformation.

[0063] Therefore, the state vector of the extended Kalman filter is expressed as: ; where k is the current moment, is the respiratory phase angle at the current moment k, is the respiratory phase change rate at the current moment k, is the tidal volume at the current moment k, is the deformation parameter of the i-th B-spline control point at the current time k, and N is the total number of B-spline control points;

[0064] The state transition model is used to describe the evolution of the state over time, and the following predictions are made using the approximate periodicity of breathing: phase angle according to Advancement, that is: ,Will Set to be basically constant, reflecting the slow drift of respiratory rate, tidal volume state V and Correlation, according to the respiratory tidal volume with the phase of the sinusoidal change, through the last filtering estimate Calculate the approximate tidal volume at the next moment. , since the deformation pattern is assumed to be constant when there is no observation correction, a constant model is adopted: ;in, is the deformation parameter of the i-th B-spline control point at the current moment k, is process noise, capturing possible slow deformation drift;

[0065] Tidal volume measurement values ​​were used as observation , thus constructing the observation equation , meaning that the tidal volume in the state should be consistent with the value reported by the ventilator. Similarly, airway pressure and flow are related to or The filter is modified according to the actual respiratory dynamics based on the state components such as the rate of change of

[0066] The difference between the electromagnetically tracked surgical instrument tip pose and the image-registered position is used as the observation to map the state to the expected instrument position error: the filter maintains a current deformation field, which is given by Determine, given the position of a surgical instrument in the patient coordinate system, map the current deformation field to the preoperative image coordinates, and predict the theoretical corresponding position of the instrument in the image;

[0067] If there is a difference between the actual electromagnetically tracked instrument position and the image-predicted position, i.e., the registration error, this difference is used as an observation to correct the control point parameters. Or the estimate of the overall tidal volume V, which is compared with the device position identified in the actual image to obtain the error ;

[0068] The extended Kalman filter adjusts the state by minimizing the error, thereby correcting the deformation field to make it more consistent with the actual anatomical position under the current breathing;

[0069] Through the above state transition and observation model, EKF performs prediction and update in each fixed cycle: the prediction step uses the last state estimate Computational priors , the update step obtains the current observation and corrects the state to obtain the posterior ;

[0070] As time progresses, the filter continuously outputs state estimates, the most important of which are the corresponding full-lung baseline deformation field and the current respiratory phase, reflecting the approximate deformation of the lungs according to the current respiratory periodic motion.

[0071] EKF provides a low-frequency updated, stable and smooth deformation background: it ensures the continuity and rationality of the deformation based on the prior model, and uses real-time data to gradually correct the model parameters, so that even if the patient's breathing amplitude or frequency drifts slowly, it can track and adapt.

[0072] In the residual deformation learning module, the respiratory waveform and electromagnetic posture sequence are intercepted, and the registration error generated by the filter is used as the teacher signal for online distillation. The local residual deformation field is predicted through a multi-layer time series transformation network. The specific content includes:

[0073] The reference deformation field provides an estimate of overall respiratory motion. However, due to individual anatomical differences, nonlinear deformation characteristics, and aperiodic variations, residual components of actual organ deformation and registration errors may remain uncaptured by the reference field. Therefore, a temporal transformation neural network is introduced to learn and predict these residual deformations—the difference between the reference and true deformation fields. The neural network takes a recent series of respiratory and navigation data as input and outputs subtle corrections to the current local deformation. Through online learning, it continuously adjusts its parameters to accommodate long-term respiratory pattern evolution and chronic drift.

[0074] Input sequence: Use a time window of length T as input, including the respiratory waveform signal recorded as The tip posture sequence of the instrument obtained by electromagnetic tracking is denoted as , the input sequence set of the neural network is obtained by combining the respiratory waveform signal and the instrument shortening posture sequence ;

[0075] After normalizing and differencing the respiratory waveform, the instrument position sequence is also expressed as a deviation sequence relative to the baseline deformation prediction position , and then align the two in time to form a multi-channel sequence signal;

[0076] Mapping T-length sequences into high-dimensional feature vectors captures the dynamic relationship between breathing pattern changes and instrument movement. After multiple layers are stacked, the network internally forms a representation of recent breathing trends.

[0077] The network finally outputs the predicted local residual deformation field, which is the deformation correction value for a specific area or specific coordinate, so that the network outputs a residual field with the same resolution as the control point grid. , represents the displacement vector that needs to be added or subtracted at each position based on the reference deformation field;

[0078] An online distillation method based on teacher signals is used to obtain training signals from real-time data. The registration error generated by the extended Kalman filter is treated as a teacher signal and continuously fed into the neural network for learning.

[0079] Specifically, each time the EKF is updated, the registration error under the current reference deformation field is obtained, which reflects the deformation part that has not been explained by the reference model. Therefore, it can be approximately regarded as a manifestation of the true residual deformation, which is expressed as: ;in, The position of the surgical instrument predicted by EKF based on the current deformation field is the tip posture of the instrument obtained by actual electromagnetic tracking, is the registration error output by the extended Kalman filter, and the error As the output label of the training sample, combined with the input sequence set of the neural network The most recent sequence in the input is used for prediction , and calculate the loss ;

[0080] Through online gradient descent, the network parameters are gradually optimized so that its output approaches the error compensation value measured by the EKF. The knowledge that the EKF fails to process is "distilled" and given to the neural network, giving it the ability to predict these residuals.

[0081] Breathing has chronic pattern drift and non-periodic behavior, requiring not only rapid response to short-term changes but also adaptation to long-term distribution changes. Therefore, a dual strategy is adopted for the learning rate of the neural network: within a short time window, a certain learning rate is retained to quickly learn new error patterns; while over longer time scales, the learning rate is gradually reduced to integrate and fine-tune historical data to avoid overfitting to temporary anomalies.

[0082] A certain length of historical sequences and corresponding errors are saved in the background. Through playback or experience replay technology, the network is prevented from forgetting the previously learned rules. This ensures that even if the patient's breathing changes from periodic to irregular, or the internal organs' posture changes slowly due to adjustment of the bed angle in the middle of the operation, the network can gradually correct its own predictions to provide reliable residual compensation.

[0083] In the fusion and closed-loop correction module, the baseline deformation field and the local residual deformation field are added to generate a total deformation field. If the deformation difference of the total deformation field exceeds the preset deformation threshold for two consecutive frames, the dynamic window smoothing path is triggered only for the area exceeding the preset deformation threshold. At the same time, the reprojection positioning deviation of the surgical instrument is continuously monitored. If the reprojection deviation trigger frequency indicator exceeds the preset threshold for several consecutive frames, the adaptive mode is first downgraded to the pure electromagnetic positioning mode. After additional image correction and update, it is restored to the adaptive mode. The specific contents include:

[0084] The final total deformation field is obtained by fusing the two-step prediction of the baseline deformation and the residual deformation for navigation. At the same time, closed-loop monitoring and necessary corrections are implemented. This includes three key elements: deformation fusion, local dynamic replanning, and mode switching and correction.

[0085] Deformation fusion: The baseline deformation field generated by the extended Kalman filter is combined with the Residual deformation predicted by neural network Add together to get the total deformation field ;

[0086] Local dynamic replanning: Although the fusion of baseline and residual usually evolves continuously and smoothly, in some cases, the total deformation field estimated between two consecutive frames may differ significantly. If the deformation difference between the two frames exceeds the preset deformation threshold, especially when the change is abrupt in a local area, direct application may cause the navigation display or robot execution path to jump, which is not conducive to surgical safety.

[0087] To this end, the present invention proposes to enable a local dynamic window algorithm in this case to perform path replanning and smoothing on the affected area, and the preset deformation threshold can be set according to surgical requirements;

[0088] When a significant change in the total deformation field is detected in a certain area, the target position is not immediately changed. Instead, a small-scale optimization is initiated in that area. The path in the surrounding unchanged areas remains unchanged, and a new feasible path is replanned for the local area with the sudden change, so that its starting point connects to the old trajectory and its end point reaches the target under the new deformation. At the same time, the path curvature and speed changes are controlled within a safe range.

[0089] Mode switching and correction: While the deformation field is being updated and the path is being adjusted, safety indicators such as reprojection error are continuously monitored. If the navigation reprojection error continues to exceed the threshold, or if the deformation field changes abnormally, even after the aforementioned baseline + residual fusion and local smoothing, a higher-level correction mechanism is triggered.

[0090] This includes two levels: first, automatically switching to pure electromagnetic positioning mode, that is, temporarily ignoring the position information provided by image registration, relying solely on electromagnetic tracking to directly guide surgical instruments, ensuring that in extreme cases, at least the position of the instruments can be guided according to the actual measurements of the sensors. Second, a series of CBCT images are acquired to reconstruct a new image of the patient's current anatomy. If necessary, the surgical team is reminded to obtain a new CBCT image under safe conditions and automatically update the registration with the preoperative image. The new image correction can correct the model deviation caused by long-term drift or accidental movement, and then switch back to the image + electromagnetic fusion navigation mode to re-enter the adaptive process;

[0091] The entire closed-loop process ensures that even in the worst case scenario, there are redundant safety mechanisms to maintain navigation reliability.

[0092] The above formulas are all dimensionless and numerical calculations. The formulas are obtained by collecting a large amount of data and performing software simulation to obtain the most recent real situation. The preset parameters in the formulas are set by technicians in this field according to actual conditions.

[0093] The above embodiments may be implemented in whole or in part through software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments may be implemented in whole or in part in the form of a computer program product.

[0094] Those skilled in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application of the technical solution and the invention constraints. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0095] In addition, each functional module in each embodiment of the present application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0096] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0097] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A real-time image registration and path planning system for bronchoscopic surgical navigation, characterized in that: include: Impact assessment module, baseline deformation generation module, residual deformation learning module, and fusion and closed-loop correction module, with signal connections between each module; The impact assessment module obtains tidal volume and respiratory cycle, monitors the reprojection positioning deviation of surgical instruments, and uses the Pearson correlation coefficient algorithm to calculate the respiratory-registration coupling coefficient index based on the tidal volume fluctuation and the reprojection positioning deviation of surgical instruments. The trigger events of the surgical instrument reprojection positioning deviation exceeding the preset safety threshold are counted and combined with the respiratory cycle to obtain the reprojection deviation trigger frequency index. When any index exceeds the corresponding preset threshold, it is determined that respiratory interference has interfered with registration, and the navigation mode is switched from conventional mode to adaptive mode. The reference deformation generation module obtains the respiratory phase angle and phase change rate, and combines the tidal volume with multiple sets of B-spline control points to encapsulate them into the extended Kalman filter state vector. The respiratory cycle advances the respiratory phase prediction, the tidal volume observation value corrects the respiratory displacement error, and the registration reprojection positioning deviation is reversely optimized to continuously output the reference deformation field that changes with respiratory fluctuations. The residual deformation learning module intercepts the respiratory waveform and electromagnetic posture sequence, uses the registration error generated by the filter as the teacher signal for online distillation, and predicts the local residual deformation field through a multi-layer time series transformation network; The fusion and closed-loop correction module adds the baseline deformation field and the local residual deformation field to generate a total deformation field; if the deformation difference of the total deformation field for two consecutive frames exceeds the preset deformation threshold, the dynamic window smoothing path is triggered only for the area exceeding the preset deformation threshold, and the reprojection positioning deviation of the surgical instrument is continuously monitored. When the reprojection deviation trigger frequency index exceeds the preset threshold for several consecutive frames, the adaptive mode is first downgraded to a pure electromagnetic positioning mode, and then restored to the adaptive mode after additional image correction and update.

2. The real-time image registration and path planning system for bronchoscopic surgery navigation according to claim 1, characterized in that: The tidal volume fluctuation is recorded as the respiratory signal change x(t) and the reprojection positioning deviation of the surgical instrument is recorded as the navigation registration error e(t). The Pearson correlation coefficient between the respiratory signal change and the navigation registration error is calculated and defined as the respiratory-registration coupling coefficient index. Monitor the reprojection positioning deviation of surgical instruments. When it exceeds the preset safety threshold, it is defined as a trigger event. The reprojection deviation trigger frequency index is measured by combining the number of trigger events and the respiratory cycle.

3. The real-time image registration and path planning system for bronchoscopic surgery navigation according to claim 1, characterized in that: The respiratory phase angle , phase change rate , tidal volume V and N B-spline control point offsets are encapsulated as an extended Kalman filter state vector, expressed as: ; where k is the current moment, is the respiratory phase angle at the current moment k, is the respiratory phase change rate at the current moment k, is the tidal volume at the current moment k, The deformation parameter of the i-th B-spline control point at the current time k, N is the total number of B-spline control points; The state transition model of the extended Kalman filter algorithm is used to predict, update and output a continuous reference deformation field that changes with the respiratory phase.

4. The real-time image registration and path planning system for bronchoscopic surgery navigation according to claim 3, characterized in that: The state transition model of the extended Kalman filter algorithm includes: The state transition model prediction steps of the extended Kalman filter algorithm are as follows; The respiratory phase angle advances according to the phase change rate: ; where k is the time step index of the current moment in the extended Kalman filter algorithm, is the time step; Tidal volume is based on a sinusoidal correlation, using the maximum tidal volume of the previous respiratory cycle. Predicted tidal volume; The B-spline control point parameters adopt a constant model: ;in, is the deformation parameter of the i-th B-spline control point at the current moment k, is the process noise; And update and output the reference deformation field based on the predicted value.

5. The real-time image registration and path planning system for bronchoscopic surgery navigation according to claim 1 is characterized in that ； The data of a time window of length T is used as input, including the respiratory waveform signal recorded as The tip posture sequence of the instrument obtained by electromagnetic tracking is denoted as , the input sequence set of the neural network is obtained by combining the respiratory waveform signal and the instrument tip posture sequence ; Predict local residual deformation through a multi-layer temporal transformation network; The registration error generated by the state transition model of the extended Kalman filter algorithm is recorded as , expressed as: ;in, The position of the surgical instrument predicted by EKF based on the current deformation field is the tip posture of the instrument obtained by actual electromagnetic tracking, is the registration error output by the extended Kalman filter; And combined with the input sequence set of the neural network The most recent sequence in the input is used for prediction ; Calculating losses ; Gradient descent optimizes the parameters of the multi-layer time series transformation network, distills the residual deformation not processed by the extended Kalman filter algorithm, and outputs the local residual deformation field.

6. The real-time image registration and path planning system for bronchoscopic surgery navigation according to claim 1, characterized in that: Add the reference deformation field and the local residual deformation field to generate the total deformation field; If the deformation difference between two consecutive frames of the total deformation field exceeds the preset deformation threshold, the dynamic window smoothing path is triggered for the affected area; Keep the paths of unchanged areas unchanged; Replan the path in bronchoscopic surgical navigation for the mutation area, ensuring that the starting point connects to the old trajectory and the end point reaches the target under the new deformation; Control the path curvature and speed changes in bronchoscopic surgical navigation within a safe range.

7. The real-time image registration and path planning system for bronchoscopic surgery navigation according to claim 6, characterized in that: Continuously monitor the re-projection positioning deviation of surgical instruments. If it exceeds the limit continuously, switch to pure electromagnetic positioning mode; New CT images were acquired and registered with the preoperative images, and the system was switched back to adaptive mode.

Images