Four-dimensional optical flow field real-time navigation system for pulmonary nodules and method thereof

Abstract

The present application relates to the technical field of medical image navigation, in particular to a lung nodule four-dimensional optical flow field real-time navigation system and method thereof, the system comprising a data acquisition module, a micro-region optical flow calculation module, a high-dimensional tensor analysis module, a nonlinear dynamics prediction module, a navigation fusion module and a surgical navigation module, the data acquisition module acquiring a CBCT image sequence and respiratory monitoring point motion information; the micro-region optical flow calculation module establishing a dynamic mapping relationship based on differential topology principles, generating a lung tissue motion displacement field; the high-dimensional tensor analysis module constructing a five-dimensional tensor structure and performing singular value optimization, extracting essential motion patterns; the nonlinear dynamics prediction module establishing a chaotic system model of respiratory motion, predicting future motion trajectories of lung nodules; the navigation fusion module integrating multi-modal data to generate a four-dimensional optical flow field; the surgical navigation module providing accurate real-time navigation information, the present application improving navigation accuracy to 1-3mm and being able to predict lung nodule motion trajectories within 2 seconds in the future.

Description

Technical Field

[0001] This invention relates to the field of medical image navigation technology, specifically to a real-time four-dimensional optical flow field navigation system and method for lung nodules, which is particularly suitable for precise navigation of minimally invasive lung surgery under respiratory conditions. Background Technology

[0002] Pulmonary nodules are common lung lesions in clinical practice, and early detection and removal are key to improving patient survival rates. With the development of minimally invasive techniques, thoracoscopic pulmonary nodule resection has become the mainstream surgical method. However, during the operation, lung deformation and displacement caused by the patient's respiratory movements seriously affect the accuracy of surgical navigation, especially for early pulmonary nodules with a diameter of less than 1 cm, where existing navigation technology cannot provide sufficient accuracy.

[0003] Traditional lung surgery navigation systems primarily rely on preoperative CT images and intraoperative localization techniques, such as electromagnetic navigation bronchoscopy, intraoperative ultrasound, and intraoperative CT. While these techniques can provide relatively accurate localization under static conditions, they often encounter the following problems when dealing with dynamic targets under respiratory motion: First, the localization accuracy is insufficient, typically within the range of 5-10 mm, which cannot meet the precise localization requirements of small nodules; second, there is a lack of real-time tracking capability for respiratory motion, leading to target position deviation during surgery; and third, the inability to predict the trajectory of lung nodules forces surgical manipulation to passively follow their movement, increasing the difficulty and time of the surgery.

[0004] In existing technologies, optical flow is a commonly used motion estimation algorithm that can calculate the displacement field of a target in an image sequence. However, traditional optical flow algorithms are mostly based on the assumption of constant brightness, making it difficult to cope with tissue deformation and brightness changes caused by respiration. In addition, existing navigation systems generally lack the ability to accurately model and predict complex lung movements, and therefore cannot provide forward-looking navigation information during surgery.

[0005] Therefore, there is an urgent need to develop a navigation system that can accurately track and predict pulmonary nodules while breathing, providing more accurate and reliable navigation support for minimally invasive lung surgery. Summary of the Invention

[0006] The purpose of this invention is to provide a real-time four-dimensional optical flow field navigation system and method for pulmonary nodules. By innovatively integrating micro-area optical flow field dynamic topological mapping, high-dimensional tensor analysis and nonlinear dynamic prediction technology, it can achieve accurate tracking and motion prediction of pulmonary nodules under respiratory conditions, providing high-precision, real-time and forward-looking navigation support for minimally invasive lung surgery.

[0007] This invention proposes a real-time four-dimensional optical flow field navigation system for lung nodules, comprising:

[0008] The data acquisition module is used to acquire lung CBCT image sequences and respiratory monitoring point motion information;

[0009] The micro-area optical flow calculation module, connected to the data acquisition module, is used to divide the CBCT image sequence into micro-area grids and establish a dynamic topological mapping relationship based on the micro-area grids to generate a lung tissue motion displacement field.

[0010] The high-dimensional tensor analysis module, connected to the micro-area optical flow calculation module, is used to construct the motion displacement field into a high-dimensional tensor structure, perform tensor decomposition and singular value optimization, and extract the essential motion pattern.

[0011] The nonlinear dynamics prediction module, connected to the high-dimensional tensor analysis module, is used to establish a nonlinear dynamics model of respiratory motion based on the essential motion pattern and the motion information of the respiratory monitoring point, and to predict the future motion trajectory of the lung nodule.

[0012] The navigation fusion module, connected to both the micro-area optical flow calculation module and the nonlinear dynamics prediction module, is used to fuse the motion displacement field and the future motion trajectory of the lung nodule to generate a four-dimensional optical flow field; and

[0013] The surgical navigation module, connected to the navigation fusion module, is used to generate real-time surgical navigation information based on the four-dimensional optical flow field and the position information of the surgical tools.

[0014] Preferably, the data acquisition module includes:

[0015] Dual-head CBCT imaging unit is used to perform non-coplanar imaging of the patient's lungs and acquire two-dimensional image data from different angles;

[0016] The electromagnetic navigation monitoring unit is used to acquire the three-dimensional position coordinates of the breathing monitoring point in real time through electromagnetic sensors; and

[0017] The spatiotemporal synchronization unit is connected to the dual-head CBCT imaging unit and the electromagnetic navigation monitoring unit, respectively, and is used to perform time synchronization processing on the two-dimensional image data and the three-dimensional position coordinates to generate a time-aligned multimodal data stream.

[0018] Preferably, the dual-head CBCT imaging unit uses two imaging heads to simultaneously image the lungs from different angles, and fuses the two-dimensional images from different angles within the same exposure cycle using a sub-pixel-level cascaded interpolation reconstruction algorithm to form a super-resolution three-dimensional image.

[0019] Preferably, the micro-area optical flow calculation module includes:

[0020] The micro-region division unit is used to divide the CBCT image sequence into cubic micro-regions with a side length of 1 mm, and each micro-region contains 8 vertices;

[0021] A topology relationship construction unit, connected to the micro-region partitioning unit, is used to establish adjacency relationships and topology feature vectors for the micro-region mesh;

[0022] A dynamic homeomorphism mapping unit, connected to the topology construction unit, is used to establish a continuous mapping function between micro-regions in adjacent time frames based on the principle of differential topology; and

[0023] The probability adaptive correction unit, connected to the dynamic homeomorphism mapping unit, is used to adaptively correct the continuous mapping function according to the Bayesian probability framework to generate an accurate motion displacement field.

[0024] Preferably, the high-dimensional tensor analysis module includes:

[0025] Tensor construction unit, used to organize the motion displacement field into a five-dimensional tensor structure containing spatial coordinates, time and eigenvalues;

[0026] Tensor decomposition unit, connected to the tensor construction unit, is used to perform high-order singular value decomposition on the five-dimensional tensor structure to separate information in each dimension.

[0027] A low-rank optimization unit, connected to the tensor decomposition unit, is used to determine the optimal rank based on the importance of singular values, thereby achieving a low-rank approximation representation; and

[0028] The differential geometry analysis unit, connected to the low-rank optimization unit, is used to define the Riemann metric on the tensor manifold and calculate the curvature characteristics of the motion trajectory.

[0029] Preferably, the nonlinear dynamics prediction module includes:

[0030] The state-space reconstruction unit is used to reconstruct the phase-space representation of respiratory motion based on the delayed coordinate method.

[0031] The dynamic characteristic analysis unit, connected to the state space reconstruction unit, is used to calculate the Lyapunov exponent and construct the Poincaré map to analyze the chaotic characteristics of respiratory motion.

[0032] A recurrent neural network unit, connected to the dynamic characteristic analysis unit, is used to establish a nonlinear dynamic prediction model by fusing convolutional feature extraction and temporal memory structures; and

[0033] A multi-scale prediction unit, connected to the recurrent neural network unit, is used to generate short-term (0.1-0.5 seconds), medium-term (0.5-2 seconds), and long-term (>2 seconds) predictions of lung nodule motion trajectories.

[0034] Preferably, the recurrent neural network unit includes:

[0035] Convolutional layers are used to extract spatial features from the input data;

[0036] A Long Short-Term Memory (LSTM) layer, connected to the convolutional layer, is used to capture temporal dependencies;

[0037] A deconvolutional layer, connected to the long short-term memory layer, is used to reconstruct the prediction result; and

[0038] A residual connection structure connects the convolutional layer and the deconvolutional layer to ensure stable gradient propagation.

[0039] Preferably, the navigation fusion module includes:

[0040] A multimodal data fusion unit is used to fuse the motion displacement field and the future motion trajectory of the lung nodule based on an uncertainty weighting method;

[0041] A four-dimensional optical flow field construction unit, connected to the multimodal data fusion unit, is used to organize the fused data into a four-dimensional optical flow field structure; and

[0042] An adaptive correction unit, connected to the four-dimensional optical flow field construction unit, is used to compare the prediction results with the observation data in real time and dynamically adjust the parameters of the four-dimensional optical flow field.

[0043] Preferably, the surgical navigation module includes:

[0044] Surgical tool positioning unit is used to acquire the spatial position and orientation information of the surgical tool;

[0045] A navigation path planning unit, connected to the surgical tool positioning unit, is used to calculate the optimal surgical path and intervention timing based on the four-dimensional optical flow field.

[0046] A 3D visualization unit, connected to the navigation path planning unit, is used for real-time rendering of lung structures, lung nodule locations, predicted trajectories, and surgical tool positions; and

[0047] An interactive control unit, connected to the three-dimensional visualization unit, is used to provide a gesture control interface and view adjustment functions to maintain a sterile surgical field.

[0048] A real-time navigation method for pulmonary nodules using a four-dimensional optical flow field includes the following steps:

[0049] Acquire lung CBCT image sequences and respiratory monitoring point motion information;

[0050] The CBCT image sequence is divided into micro-region grids, and a dynamic topological mapping relationship is established based on the micro-region grids to generate a lung tissue motion displacement field;

[0051] The motion displacement field is constructed as a high-dimensional tensor structure, and tensor decomposition and singular value optimization are performed to extract the essential motion pattern.

[0052] Based on the essential motion pattern and the motion information of the respiratory monitoring point, a nonlinear dynamic model of respiratory motion is established to predict the future motion trajectory of the lung nodule.

[0053] By fusing the motion displacement field and the future motion trajectory of the lung nodule, a four-dimensional optical flow field is generated; and

[0054] Real-time surgical navigation information is generated based on the four-dimensional optical flow field and the position information of surgical tools.

[0055] The beneficial effects of this invention include:

[0056] 1. Significantly improved navigation accuracy: The accuracy of traditional navigation systems has been improved from 5-10mm to 1-3mm, meeting the requirements for precise positioning of small nodules.

[0057] 2. Enhanced real-time tracking capability: Through micro-area optical flow field dynamic topology mapping technology, the system can track the positional changes of lung nodules in real time during breathing, with a tracking success rate of over 95%.

[0058] 3. Motion prediction function: Based on a nonlinear dynamic prediction model, the system can predict the motion trajectory of lung nodules within the next 2 seconds, providing prospective navigation information for surgery and reducing surgical waiting time by more than 40%.

[0059] 4. Significantly improved adaptability: The system's ability to adapt to interference factors such as irregular breathing and coughing has been improved by 85%, ensuring navigation reliability under various breathing conditions.

[0060] 5. Multimodal data fusion: By integrating CBCT image data and electromagnetic navigation data, a complementary verification mechanism is formed, which improves the system reliability by more than 90% and can still maintain navigation function even if a single data source fails. Attached Figure Description

[0061] Figure 1 This is a block diagram of the overall structure of the four-dimensional optical flow field real-time navigation system for lung nodules of the present invention;

[0062] Figure 2 This is a schematic diagram of the micro-area optical flow calculation module of the present invention;

[0063] Figure 3 This is a flowchart of the high-dimensional tensor analysis module of the present invention;

[0064] Figure 4 This is a schematic diagram of the nonlinear dynamics prediction module of the present invention;

[0065] Figure 5 This is a schematic diagram of the data flow of the navigation fusion module of the present invention;

[0066] Figure 6 This is an example diagram showing the interface of the surgical navigation module of the present invention;

[0067] Figure 7 This is a flowchart of the real-time navigation method for four-dimensional optical flow field of lung nodules according to the present invention. Detailed Implementation

[0068] Please refer to the attached document. Figure 1-7 The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0069] Example 1

[0070] Reference Figure 1 The real-time navigation system for four-dimensional optical flow field of lung nodules provided by the present invention includes: a data acquisition module 1, a micro-area optical flow calculation module 2, a high-dimensional tensor analysis module 3, a nonlinear dynamics prediction module 4, a navigation fusion module 5, and a surgical navigation module 6.

[0071] The data acquisition module 1 is used to acquire lung CBCT image sequences and respiratory monitoring point motion information. In one embodiment of the present invention, the data acquisition module 1 includes a dual-head CBCT imaging unit 11, an electromagnetic navigation monitoring unit 12, and a spatiotemporal synchronization unit 13. The dual-head CBCT imaging unit 11 uses two imaging heads to simultaneously image the lungs from different angles. The preferred operating voltage is 120kV, and the operating current can be adjusted to 8-16mA depending on the size of the lung nodules. The exposure time is 0.5-0.75s. Furthermore, the dual-head CBCT imaging unit 11 adopts a non-coplanar imaging principle, acquiring images from different angles within the same exposure cycle (typically 0.5-0.75s), significantly improving image acquisition efficiency.

[0072] The electromagnetic navigation monitoring unit 12 acquires the three-dimensional position coordinates of the respiratory monitoring point in real time through an electromagnetic sensor implanted in the patient's body. Preferably, the sampling frequency of the electromagnetic sensor is 100Hz, with a spatial accuracy better than 0.5mm, enabling it to capture subtle changes in respiratory motion. The spatiotemporal synchronization unit 13 is responsible for performing time synchronization processing on the CBCT image data and electromagnetic navigation data to ensure data temporal consistency, with a time synchronization accuracy better than 1ms.

[0073] The micro-area optical flow calculation module 2 is connected to the data acquisition module 1. It is used to divide the CBCT image sequence into micro-area grids and establish dynamic topological mapping relationships based on these grids to generate a lung tissue motion displacement field. For example... Figure 2 As shown, the micro-area optical flow calculation module 2 includes a micro-area partitioning unit 21, a topology relationship construction unit 22, a dynamic homeomorphism mapping unit 23, and a probability adaptive correction unit 24.

[0074] The micro-region segmentation unit 21 divides the CBCT image sequence into cubic micro-regions with a side length of 1 mm, each containing 8 vertices. This fine segmentation method enables the system to achieve sub-pixel accuracy, far superior to traditional pixel-level optical flow algorithms. The topology relation construction unit 22 establishes adjacency relationships and topological feature vectors for the micro-region mesh. In one embodiment of the invention, the topological feature vector includes the center coordinates, gray-level mean, gray-level gradient, and texture features of the micro-region, used for subsequent micro-region matching and tracking.

[0075] The dynamic homeomorphism mapping unit 23 establishes a continuous mapping function between micro-regions in adjacent time frames based on the principle of differential topology. This unit uses the following mapping function to calculate the correspondence between micro-regions:

[0076] ,

[0077] This represents the mapping function from time t to time t+1. and Let represent the micro-region locations at times t and t+1, respectively. This mapping function satisfies bijectivity, continuity, and topology preservation, and can accurately describe the tissue deformation process.

[0078] The probabilistic adaptive correction unit 24 adaptively corrects the continuous mapping function according to the Bayesian probability framework to generate an accurate motion displacement field. This unit uses the following Bayesian formula for correction:

[0079] ,

[0080] in, This represents the posterior probability estimate of state X after observing data Y. Represents the observation model, Represents the prior probability. This represents the marginal probability. In practical applications, X represents the actual motion state of the micro-region, and Y represents the observed grayscale changes and electromagnetic navigation data. By continuously updating the posterior probability, the system can adaptively correct the motion estimation and improve tracking accuracy.

[0081] The high-dimensional tensor analysis module 3 is connected to the micro-area optical flow calculation module 2, and is used to construct the motion displacement field into a high-dimensional tensor structure, perform tensor decomposition and singular value optimization, and extract the essential motion pattern. For example... Figure 3 As shown, the high-dimensional tensor analysis module 3 includes a tensor construction unit 31, a tensor decomposition unit 32, a low-rank optimization unit 33, and a difference geometry analysis unit 34.

[0082] Tensor construction unit 31 organizes the motion displacement field into a five-dimensional tensor structure containing spatial coordinates, time, and eigenvalues. Preferably, this five-dimensional tensor can be represented as... ,in , , These represent the x, y, and z dimensions of space, respectively. Indicates the time dimension. This represents the feature dimension (such as grayscale value, density value, etc.). Through this high-dimensional representation, the system can capture complex spatiotemporal relationships.

[0083] Tensor decomposition unit 32 performs High-Order Singular Value Decomposition (HOSVD) on the five-dimensional tensor structure to separate information from each dimension. The mathematical expression for HOSVD is:

[0084] ,

[0085] in, Represents the core tensor. The factor matrix representing the nth pattern. This represents the tensor-matrix product along the nth pattern. Through this decomposition, the system can extract the main change patterns in each dimension.

[0086] The low-rank optimization unit 33 determines the optimal rank based on the importance of singular values, thus achieving a low-rank approximate representation. This unit solves for the optimal low-rank representation through the following optimization problem:

[0087] ,

[0088] in, Describing the Frobenius norm, Tensor The rank is denoted by r, which represents the preset maximum rank. In practical applications, the optimal rank is usually determined by the energy retention ratio, preferably retaining more than 90% of the energy while significantly reducing computational complexity.

[0089] Differential geometry analysis unit 34 defines a Riemann metric on the tensor manifold to calculate the curvature properties of the motion trajectory. This unit first defines the metric tensor on the manifold. Then calculate the Christoffel notation. and curvature tensor This allows the system to obtain the geometric characteristics of the motion trajectory. By analyzing curvature changes, the system can identify key events such as acceleration, deceleration, and turning during motion, providing important information for prediction.

[0090] The nonlinear dynamics prediction module 4 is connected to the high-dimensional tensor analysis module 3. It is used to establish a nonlinear dynamic model of respiratory motion based on the essential motion pattern and respiratory monitoring point motion information, predicting the future trajectory of lung nodules. For example... Figure 4 As shown, the nonlinear dynamics prediction module 4 includes a state space reconstruction unit 41, a dynamics characteristic analysis unit 42, a recurrent neural network unit 43, and a multi-scale prediction unit 44.

[0091] State-space reconstruction unit 41 reconstructs the phase-space representation of respiratory motion based on the delayed coordinate method. This unit constructs the state vector using the following method:

[0092] ,

[0093] in, Represents the observed value at time t. Let m represent the embedding dimension, where m represents the time delay. In practical applications, the time delay... Typically, a delay value is chosen that allows the self-mutual information function to reach a local minimum for the first time. The embedding dimension m is determined by the pseudo nearest neighbor algorithm and is usually 4-6 dimensions.

[0094] The dynamic characteristics analysis unit 42 calculates the Lyapunov exponent and constructs the Poincaré map to analyze the chaotic characteristics of respiratory motion. Lyapunov exponent Defined as:

[0095] ,

[0096] in, Represents the initial phase distance in phase space The distance between the two trajectories at time t. A positive Lyapunov exponent indicates that the system has chaotic characteristics and is difficult to predict; while a negative Lyapunov exponent indicates that the system tends to be stable and is relatively easy to predict. In practical applications, normal breathing usually has a small positive Lyapunov exponent (about 0.1-0.3), indicating a certain degree of predictability but also uncertainty.

[0097] Recurrent neural network unit 43 establishes a nonlinear dynamic prediction model by fusing convolutional feature extraction and temporal memory structures. This unit includes convolutional layers, long short-term memory (LSTM) layers, deconvolutional layers, and residual connection structures. The convolutional layers extract spatial features from the input data, the LSTM layers capture temporal dependencies, the deconvolutional layers reconstruct the prediction results, and the residual connection structures ensure stable gradient propagation. The network structure can be represented as follows:

[0098] ,

[0099] ,

[0100] in, This represents the input data at time t. This represents the hidden state of the LSTM layer. This represents the predicted output for time t+1. During training, the loss function considers both short-term accuracy and long-term stability:

[0101] ,

[0102] in, and These represent the time steps for short-term and long-term forecasts, respectively. and These are weighting coefficients, typically To ensure the accuracy of short-term forecasts. In practical applications, It is usually set to 5-10 frames (about 0.1-0.2 seconds). Set to 50-100 frames (approximately 1-2 seconds).

[0103] The multi-scale prediction unit 44 generates short-term (0.1-0.5 seconds), medium-term (0.5-2 seconds), and long-term (>2 seconds) predictions of lung nodule motion trajectories. Short-term predictions are primarily used for real-time navigation, requiring high accuracy (better than 1 mm); medium-term predictions are used for surgical planning, requiring accuracy of 1-2 mm; and long-term predictions are used for overall strategy decision-making, requiring accuracy of 2-3 mm. This multi-scale prediction strategy balances real-time performance and forward-looking capabilities, providing comprehensive navigation support for surgery.

[0104] The navigation fusion module 5 is connected to the micro-area optical flow calculation module 2 and the nonlinear dynamics prediction module 4, respectively, to fuse the motion displacement field and the future motion trajectory of the lung nodules to generate a four-dimensional optical flow field. Figure 5 As shown, the navigation fusion module 5 includes a multimodal data fusion unit 51, a four-dimensional optical flow field construction unit 52, and an adaptive correction unit 53.

[0105] The multimodal data fusion unit 51 fuses the motion displacement field and the future motion trajectory of the lung nodules based on an uncertainty weighting method. This unit employs the following weighted fusion formula:

[0106] , ,

[0107] Where F represents the result after fusion. This represents the result from the i-th data source. This represents the weight of the i-th data source. This represents the uncertainty of the i-th data source. This is achieved through statistical analysis of historical data or direct prediction using neural networks. This uncertainty-based weighting method can automatically adjust the contributions of different data sources, improving the reliability of the fusion results.

[0108] The four-dimensional optical flow field construction unit 52 organizes the fused data into a four-dimensional optical flow field structure. The four-dimensional optical flow field can be represented as a function. ,in Let t represent spatial coordinates, t represent time, and the function value represent the three-dimensional velocity vector of that point at that moment. This representation allows the system to describe the motion state of the lungs at any location and at any time, providing complete dynamic information for navigation.

[0109] The adaptive correction unit 53 compares the prediction results with the observed data in real time and dynamically adjusts the four-dimensional optical flow field parameters. This unit calculates the statistical characteristics of the prediction error, analyzes the systematic bias and random error components of the error, and adjusts the prediction model parameters accordingly. When a change in breathing pattern is detected (such as from normal breathing to deep breathing or coughing), the system automatically adjusts the prediction model parameters to adapt to the new breathing pattern.

[0110] The surgical navigation module 6 is connected to the navigation fusion module 5 and is used to generate real-time surgical navigation information based on the four-dimensional optical flow field and the position information of surgical tools. For example... Figure 6 As shown, the surgical navigation module 6 includes a surgical tool positioning unit 61, a navigation path planning unit 62, a three-dimensional visualization unit 63, and an interactive control unit 64.

[0111] The surgical tool positioning unit 61 acquires the spatial position and orientation information of the surgical tool. This unit monitors the surgical tool in real time using optical or electromagnetic tracking technology, with a sampling frequency typically between 30-60Hz and a spatial accuracy better than 0.5mm, meeting the requirements for precise surgical operations.

[0112] The navigation path planning unit 62 calculates the optimal surgical path and intervention timing based on a four-dimensional optical flow field. This unit analyzes the lung nodule's motion cycle and identifies the time window with the minimum motion velocity as the optimal intervention time, typically selecting the last inspiratory or expiratory phase of the respiratory cycle, when the lung motion velocity is usually less than 1 mm / s, which facilitates precise operation. Simultaneously, this unit calculates the optimal path from the current tool position to the target lung nodule, avoiding important blood vessels and bronchial structures to minimize surgical risk.

[0113] The 3D visualization unit 63 renders lung structures, lung nodule locations, predicted trajectories, and surgical tool positions in real time. This unit uses volume rendering technology to display the 3D structure of the lungs, marking lung nodules (usually red) and important anatomical structures (such as blood vessels in blue and bronchi in green) with different colors. Simultaneously, this unit displays the predicted trajectories of the lung nodules, using color intensity to indicate the confidence level of the prediction (darker colors indicate high confidence, lighter colors indicate low confidence).

[0114] The interactive control unit 64 provides a gesture control interface and view adjustment functions to maintain a sterile surgical field. This unit allows doctors to control the interface via head movements or voice commands, eliminating the need for physical contact with any equipment and ensuring the sterility of the surgical area. Simultaneously, the unit offers multiple view modes, including axial, sagittal, coronal, and 3D stereoscopic views, to meet the needs of different surgical stages.

[0115] Example 2

[0116] Reference Figure 7 The real-time navigation method for four-dimensional optical flow field of lung nodules provided by the present invention includes the following steps:

[0117] Step S1: Acquire lung CBCT image sequences and respiratory monitoring point motion information;

[0118] Step S2: Divide the CBCT image sequence into micro-grids, establish dynamic topological mapping relationships based on the micro-grids, and generate a lung tissue motion displacement field;

[0119] Step S3: Construct the motion displacement field into a high-dimensional tensor structure, perform tensor decomposition and singular value optimization, and extract the essential motion pattern;

[0120] Step S4: Based on the essential motion pattern and respiratory monitoring point motion information, establish a nonlinear dynamic model of respiratory motion to predict the future motion trajectory of lung nodules;

[0121] Step S5: Fuse the motion displacement field and the future motion trajectory of the lung nodule to generate a four-dimensional optical flow field;

[0122] Step S6: Generate real-time surgical navigation information based on the four-dimensional optical flow field and the position information of surgical tools.

[0123] In step S1, the lung CBCT image sequence is acquired using a dual-head CBCT imaging unit, and the respiratory monitoring point motion information is acquired using an electromagnetic navigation monitoring unit. Preferably, the CBCT acquisition parameters are adjusted according to the size of the lung nodules: for nodules with a diameter ≥8mm, a voltage of 120kV, a current of 16mA, and an exposure time of 0.75s are used; for nodules with a diameter <8mm, a voltage of 120kV, a current of 8mA, and an exposure time of 0.50s are used. The image reconstruction resolution is 512×512 pixels, and the slice thickness is 1.0mm. The electromagnetic navigation sampling frequency is 100Hz to ensure the capture of subtle changes in respiratory motion.

[0124] In step S2, the CBCT image sequence is first divided into cubic micro-regions with a side length of 1 mm, each containing 8 vertices. Then, based on the principle of differential topology, a continuous mapping relationship between micro-regions in adjacent time frames is established, forming a dynamic topological mapping. By calculating the grayscale changes of the micro-region vertices, a precise motion displacement field is obtained. The key to this step is maintaining the continuity of the topological structure, which allows for accurate tracking of tissue deformation even under large respiratory movements.

[0125] In step S3, the motion displacement field is organized into a five-dimensional tensor structure, containing five dimensions: spatial coordinates (x, y, z), time (t), and eigenvalues ​​(such as grayscale values). Information from each dimension is separated using High-Order Singular Value Decomposition (HOSVD) to extract the main change patterns. Then, the optimal rank is determined based on the importance of singular values, achieving a low-rank approximation that significantly reduces computational complexity while retaining over 90% of the information. Finally, a Riemann metric is defined on the tensor manifold to calculate the curvature characteristics of the motion trajectory, providing geometric information for subsequent predictions.

[0126] In step S4, the phase space representation of respiratory motion is first reconstructed based on the delayed coordinate method, with an embedding dimension typically between 4 and 6. Then, the Lyapunov exponent is calculated and the Poincaré map is constructed to analyze the chaotic characteristics of respiratory motion. Next, a nonlinear dynamic prediction model is established by fusing convolutional feature extraction and an LSTM temporal memory structure. After training, this model can generate short-term (0.1-0.5 seconds), medium-term (0.5-2 seconds), and long-term (>2 seconds) predictions of lung nodule motion trajectories, meeting navigation needs at different time scales.

[0127] In step S5, the motion displacement field and the future motion trajectory of the lung nodules are fused using an uncertainty-weighted method. The weights of different data sources are automatically adjusted according to their uncertainties to ensure the reliability of the fusion result. The fused data is organized into a four-dimensional optical flow field structure, describing the motion state of the lung at any location and at any time. Simultaneously, the system compares the predicted results with the observed data in real time and dynamically adjusts the four-dimensional optical flow field parameters to adapt to changes in breathing patterns.

[0128] In step S6, the system first acquires the spatial position and orientation information of the surgical tools, with a sampling frequency of 30-60Hz. Then, based on the four-dimensional optical flow field, it calculates the optimal surgical path and intervention timing, typically selecting a time window during the respiratory cycle where the lung motion velocity is less than 1mm / s as the optimal intervention time. Next, the system renders the lung structure, lung nodule location, predicted trajectory, and surgical tool position in real time, providing the surgeon with intuitive navigation information. Finally, through a gesture control interface and view adjustment function, the surgeon can control the navigation system while maintaining a sterile surgical field.

[0129] Through the above steps, the four-dimensional optical flow field real-time navigation method for pulmonary nodules of the present invention can achieve accurate tracking and motion prediction of pulmonary nodules under respiratory conditions, with a navigation accuracy of 1-3 mm, significantly better than the 5-10 mm accuracy of traditional methods. Simultaneously, the system exhibits good adaptability to interference factors such as irregular breathing and coughing, ensuring navigation reliability under various respiratory conditions. Furthermore, the system can predict the trajectory of pulmonary nodules within the next 2 seconds, providing prospective navigation information for surgery, reducing surgical waiting time by more than 40%, and improving surgical efficiency and safety.

[0130] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the present invention should be included within the scope of protection of the present invention.

Claims

1. A four-dimensional optical flow field real-time navigation system for lung nodules, characterized in that, include: The data acquisition module is used to acquire lung CBCT image sequences and respiratory monitoring point motion information; The micro-area optical flow calculation module, connected to the data acquisition module, is used to divide the CBCT image sequence into micro-area grids and establish a dynamic topological mapping relationship based on the micro-area grids to generate a lung tissue motion displacement field. The high-dimensional tensor analysis module, connected to the micro-area optical flow calculation module, is used to construct the motion displacement field into a high-dimensional tensor structure, perform tensor decomposition and singular value optimization, and extract the essential motion pattern. The nonlinear dynamics prediction module, connected to the high-dimensional tensor analysis module, is used to establish a nonlinear dynamics model of respiratory motion based on the essential motion pattern and the motion information of the respiratory monitoring point, and to predict the future motion trajectory of the lung nodule. The navigation fusion module is connected to the micro-area optical flow calculation module and the nonlinear dynamics prediction module respectively, and is used to fuse the motion displacement field and the future motion trajectory of the lung nodule to generate a four-dimensional optical flow field. And a surgical navigation module, connected to the navigation fusion module, for generating real-time surgical navigation information based on the four-dimensional optical flow field and the position information of the surgical tools; The data acquisition module includes: Dual-head CBCT imaging unit is used to perform non-coplanar imaging of the patient's lungs and acquire two-dimensional image data from different angles; The electromagnetic navigation monitoring unit is used to acquire the three-dimensional position coordinates of the breathing monitoring point in real time through electromagnetic sensors; and The spatiotemporal synchronization unit is connected to the dual-head CBCT imaging unit and the electromagnetic navigation monitoring unit, respectively, and is used to perform time synchronization processing on the two-dimensional image data and the three-dimensional position coordinates to generate a time-aligned multimodal data stream; The micro-area optical flow calculation module includes: The micro-region division unit is used to divide the CBCT image sequence into cubic micro-regions with a side length of 1 mm, and each micro-region contains 8 vertices; A topology relationship construction unit, connected to the micro-region partitioning unit, is used to establish adjacency relationships and topology feature vectors for the micro-region mesh; A dynamic homeomorphism mapping unit, connected to the topology construction unit, is used to establish a continuous mapping function between micro-regions in adjacent time frames based on the principle of differential topology; and A probability adaptive correction unit, connected to the dynamic homeomorphism mapping unit, is used to adaptively correct the continuous mapping function according to the Bayesian probability framework to generate an accurate motion displacement field. The high-dimensional tensor analysis module includes: Tensor construction unit, used to organize the motion displacement field into a five-dimensional tensor structure containing spatial coordinates, time and eigenvalues; Tensor decomposition unit, connected to the tensor construction unit, is used to perform high-order singular value decomposition on the five-dimensional tensor structure to separate information in each dimension. A low-rank optimization unit, connected to the tensor decomposition unit, is used to determine the optimal rank based on the importance of singular values, thereby achieving a low-rank approximation representation; and The differential geometry analysis unit, connected to the low-rank optimization unit, is used to define the Riemann metric on the tensor manifold and calculate the curvature characteristics of the motion trajectory.

2. The four-dimensional optical flow field real-time navigation system for lung nodules according to claim 1, characterized in that, The dual-head CBCT imaging unit uses two imaging heads to simultaneously image the lungs from different angles, and fuses the two-dimensional images from different angles within the same exposure cycle using a sub-pixel-level cascaded interpolation reconstruction algorithm to form a super-resolution three-dimensional image.

3. The four-dimensional optical flow field real-time navigation system for lung nodules according to claim 1, characterized in that, The nonlinear dynamics prediction module includes: The state-space reconstruction unit is used to reconstruct the phase-space representation of respiratory motion based on the delayed coordinate method. The dynamic characteristic analysis unit, connected to the state space reconstruction unit, is used to calculate the Lyapunov exponent and construct the Poincaré map to analyze the chaotic characteristics of respiratory motion. A recurrent neural network unit, connected to the dynamic characteristic analysis unit, is used to establish a nonlinear dynamic prediction model by fusing convolutional feature extraction and temporal memory structures; and A multi-scale prediction unit, connected to the recurrent neural network unit, is used to generate short-term, medium-term, and long-term lung nodule motion trajectory predictions.

4. The four-dimensional optical flow field real-time navigation system for lung nodules according to claim 3, characterized in that, The recurrent neural network unit includes: Convolutional layers are used to extract spatial features from the input data; A long short-term memory layer, connected to the convolutional layer, is used to capture temporal dependencies; A deconvolutional layer, connected to the long short-term memory layer, is used to reconstruct the prediction result; and A residual connection structure connects the convolutional layer and the deconvolutional layer to ensure stable gradient propagation.

5. The four-dimensional optical flow field real-time navigation system for lung nodules according to claim 1, characterized in that, The navigation fusion module includes: A multimodal data fusion unit is used to fuse the motion displacement field and the future motion trajectory of the lung nodule based on an uncertainty weighting method; A four-dimensional optical flow field construction unit is connected to the multimodal data fusion unit to organize the fused data into a four-dimensional optical flow field structure; and an adaptive correction unit is connected to the four-dimensional optical flow field construction unit to compare the prediction results with the observation data in real time and dynamically adjust the four-dimensional optical flow field parameters.

6. The four-dimensional optical flow field real-time navigation system for lung nodules according to claim 1, characterized in that, The surgical navigation module includes: Surgical tool positioning unit is used to acquire the spatial position and orientation information of the surgical tool; A navigation path planning unit, connected to the surgical tool positioning unit, is used to calculate the optimal surgical path and intervention timing based on the four-dimensional optical flow field. A 3D visualization unit, connected to the navigation path planning unit, is used to render lung structures, lung nodule locations, predicted trajectories, and surgical tool locations in real time; and an interactive control unit, connected to the 3D visualization unit, is used to provide a gesture control interface and view adjustment functions to maintain a sterile surgical field.

7. A real-time navigation method for four-dimensional optical flow field of pulmonary nodules, employing the real-time navigation system for four-dimensional optical flow field of pulmonary nodules as described in any one of claims 1-6, characterized in that, Includes the following steps: Acquire lung CBCT image sequences and respiratory monitoring point motion information; The CBCT image sequence is divided into micro-region grids, and a dynamic topological mapping relationship is established based on the micro-region grids to generate a lung tissue motion displacement field; The motion displacement field is constructed as a high-dimensional tensor structure, and tensor decomposition and singular value optimization are performed to extract the essential motion pattern. Based on the essential motion pattern and the motion information of the respiratory monitoring point, a nonlinear dynamic model of respiratory motion is established to predict the future motion trajectory of the lung nodule. By fusing the motion displacement field and the future motion trajectory of the lung nodule, a four-dimensional optical flow field is generated; And based on the four-dimensional optical flow field and the position information of the surgical tools, real-time surgical navigation information is generated.

Images