A method for calculating time delay between a navigation positioning device and a three-dimensional perspective device

Abstract

The present invention discloses a method for calculating the time delay between a navigation and positioning device and a three-dimensional fluoroscopy device. The method comprises the following steps: controlling the target's motion and scanning its 4D image with the three-dimensional fluoroscopy device, while simultaneously acquiring a corresponding respiratory motion curve of the target's motion in real time through a sensor; identifying the target's position in the 4D image, thereby obtaining a target motion curve, and combining the obtained respiratory motion curve with the calculated time delay between the navigation and positioning device and the three-dimensional fluoroscopy device. This method can accurately determine the time delay between a surgical robot and the three-dimensional fluoroscopy device, enabling precise manipulation of lesions affected by respiration.

Description

Technical Field

[0001] The present invention relates to the technical field of surgical robots, and in particular to a method for calculating time delay between a navigation positioning device and a three-dimensional perspective device. Background Art

[0002] Patient images captured by 3D fluoroscopy equipment (typically CT) contain both spatial and temporal information. Surgical robotic systems use spatial information for navigation and positioning, while temporal information is used to understand lesion motion data, enabling precise execution of procedures such as lung puncture and tumor radiotherapy. Time synchronization between the surgical robotic system and the CT system is crucial for utilizing this temporal information. Summary of the Invention

[0003] Purpose of the invention: In view of the above-mentioned deficiencies, the present invention provides a time delay calculation method for a navigation and positioning device and a three-dimensional perspective device based on time synchronization, which realizes accurate time synchronization tasks by performing time delay analysis and calculation.

[0004] Technical solution:

[0005] A method for calculating the time delay between a navigation and positioning device and a three-dimensional perspective device, comprising the steps of:

[0006] Control the target's movement and scan its 4D image through a 3D perspective device. At the same time, the sensor obtains the corresponding respiratory motion curve of the target's movement in real time.

[0007] The target position in the 4D image is identified, and the target motion curve is obtained based on this. The time delay between the navigation positioning device and the 3D fluoroscopy device is calculated in combination with the respiratory motion curve obtained above.

[0008] The time delay between the calculation navigation positioning device and the 3D perspective device is specifically as follows:

[0009] The target motion curve and the respiratory motion curve are intercepted separately so that they have the same start and end time;

[0010] Normalize the two motion curves;

[0011] A generalized cross-correlation analysis is performed on the two motion curves, and the product of the index position corresponding to the maximum cross-correlation value and the sampling step time is selected as the time delay.

[0012] After normalizing the two motion curves, the data sampling rate conversion step is also included:

[0013] The sampling frequencies of the two motion curves are calculated respectively, and the motion curve with the low sampling frequency is subjected to linear interpolation and upsampling processing, so that the two motion curves are synchronized with the same frequency.

[0014] The generalized cross-correlation analysis is specifically as follows:

[0015]

[0016]

[0017] Among them, F Y (ω) is the frequency domain signal obtained by Fourier transforming the target motion curve, is the frequency domain signal obtained by conjugate Fourier transform of the respiratory motion curve, P(ω) is the cross-correlation data of the two motion curves after Fourier transform, R(τ) is the cross-correlation value of the two motion curves after P(ω) is inverse Fourier transform and the zero frequency point is shifted to the center of the spectrum, ω is the sampling frequency of the frequency domain signal obtained after Fourier transform, and τ is the sampling index position corresponding to the frequency domain signal obtained after inverse Fourier transform.

[0018] The middle point of the number of sampling points of the two motion curves is taken as the zero position, the time difference between adjacent sampling points is taken as the sampling time step, and the product of the sampling point corresponding to the maximum value of the cross-correlation of the corresponding sampling points in the two motion curves relative to the zero position and the sampling time step is the time delay of the two motion curves, that is, the time delay between the navigation positioning device and the three-dimensional perspective device.

[0019] Before intercepting the two motion curves, the method further includes the step of reducing the dimension of the target motion curve identified in the 4D image to a one-dimensional motion curve in its maximum motion direction.

[0020] The target motion curve obtained by identifying the target position in the 4D image is specifically:

[0021] According to the time tags in the 4D images, the 4D images are divided into 3D image sequences sorted by sampling time, and target recognition and extraction are performed on each 3D image one by one, and the target motion curve is obtained by combining the sampling time.

[0022] The target recognition and extraction of each 3D image is specifically as follows:

[0023] Calculate the gradient of each 3D image and use the watershed algorithm to segment the 3D image to obtain connected regions;

[0024] The target area is screened according to the above segmentation results, and the target center of mass is calculated.

[0025] The screening target area is specifically:

[0026] Calculate the mean and maximum grayscale values ​​of each connected region obtained by segmentation, delete the connected regions with a mean value less than 0.1 times the maximum value, and obtain the candidate target region. Calculate the circularity of each candidate target region based on the position of each point in the candidate target region. The candidate target region with the largest circularity is the target region.

[0027] Its characteristics are:

[0028] The target is mounted on a tooling, and the sensor is mounted on a simulated skin on the tooling for simulating the patient's respiratory movement. While the three-dimensional perspective device scans the 4D image of the target, the sensor collects the movement data of the simulated skin in real time to obtain a respiratory movement curve.

[0029] Beneficial effects: The present invention obtains the motion curve of the target in the 4D image by performing 4D scanning on the tooling, and performs time delay analysis on it and the respiratory motion curve acquired by the sensor in real time obtained by the navigation and positioning device, thereby obtaining a precise time delay between the navigation and positioning device and the three-dimensional perspective device, and realizing precise operation on lesions affected by breathing. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A flow chart of the time delay calculation of the present invention;

[0031] Figure 2 Flowchart of the algorithm for extracting targets from 3D images;

[0032] Figure 3 Flowchart of the generalized cross-correlation time delay analysis algorithm. DETAILED DESCRIPTION

[0033] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments.

[0034] The time delay calculation method of the navigation positioning device and the three-dimensional perspective device of the present invention includes the following steps:

[0035] (1)NTP time synchronization;

[0036] Connect the navigation and positioning device and the 3D perspective device through the network and perform NTP time synchronization.

[0037] (2) Installing sensors on the simulated skin of the respiratory motion tooling that simulates the patient's respiratory motion, controlling the movement of the target (i.e., the simulated lesion) on the respiratory motion tooling, performing a 4D scan of the target using a three-dimensional perspective device to obtain its 4D image, while the sensor collects data on the simulated skin movement in real time;

[0038] In the present invention, the breathing exercise tooling adopts the motion curve simulation body membrane disclosed in the Chinese invention patent CN114176725A;

[0039] In the present invention, the three-dimensional perspective device adopts a CT device, and the 4D scanning of the respiratory motion tooling is performed using a Cine mode;

[0040] In the present invention, the sensor installed on the simulated skin is a respiratory abdominal belt.

[0041] In the present invention, the scanning area of ​​the three-dimensional fluoroscopy equipment needs to cover the movement range of the target. The scanning speed and reconstruction mode are coordinated to make the time resolution of the 3D reconstructed image the lowest, and the scanning duration is not less than one movement cycle of the target. In the present invention, the scanning duration is set to 10s; the movement speed of the target on the respiratory motion tooling needs to be as large as possible without causing artifacts in the image extraction target to ensure image recognition accuracy.

[0042] In the present invention, the navigation and positioning device adopts a lung nodule puncture robot, and the target simulates the movement of the lung nodule.

[0043] (3) Processing the 4D images to obtain a 3D image sequence, identifying the target position in each 3D image, and obtaining the target motion curve;

[0044] According to the time tags in the 4D images, the 4D images are divided into 3D image sequences sorted by sampling time, and target recognition and extraction are performed on each 3D image one by one. The specific extraction process is as follows:

[0045] (31) determining a region of interest in each 3D image;

[0046] In the present invention, this step can be omitted, but in order to speed up the target recognition speed, the target's motion range can be framed on each 3D image as the region of interest;

[0047] (32) performing filtering on each 3D image to smooth the image;

[0048] In the present invention, the filtering adopts the thermal diffusion anisotropic filtering; wherein, the number of iterations of the thermal diffusion anisotropic filtering is set to 3, the thermal conductivity is set to 5, and the lambda coefficient is set to 0.02;

[0049] In the present invention, this step can also be omitted;

[0050] (33) Calculate the gradient of each 3D image and use the watershed algorithm to segment the 3D image to obtain connected regions;

[0051] Use the first-order Gaussian convolution kernel in three different directions to perform convolution operations respectively, and then perform the 2-norm merging on the calculated dimensional data in the three directions to obtain a 3D data, thereby obtaining the gradient amplitude image;

[0052] Among them, the variance factor of the first-order Gaussian convolution kernel is σ=1.2;

[0053] The watershed algorithm is used to segment the 3D image to obtain connected regions. Specifically, the maximum grayscale value in the gradient amplitude image is multiplied by 0.02 as the starting seed point, and the segmentation is gradually carried out to form a tree structure. Each sub-branch represents the isolated region of the segmentation. Then, the branches with a spanning tree level less than 0.6 are merged to obtain the segmented region.

[0054] (34) Filtering the target area according to the segmentation result of step (33) and calculating the target centroid;

[0055] Calculate the mean and maximum grayscale values ​​of each foreground area segmented in step (33), delete the connected areas whose mean is less than 0.1 times the maximum grayscale value, and obtain candidate target areas. Calculate the circularity of each candidate target area based on the position of each point in the candidate target area. The candidate target area with the largest circularity is the target area, and calculate the centroid of the area as the target centroid.

[0056] (35) The target motion curve in each 3D image is obtained based on the target center of mass position in each 3D image and the sampling time.

[0057] (4) obtaining simulated skin motion data collected by the sensor and obtaining a respiratory motion curve, and calculating the time delay between the navigation positioning device and the three-dimensional perspective device based on the target motion curve in the image obtained in step (3);

[0058] (41) reducing the target motion curve in the image obtained in step (3) to a one-dimensional motion curve in its maximum motion direction;

[0059] (42) intercepting data according to the intersection of the time axes of the target motion curve and the respiratory motion curve so that the two motion curves have the same start and end time;

[0060] (43) The two motion curves are standardized to eliminate the influence caused by the inconsistency of physical scales of different data;

[0061] (44) Converting the data of the two motion curves to the same sampling rate: Calculate the sampling frequencies of the two motion curves respectively, and perform linear interpolation on the motion curve with the low sampling frequency to perform upsampling processing to ensure the same frequency synchronization of the two motion curves;

[0062] (45) Perform a generalized cross-correlation analysis on the two motion curves, and select the product of the index position corresponding to the maximum amplitude and the sampling time step as the time delay;

[0063] Specifically:

[0064] The following formula is used to calculate the conjugate multiplication of the Fourier transform of the two motion curves, and then the inverse Fourier transform and shift operation can be used to obtain the cross-correlation of the two motion curves:

[0065]

[0066]

[0067] Among them, F Y (ω) is the frequency domain signal obtained by Fourier transforming the target motion curve, is the frequency domain signal obtained by conjugate Fourier transform of the respiratory motion curve, P(ω) is the cross-power spectral density of the two motion curves after Fourier transform, R(τ) is the cross-correlation value of the two motion curves after P(ω) is inverse Fourier transform and the zero frequency point is shifted to the center of the spectrum, ω is the sampling frequency of the frequency domain signal obtained after Fourier transform, and τ is the sampling index position corresponding to the frequency domain signal obtained after inverse Fourier transform;

[0068] (46) The middle point of the R(τ) data length is taken as the zero position, and the time difference between adjacent sampling points is taken as the sampling time step. The product of the sampling point corresponding to the maximum value of the cross-correlation of the corresponding sampling point in the R(τ) data relative to the zero position and the sampling time step is the time delay of the two motion curves, that is, the time delay between the surgical robot and the three-dimensional fluoroscopy device.

[0069] The present invention takes the lung nodule puncture surgical robot as an example. In order to accurately realize the respiratory gating function, the time when the image slice is scanned to the lung nodule corresponds to the patient's respiratory phase. If the patient reaches this phase again, it is possible to grasp the time of the nodules affected by breathing and achieve accurate puncture. Therefore, the precise alignment of the axial slice sampling time of the lung nodule and the real-time respiratory acquisition time of the robot system is crucial. Under the NTP time alignment condition, the present invention designs a respiratory motion tooling with a target. The respiratory motion tooling is imaged in CT continuous Cine mode to obtain a 4D image, and the target position in the 4D image is extracted to generate continuous sine-like motion data (i.e., the motion curve of the target), and the respiratory motion data collected in real time by the surgical robot acquisition sensor is subjected to time delay analysis, thereby obtaining a precise time delay between the surgical robot and the three-dimensional perspective setting.

[0070] The preferred embodiments of the present invention are described in detail above, but the present invention is not limited to the specific details in the above embodiments. Within the technical concept of the present invention, various equivalent transformations (such as quantity, shape, position, etc.) can be made to the technical solution of the present invention, and these equivalent transformations all fall within the scope of protection of the present invention.

Claims

1. A method for calculating the time delay between a navigation and positioning device and a three-dimensional perspective device, characterized by: Including steps: Control the target's movement and scan its 4D image through a 3D perspective device. At the same time, the sensor obtains the corresponding respiratory motion curve of the target's movement in real time. The target position in the 4D image is identified, and the target motion curve is obtained based on this. The target motion curve and the respiratory motion curve are respectively intercepted so that they have the same start and end time. The two motion curves are normalized and then subjected to generalized cross-correlation analysis. The midpoint of the number of sampling points of the two motion curves is used as the zero position, and the time difference between adjacent sampling points is used as the sampling time step. The product of the position of the sampling point corresponding to the maximum cross-correlation value of the corresponding sampling point in the two motion curves relative to the zero position and the sampling time step is the time delay between the two motion curves, that is, the time delay between the navigation positioning device and the 3D fluoroscopy device.

2. The time delay calculation method for a navigation and positioning device and a three-dimensional perspective device according to claim 1, characterized in that: After normalizing the two motion curves, the data sampling rate conversion step is also included: The sampling frequencies of the two motion curves are calculated respectively, and the motion curve with the low sampling frequency is subjected to linear interpolation and upsampling processing, so that the two motion curves are synchronized with the same frequency.

3. The time delay calculation method for a navigation and positioning device and a three-dimensional perspective device according to claim 1, characterized in that: The generalized cross-correlation analysis is specifically as follows: ； ； in, is the frequency domain signal obtained by Fourier transforming the target motion curve. is the frequency domain signal obtained by conjugate Fourier transform of the respiratory motion curve. is the cross-correlation data of the two motion curves after Fourier transform, R(τ) is The cross-correlation value of the two motion curves obtained after inverse Fourier transform and zero frequency point shifted to the center of the spectrum, is the sampling frequency of the frequency domain signal obtained after Fourier transform, is the sampling index position corresponding to the frequency domain signal obtained after inverse Fourier transform.

4. The method for calculating time delay between a navigation and positioning device and a three-dimensional perspective device according to claim 1, wherein: Before intercepting the two motion curves, the method further includes the step of reducing the dimension of the target motion curve identified in the 4D image to a one-dimensional motion curve in its maximum motion direction.

5. The method for calculating time delay between a navigation and positioning device and a three-dimensional perspective device according to claim 1, wherein: The target motion curve obtained by identifying the target position in the 4D image is specifically: According to the time tags in the 4D images, the 4D images are divided into 3D image sequences sorted by sampling time, and target recognition and extraction are performed on each 3D image one by one, and the target motion curve is obtained by combining the sampling time.

6. The time delay calculation method for a navigation and positioning device and a three-dimensional perspective device according to claim 5, characterized in that: The target recognition and extraction of each 3D image is specifically as follows: Calculate the gradient of each 3D image and use the watershed algorithm to segment the 3D image to obtain connected regions; The target area is screened according to the segmentation results, and the target center of mass is calculated.

7. The time delay calculation method for a navigation and positioning device and a three-dimensional perspective device according to claim 6, characterized in that: The screening target area is specifically: Calculate the mean and maximum grayscale values ​​of each connected region obtained by segmentation, delete the connected regions with a mean value less than 0.1 times the maximum value, and obtain the candidate target region. Calculate the circularity of each candidate target region based on the position of each point in the candidate target region. The candidate target region with the largest circularity is the target region.

8. The method for calculating time delay between a navigation and positioning device and a three-dimensional perspective device according to claim 1, wherein: The target is mounted on a tooling, and the sensor is mounted on a simulated skin on the tooling for simulating the patient's respiratory movement. While the three-dimensional perspective device scans the 4D image of the target, the sensor collects the movement data of the simulated skin in real time to obtain a respiratory movement curve.

Images