Thoracic surgery navigation data registration method and system

Abstract

The invention provides a thoracic surgery navigation data registration method and system. The method comprises the following steps: constructing a preoperative thoracic surgery three-dimensional model of a patient; extracting a first feature set of the three-dimensional model and a second feature set of the intraoperative real-time image; performing initial registration of surgical navigation data on the first feature set and the second feature set; the feature matching pairs in the initial registration result serve as input of a preset optimization algorithm, and real-time registration of the preoperative image data and the intraoperative image data is achieved by adjusting the position and the posture of the three-dimensional model through iteration; through real-time registration of the preoperative image data and the intraoperative image data in real time, the three-dimensional model can still be adjusted in real time when the body posture of the patient changes in the operation, so that the stability and the real-time performance of the preoperative image data and the intraoperative image data are realized, and the accuracy and the reliability of the patient are improved. And the iterative nearest point is adopted for final registration, so that the accuracy of surgical navigation data registration is improved.

Description

Technical Field

[0001] This application relates to the technical field of data processing, and more specifically, to a method and system for registering thoracic surgery navigation data. Background Art

[0002] In thoracic surgery, in order to achieve precise surgical operations, surgical navigation systems are widely used. These systems usually need to integrate preoperative imaging data (such as CT, MRI, etc.) and intraoperative actual anatomical structure information to provide doctors with accurate three-dimensional position and orientation guidance of the surgical site. However, since the preoperative imaging data is obtained when the patient is in a static and specific position, and the patient's position may change during the operation, there is a difference between the preoperative imaging data and the intraoperative actual situation. Therefore, an effective data registration method is needed to accurately map the preoperative imaging data to the intraoperative actual anatomical structure, so as to provide a reliable basis for surgical navigation and improve the accuracy and safety of the operation. At present, some existing data registration methods have deficiencies in accuracy, real-time performance, and stability, and are difficult to meet the high requirements of thoracic surgery.

[0003] Therefore, how to overcome the above-mentioned technical problems and deficiencies has become a problem that needs to be solved. Summary of the Invention

[0004] In order to overcome the deficiencies of some existing data registration methods in accuracy, real-time performance, and stability, which are difficult to meet the high requirements of thoracic surgery, this application provides a method and system for registering thoracic surgery navigation data, and adopts the following technical solutions:

[0005] In the first aspect, this application provides a method for registering thoracic surgery navigation data, including:

[0006] Construct a preoperative three-dimensional model of the patient's thoracic surgery;

[0007] Extract the first feature set of the three-dimensional model and the second feature set of the intraoperative real-time image;

[0008] Perform initial registration of the surgical navigation data on the first feature set and the second feature set;

[0009] Use the feature matching pairs in the initial registration result as the input of a preset optimization algorithm, and realize the real-time registration of the preoperative imaging data and the intraoperative imaging data by iteratively adjusting the position and posture of the three-dimensional model.

[0010] Further, constructing a preoperative three-dimensional model of the patient's thoracic surgery includes:

[0011] Segment the patient's imaging data based on a preset image segmentation model to obtain the image data of different tissues and organs in the imaging data; construct a three-dimensional model based on the segmented image data.

[0012] Further, segment the imaging data of the patient based on a preset image segmentation model to obtain the image data of different tissues and organs in the imaging data, including:

[0013] Based on the range of the chest imaging and the positions of different tissues and organs, initialize the control points of the contour curve, and construct the initial contour curve of the parametric B-spline based on the initialized control points and the B-spline order.

[0014] Based on a preset energy function, iteratively update the initial contour curve. During the iteration of the initial contour curve, calculate the gradient of the energy function with respect to the parameters of the contour curve, and adjust the parameters of the contour curve based on the negative direction of the gradient to minimize the energy function. When the stopping condition is reached, stop the iteration of the contour curve to obtain the final segmentation result.

[0015] Based on the contour boundary of the final segmentation result, segment the chest imaging data into different regions, with each region corresponding to a tissue or organ. Mark the pixels inside the contour as the categories of the corresponding tissues or organs to obtain the data of different tissues or organs.

[0016] Further, construct a three-dimensional model based on the segmented image data, including:

[0017] Extract the key points on the contours of the tissues or organs in the segmented image, and use the key points as the point cloud data for constructing the three-dimensional model.

[0018] Construct a three-dimensional control point grid based on the distribution of the point cloud data.

[0019] Perform surface fitting on the point cloud data based on the B-spline surface fitting algorithm.

[0020] Smooth the fitted three-dimensional model to complete the construction of the three-dimensional model.

[0021] Further, perform initial registration of the surgical navigation data for the first feature set and the second feature set, including:

[0022] Obtain the corresponding relationship between the first feature set and the second feature set based on a preset similarity algorithm; obtain the deformation parameters of the control points based on the matched feature points; apply the obtained deformation parameters to the control point grid to make the control points undergo corresponding deformations, and obtain the deformations of other feature points in the entire deformation space through the interpolation algorithm, thereby realizing the deformation of the first feature set and completing the initial registration of the first feature set and the second feature set.

[0023] Further, use the feature matching pairs in the initial registration result as the input of a preset optimization algorithm, where the preset optimization algorithm is the Iterative Closest Point (ICP) algorithm, and based on the ICP algorithm, realize the real-time registration of preoperative image data and intraoperative image data.

[0024] Further, based on the ICP algorithm, realize the real-time registration of preoperative image data and intraoperative image data, including:

[0025] Set an initial transformation matrix for the preoperative image data of the initial registration.

[0026] Based on the nearest neighbor search in the intraoperative image data, obtain the corresponding points closest to each point of the preoperative image data of the initial registration, and obtain the matching point pairs.

[0027] Based on the matching point pairs, obtain the optimal rotation matrix and translation vector of the preoperative image data of the initial registration, so that the error between the intraoperative image data and the preoperative image data of the initial registration is minimized.

[0028] Apply the optimal rotation matrix and translation vector to the preoperative image data of the initial registration to complete the real-time registration of the preoperative image data and the intraoperative image data.

[0029] In a second aspect, the present application also provides a thoracic surgery navigation data registration system, including:

[0030] A three-dimensional model construction module for constructing a three-dimensional model of the patient's preoperative thoracic surgery.

[0031] A feature extraction module for extracting the first feature set of the three-dimensional model and the second feature set of the intraoperative real-time image.

[0032] An initial registration module for performing initial registration of the surgical navigation data on the first feature set and the second feature set.

[0033] A real-time registration module for using the feature matching pairs in the initial registration as the input of a preset optimization algorithm, and realizing the real-time registration of the preoperative image data and the intraoperative image data by iteratively adjusting the position and pose of the three-dimensional model.

[0034] In a third aspect, the present application provides an electronic device, including:

[0035] One or more processors; a memory; and one or more computer programs, where the one or more computer programs are stored in the memory, and the one or more computer programs include instructions that, when executed by the device, cause the device to execute the method described in the first aspect.

[0036] Fourthly, the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program runs on a computer, the computer is enabled to execute the method described in the first aspect.

[0037] Fifthly, the present application provides a computer program which is used to execute the method described in the first aspect when the computer program is executed by a computer.

[0038] In a possible design, the program in the fifth aspect can be stored in whole or in part on a storage medium packaged together with the processor, or can be stored in whole or in part on a memory not packaged together with the processor.

[0039] The present application has the following beneficial effects:

[0040] 1. The present application constructs a three-dimensional model of the patient's thoracic surgery before surgery; extracts the first feature set of the three-dimensional model and the second feature set of the intraoperative real-time image; performs initial registration of the surgical navigation data on the first feature set and the second feature set; uses the feature matching pairs in the initial registration result as the input of a preset optimization algorithm, and realizes real-time registration of the preoperative image data and the intraoperative image data by iteratively adjusting the position and posture of the three-dimensional model. By performing real-time registration of the preoperative image data and the intraoperative image data in real time, the present application enables the three-dimensional model to be adjusted in real time when the patient's body position and posture change during the operation, so as to achieve the stability and real-time performance of the preoperative image data and the intraoperative image data. The use of the iterative closest point for final registration improves the accuracy of the registration of the surgical navigation data.

[0041] 2. The preliminary registration of the three-dimensional model and the real-time position of the patient determines a preliminary matching range during the surgical navigation process, reducing the search space for subsequent precise registration; further registration based on the preliminary registration enables real-time adjustment of the three-dimensional model when the patient's body position changes, which better meets the real-time requirements of the surgical navigation registration data. Description of the Drawings

[0042] Figure 1 It is an exemplary system architecture diagram to which the embodiments of the present application can be applied;

[0043] Figure 2 It is a flowchart of the method for registering surgical navigation data in the thoracic surgery of the embodiments of the present application;

[0044] Figure 3 It is a flowchart of constructing a preoperative three-dimensional model in the embodiments of the present application;

[0045] Figure 4 It is a flowchart of preoperative image pixel segmentation in the embodiments of the present application;

[0046] Figure 5 Flow chart for constructing a three-dimensional model after preoperative image pixel segmentation according to an embodiment of the present application;

[0047] Figure 6 System flow chart according to an embodiment of the present application;

[0048] Figure 7 Schematic diagram of a computer device according to an embodiment of the present application. Detailed implementation manners

[0049] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the technical field to which this application belongs; the terms used in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application; the terms "including" and "having" and any variations thereof in the specification and claims of this application and the above drawings are intended to cover non-exclusive inclusion. The terms "first", "second", etc. in the specification and claims of this application or the above drawings are used to distinguish different objects and not to describe a specific order.

[0050] Reference to "embodiment" herein means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of this application. The phrase appears in various places in the specification and does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments.

[0051] To enable those skilled in the art to better understand the solution of this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0052] As Figure 1 shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0053] Users can use the terminal devices 101, 102, 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications may be installed on the terminal devices 101, 102, 103, such as a web browser application, a shopping application, a search application, an instant messaging tool, an email client, a social platform software, etc.

[0054] The terminal devices 101, 102, and 103 may be various electronic devices with a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 (Moving Picture Experts Group Audio Layer IV) players, laptop computers, desktop computers, and so on.

[0055] The server 105 may be a server providing various services, such as a background server that supports the pages displayed on the terminal devices 101, 102, and 103.

[0056] It should be noted that the method for registering thoracic surgery navigation data provided in the embodiments of the present application is generally executed by a server / terminal device. Correspondingly, the thoracic surgery navigation data registration system is generally set in the server / terminal device.

[0057] It should be understood that Figure 1 the numbers of the terminal devices, network, and server in

[0058] Continuing to refer to Figure 2 , the figure shows a flowchart of a method for registering thoracic surgery navigation data according to the present application. The method includes the following steps:

[0059] Step 201, construct a three-dimensional model of the patient's preoperative thoracic surgery.

[0060] In a possible implementation manner, constructing a three-dimensional model of the patient's preoperative thoracic surgery includes: obtaining the imaging data of the thoracic surgery patient, preprocessing the imaging data, and performing three-dimensional reconstruction on the preprocessed imaging data to generate a three-dimensional model of the patient's preoperative thoracic surgery.

[0061] In the embodiments of the present application, the preoperative imaging data of the patient is obtained through a three-dimensional scanning device, and the three-dimensional structures and specific position coordinate points of different organs of the thoracic surgery of the patient are obtained based on the three-dimensional scanning device. In the embodiments of the present application, the preoperative imaging data of the patient can be obtained through a three-dimensional scanning device, and the preoperative imaging data of the patient can be obtained through a CT scanner, a magnetic resonance imaging device, a three-dimensional ultrasonic imaging device, etc. In the embodiments of the present application, obtaining the preoperative imaging data of the patient through a CT scanner, a magnetic resonance imaging device, a three-dimensional ultrasonic imaging device, etc. includes structures such as the lungs, heart, blood vessels, and bones of the patient.

[0062] In the embodiments of the present application, preprocessing the image data includes: performing noise reduction, enhancement, and normalization operations on the image data. Noise reduction of the image data can be achieved by using a filtering algorithm, and the details of noise reduction processing of the image data are not described herein. Image enhancement can be performed by methods such as histogram equalization and deep learning-based image enhancement, and the specific steps of image enhancement of the image data are not described herein. If the obtained image data is in color, then the color image data needs to be grayscale processed to facilitate subsequent three-dimensional model construction. Normalize the grayscale values of the grayscale image data to a preset unified range to obtain the preprocessed image data.

[0063] In the embodiments of the present application, three-dimensional reconstruction is performed on the preprocessed image data to generate a three-dimensional model of the patient's thoracic surgery. Please refer to Figure 3 , and the specific content includes:

[0064] Step 31: Segment the patient's image data based on a preset image segmentation model to obtain the image data of different tissues and organs in the image data.

[0065] In the embodiments of the present application, segmenting the patient's image data based on a preset image segmentation model is to convert the segmented patient image data into voxel data and further construct a three-dimensional model of thoracic surgery. Based on a preset image segmentation model is a process of dividing the image data into several sub-regions or objects; the pixels within each region have similar characteristics, and there are significant differences between the pixels in different regions.

[0066] In the embodiments of the present application, a geometric active contour model is used to segment the patient's image data. The specific implementation steps are as follows:

[0067] During the preprocessing stage, the image data has been grayscale normalized. Here, the grayscale-normalized image data is directly used for segmentation processing.

[0068] Step 311: Based on the range of the chest image and the positions of different tissues and organs, initialize the control points of the contour curve, and construct an initial contour curve of a parametric B-spline based on the initialized control points and the B-spline order. The B-spline curve is expressed as: where P i is the control point, N i,p (u) is the p-order B-spline basis function, n is the number of control points minus one, u is a parameter, and its value range is usually [0,1]. By calculating the curve points corresponding to different u values, the initial contour curve is obtained.

[0069] Step 312: Based on a preset energy function, iteratively update the initial contour curve. During the iteration of the initial contour curve, by calculating the gradient of the energy function with respect to the contour curve parameters and adjusting the contour curve parameters based on the negative direction of the gradient, the energy function is decreased; when the stopping condition is reached, the iteration of the contour curve is stopped, and the final segmentation result is obtained. The iteration stops when one of the following conditions is met: ① The change amount of the energy function is less than a preset threshold; ② The change amount of the control points is less than a preset threshold; ③ The preset maximum number of iterations is reached.

[0070] In the embodiments of the present application, an internal energy term is defined based on curve length, curvature, and control point variation. By defining the internal energy term, the initial contour curve can be controlled to avoid excessive stretching or deformation. An external energy term is defined based on image gradient and regional gray information. The external energy term guides the contour curve to move towards the target edge, enhancing the segmentation ability for different tissues and organs.

[0071] In the embodiments of the present application, the preset energy function is:

[0072] E = w1E l + w2E c + w3E control + w4E edge + w5E region

[0073] where w1, w2, w3, w4, and w5 are the weight coefficients of each term. E l is the energy term based on curve length, E c is the energy term based on curvature, and E control is the energy term based on control points, E edge is the energy term based on image gradient, and E region is the energy term based on regional gray information.

[0074] The energy function in the present application includes an internal energy term and an external energy term. There is an adversarial relationship between the internal energy term and the external energy term. During the process of image data segmentation, when the internal energy term and the external energy term reach equilibrium, the contour curve at this time can accurately evolve according to the image features, and at the same time can maintain smoothness and stability, avoiding unreasonable shape changes.

[0075] Among them, the energy term based on curve length is E l = ∫S(u)ds, where ds is the arc length element of the curve. Then the gradient of the curve length energy term with respect to the control point P i is The energy term based on curvature is E c = ∫S(u)k 2ds, where k is the curvature of the curve, then the gradient of the curvature energy term with respect to the control point P i is The energy term based on the change of the control point is where ΔP i is the position change of the control point P i , where ΔP i =|P i -P i-1 |, P i-1 is the control point of the previous iteration, then the gradient of the control point energy term is

[0076] where the external energy term based on the image gradient is where ▽I is the gradient of the image, I(x, y) is the gray function of the image, g is a decreasing function, then the gradient of the energy term based on the image gradient is The energy term based on the gray information is E region =∫R I(x, y)-c 2 dxdy, where R is the region enclosed by the curve, and c is the average gray value in the region R. Adjust the position according to the regional gray characteristics to segment the region with specific gray characteristics, then the gradient of the energy term based on the gray information is

[0077] After obtaining the gradient i of the preset energy function with respect to the control point P , adjust the position of the control point based on the negative direction of the gradient, and the update formula is: where is the position of the updated control point, is the current position of the control point, and α is the learning rate.

[0078] Step 313, based on the contour boundary of the final segmentation result, segment the chest image data into different regions, each region corresponding to a tissue or organ, label the pixels inside the contour as the categories of the corresponding tissues or organs, and obtain the data of different tissues or organs.

[0079] Step 32, construct a three-dimensional model based on the segmented image data.

[0080] In the embodiment of the present application, to construct a three-dimensional model based on the segmented image data, please refer to Figure 5 , and the specific content includes:

[0081] Step 321, extract the key points on the contours of the tissues or organs in the segmented image, and use the key points as the point cloud data for constructing the three-dimensional model.

[0082] Step 322: Based on the distribution of the point cloud data, construct a three-dimensional control point grid, where the three-dimensional control point grid is used to define the control points of the B-spline surface.

[0083] Step 323: Perform surface fitting on the point cloud data based on the B-spline surface fitting algorithm. By adjusting the control point grid and basis functions, the fitted surface approximates the surface shape of the tissue and organ represented by the point cloud data. The basis function in this application is the basis function in Step 311.

[0084] Step 324: Smooth the fitted three-dimensional model to eliminate surface noise and redundant details, and complete the construction of the three-dimensional model. In this application, the smoothing process can be performed using mean filtering and Gaussian filtering, which will not be elaborated here.

[0085] In this application, by establishing a three-dimensional model, during the subsequent real-time registration process, a model basis can be extracted. By adding the patient's physiological parameter information and body position change information to the three-dimensional model, more accurate three-dimensional model data can be further provided, thereby further improving the registration of thoracic surgery navigation data.

[0086] Step 202: Extract the first feature set of the three-dimensional model and the second feature set of the intraoperative real-time image.

[0087] In the embodiments of this application, feature extraction is performed on the three-dimensional model and the intraoperative real-time image respectively based on deep learning to obtain the first feature set and the second feature set. Feature extraction through deep learning can automatically learn features through a multi-layer neural network, discover useful feature representations from a large amount of raw data, and have good generalization ability on unseen data.

[0088] Step 203: Perform initial registration of the surgical navigation data for the first feature set and the second feature set.

[0089] In the embodiments of this application, performing initial registration of the surgical navigation data for the first feature set and the second feature set includes:

[0090] Obtain the corresponding relationship between the first feature set and the second feature set based on a preset similarity algorithm. The preset similarity algorithm in this application can use algorithms such as Euclidean distance and cosine similarity.

[0091] Based on the matched feature points, obtain the deformation parameters of the control points. In this application, the least squares method can be used to minimize the error between the feature points to solve the deformation parameters. The deformation parameters in this application describe the deformation amounts of the control points in each direction.

[0092] Apply the obtained deformation parameters to the control point grid, causing the corresponding deformation of the control points. Obtain the deformation of other feature points in the entire deformation space through the interpolation algorithm, and then realize the deformation of the first feature set, so that the first feature set and the second feature set complete the initial registration. By performing the initial registration on the preoperative image data and the intraoperative image data in this application, the difference in the spatial range between the preoperative image data and the intraoperative image data is reduced.

[0093] Step 204: Use the feature matching pairs in the initial registration as the input of the preset optimization algorithm, and realize the real-time registration of the preoperative image data and the intraoperative image data by iteratively adjusting the position and attitude of the three-dimensional model.

[0094] In the embodiment of this application, the preset optimization algorithm is the Iterative Closest Point (ICP) algorithm. The core idea of the ICP algorithm is to gradually reduce the distance difference between two point clouds through iterative optimization, so as to achieve high-precision registration.

[0095] In the embodiment of this application, based on the ICP algorithm, the real-time registration of the preoperative image data and the intraoperative image data is realized, including:

[0096] Set an initial transformation matrix for the preoperative image data of the initial registration.

[0097] Based on the nearest neighbor search in the intraoperative image data, obtain the corresponding points closest to each point of the preoperative image data of the initial registration, and obtain the matching point pairs. By using the nearest neighbor search in this application, the calculation speed of the corresponding points, that is, the matching point pairs, between the preoperative image data and the intraoperative image data is accelerated. In this application, the KD tree is used to accelerate the calculation of the nearest neighbor search.

[0098] Based on the matching point pairs, obtain the optimal rotation matrix and translation vector of the preoperative image data of the initial registration, so that the error between the intraoperative image data and the preoperative image data of the initial registration is minimized.

[0099] Apply the optimal rotation matrix and translation vector to the preoperative image data of the initial registration, so that the preoperative image data of the initial registration is closer to the intraoperative image data, and complete the real-time registration of the preoperative image data and the intraoperative image data.

[0100] In the embodiment of this application, the real-time registration of the preoperative image data and the intraoperative image data can be realized in the following way:

[0101] (1) In the three-dimensional space, the transformation matrix T is composed of the rotation matrix R and the translation vector t, and the form is: Where the initial rotation matrix R is set as the identity matrix, and the initial translation vector t is set as the zero vector. The initial transformation matrix T is a 4x4 identity matrix. The identity matrix in this application does not perform any rotation and translation operations at the initial moment.

[0102] (2) Create a KD tree for the intraoperative image data point cloud Q. The KD tree divides the k-dimensional space of the intraoperative image data point cloud into different regions, and each node represents a hyper-rectangular region. During the construction process, the splitting axis is alternately selected in different dimensions according to the coordinate values of the point cloud, and the point cloud is recursively divided into left and right subtrees until a certain stopping condition is met (such as the number of points in the subtrees is less than a preset threshold).

[0103] For each point p in the preoperative image data point cloud P i , perform a nearest neighbor search in the KD tree of the intraoperative image data point cloud Q. Starting from the root node of the KD tree, determine whether to search in the left subtree or the right subtree according to the coordinate value of point p i on the current splitting axis, and gradually approach the nearest neighbor point. By calculating the distance (such as Euclidean distance) between point p i and the points within the region represented by the node in the intraoperative image data point cloud Q, find the point q i that is closest to point p i , and take (p i , q i ) as a set of matching point pairs.

[0104] (3) Based on the obtained matching point pairs, calculate the optimal rotation matrix R and translation vector t to minimize the error between the preoperative image data point cloud P after transformation and the intraoperative image data point cloud Q.

[0105] For the matching points (p i , q i ), i = 1, 2, 3..., k, where k is the number of matching point pairs. Calculate the centroids cp and cq of the preoperative image data point cloud P and the intraoperative image data point cloud Q respectively.

[0106] Calculate the covariance matrix Perform singular value decomposition on the covariance matrix H = UΣV T , then the rotation matrix R is R = VU T , and the translation vector t is t = cq - Rcp.

[0107] (4) Combine the calculated rotation matrix R and translation vector t to form a transformation matrix T, and transform each point p i in the preoperative image data point cloud P: p i ' = Rp i + t, to obtain the transformed preoperative image data point cloud P i ' = {p1', p2',..., p n '}, making the preoperative image data point cloud closer to the intraoperative image data point cloud Q.

[0108] (5) Repeat the above steps 2 to 4 until the convergence condition is met. The convergence condition can be that the number of iterations reaches a preset value (such as 100 times), or the change in the error between the preoperative image data point cloud and the intraoperative image data point cloud is less than a preset threshold. Each iteration can make the preoperative image data point cloud more accurately registered to the intraoperative image data point cloud. After multiple iterations, the final registration result is obtained.

[0109] In this application, through the initial registration of the preoperative matrix and real-time registration through the iterative closest point algorithm, the positions between the preoperative image data and the intraoperative image data are closer. An initial transformation matrix is set for the preoperative image data through the initial registration, avoiding the problem of local optimal solutions caused by the non-approximation of the initial positions of the preoperative image data and the intraoperative image data.

[0110] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above method embodiments. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, an optical disc, a read-only memory (ROM), or a random access memory (RAM), etc.

[0111] It should be understood that although the steps in the flowchart of the accompanying drawings are shown in sequence according to the arrows, these steps do not necessarily have to be executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings can include multiple sub-steps or multiple stages. These sub-steps or stages do not necessarily have to be executed at the same time, but can be executed at different times. Their execution order does not necessarily have to be sequential, but can be executed alternately or alternately with at least a part of other steps or sub-steps or stages of other steps.

[0112] Continue to refer to Figure 6 , the thoracic surgery navigation data registration system described in this embodiment includes:

[0113] A three-dimensional model construction module 601 for constructing a three-dimensional model of the patient's preoperative thoracic surgery;

[0114] A feature extraction module 602 for extracting the first feature set of the three-dimensional model and the second feature set of the intraoperative real-time image;

[0115] An initial registration module 603 for performing initial registration of the surgical navigation data on the first feature set and the second feature set;

[0116] The real-time registration module 604 is configured to use the feature matching pairs in the initial registration as the input of a preset optimization algorithm, and realize the real-time registration of the preoperative image data and the intraoperative image data by iteratively adjusting the position and posture of the three-dimensional model.

[0117] This application constructs a three-dimensional model of the patient's preoperative thoracic surgery; extracts the first feature set of the three-dimensional model and the second feature set of the intraoperative real-time image; performs initial registration of the surgical navigation data on the first feature set and the second feature set; uses the feature matching pairs in the initial registration result as the input of a preset optimization algorithm, and realizes the real-time registration of the preoperative image data and the intraoperative image data by iteratively adjusting the position and posture of the three-dimensional model. By performing real-time registration of the preoperative image data and the intraoperative image data in real time, this application enables the three-dimensional model to be adjusted in real time when the patient's body position and posture change during the operation, so as to achieve the stability and real-time performance of the preoperative image data and the intraoperative image data. Using the iterative closest point for the final registration improves the accuracy of the registration of the surgical navigation data.

[0118] To solve the above technical problems, an embodiment of this application also provides a computer device. For details, please refer to Figure 7 , Figure 7 which is the basic structural block diagram of the computer device in this embodiment.

[0119] The computer device 7 includes a memory 7a, a processor 7b, and a network interface 7c that are communicatively connected to each other through a system bus. It should be noted that only the computer device 7 with components 7a - 7c is shown in the figure, but it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Among them, those skilled in the art of this technology can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0120] The computer device can be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. The computer device can perform human-computer interaction with the user through means such as a keyboard, a mouse, a remote control, a touchpad, or a voice control device.

[0121] The memory 7a at least includes one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 7a may be an internal storage unit of the computer device 7, such as the hard disk or memory of the computer device 7. In other embodiments, the memory 7a may also be an external storage device of the computer device 7, such as a plug-in hard disk, SmartMediaCard (SMC), Secure Digital (SD) card, FlashCard, etc. equipped on the computer device 7. Of course, the memory 7a may also include both the internal storage unit and the external storage device of the computer device 7. In this embodiment, the memory 7a is generally used to store the operating system and various application software installed on the computer device 7, such as the program code of the thoracic surgical navigation data registration method. In addition, the memory 7a may also be used to temporarily store various data that have been output or will be output.

[0122] In some embodiments, the processor 7b may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips. The processor 7b is generally used to control the overall operation of the computer device 7. In this embodiment, the processor 7b is used to run the program code stored in the memory 7a or process data, such as running the program code of the thoracic surgical navigation data registration method.

[0123] The network interface 7c may include a wireless network interface or a wired network interface, and the network interface 7c is generally used to establish a communication connection between the computer device 7 and other electronic devices.

[0124] The present application also provides another implementation manner, that is, to provide a non-volatile computer-readable storage medium storing a program of a thoracic surgical navigation data registration method, and the thoracic surgical navigation data registration can be executed by at least one processor, so that the at least one processor executes the steps of the thoracic surgical navigation data registration method as described above.

[0125] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-described example methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solution of the present application, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions for causing a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in various embodiments of the present application.

[0126] Obviously, the above-described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The accompanying drawings show the preferred embodiments of the present application, but do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosed content of the present application more thorough and comprehensive. Although the present application has been described in detail with reference to the foregoing embodiments, for those skilled in the art, they can still modify the technical solutions described in the foregoing specific embodiments or equivalently replace some of the technical features. Any equivalent structure directly or indirectly using the content of the specification and drawings of the present application in other related technical fields is equally within the scope of the patent protection of the present application.

Claims

1. A method for registering thoracic surgery navigation data, characterized in that, Including: Constructing a three-dimensional model of the patient's preoperative thoracic surgery; Extracting the first feature set of the three-dimensional model and the second feature set of intraoperative real-time images; Performing initial registration of surgical navigation data for the first feature set and the second feature set; Taking the feature matching pairs in the initial registration result as the input of a preset optimization algorithm, and realizing real-time registration of preoperative image data and intraoperative image data by iteratively adjusting the position and pose of the three-dimensional model.

2. The method for registering thoracic surgery navigation data according to claim 1, characterized in that, Constructing a three-dimensional model of the patient's preoperative thoracic surgery, including: Segmenting the patient's image data based on a preset image segmentation model to obtain image data of different tissues and organs in the image data; constructing a three-dimensional model based on the segmented image data.

3. The method for registering thoracic surgery navigation data according to claim 2, wherein Segmenting the patient's image data based on a preset image segmentation model to obtain image data of different tissues and organs in the image data, including: Initializing the control points of the contour curve based on the range of chest images and the positions of different tissues and organs, and constructing an initial contour curve of a parametric B-spline based on the initialized control points and the B-spline order; Iteratively updating the initial contour curve based on a preset energy function. During the iteration of the initial contour curve, by calculating the gradient of the energy function with respect to the contour curve parameters and adjusting the parameters of the contour curve based on the negative direction of the gradient, the energy function is reduced; when the stopping condition is reached, the iteration of the contour curve is stopped to obtain the final segmentation result; Based on the contour boundary of the final segmentation result, segmenting the chest image data into different regions, each region corresponding to a tissue or organ, and marking the pixels inside the contour as the categories of the corresponding tissues or organs to obtain data of different tissues or organs.

4. The method for registering thoracic surgery navigation data according to claim 2, wherein Constructing a three-dimensional model based on the segmented image data, including: Extracting key points on the contours of tissues or organs in the segmented image, and taking the key points as the point cloud data for constructing the three-dimensional model; Constructing a three-dimensional control point grid based on the distribution of the point cloud data; Performing surface fitting on the point cloud data based on the B-spline surface fitting algorithm; Smoothing the fitted three-dimensional model to complete the construction of the three-dimensional model.

5. The method for registering thoracic surgery navigation data according to claim 1, wherein Performing initial registration of surgical navigation data for the first feature set and the second feature set, including: Obtaining the corresponding relationship between the first feature set and the second feature set based on a preset similarity algorithm; obtaining the deformation parameters of the control points based on the matched feature points; applying the obtained deformation parameters to the control point grid to make the control points undergo corresponding deformations, and obtaining the deformations of other feature points in the entire deformation space through an interpolation algorithm, thereby realizing the deformation of the first feature set and completing the initial registration of the first feature set and the second feature set.

6. The method for registering thoracic surgery navigation data according to claim 1, wherein Taking the feature matching pairs in the initial registration result as the input of a preset optimization algorithm, where the preset optimization algorithm is the iterative closest point algorithm, and realizing real-time registration of preoperative image data and intraoperative image data based on the iterative closest point algorithm.

7. The method for registering thoracic surgery navigation data according to claim 6, wherein Realizing real-time registration of preoperative image data and intraoperative image data based on the iterative closest point algorithm, including: Setting an initial transformation matrix for the preoperative image data of the initial registration; Obtaining the corresponding points closest to each point of the preoperative image data of the initial registration based on the nearest neighbor search in the intraoperative image data to obtain matching point pairs; Based on the matching point pairs, obtain the optimal rotation matrix and translation vector of the pre-operative image data for initial registration, so that the error between the intra-operative image data and the pre-operative image data for initial registration is minimized; Apply the optimal rotation matrix and translation vector to the pre-operative image data for initial registration to complete the real-time registration of the pre-operative image data and the intra-operative image data.

8. A thoracic surgery navigation data registration system for implementing the thoracic surgery navigation data registration method of claims 1-7, characterized in that, It includes: A three-dimensional model construction module for constructing a three-dimensional model of the patient's pre-operative thoracic surgery; A feature extraction module for extracting the first feature set of the three-dimensional model and the second feature set of the intra-operative real-time image; An initial registration module for initially registering the first feature set and the second feature set for surgical navigation data; A real-time registration module for using the feature matching pairs in the initial registration as the input of a preset optimization algorithm, and realizing the real-time registration of the pre-operative image data and the intra-operative image data by iteratively adjusting the position and pose of the three-dimensional model.

9. An electronic device, characterized in that, It includes: One or more processors; A memory; And one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions that, when executed by the device, cause the device to perform the steps of the thoracic surgery navigation data registration method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, A computer program is stored in the computer-readable storage medium, and when it runs on a computer, it causes the computer to perform the steps of the thoracic surgery navigation data registration method according to any one of claims 1 to 7.

Images