A bronchoscopic electromagnetic navigation real-time registration method, system, terminal and storage medium

Abstract

The application belongs to the technical field of medical apparatus and instruments, and discloses a bronchus intraoperative electromagnetic navigation real-time registration method, a system, a terminal and a storage medium, which comprises the following steps: acquiring an airway tree center line point cloud; acquiring first pose information of an electromagnetic sensor, registering the first pose information and the airway tree center line point cloud, and determining a pose transformation relationship between a magnetic field coordinate system and a CT coordinate system; acquiring second pose information of the electromagnetic sensor, transforming the second pose information to the CT coordinate system based on the pose transformation relationship; and registering the transformed second pose information and the airway tree center line point cloud, and updating the pose transformation relationship between the magnetic field coordinate system and the CT coordinate system. The application can realize real-time registration of electromagnetic navigation in a bronchus, constantly update the transformation relationship between the magnetic field coordinate system and the CT coordinate system, and improve the positioning accuracy of electromagnetic navigation.

Description

Technical Field

[0001] This invention relates to the field of medical device technology, and in particular to a method, system, terminal and storage medium for real-time electromagnetic navigation registration during bronchial surgery. Background Technology

[0002] In clinical settings for respiratory diseases such as lung cancer, bronchoscopy has become a key tool for clinical diagnosis and treatment due to its advantages of minimal invasiveness and good visualization.

[0003] Existing bronchoscopy techniques still have significant limitations. When faced with the complex bronchial network, conventional bronchoscopes are difficult to reach the distal lesion areas of the lungs due to their large instrument diameter. At the same time, operators often have difficulty judging the orientation in multi-branched pathways, which leads to the problem of prolonged operation time.

[0004] The existing electromagnetic navigation positioning technology has become quite popular in recent years. It can solve the problem of operators having difficulty judging the location in multi-branch paths to a certain extent. However, the accuracy of electromagnetic navigation positioning technology is greatly affected by the difference between the preoperative three-dimensional reconstruction and the actual anatomical location during the operation. Especially in the case of tissue displacement caused by respiratory movements or changes in patient posture, the positioning deviation of distal small bronchi is more obvious, which directly reduces the diagnosis and treatment effect.

[0005] Current technology still suffers from the problem that the accuracy of electromagnetic navigation positioning is greatly affected by the difference between the preoperative three-dimensional reconstruction and the actual anatomical position during the operation. Therefore, the current technology needs to be improved. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide a method, system, terminal and storage medium for real-time registration of electromagnetic navigation during bronchial surgery, in order to solve the problem that the accuracy of existing electromagnetic navigation positioning technology is greatly affected by the difference between preoperative three-dimensional reconstruction and intraoperative anatomical position.

[0007] The technical solution adopted by this invention to solve the technical problem is as follows:

[0008] In a first aspect, the present invention provides a method for real-time registration of electromagnetic navigation during bronchial surgery, comprising:

[0009] Obtain the point cloud of the airway tree centerline;

[0010] The first pose information of the electromagnetic sensor is obtained, and the first pose information is registered with the point cloud of the airway tree centerline to determine the pose transformation relationship between the magnetic field coordinate system and the CT coordinate system.

[0011] Acquire the second pose information of the electromagnetic sensor, and transform the second pose information to the CT coordinate system based on the pose transformation relationship;

[0012] The transformed second pose information is registered with the airway tree centerline point cloud to update the pose transformation relationship between the magnetic field coordinate system and the CT coordinate system.

[0013] In one implementation, obtaining the airway tree centerline point cloud includes:

[0014] Preoperative image reconstruction was performed based on CT data segmentation and calculation methods to obtain the airway tree centerline point cloud generated by the preoperative image reconstruction.

[0015] In one implementation, the step of acquiring the first pose information of the electromagnetic sensor, registering the first pose information with the airway tree centerline point cloud, and determining the pose transformation relationship between the magnetic field coordinate system and the CT coordinate system includes:

[0016] Acquire the first pose information generated by the movement of the electromagnetic sensor in the airway tree;

[0017] The first trajectory point cloud of the electromagnetic sensor is obtained based on the first pose information;

[0018] The first trajectory point cloud is registered with the airway tree centerline point cloud to determine the pose transformation relationship between the magnetic field coordinate system and the CT coordinate system.

[0019] In one implementation, before registering the transformed second pose information with the airway tree centerline point cloud, the method further includes:

[0020] Obtain the current airway branch where the electromagnetic sensor is located;

[0021] The airway branch where the electromagnetic sensor is currently located is included in the point cloud of the airway tree centerline.

[0022] In one implementation, obtaining the airway branch currently occupied by the electromagnetic sensor includes:

[0023] Obtain a set of airway branch centerlines; the set of airway branch centerlines includes at least the starting point and the ending point of the airway branch centerlines.

[0024] Based on the second pose information and the set of airway branch centerlines, the first distance between the airway branch centerline and the electromagnetic sensor is calculated.

[0025] Minimize the first distance, determine the nearest airway branch corresponding to the minimized first distance, and take the nearest airway branch as the airway branch where the electromagnetic sensor is currently located.

[0026] In one implementation, the step of obtaining the airway branch where the electromagnetic sensor is currently located further includes:

[0027] When there are multiple nearest airway branches, obtain the airway branch centerline direction of each nearest airway branch;

[0028] Based on the second pose information and the centerline direction of each airway branch, the nearest airway branch to the target is selected.

[0029] In one implementation, the real-time registration method for electromagnetic navigation during bronchial surgery further includes:

[0030] Determine whether the current airway branch is a new branch; wherein, a new branch is defined as an airway branch that the electromagnetic sensor has not entered before;

[0031] If the current airway branch is a new branch, determine whether the number of trajectory points collected in the new branch meets the preset requirements;

[0032] If the preset requirements are met, the transformed second pose information is registered with the airway tree centerline point cloud based on the radius weighted point cloud registration method, so as to update the pose transformation relationship between the magnetic field coordinate system and the CT coordinate system.

[0033] In one implementation, determining whether the trajectory points collected in the new branch meet preset requirements includes:

[0034] Based on the second pose information, obtain the start and end points of the center line of the airway branch where the electromagnetic sensor is currently located;

[0035] Based on the second pose information and the starting point, a first vector is constructed;

[0036] Calculate the first length of the first vector projected onto the center line, and calculate the vector length of the center line based on the starting point and the ending point;

[0037] Determine whether the first length is greater than half the vector length of the center line;

[0038] When the first length is greater than half the vector length of the center line, it is determined that the number of trajectory points collected in the new branch meets the preset requirement.

[0039] In one implementation, the step of registering the transformed second pose information with the airway tree centerline point cloud to update the pose transformation relationship between the magnetic field coordinate system and the CT coordinate system includes:

[0040] Based on the transformed second pose information, select the trajectory point, obtain the first centerline point that is closest to the selected trajectory point in terms of Euclidean distance, and calculate the distance between the two points.

[0041] The minimum distance is obtained by minimizing the distance between the two points using the least squares method.

[0042] The pose transformation relationship increment between the magnetic field coordinate system and the CT coordinate system is calculated based on the minimum distance, and the pose transformation relationship between the magnetic field coordinate system and the CT coordinate system is updated according to the pose transformation relationship increment.

[0043] In a second aspect, the present invention provides a real-time electromagnetic navigation registration system for bronchial surgery, comprising:

[0044] The preoperative data collection module is used to acquire the point cloud of the airway tree centerline;

[0045] The registration module is used to obtain the first pose information of the electromagnetic sensor, register the first pose information with the point cloud of the airway tree centerline, and determine the pose transformation relationship between the magnetic field coordinate system and the CT coordinate system.

[0046] The positioning module is used to acquire the second pose information of the electromagnetic sensor and transform the second pose information to the CT coordinate system based on the pose transformation relationship.

[0047] The real-time registration module is used to register the transformed second pose information with the airway tree centerline point cloud and update the pose transformation relationship between the magnetic field coordinate system and the CT coordinate system.

[0048] Thirdly, the present invention provides a terminal, comprising: a processor and a memory, wherein the memory stores a real-time registration program for electromagnetic navigation during bronchial surgery, and the real-time registration program for electromagnetic navigation during bronchial surgery, when executed by the processor, is used to implement the operation of the real-time registration method for electromagnetic navigation during bronchial surgery as described in the first aspect.

[0049] Fourthly, the present invention also provides a computer-readable storage medium storing a real-time registration program for electromagnetic navigation during bronchial surgery, which, when executed by a processor, is used to implement the operation of the real-time registration method for electromagnetic navigation during bronchial surgery as described in the first aspect.

[0050] The present invention, by employing the above technical solution, has the following effects:

[0051] This invention, based on the transformation relationship between the magnetic field coordinate system and the CT coordinate system obtained during the registration process, converts the pose information of each newly received electromagnetic sensor frame into the CT coordinate system. Then, based on the branch proximity filtering method, it calculates the airway branch where the end of the bronchoscope is located and incorporates this branch into the centerline point cloud. It determines whether the current airway branch is a new branch, and after collecting enough trajectory points in the new branch, it uses the radius weight point cloud registration method to re-register the trajectory point cloud and the centerline point cloud. Both the trajectory point cloud and the centerline point cloud are continuously updated and re-registered, which can continuously update the transformation relationship between the magnetic field coordinate system and the CT coordinate system, thereby improving the positioning accuracy of electromagnetic navigation. Furthermore, by setting the constraint of collecting enough points, it can improve the utilization rate of system resources while ensuring the optimization effect. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0053] Figure 1 This is a flowchart of the real-time registration method for electromagnetic navigation during bronchial surgery in this invention.

[0054] Figure 2 This is a schematic diagram of the positioning steps and algorithm flow of the real-time registration method for electromagnetic navigation during bronchial surgery in this invention.

[0055] Figure 3 This is a schematic diagram of the location and structure of the electromagnetic sensor in the airway tree, representing one implementation of the present invention.

[0056] Figure 4 This is a functional schematic diagram of the terminal in one implementation of the present invention.

[0057] The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0059] Exemplary methods

[0060] Existing electromagnetic navigation technologies generally consist of two steps: registration and localization. Current technologies typically focus more on the registration step, neglecting the localization step. During registration, the operator moves the bronchoscope from the main carina position to the upper and lower lobes of the left and right lungs, respectively. During this process, the operator receives real-time 6DOF pose information from the electromagnetic sensors fixed to the bronchoscope end. Then, the collected point clouds of all electromagnetic sensor positions are registered with the airway centerline point cloud to obtain the relative transformation relationship between the magnetic field coordinates and the CT coordinate system. In the localization step, this coordinate transformation relationship is applied to the real-time electromagnetic sensors, thereby transforming the head position from the magnetic field coordinate system to the CT coordinate system, achieving electromagnetic localization. However, the accuracy of existing electromagnetic navigation technologies is significantly affected by the difference between preoperative 3D reconstruction and the actual intraoperative position; therefore, the current technology needs further improvement.

[0061] To address the above technical problems, this invention provides a real-time registration method for electromagnetic navigation during bronchial surgery, comprising: acquiring an airway tree centerline point cloud; acquiring first pose information of an electromagnetic sensor, registering the first pose information with the airway tree centerline point cloud, and determining the pose transformation relationship between the magnetic field coordinate system and the CT coordinate system; acquiring second pose information of the electromagnetic sensor, transforming the second pose information to the CT coordinate system based on the pose transformation relationship; registering the transformed second pose information with the airway tree centerline point cloud, and updating the pose transformation relationship between the magnetic field coordinate system and the CT coordinate system; this invention enables real-time registration of electromagnetic navigation during bronchial surgery, continuously updating the transformation relationship between the magnetic field coordinate system and the CT coordinate system, and improving the positioning accuracy of electromagnetic navigation.

[0062] like Figure 1 As shown, this embodiment of the invention provides a real-time electromagnetic navigation registration method for bronchial surgery, comprising the following steps:

[0063] Step S100: Obtain the point cloud of the airway tree centerline.

[0064] It should be noted that the electromagnetic navigation bronchoscope is a minimally invasive diagnostic and treatment tool that combines electromagnetic positioning technology, 3D CT imaging, and virtual bronchoscopy. An electromagnetic sensor is embedded at the tip of the bronchoscope. The method of using this tool typically includes two steps: registration and positioning. The registration step is mainly used to obtain the transformation relationship between the magnetic field coordinate system and the CT coordinate system. In the positioning step, this transformation relationship is used to transform the real-time acquired pose information of the electromagnetic sensor into the CT coordinate system. This allows the position and orientation of the bronchoscope tip in the CT coordinate system to be determined based on the pose information of the electromagnetic sensor, achieving the desired positioning effect.

[0065] Specifically, in one implementation of this embodiment, obtaining the airway tree centerline point cloud includes: completing preoperative image reconstruction based on CT data segmentation calculation method, and obtaining the airway tree centerline point cloud generated by the preoperative image reconstruction.

[0066] In this embodiment, preoperative image reconstruction based on CT data segmentation calculation method includes: acquiring two-dimensional and / or three-dimensional CT data, extracting the precise three-dimensional structure of the target tissue from the CT data using CT data segmentation calculation method, and performing preoperative image reconstruction.

[0067] As an example, in this embodiment, the three-dimensional structure of the target tissue that has already undergone preoperative image reconstruction can also be directly obtained.

[0068] It should be noted that the three-dimensional precise structure of the target tissue is obtained, and the target tissue depends on the acquired CT data. In this embodiment, the acquired CT data is the CT data of the airway tree, so that the airway tree generated by the preoperative image reconstruction can be obtained. The airway tree centerline point cloud is formed by processing the three-dimensional airway tree structure.

[0069] like Figure 1 As shown, this embodiment of the invention provides a real-time electromagnetic navigation registration method for bronchial surgery, comprising the following steps:

[0070] Step S200: Obtain the first pose information of the electromagnetic sensor, register the first pose information with the point cloud of the airway tree centerline, and determine the pose transformation relationship between the magnetic field coordinate system and the CT coordinate system.

[0071] Specifically, in one implementation of this embodiment, step S200 includes the following steps:

[0072] Step S201: Obtain the first pose information generated by the electromagnetic sensor moving in the airway tree.

[0073] In this embodiment, the position and attitude information generated by the electromagnetic sensor moving in the airway tree during the registration step are acquired in real time. To distinguish it from the positioning step, the position and attitude information of the electromagnetic sensor in the registration step are recorded as the first position information and the first attitude information. The first position information and the first attitude information are combined to obtain the first attitude information of the electromagnetic sensor, which can completely describe the state of the electromagnetic sensor in the three-dimensional airway tree.

[0074] Step S202: Obtain the first trajectory point cloud of the electromagnetic sensor based on the first pose information.

[0075] In this embodiment, by moving the bronchoscope within the airway tree, the movement range of the electromagnetic sensor will cover the entire airway tree as much as possible. During the movement, the first pose information of each frame of the electromagnetic sensor can be acquired, and the first trajectory point cloud of the electromagnetic sensor in the airway tree can be obtained by processing the first pose information in the time sequence of the corresponding frames.

[0076] Step S203: Register the first trajectory point cloud with the airway tree centerline point cloud to determine the pose transformation relationship between the magnetic field coordinate system and the CT coordinate system.

[0077] It should be noted that during the registration process, the first trajectory point cloud and the airway tree centerline point cloud are registered to obtain the initial pose transformation relationship between the magnetic field coordinate system and the CT coordinate system.

[0078] In this embodiment, point cloud registration technology is used to construct the pose transformation relationship between the magnetic field coordinate system and the CT coordinate system during the registration step. It can realize the positioning of electromagnetic sensors during bronchial surgery, thereby completing electromagnetic navigation during bronchial surgery.

[0079] like Figure 1 As shown, this embodiment of the invention provides a real-time electromagnetic navigation registration method for bronchial surgery, comprising the following steps:

[0080] Step S300: Obtain the second pose information of the electromagnetic sensor, and transform the second pose information to the CT coordinate system based on the pose transformation relationship.

[0081] like Figure 2 The diagram shows the positioning steps of bronchoscopic electromagnetic navigation technology. First, the second position and second attitude information of the electromagnetic sensor in the airway tree are acquired in real time to obtain the second pose information of the electromagnetic sensor. Then, the airway branch at the end of the bronchoscope, i.e., where the electromagnetic sensor is located, is obtained based on the branch proximity filtering method. It is then determined whether the airway branch is a new branch. If it is a new branch, the centerline point cloud is directly acquired and included in the target centerline point cloud. If the airway branch is not a new branch, the second trajectory point cloud is re-registered with the centerline point cloud based on the radius weight point cloud registration method to update the transformation relationship between the magnetic field coordinate system and the CT coordinate system. This allows the pose information of the electromagnetic sensor acquired subsequently to be transformed into the CT coordinate system based on the updated transformation relationship, thereby improving the positioning accuracy of the electromagnetic sensor in the airway tree.

[0082] It should be noted that the text description distinguishes it from the first step of the registration process by using the term "second," and it is not included in the text. Figure 2 stated in .

[0083] Specifically, in one implementation of this embodiment, step S300 includes the following steps:

[0084] Step S301: Obtain the second pose information of the electromagnetic sensor, and transform the second pose information to the CT coordinate system based on the pose transformation relationship.

[0085] In this embodiment, the second pose information of the magnetic field coordinate system includes the second position information. Second attitude information The second position information is transformed into the CT coordinate system. Second attitude information The corresponding transformation formula is as follows:

[0086] ;

[0087] .

[0088] Using the initial pose transformation relationship between the magnetic field coordinate system and the CT coordinate system obtained in step S200, the second pose information of the electromagnetic sensor is transformed from the magnetic field coordinate system to the CT coordinate system; at the same time, the second trajectory point cloud of the electromagnetic sensor in the airway tree is obtained based on the second pose information of the electromagnetic sensor, that is, the trajectory point cloud of the electromagnetic sensor in the positioning step.

[0089] Step S302: Obtain the current airway branch where the electromagnetic sensor is located.

[0090] It should be noted that the branch proximity screening method is a technique for identifying adjacent branches in a spatial or topological structure. The core objective is to filter out spatially adjacent branch nodes or paths by example or topological relationship. In this embodiment, the branch proximity screening method is used to screen the nearest branch of the bronchoscope terminal electromagnetic sensor in the airway tree, thereby obtaining the airway branch where the electromagnetic sensor is currently located.

[0091] Specifically, in one implementation of this embodiment, step S302 includes the following steps:

[0092] Step S302a: Obtain the set of airway branch centerlines; the set of airway branch centerlines includes at least the starting point and ending point of the airway branch centerlines.

[0093] like Figure 3 The diagram shows the location and structure of the electromagnetic sensor within the airway tree, centered on the airway branch centerlines. Represents the airway tree, the set of airway branch centerlines. Includes Central line of airway branch , By adding subscripts to indicate the corresponding lines, the starting point of the airway branch centerline is used. and termination point This indicates the centerline of the airway branch; superscripts are used to denote the corresponding points. It actually contains a set of three-dimensional coordinate points. Therefore, it indicates the first The set of three-dimensional coordinate points of the airway branch centerlines can also be called a point cloud. In summary, the set of airway branch centerlines... It can be represented as: .

[0094] Step S302b: Based on the second pose information and the set of airway branch centerlines, calculate the first distance between the airway branch centerline and the electromagnetic sensor.

[0095] In this embodiment, the second pose information is the second pose information of the electromagnetic sensor in the CT coordinate system, and the current second position information of the electromagnetic sensor is obtained based on the second pose information. The second location information also represents the position of the end of the bronchoscope.

[0096] In this embodiment, the second location information is obtained. airway branch centerline convergence Calculate the first distance between the airway branch centerline and the electromagnetic sensor. First distance The calculation formula is as follows:

[0097] ;

[0098] in, Indicates the center line of the airway branch The length of the vector. Representing vectors Projected onto the center line of the airway branch The length of the length, and there exists a projection point. ;when When, it represents the projection point. On the center line of the parent branch of the current airway branch; when When, it represents the projection point. On the center line of the sub-branch of the current airway branch; otherwise, it indicates the projection point. Falling on the center line of the current airway branch superior.

[0099] Furthermore, The calculation formula is as follows:

[0100] ;

[0101] The calculation formula is as follows:

[0102] .

[0103] Step S302c: Minimize the first distance, determine the nearest airway branch corresponding to the minimized first distance, and take the nearest airway branch as the airway branch where the electromagnetic sensor is currently located.

[0104] In this embodiment, the first distance is minimized. This allows us to obtain the nearest airway branch corresponding to the minimized first distance. The nearest airway branch is taken as the airway branch where the electromagnetic sensor is currently located, that is, the nearest airway branch at the end of the bronchoscope. The formula for calculating this nearest airway branch is as follows:

[0105] .

[0106] As an example, in this embodiment, when there are multiple nearest neighbor airway branches, it is necessary to determine a unique nearest neighbor airway branch. The determination method includes:

[0107] Step S302d: When there are multiple nearest airway branches, obtain the direction of the airway branch centerline of each nearest airway branch.

[0108] It should be noted that, through step S404, the calculation may simultaneously obtain the first distance of multiple electromagnetic sensors relative to the current position of the bronchoscope tip. Equal nearest branches When there are multiple nearest neighbors, it is denoted as Therefore, it is necessary to determine the unique nearest airway branch. The method for determining this branch first requires obtaining the direction of the centerline of each nearest airway branch.

[0109] The direction of the airway branch centerline is represented as follows: .

[0110] Step S302e: Based on the second pose information and the centerline direction of each airway branch, filter the nearest airway branch to the target.

[0111] In this embodiment, when multiple nearest airway branches exist, the current second pose information of the electromagnetic sensor needs to be obtained first based on the second pose information, and then the nearest airway branch of the target is selected based on the second pose information. The method for obtaining the second pose information is as follows:

[0112] The current second pose information of the electromagnetic sensor is obtained based on the second pose information. The second pose information represents the pose information of the electromagnetic sensor. That is, the end position of the bronchoscope. and orientation ,in This indicates the orientation of the z-axis in the coordinate system at the end of the bronchoscope, while Represents the terminal pose.

[0113] In this embodiment, after obtaining the second pose information, the nearest airway branch to the target is selected based on the second pose information and the centerline direction of each airway branch. The method is as follows:

[0114] According to the direction of the airway branch centerline Second attitude information Determine the optimal nearest airway branch and use the centerline of the nearest airway branch. The formula for calculating the centerline of the nearest airway branch is as follows:

[0115] .

[0116] Step S303: Incorporate the airway branch where the electromagnetic sensor is currently located into the point cloud of the airway tree centerline.

[0117] like Figure 1 As shown, this embodiment of the invention provides a real-time electromagnetic navigation registration method for bronchial surgery, comprising the following steps:

[0118] Step S400: Register the transformed second pose information with the airway tree centerline point cloud, and update the pose transformation relationship between the magnetic field coordinate system and the CT coordinate system.

[0119] Specifically, in one implementation of this embodiment, step S400 includes the following steps:

[0120] Step S400a: Select trajectory points based on the transformed second pose information, obtain the first centerline point that is closest to the selected trajectory point in terms of Euclidean distance, and calculate the distance between the two points.

[0121] In this embodiment, trajectory points are selected based on the transformed second pose information, i.e., the transformed second pose information is obtained, and a second trajectory point cloud is obtained based on the transformed second pose information. The second trajectory point cloud is the trajectory point cloud of the electromagnetic sensor in the positioning step. Trajectory points are selected from this second trajectory point cloud, and the second position information in the second pose information is used. Representing the trajectory point, obtain the first centerline point that has the closest Euclidean distance to the trajectory point. ,in, This allows us to calculate the distance between the trajectory point and the first centerline point. The calculation formula is as follows:

[0122] ;

[0123] in, The symbol represents the antisymmetric matrix of the Lie algebra. The function represents the exponential mapping of Lie algebras; Indicates the current centerline point The radius of the airway to which it belongs; Indicates the angle increment. This represents the displacement increment, which is equivalent to the pose increment. .

[0124] In addition, distance It also satisfies the condition of being the current centerline point. airway radius Normalization. The first trajectory point and the first centerline point. Distance between The calculation formula also shows that the closer the first trajectory point is to the first centerline, the more significant the transformation relationship between the magnetic field coordinate system and the CT coordinate system. The more accurate.

[0125] Step S400b: Minimize the distance between the two points using the least squares method to obtain the minimum distance.

[0126] In this embodiment, the least squares method is used to minimize the distance between the trajectory point and the first centerline point. Distance between The formula for the least squares equation, obtained by finding the minimum distance, is as follows:

[0127] ;

[0128] in, This represents the objective function.

[0129] Furthermore, due to the objective function The Jacobian matrix can be calculated and defined based on the Lie algebra perturbation model:

[0130] ;

[0131] in, for Identity matrix.

[0132] Therefore, the above least squares equation can be solved by any gradient-based optimization method, such as Newton's method, LBFGS, Gauss-Newton method, Levenberg-Marquardt method, etc.

[0133] Step S400c: Calculate the pose transformation relationship increment between the magnetic field coordinate system and the CT coordinate system based on the minimum distance, and update the pose transformation relationship between the magnetic field coordinate system and the CT coordinate system according to the pose transformation relationship increment.

[0134] In this embodiment, the incremental relationship between the pose change of the magnetic field coordinate system and the CT coordinate system is calculated based on the minimum distance. Specifically, based on the solution of the minimum distance, the corresponding angle increment can be obtained. and displacement increment To determine the incremental relationship of pose change :

[0135] ;

[0136] In this embodiment, the pose transformation relationship between the magnetic field coordinate system and the CT coordinate system is updated incrementally according to the pose transformation relationship. The corresponding formula is as follows:

[0137] .

[0138] In this embodiment, the airway radius weight is used in the weighted axis vector point cloud registration method designed for airway characteristics to normalize the error. This results in the small errors in the small airway being relatively amplified, so that the optimization of the transformation matrix is ​​not tilted towards the large airway, thereby improving the positioning accuracy of the small airway.

[0139] As an example, in this embodiment, before registering the transformed second pose information with the airway tree centerline point cloud, the method further includes: confirming whether the current airway branch is a new branch and determining whether the current collected trajectory point meets the preset conditions. Based on this step, step S400 is optimized, and the optimized step S400 includes:

[0140] Step S401: Determine whether the current airway branch is a new branch; wherein, the new branch is defined as an airway branch that the electromagnetic sensor has not entered before.

[0141] In this embodiment, an algorithm determines whether the current airway branch is a new branch. The algorithm can identify whether the airway branch entered by the electromagnetic sensor is an airway branch that has not been entered before, i.e., an airway branch that was not entered during the registration and positioning steps. Specifically, whenever the algorithm identifies that the bronchoscope has entered an unfamiliar airway branch (a branch that was not entered during registration and positioning), the algorithm will register the trajectory point cloud and the target point cloud based on the radius-weighted point cloud registration method, thereby continuously updating the transformation relationship between the magnetic field coordinate system and the CT coordinate system during the positioning process, improving the electromagnetic navigation positioning accuracy. In addition, due to the presence of airway radius weights in the weighted axis vector point cloud registration method designed for airway characteristics, small errors in small airways will be relatively amplified through error normalization, so that the optimization of the transformation matrix will not be tilted towards the large airways, thereby improving the positioning accuracy of small airways.

[0142] Step S402: If the current airway branch is a new branch, determine whether the number of trajectory points collected in the new branch meets the preset requirements.

[0143] In this embodiment, when the airway branch into which the electromagnetic sensor enters or when the airway branch it is in is a new branch, it is necessary to determine whether the number of trajectory points of the electromagnetic sensor collected in the new branch meets the preset requirements, that is, whether enough trajectory points are collected in the new branch. Collecting enough trajectory points is a prerequisite for using the radius-based weighted cloud registration method.

[0144] Specifically, in one implementation of this embodiment, step S402 includes the following steps:

[0145] Step S402a: Obtain the start and end points of the center line of the airway branch where the electromagnetic sensor is currently located based on the second pose information.

[0146] In this embodiment, the current second position information of the electromagnetic sensor is obtained based on the second pose information. The second location information also represents the position of the end of the bronchoscope; obtain the center line of the airway branch where the electromagnetic sensor is currently located. starting point and termination point .

[0147] Step S402b: Construct a first vector based on the second pose information and the starting point.

[0148] In this embodiment, based on the second position information in the second pose information and starting point Construct the first vector .

[0149] Step S402c: Calculate the first length of the first vector projected onto the center line, and calculate the vector length of the center line based on the starting point and the ending point.

[0150] In this embodiment, the first vector is calculated. Projected onto the center line The first length on and based on the starting point and termination point Calculate the center line vector length The vector length of the centerline The calculation formula is as follows:

[0151] .

[0152] Step S402d: Determine whether the first length is greater than half the vector length of the center line.

[0153] In this embodiment, the first length is determined. Is it greater than the center line? vector length Half of it.

[0154] Step S402e: When the first length is greater than half the vector length of the center line, it is determined that the number of trajectory points collected in the new branch meets the preset requirements.

[0155] In this embodiment, when the first length Greater than the center line vector length When half of the points are collected, it is determined that the number of trajectory points collected in the new branch meets the preset requirement. Therefore, the preset requirement is: .

[0156] Step S403: If the preset requirements are met, the transformed second pose information is registered with the airway tree centerline point cloud based on the radius weighted point cloud registration method to update the pose transformation relationship between the magnetic field coordinate system and the CT coordinate system.

[0157] Specifically, in one implementation of this embodiment, step S403 includes the following steps:

[0158] Step S403a: Select trajectory points based on the transformed second pose information, obtain the first centerline point that is closest to the selected trajectory point in terms of Euclidean distance, and calculate the distance between the two points;

[0159] Step S403b: Minimize the distance between the two points using the least squares method to obtain the minimum distance;

[0160] Step S403c: Calculate the pose transformation relationship increment between the magnetic field coordinate system and the CT coordinate system based on the minimum distance, and update the pose transformation relationship between the magnetic field coordinate system and the CT coordinate system according to the pose transformation relationship increment.

[0161] Furthermore, the real-time registration method for electromagnetic navigation during bronchial surgery provided in this embodiment of the invention also includes the following steps:

[0162] In step S500, the pose information of the electromagnetic sensor is transformed into the CT coordinate system using the updated pose transformation relationship between the magnetic field coordinate system and the CT coordinate system, thereby realizing the positioning of the electromagnetic sensor in the CT coordinate system.

[0163] In this embodiment, the updated transformation relationship between the magnetic field coordinate system and the CT coordinate system is used. That is, each time a new frame of electromagnetic sensor pose information is received, the latest updated transformation relationship between the magnetic field coordinate system and the CT coordinate system is used. This transformation relationship is based on real-time registration of electromagnetic navigation, thereby transforming the pose information of the electromagnetic sensor to the CT coordinate system and realizing the real-time positioning of the electromagnetic sensor.

[0164] This embodiment achieves the following technical effects through the above technical solution:

[0165] This embodiment, based on the transformation relationship between the magnetic field coordinate system and the CT coordinate system obtained in the registration step, converts the pose information of each new frame of electromagnetic sensor received into the CT coordinate system; then, based on the branch proximity screening method, it calculates the airway branch where the end of the bronchoscope is located and includes this branch in the centerline point cloud; it determines whether the current airway branch is a new branch, and after collecting enough trajectory points in the new branch, it uses the radius weight point cloud registration method to re-register the trajectory point cloud and the centerline point cloud. The trajectory point cloud and the centerline point cloud are continuously updated and re-registered, which can realize the continuous updating of the transformation relationship between the magnetic field coordinate system and the CT coordinate system, improving the positioning accuracy of electromagnetic navigation; and setting the constraint of collecting enough points can improve the utilization rate of system resources while ensuring the optimization effect.

[0166] Exemplary device

[0167] Based on the above embodiments, the present invention also provides a real-time electromagnetic navigation registration system for bronchial surgery, comprising:

[0168] The preoperative data collection module is used to acquire the point cloud of the airway tree centerline;

[0169] The registration module is used to obtain the first pose information of the electromagnetic sensor, register the first pose information with the point cloud of the airway tree centerline, and determine the pose transformation relationship between the magnetic field coordinate system and the CT coordinate system.

[0170] The positioning module is used to acquire the second pose information of the electromagnetic sensor and transform the second pose information to the CT coordinate system based on the pose transformation relationship.

[0171] The real-time registration module is used to register the transformed second pose information with the airway tree centerline point cloud and update the pose transformation relationship between the magnetic field coordinate system and the CT coordinate system.

[0172] As an example, the real-time electromagnetic navigation registration system during bronchial surgery in this embodiment also includes:

[0173] The airway branch acquisition module is used to acquire the airway branch where the electromagnetic sensor is currently located; and to incorporate the airway branch where the electromagnetic sensor is currently located into the airway tree centerline point cloud.

[0174] The new branch determination module is used to determine whether the current airway branch is a new branch; wherein, the new branch is defined as an airway branch that the electromagnetic sensor has not entered before;

[0175] The trajectory point determination module is used to determine whether the number of trajectory points collected in the new branch meets the preset requirements when the current airway branch is a new branch.

[0176] In this embodiment, after using the trajectory point determination module, the real-time registration module is updated as follows:

[0177] The updated real-time registration module is used to register the transformed second pose information with the airway tree centerline point cloud based on the radius weighted point cloud registration method when the preset requirements are met, so as to update the pose transformation relationship between the magnetic field coordinate system and the CT coordinate system.

[0178] This embodiment achieves the following technical effects through the above technical solution:

[0179] The transformation relationship between the magnetic field coordinate system and the CT coordinate system obtained in the registration step of this embodiment is used to convert the pose information of the electromagnetic sensor in each new frame into the CT coordinate system; then, the airway branch at the end of the bronchoscope is calculated based on the branch proximity screening method, and the branch is included in the centerline point cloud; the trajectory point cloud and the centerline point cloud are re-registered using the radius weight point cloud registration method, which can continuously update the transformation relationship between the magnetic field coordinate system and the CT coordinate system and improve the positioning accuracy of electromagnetic navigation.

[0180] Based on the above embodiments, the present invention also provides a terminal, the principle block diagram of which can be as follows: Figure 4 As shown.

[0181] The terminal includes: a processor, a memory, an interface, a display screen, and a communication module connected via a system bus; wherein, the processor of the terminal provides computing and control capabilities; the memory of the terminal includes a computer-readable storage medium and internal memory; the computer-readable storage medium stores an operating system and computer programs; the internal memory provides an environment for the operation of the operating system and computer programs in the computer-readable storage medium; the interface is used to connect to external devices; the display screen is used to display relevant information; and the communication module is used to communicate with a cloud server or other devices.

[0182] When executed by the processor, this computer program is used to implement the real-time registration method of electromagnetic navigation during bronchial surgery.

[0183] It will be understood by those skilled in the art that Figure 4The schematic diagram shown is merely a partial structural diagram related to the present invention and does not constitute a limitation on the terminal to which the present invention is applied. A specific terminal may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0184] In one embodiment, a terminal is provided, comprising: a processor and a memory, the memory storing a real-time registration program for electromagnetic navigation during bronchial surgery, which, when executed by the processor, is used to implement the operation of the real-time registration method for electromagnetic navigation during bronchial surgery as described above.

[0185] In one embodiment, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a real-time registration program for intrabronchial electromagnetic navigation, which, when executed by a processor, is used to implement the operation of the above-described real-time registration method for intrabronchial electromagnetic navigation.

[0186] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, database, or other media used in the embodiments provided by this invention can include both non-volatile and volatile memory.

[0187] In summary, this invention provides a method, system, terminal, and storage medium for real-time registration of electromagnetic navigation during bronchial surgery, comprising: acquiring an airway tree centerline point cloud; acquiring first pose information of an electromagnetic sensor, registering the first pose information with the airway tree centerline point cloud, and determining the pose transformation relationship between the magnetic field coordinate system and the CT coordinate system; acquiring second pose information of the electromagnetic sensor, transforming the second pose information to the CT coordinate system based on the pose transformation relationship; registering the transformed second pose information with the airway tree centerline point cloud, and updating the pose transformation relationship between the magnetic field coordinate system and the CT coordinate system; this invention can achieve real-time registration of electromagnetic navigation in the bronchi, continuously update the transformation relationship between the magnetic field coordinate system and the CT coordinate system, and improve the positioning accuracy of electromagnetic navigation.

[0188] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A method for real-time registration of electromagnetic navigation during bronchial surgery, characterized in that, include: Obtain the point cloud of the airway tree centerline; The first pose information of the electromagnetic sensor is obtained, and the first pose information is registered with the point cloud of the airway tree centerline to determine the pose transformation relationship between the magnetic field coordinate system and the CT coordinate system. Obtain the airway branch where the electromagnetic sensor is currently located, and incorporate the airway branch where the electromagnetic sensor is currently located into the point cloud of the airway tree centerline; Acquire the second pose information of the electromagnetic sensor, and transform the second pose information to the CT coordinate system based on the pose transformation relationship; The transformed second pose information is registered with the airway tree centerline point cloud to update the pose transformation relationship between the magnetic field coordinate system and the CT coordinate system. The step of incorporating the airway branch where the electromagnetic sensor is currently located into the airway tree centerline point cloud specifically involves: Based on the branch proximity screening method, the airway branch where the electromagnetic sensor is currently located is included in the point cloud of the airway tree centerline; The step of registering the transformed second pose information with the airway tree centerline point cloud to update the pose transformation relationship between the magnetic field coordinate system and the CT coordinate system includes: Based on the transformed second pose information, select the trajectory point, obtain the first centerline point that is closest to the selected trajectory point in terms of Euclidean distance, and calculate the distance between the two points. The minimum distance is obtained by minimizing the distance between the two points using the least squares method. The pose transformation relationship increment between the magnetic field coordinate system and the CT coordinate system is calculated based on the minimum distance, and the pose transformation relationship between the magnetic field coordinate system and the CT coordinate system is updated according to the pose transformation relationship increment.

2. The real-time registration method for intraoperative electromagnetic navigation in bronchial surgery according to claim 1, characterized in that, The acquisition of the airway tree centerline point cloud includes: Preoperative image reconstruction was performed based on CT data segmentation and calculation methods to obtain the airway tree centerline point cloud generated by the preoperative image reconstruction.

3. The real-time registration method for electromagnetic navigation during bronchial surgery according to claim 1, characterized in that, The step of acquiring the first pose information of the electromagnetic sensor, registering the first pose information with the airway tree centerline point cloud, and determining the pose transformation relationship between the magnetic field coordinate system and the CT coordinate system includes: Acquire the first pose information generated by the movement of the electromagnetic sensor in the airway tree; The first trajectory point cloud of the electromagnetic sensor is obtained based on the first pose information; The first trajectory point cloud is registered with the airway tree centerline point cloud to determine the pose transformation relationship between the magnetic field coordinate system and the CT coordinate system.

4. The real-time registration method for electromagnetic navigation during bronchial surgery according to claim 1, characterized in that, The step of obtaining the current airway branch where the electromagnetic sensor is located includes: Obtain a set of airway branch centerlines; the set of airway branch centerlines includes at least the starting point and the ending point of the airway branch centerlines. Based on the second pose information and the set of airway branch centerlines, the first distance between the airway branch centerline and the electromagnetic sensor is calculated. Minimize the first distance, determine the nearest airway branch corresponding to the minimized first distance, and take the nearest airway branch as the airway branch where the electromagnetic sensor is currently located.

5. The real-time registration method for electromagnetic navigation during bronchial surgery according to claim 4, characterized in that, The method of obtaining the current airway branch where the electromagnetic sensor is located also includes: When there are multiple nearest airway branches, obtain the airway branch centerline direction of each nearest airway branch; Based on the second pose information and the centerline direction of each airway branch, the nearest airway branch to the target is selected.

6. The real-time registration method for electromagnetic navigation during bronchial surgery according to claim 1, characterized in that, The intraoperative electromagnetic navigation real-time registration method for bronchial surgery also includes: Determine whether the current airway branch is a new branch; wherein, a new branch is defined as an airway branch that the electromagnetic sensor has not entered before; If the current airway branch is a new branch, determine whether the number of trajectory points collected in the new branch meets the preset requirements; If the preset requirements are met, the transformed second pose information is registered with the airway tree centerline point cloud based on the radius weighted point cloud registration method, so as to update the pose transformation relationship between the magnetic field coordinate system and the CT coordinate system.

7. The real-time registration method for electromagnetic navigation during bronchial surgery according to claim 6, characterized in that, The determination of whether the trajectory points collected in the new branch meet the preset requirements includes: Based on the second pose information, obtain the start and end points of the center line of the airway branch where the electromagnetic sensor is currently located; Based on the second pose information and the starting point, a first vector is constructed; Calculate the first length of the first vector projected onto the center line, and calculate the vector length of the center line based on the starting point and the ending point; Determine whether the first length is greater than half the vector length of the center line; When the first length is greater than half the vector length of the center line, it is determined that the number of trajectory points collected in the new branch meets the preset requirement.

8. A real-time registration system for electromagnetic navigation during bronchial surgery, used to implement the real-time registration method for electromagnetic navigation during bronchial surgery as described in any one of claims 1-7, characterized in that, include: The preoperative data collection module is used to acquire the point cloud of the airway tree centerline; The registration module is used to obtain the first pose information of the electromagnetic sensor, register the first pose information with the point cloud of the airway tree centerline, and determine the pose transformation relationship between the magnetic field coordinate system and the CT coordinate system. The positioning module is used to acquire the second pose information of the electromagnetic sensor and transform the second pose information to the CT coordinate system based on the pose transformation relationship. The real-time registration module is used to register the transformed second pose information with the airway tree centerline point cloud and update the pose transformation relationship between the magnetic field coordinate system and the CT coordinate system.

9. A terminal, characterized in that, include: The processor and memory, wherein the memory stores a real-time registration program for electromagnetic navigation during bronchial surgery, which, when executed by the processor, is used to implement the operation of the real-time registration method for electromagnetic navigation during bronchial surgery as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a real-time registration program for intrabronchial electromagnetic navigation, which, when executed by a processor, is used to implement the operation of the real-time registration method for intrabronchial electromagnetic navigation as described in any one of claims 1-7.

Images