A three-dimensional positioning method and apparatus based on a general model

Abstract

The application discloses a three-dimensional positioning method and device based on a general model, and the method comprises the following steps: collecting a two-dimensional projection image in an operation, wherein the two-dimensional projection image comprises projection data of a target tissue and a target instrument which penetrates into the target tissue; generating an auxiliary mark on the basis of the two-dimensional projection image; acquiring an estimated pose of the target instrument in a three-dimensional space based on the auxiliary mark; the auxiliary mark is established based on a two-dimensional projection feature and is used for indicating depth information of the target instrument in the three-dimensional space; and comparing the estimated pose with a general model of the target instrument to obtain a real pose of the target instrument in the three-dimensional space. According to the method, only a two-dimensional image is collected, and real-time pose of a target instrument in a three-dimensional space in an operation can be obtained, the algorithm structure is simple, and the calculation efficiency is high.

Description

Technical Field

[0001] This application relates to the field of medical device technology, and more specifically, to a three-dimensional positioning method and device based on a universal model. Background Technology

[0002] In clinical practice, current procedures for biopsy typically rely on preoperative 3D imaging data as initial navigation. During the procedure, to confirm the real-time position of the needle within the patient, another set of 3D images needs to be acquired intraoperatively, or spatial registration of two 2D images (anteroposterior and lateral views) needs to be performed and projected into 3D space for positioning reference. However, this method has significant limitations: firstly, repeatedly acquiring 3D images increases radiation exposure and prolongs the procedure; secondly, relying on 2D projection for spatial positioning cannot achieve continuous updates of positional information, resulting in significant lag in intraoperative navigation. This limits the accuracy and real-time interactive capabilities of the procedure, impacting overall surgical efficiency and safety. Summary of the Invention

[0003] To address the aforementioned technical issues, this application discloses a three-dimensional positioning method and device based on a general model. By acquiring only a single two-dimensional image, the real-time pose of the target instrument in three-dimensional space during surgery can be determined. The algorithm structure is simplified and the computational efficiency is high.

[0004] Specifically, the technical solution of this application is as follows:

[0005] In a first aspect, this application discloses a three-dimensional positioning method based on a general model, comprising the following steps:

[0006] A single two-dimensional projection image is acquired during the procedure. The two-dimensional projection image includes the projection data of the target tissue and the target instrument inserted into the target tissue.

[0007] An auxiliary label is generated based on the two-dimensional projection image; based on the auxiliary label, the estimated pose of the target instrument in three-dimensional space is obtained; the auxiliary label is established based on two-dimensional projection features and is used to indicate the depth information of the target instrument in the three-dimensional space.

[0008] By comparing the estimated pose with the general model of the target instrument, the true pose of the target instrument in the three-dimensional space is calculated.

[0009] In some embodiments, the target device is a rigid cylinder that does not contain any marking points.

[0010] Furthermore, the generation of auxiliary labels based on the two-dimensional projected image includes:

[0011] Based on the width of the target device in the two-dimensional projection image, the target device is divided into several sub-interval segments;

[0012] Obtain the projection matrix of the imaging system, and use the perspective projection characteristic of imaging to calculate the depth information of each sub-interval segment in three-dimensional space.

[0013] Determine the point of tangency between the cross-section of the target instrument and the ray, as well as the center of the ellipse, by taking any plane passing through the ray source that is not perpendicular to the central axis of the target instrument.

[0014] On each of the cross sections, the line segment connecting the center of the circle and the point of tangency is used as a first auxiliary identifier.

[0015] Furthermore, the step of obtaining the estimated pose of the target instrument in three-dimensional space based on the auxiliary identifier includes:

[0016] Extract the endpoints of each of the first auxiliary identifiers to form a feature point set;

[0017] The fitting axis of the target instrument is obtained by fitting the feature point set.

[0018] The estimated pose of the target instrument in three-dimensional space is determined based on the fitted axis.

[0019] In other embodiments, the target instrument includes a plurality of marker points, the distribution of which is fixed.

[0020] Furthermore, the generation of auxiliary labels based on the two-dimensional projected image includes:

[0021] Obtain the projection matrix of the imaging system, and use the perspective projection characteristic of imaging to calculate the depth information of the marked points in three-dimensional space;

[0022] Connect adjacent marker points with line segments to form a second auxiliary identifier.

[0023] Furthermore, the step of obtaining the estimated pose of the target instrument in three-dimensional space based on the auxiliary identifier includes:

[0024] Extract the endpoints and / or midpoints of each of the second auxiliary identifiers to form a feature point set;

[0025] The fitting axis of the target instrument is obtained by fitting the feature point set.

[0026] The estimated pose of the target instrument in three-dimensional space is determined based on the fitted axis.

[0027] In some implementations, the step of comparing the estimated pose with a general model of the target instrument to calculate the true pose of the target instrument in the three-dimensional space specifically includes the following steps:

[0028] Obtain the general model of the target instrument;

[0029] The general model is translated and rotated in the three-dimensional space so that the central axis of the general model is aligned with the fitting axis.

[0030] The general model is simulated by orthographic projection, and it is determined whether there is a deviation between the orthographic projection result and the projection data of the target device in the two-dimensional projection image. If so, the general model is translated until the orthographic projection result of the general model coincides with the projection data of the target device in the two-dimensional projection image. At this time, the pose of the general model is the true pose of the target device in three-dimensional space.

[0031] Secondly, this application also discloses a three-dimensional positioning device based on a universal model, which is used to implement the steps of a three-dimensional positioning method based on a universal model as described in any of the above embodiments; specifically including:

[0032] An imaging system for acquiring a single two-dimensional projection image during surgery, wherein the two-dimensional projection image includes projection data of the target tissue and the target instrument inserted into the target tissue;

[0033] Processor system; including:

[0034] An auxiliary labeling module is used to generate auxiliary labels based on the two-dimensional projected image;

[0035] The pose estimation module is used to obtain the estimated pose of the target instrument in three-dimensional space based on the auxiliary identifier; the auxiliary identifier is established based on two-dimensional projection features and is used to indicate the depth information of the target instrument in the three-dimensional space.

[0036] The model correction module is used to compare the estimated pose with the general model of the target instrument to calculate the true pose of the target instrument in the three-dimensional space.

[0037] In some embodiments, the target device is a rigid cylinder that does not contain any marking points;

[0038] Alternatively, the target instrument may include a number of marker points, and the distribution of the marker points on the target instrument may be fixed.

[0039] Compared with the prior art, this application has at least one of the following beneficial effects:

[0040] The technical solution proposed in this application, by utilizing markers on the instruments and a general model of the instruments, can calculate the pose of the target instruments in three-dimensional space in real time by acquiring a single two-dimensional projection image. This result can be deeply integrated with preoperative planning data, providing core support for achieving precise surgical instrument navigation. Compared to traditional methods that rely on multiple sets of three-dimensional scans or dual-view two-dimensional imaging, this technology significantly reduces intraoperative radiation dose and effectively improves surgical safety. Simultaneously, its algorithm structure is simplified and computationally efficient, ensuring both the real-time requirements for pose updates during surgery and possessing good system compatibility and clinical application potential. Attached Figure Description

[0041] The preferred embodiments will now be described in a clear and easy-to-understand manner, in conjunction with the accompanying drawings, to further explain the above-mentioned characteristics, technical features, advantages, and implementation methods of this application.

[0042] Figure 1 This is a flowchart illustrating the steps of an embodiment of a three-dimensional positioning method based on a general model according to this application.

[0043] Figure 2 This is a flowchart illustrating one embodiment of step S200 of this application;

[0044] Figure 3 This is a schematic diagram of the projection of the target device in a two-dimensional image in an embodiment of this application;

[0045] Figure 4 This is a cross-sectional schematic diagram of the target device in an embodiment of this application;

[0046] Figure 5 This is a schematic diagram illustrating the establishment of the first auxiliary identifier in an embodiment of this application;

[0047] Figure 6 This is a schematic diagram of the first auxiliary identifier in the target device according to an embodiment of this application;

[0048] Figure 7 This is a flowchart illustrating another embodiment of step S200 of this application;

[0049] Figure 8 This is a schematic diagram of the structure of the marking points of the target device in the embodiments of this application;

[0050] Figure 9 This is a schematic diagram illustrating the estimation of marker point positions in an embodiment of this application;

[0051] Figure 10 This is a schematic diagram illustrating the establishment of a second auxiliary identifier in an embodiment of this application;

[0052] Figure 11This is a flowchart illustrating one embodiment of step S400 of this application. Detailed Implementation

[0053] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application can also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0054] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or sets.

[0055] To keep the drawings concise, each figure only schematically shows the parts relevant to the invention, and these do not represent the actual structure of the product. Furthermore, to facilitate understanding, in some figures, only one of components with the same structure or function is schematically depicted, or only one is labeled. In this document, "one" not only means "only one," but can also mean "more than one."

[0056] It should also be further understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0057] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the specific implementation methods of this application will be described below with reference to the accompanying drawings. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings and other implementation methods can be obtained based on these drawings without creative effort.

[0058] In existing technologies, reconstructing an object's pose in three-dimensional space from a two-dimensional image theoretically requires at least two two-dimensional images. This is because a single two-dimensional image loses depth information, and a projection from only one viewpoint cannot distinguish the object's translation along the optical axis or its rotation around certain axes in three-dimensional space. When using two images from different viewpoints, the three-dimensional coordinates of feature points can be calculated using the n-point perspective pose estimation (PnP) algorithm, thereby establishing the correspondence between the object's coordinate system and the camera's coordinate system, and ultimately solving for the complete degree of freedom pose. While this method achieves pose estimation, it cannot be performed in real-time, causing inconvenience.

[0059] This application innovates upon the aforementioned technical shortcomings by leveraging the characteristics of imaging perspective and a universal instrument model to estimate the intraoperative target instrument's pose in three-dimensional space in real time using a single two-dimensional projection image. This result can be deeply integrated with preoperative planning data, providing core support for precise surgical instrument navigation. Compared to traditional methods relying on multiple sets of three-dimensional scans or dual-view two-dimensional imaging, this technology significantly reduces intraoperative radiation dose and effectively improves surgical safety. Furthermore, its algorithm structure is simplified and computationally efficient, ensuring both real-time pose updates during surgery and good system compatibility and clinical application potential.

[0060] Reference manual attached Figure 1 As shown, an embodiment of a three-dimensional positioning method based on a general model according to this application specifically includes the following steps:

[0061] S100, a single two-dimensional projection image is acquired during the procedure, the two-dimensional projection image including the target tissue and the projection data of the target instrument inserted into the target tissue.

[0062] Specifically, in this embodiment, the target instrument includes, but is not limited to, puncture needles, bone screws, and artificial stents. This embodiment employs different design schemes for target instruments with different structures. In some implementations, several marker points are present on the target instrument. These marker points provide a known three-dimensional spatial reference in the two-dimensional projection image. By identifying and tracking the projection positions of these marker points in the image, combined with their inherent spatial geometric relationships, the spatial pose of the instrument can be tracked.

[0063] In other implementations, the target instrument has no markers but has a regular shape or aggregate structure, such as a cylinder. By analyzing the geometric features and projection patterns of the target instrument in a two-dimensional image, its position and orientation in three-dimensional space can be directly calculated using computer vision algorithms without relying on external markers.

[0064] S200, generate auxiliary labels based on the two-dimensional projected image.

[0065] S300, based on the auxiliary identifier, obtain the estimated pose of the target instrument in three-dimensional space. The auxiliary identifier is established based on two-dimensional projection features and is used to indicate the depth information of the target instrument in the three-dimensional space.

[0066] Specifically, the auxiliary identifiers established in this application realize autonomous pose estimation of the device by structurally mapping two-dimensional projection features into three-dimensional spatial cues. Its advantage lies in explicitly representing the geometric information implicit in the image, including width variations and endpoint positions, as feature identifiers that can be directly used for mathematical calculations, such as cross-sectional diameters and lines connecting circle centers. This allows for the construction of a spatial skeleton model of the device.

[0067] S400, compare the estimated pose with the general model of the target instrument to calculate the true pose of the target instrument in the three-dimensional space.

[0068] Specifically, after obtaining the estimated pose, registration can be performed by combining it with the general model of the target instrument to obtain the accurate true pose.

[0069] The method described in this application effectively overcomes the inherent limitation of traditional two-dimensional images lacking depth information, and can simultaneously calculate parameters such as the length, width, spatial angle, and depth of the device, significantly improving the completeness and robustness of pose estimation. Furthermore, in some embodiments, the auxiliary markers are entirely derived from the device's own projection, eliminating the need for pre-set markers or external tracking devices. This simplifies the system and avoids tracking failures caused by marker occlusion or detachment, thus improving the method's practicality and reliability.

[0070] This application provides some optional embodiments. Based on the above embodiments, the target instrument is a rigid cylinder that does not contain any marking points. More preferably, the target instrument does not contain any marking points.

[0071] Specifically, the basic geometric feature of the target instrument is a regularly shaped cylindrical object, such as a straight cylinder or prism. The dimensions of its cross-section remain constant along the length of the instrument. There are no variations such as tapers, protrusions, or grooves. Optionally, the cross-section of the target instrument can be circular, rectangular, or other polygonal; this application does not specifically limit this.

[0072] In this embodiment, step S200 specifically includes the following sub-steps, as detailed in the appendix to the specification. Figure 2 As shown:

[0073] S211, based on the width of the target device in the two-dimensional projection image, divide the target device into several sub-segments.

[0074] Specifically, assume the target device is a rigid cylinder. For a cylinder with width in space, refer to... Figure 3The projected length is the same at the angle shown in the diagram. If the position is determined solely by the length on the projection diagram, there will be a significant error. However, since the distance from the ray source varies, the width of the circle projected at different positions will also vary. Therefore, the circle can be segmented based on the width on the projection diagram, and the segments with the same width can be calculated separately.

[0075] In practice, based on the pixel width variation of the target instrument's outline in the two-dimensional projection image, the instrument is divided into multiple continuous segments along its projection centerline. Each segment corresponds to an approximately equal-width portion of the instrument in three-dimensional space.

[0076] S212, obtain the projection matrix of the imaging system, and use the perspective projection characteristic of imaging to calculate the depth information of each sub-interval segment in three-dimensional space.

[0077] Specifically, by utilizing the known projection matrix of the imaging system (including intrinsic and extrinsic parameters) and combining the geometric relationship between the object width and its imaging width and depth in the perspective projection model, a solution is established for each divided sub-interval segment to obtain the depth coordinates of the endpoints or representative points of each sub-interval segment relative to the imaging plane in three-dimensional space, thereby transforming the width information measured in the two-dimensional image into a depth estimate in three-dimensional space.

[0078] S213, any plane passing through the radiation source that is not perpendicular to the central axis of the target instrument forms an ellipse with the cross-section of the target instrument, and the point of tangency between the cross-section and the radiation and the center of the circle are determined.

[0079] S214, on each of the cross sections, the line segment connecting the center of the circle and the tangent point is used as a first auxiliary identifier.

[0080] Specifically, a segmented cylinder can be considered as being composed of several circles of thickness 1 stacked together. (Refer to the attached instruction manual.) Figure 3 As shown. Therefore, any plane passing through the ray source that is not perpendicular to the cylinder forms an ellipse with the cross-section of the cylinder, and the tangent points of the ellipse and the ray are P1P2. Refer to the appendix of the instruction manual. Figure 4 As shown, when a line segment simL of length R2, parallel to the flat panel detector, is moved from the X-ray source along oRo' towards the flat panel, the projected length of this line segment will continuously decrease. Since the perpendicular distance between the tangency point P2 and oRo' is always greater than R2, when the projection of line segment simL equals the projected length on the raw diagram, this position must be to the left of P2. Similarly, this position must be to the left of o', meaning that line segment simL and P2o' must intersect at this point. Refer to the instruction manual appendix. Figure 5 As shown.

[0081] Reference manual attached Figure 6As shown, for any cylinder in space, multiple radii can be calculated from the cross-sections of multiple ellipses, and the position of the cylinder can be fitted by these multiple radii.

[0082] In some optional implementations, step S300 specifically includes the following sub-steps:

[0083] S311, extract the endpoints of each of the first auxiliary identifiers to form a feature point set.

[0084] S312, Based on the feature point set, a fitting is performed to obtain the fitting axis of the target instrument.

[0085] S313, Determine the estimated pose of the target instrument in three-dimensional space based on the fitted axis.

[0086] Specifically, the first auxiliary identifier is a line segment connecting the center of the circle and the tangent point. All key geometric feature points of the first auxiliary identifier are extracted, and an axis in space is fitted based on these points as the central axis of the general model to determine the initial position.

[0087] In practice, endpoints are extracted from all generated first auxiliary identifiers. These endpoints from different auxiliary identifiers are aggregated to form a feature point set representing the spatial distribution of the target instrument. Subsequently, a linear regression algorithm in three-dimensional space is used to fit a straight line to this feature point set, calculating the optimal fitting axis that minimizes the sum of the squared distances from all points to the fitted axis. Finally, this fitted axis is used to align with the central axis of the general model in three-dimensional space. By determining the position and orientation of the fitted axis in the reference coordinate system, the estimated position and orientation of the target instrument in space can be preliminarily obtained.

[0088] This application provides some optional embodiments. Based on the above embodiments, the target device includes a plurality of marking points, and the distribution position of the marking points on the target device is fixed.

[0089] Specifically, in this embodiment, the geometry of the target instrument may not be a rigid cylinder, but rather a polygonal prism or other irregular shape. The markers are typically made of materials with extremely strong X-ray absorption capabilities, such as platinum, gold, or titanium alloys. When X-rays penetrate the human body, these high-density markers strongly attenuate the photon energy, resulting in a signal intensity in the corresponding area received at the detector plane that is much lower than that of the surrounding tissue. This manifests as a high-contrast black shadow or a clear dotted structure in the final projected image. This ensures that the markers on the puncture needle are clearly visible in the projected image within the imaging system, providing a reference for surgical instrument positioning.

[0090] In some embodiments, the marker points are evenly distributed on the target instrument, with equidistant distances between adjacent marker points. In other embodiments, the marker points are not evenly distributed on the target instrument, but their positions are fixed. The distribution of each marker point can be directly obtained by pre-establishing a general model of the target instrument.

[0091] In this embodiment, step S200 specifically includes the following sub-steps, as detailed in the appendix to the specification. Figure 7 As shown:

[0092] S221, Obtain the projection matrix of the imaging system, and use the perspective projection characteristic of imaging to calculate the depth information of the marked point in three-dimensional space.

[0093] S222, connect adjacent marker points with line segments to form a second auxiliary identifier.

[0094] Specifically, the projection matrix encapsulates the geometric parameters of the imaging system, including: the spatial position of the X-ray source, the detection angle of the detector, and the spatial relationship between the X-ray source and the detector. More specifically, the geometric parameters include: the distance from the X-ray source to the rotation center, the distance from the X-ray source to the detector, the pixel size of the detector, the deflection and tilt angles of the detector, and the misalignment parameters between the rotation axis and the detector, etc. The projection matrix is ​​obtained through pre-calibration or other methods.

[0095] Because CBCT imaging has the characteristic of perspective projection, when the distance between the target and the X-ray source changes, the distance between the image points formed by the two endpoints of the target on the projected image also changes accordingly. This change has a clear geometric relationship with the distance. Therefore, when the projection matrix of the imaging system is known, by measuring the distance between the two endpoints of the target on the projected image, the approximate distance L between the midpoint of the target and the X-ray source can be derived in reverse.

[0096] In some embodiments, the marker point is a positioning ball (see attached instruction manual). Figure 8 As shown, the dimensions of the positioning ball are known fixed dimensions, and its shape is a standard sphere.

[0097] When acquiring 2D projection images, each positioning sphere will mostly appear as an ellipse on the projection image. Furthermore, the same radius will correspond to different radii on the projection image at different locations. Therefore, we need to calculate the target radius on the projection image to estimate its distance from the ray source in 3D space. See the attached instruction manual for details. Figure 9As shown in the figure: o is the position of the ray source in three-dimensional space, and o' is the position of the sphere in three-dimensional space. The four tangent points P1P2P3P4 between the ray and the sphere correspond to p1p2p3p4 on the projected image, respectively. The four corresponding radii are r1r2r3r4, and due to magnification, r1r2r3r4 are not equal. Connecting p1p2 and P1P2, since R1=R2, the midpoints of o' and P1P2 are collinear. The same applies to connecting p3p4. Therefore, the coordinates of the ray passing through o' projected onto the raw image are the intersection point Ro' of p1p2 and p3p4. Since p2 and Ro' are calculated from the projected image, and R2 is known, if we can find oRo' and op2, we can estimate the position of o' using the principle of similar triangles.

[0098] Since we can obtain the positional relationship between the ray source and the plate through geometric parameters—that is, the position of the point Ro where the ray source falls perpendicularly on the detector, and the distance between the ray source and the plate—we can calculate the distance between the ray and that projection point in three-dimensional space for any projection point on the projection map. In other words, the lengths of oRo' and op2 are calculable: taking the projection point Ro' as an example, we can obtain the ray source projection coordinates Ro from the geometric calibration results, and then calculate the distance RoRo' between the projection point Ro' and the ray source Ro. The straight-line distance between the ray source and the plate is SDD; according to the Pythagorean theorem, the distance oRo' in three-dimensional space can be calculated.

[0099] The position of the positioning sphere in space is estimated using the positioning sphere on the projection diagram. Connecting the centers of two positioning spheres forms the second auxiliary marker. Due to calculation errors, the general model and the auxiliary marker may not be perfectly aligned. (See attached instruction manual.) Figure 10 As shown.

[0100] In some optional implementations, step S300 specifically includes the following sub-steps:

[0101] S321, extract the endpoints and / or midpoints of each of the second auxiliary identifiers to form a feature point set.

[0102] S322, Based on the feature point set, a fitting axis of the target instrument is obtained.

[0103] S323, Determine the estimated pose of the target instrument in three-dimensional space based on the fitted axis.

[0104] Specifically, the geometric midpoints of each of the generated auxiliary markers are extracted and aggregated into a feature point set in three-dimensional space. Then, a spatial line fitting algorithm is applied to this feature point set to calculate the optimal spatial line that minimizes the sum of the squared distances from all midpoints to this line. This fitted line is the estimated central axis of the target instrument. Finally, based on the direction vector and position coordinates of this central axis in the reference coordinate system, the orientation and position of the instrument in three-dimensional space are directly determined, thus completing the estimation of its overall pose.

[0105] This application provides another embodiment of a three-dimensional positioning method based on a general model. Based on any embodiment of the above method, refer to the appendix to the specification. Figure 11 As shown, step S400 involves comparing the estimated pose with the general model of the target device to calculate the true pose of the target device in the three-dimensional space. Specifically, this includes the following steps:

[0106] S410, Obtain the general model of the target instrument.

[0107] S420, the general model is translated and rotated in the three-dimensional space so that the central axis of the general model is aligned with the fitting axis.

[0108] S430, perform a simulated orthographic projection on the general model, and determine whether there is a deviation between the orthographic projection result and the projection data of the target device in the two-dimensional projection image; if so, translate the general model. Continue until the orthographic projection result of the general model coincides with the projection data of the target device in the two-dimensional projection image. At this point, the pose of the general model is the true pose of the target device in three-dimensional space.

[0109] Specifically, the first step is to obtain a general 3D model of the target instrument and its predefined central axis. During the initialization phase, the central axis of this general model is spatially aligned with the estimated central axis fitted by the auxiliary identifier in the previous steps. After alignment, the general model is orthographically projected along its current pose to generate its simulated projection profile on the imaging plane.

[0110] By calculating the deviation between the simulated projected contour and the actual contour of the target instrument in the real 2D projected image, for example using contour distance, overlap area, or feature point error, a translation transformation is applied to the general model to correct its spatial position. This translation iteration process continues, i.e., after each translation, the orthographic projection is recalculated and compared with the real projection. This continues until the deviation between the simulated projection and the real projection meets a preset convergence threshold, or the simulated projection of the general model and the real projection image achieve optimal overlap in overall shape and contour. At this point, it is determined that the 3D pose of the general model coincides with the spatial pose of the real instrument, thus completing accurate registration. Ultimately, the actual spatial position and 3D pose of the 3D object when the general model of the projection image was captured are accurately restored.

[0111] This application provides another embodiment of a three-dimensional positioning method based on a general model. Based on any of the above embodiments, the accurate pose of the target instrument in three-dimensional space calculated in real time is combined with the preoperative three-dimensional image of the target tissue acquired before surgery. The target instrument can be reconstructed based on the preoperative three-dimensional image, so that the target instrument can be punctured and navigated based on the surgical plan.

[0112] In other embodiments, two-dimensional projections of the target tissue are continuously acquired intraoperatively. Based on the algorithm of this application, the three-dimensional pose of the surgical instruments can be updated in real time. By capturing only one projection image at a time, the location of the puncture needle can be seen in real time on the three-dimensional image, effectively reducing radiation exposure.

[0113] This application provides another embodiment of a three-dimensional positioning method based on a general model, which makes assumptions about the special circumstances of the placement of surgical instruments and simplifies the steps of the three-dimensional positioning method in the above embodiment.

[0114] Specifically: This embodiment assumes that the target instrument is placed parallel to the flat panel detector. In this case, the depth information between each part of the instrument and the detector plane is the same. Therefore, it is no longer necessary to divide the needle into multiple segments using marker points to obtain multiple virtual line segments parallel to the flat panel but not collinear in three-dimensional space. That is, this embodiment treats the target instrument as a complete interval for calculation, and the simplified steps are as follows:

[0115] S010, first, model the target device of a specific shape to obtain a general model of the target device. Then, obtain the projection matrix of the imaging system. The specific implementation method is the same as the above embodiment, and will not be repeated in this embodiment.

[0116] S020, a single two-dimensional projection image is acquired during the procedure, the two-dimensional image including the projection data of the target tissue and the target instrument inserted into the target tissue.

[0117] S030, the general model is initially registered in three-dimensional space with the captured two-dimensional projection image. The initial registration process includes translation / rotation / scaling. This ensures that the general model coincides with the projection pattern of the target instrument in the two-dimensional projection image.

[0118] Specifically, the general model M is translated in three-dimensional space and orthographically projected using the geometric parameters corresponding to the projection matrix. Due to the characteristics of CBCT, when the distance between the general model M and the ray source changes, the distances between the two ends of the general model M on the projection map will also change. Therefore, the distance L between the midpoint of the general model M and the ray source can be estimated first.

[0119] Next, after determining the distance L, rotate the general model M in three-dimensional space so that the two ends of the general model M coincide with the two ends of the general model M on the projection map K.

[0120] Furthermore, fix the two ends of the general model M in three-dimensional space, and rotate and orthographically project the general model M with the line connecting the two ends of the general model M as the axis until the orthographic projection result of the general model M coincides with the actual projection result.

[0121] Finally, the error between the general model M in three-dimensional space and the actual reconstruction result is checked to verify the rationality of the scheme.

[0122] The above solutions can only be implemented under specific preconditions and are not applicable to general situations; they are merely extended ideas.

[0123] Based on the same concept, this application also discloses a three-dimensional positioning device based on a universal model. The three-dimensional positioning device is used to implement the steps described in any of the above method embodiments. Specifically, one embodiment of a three-dimensional positioning device based on a universal model in this application includes:

[0124] An imaging system is used to acquire a single two-dimensional projection image during surgery, the two-dimensional projection image including the projection data of the target tissue and the target instrument inserted into the target tissue.

[0125] Processor system. Includes:

[0126] An auxiliary labeling module is used to generate auxiliary labels based on the two-dimensional projected image.

[0127] The pose estimation module is used to obtain the estimated pose of the target instrument in three-dimensional space based on the auxiliary identifier. The auxiliary identifier is established based on two-dimensional projection features and is used to indicate the depth information of the target instrument in the three-dimensional space.

[0128] The model correction module is used to compare the estimated pose with the general model of the target instrument to calculate the true pose of the target instrument in the three-dimensional space.

[0129] In specific implementations, the processor systems described in the embodiments of this application include, but are not limited to, medical imaging systems.

[0130] Based on the above embodiments, the target device is a rigid cylinder that does not contain any marking points.

[0131] Alternatively, the target instrument may include a number of marker points, and the distribution of the marker points on the target instrument may be fixed.

[0132] The three-dimensional positioning method and device based on a general model in this application have the same technical concept, and the technical details of the embodiments of the two are applicable to each other. In order to reduce repetition, they will not be repeated here.

[0133] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of program modules is merely an example. In practical applications, the above functions can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program units or modules to complete all or part of the functions described above. The program modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software program unit. Furthermore, the specific names of the program modules are only for easy differentiation and are not intended to limit the scope of protection of this application.

[0134] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method of three-dimensional positioning based on a general model, characterized by, The method comprises the following steps: Collecting a single two-dimensional projection image in-situ, the two-dimensional projection image comprising projection data of a target instrument; Generating auxiliary markers based on the two-dimensional projection image; Obtaining an estimated pose of the target instrument in a three-dimensional space based on the auxiliary markers; The auxiliary markers are established based on two-dimensional projection features and are used to indicate depth information of the target instrument in the three-dimensional space; Comparing the estimated pose with a general model of the target instrument to obtain a real pose of the target instrument in the three-dimensional space.

2. A general model based three-dimensional positioning method as claimed in claim 1, characterized in that, The target instrument is a rigid cylinder and does not contain marker points.

3. A general model based three-dimensional positioning method as claimed in claim 2, characterized in that, The step of generating auxiliary markers based on the two-dimensional projection image comprises the following steps: Dividing the target instrument into a plurality of sub-interval segments according to the width of the target instrument in the two-dimensional projection image; Obtaining a projection matrix of an imaging system and calculating depth information of each sub-interval segment in a three-dimensional space by utilizing the perspective projection characteristics of the imaging system; Determining the tangent point of the cross section and the ray and the center of the cross section by passing through a plane that is not perpendicular to the central axis of the target instrument and the ray source and that has an elliptical cross section with the target instrument; Connecting the line segment connecting the center and the tangent point on each cross section as a first auxiliary marker.

4. A general model based three-dimensional positioning method as claimed in claim 3, characterized in that, The step of obtaining an estimated pose of the target instrument in a three-dimensional space based on the auxiliary markers comprises the following steps: Extracting the end points of each first auxiliary marker to form a feature point set; Fitting based on the feature point set to obtain a fitted axis of the target instrument; Determining the estimated pose of the target instrument in the three-dimensional space according to the fitted axis.

5. A general model based three-dimensional positioning method as claimed in claim 1, characterized in that, The target instrument comprises a plurality of marker points, and the distribution positions of the marker points on the target instrument are fixed.

6. A general model based three-dimensional positioning method as claimed in claim 5, characterized in that, The step of generating auxiliary markers based on the two-dimensional projection image comprises the following steps: Obtaining a projection matrix of an imaging system and calculating depth information of the marker points in a three-dimensional space by utilizing the perspective projection characteristics of the imaging system; Connecting adjacent marker points by line segments to form second auxiliary markers.

7. A general model based three-dimensional positioning method as claimed in claim 6, characterized in that, The step of obtaining an estimated pose of the target instrument in a three-dimensional space based on the auxiliary markers comprises the following steps: Extracting the end points and / or midpoints of each second auxiliary marker to form a feature point set; Fitting based on the feature point set to obtain a fitted axis of the target instrument; Determining the estimated pose of the target instrument in the three-dimensional space according to the fitted axis.

8. A general model based three-dimensional positioning method according to claim 4 or 7, characterized in that, The step of comparing the estimated pose with a general model of the target instrument to obtain a real pose of the target instrument in the three-dimensional space comprises the following steps: Obtaining the general model of the target instrument; Performing translation and rotation operations on the general model in the three-dimensional space to align the central axis of the general model with the fitted axis. The general model is simulated orthographic projection, and it is judged whether there is deviation between the orthographic projection result and the projection data of the target instrument in the two-dimensional projection image; if yes, the general model is translated; until the orthographic projection result of the general model coincides with the projection data of the target instrument in the two-dimensional projection image; at this time, the pose of the general model is the real pose of the target instrument in the three-dimensional space.

9. A general model based three-dimensional positioning apparatus, characterized by, The three-dimensional positioning device based on the general model is used to realize the steps of the three-dimensional positioning method based on the general model in any one of claims 1-8; and specifically includes: An imaging system is used to collect a single two-dimensional projection image in an operation, and the two-dimensional projection image includes projection data of a target tissue and a target instrument punctured into the target tissue. A processor system includes: A projection matrix module is used to obtain a projection matrix of the imaging system; An auxiliary identification module is used to generate an auxiliary identification based on the two-dimensional projection image; A pose estimation module is used to obtain an estimated pose of the target instrument in a three-dimensional space based on the auxiliary identification; the auxiliary identification is established based on a two-dimensional projection feature and is used to indicate depth information of the target instrument in the three-dimensional space; A model correction module is used to compare the estimated pose with a general model of the target instrument, and calculate a real pose of the target instrument in the three-dimensional space.

10. A general model based three-dimensional positioning apparatus as claimed in claim 9, characterized in that, The target instrument is a rigid cylinder, and no marker point is contained in the target instrument. Or, the target instrument includes a plurality of marker points, and the distribution positions of the marker points on the target instrument are fixed.

Images