Navigation method, device, medium, and program product for surgical instruments

Abstract

This application discloses a navigation method, device, medium, and program product for surgical instruments, relating to the field of surgical technology. The disclosed navigation method for surgical instruments includes: acquiring a calibration module DR image and a calibration module optical image; determining a transformation matrix between the coordinate system of the optical tracking system and the coordinate system of the DR imaging plane based on the imaging positions of marker points in the calibration module DR image and the imaging positions of marker points in the calibration module optical image, and denoting this as the first transformation matrix; tracking the surgical instrument through the optical tracking system, and projecting the instrument feature points of the surgical instrument onto the patient's DR image based on the first transformation matrix. This allows for real-time presentation of the actual position of the surgical instrument in the patient's DR image without relying on 3D CT imaging equipment, thereby improving surgical accuracy and ease of operation for doctors.

Description

Technical Field

[0001] This application relates to the field of surgical technology, and more particularly to a navigation method, device, medium, and program product for surgical instruments. Background Technology

[0002] In trauma orthopedic surgery, surgeons often rely on intraoperative digital radiography (DR) images for lesion localization and intraoperative navigation. Common image navigation systems include computed tomography (CT) navigation and cone-beam CT (CBCT) navigation, which depend on intraoperative three-dimensional reconstruction, registration, and navigation system support to achieve real-time positioning of instruments relative to the patient's anatomical structures.

[0003] However, the aforementioned surgical navigation system relies on 3D CT imaging equipment, which increases equipment costs. Summary of the Invention

[0004] This application provides a navigation method, device, medium, and program product for surgical instruments, which enables the visualization of surgical instruments in intraoperative DR images without relying on 3D CT imaging equipment, reducing costs and ensuring the safety of surgical operations.

[0005] To achieve the above objectives, this application proposes a navigation method for surgical instruments, comprising:

[0006] Acquire a calibration module DR image and a calibration module optical image, wherein the calibration module DR image is a DR image of the calibration module captured by a DR imaging device, and the calibration module optical image is an image of the calibration module captured by an optical tracking system, and the calibration module is provided with marker points;

[0007] Based on the imaging position of the marker point in the DR image of the calibration module and the imaging position of the marker point in the optical image of the calibration module, the transformation matrix between the optical tracking system coordinate system and the DR imaging plane coordinate system is determined and denoted as the first transformation matrix.

[0008] The optical tracking system tracks the surgical instruments and projects the instrument feature points onto the patient's DR image based on the first transformation matrix.

[0009] In addition, to achieve the above objectives, this application also proposes an electronic device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the surgical instrument navigation method as described above.

[0010] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, implements the steps of the surgical instrument navigation method described above.

[0011] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the surgical instrument navigation method described above.

[0012] One or more technical solutions proposed in this application have at least the following technical effects:

[0013] By acquiring calibration module DR images and calibration module optical images, and based on the imaging positions of the marker points in the calibration module DR images and the imaging positions of the marker points in the calibration module optical images, a transformation matrix between the coordinate system of the optical tracking system and the image coordinates of the DR imaging device is determined and denoted as the first transformation matrix. The surgical instruments are tracked by the optical tracking system, and the instrument feature points are projected onto the patient's DR image based on the first transformation matrix. This allows for real-time visualization of the actual position of the surgical instruments in the patient's DR image without relying on 3D CT imaging equipment, thereby improving surgical accuracy and ease of operation for doctors. Furthermore, because the instrument position is visualized in real-time on the patient's DR image, doctors can reduce the number of times they need to repeatedly take correction images, reducing the risk of X-ray exposure for both doctors and patients, and improving work efficiency during surgery. Attached Figure Description

[0014] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0015] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 A schematic flowchart illustrating an embodiment of the navigation method for surgical instruments according to this application;

[0017] Figure 2 This diagram illustrates the spatial relationship between the C-arm, image calibrator, and calibration plate involved in the navigation method embodiment of the surgical instruments of this application.

[0018] Figure 3A schematic diagram showing the projection of the first instrument projection point onto the anteroposterior and lateral DR images of the patient, as provided in the embodiments of this application;

[0019] Figure 4 This is a diagram showing the spatial relationship between the feature points of the surgical instruments and the instrument reference coordinate system in the embodiments of this application;

[0020] Figure 5 This is a schematic diagram of the electronic device structure of the hardware operating environment involved in the navigation method of surgical instruments in the embodiments of this application. Detailed Implementation

[0021] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0022] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0023] In trauma orthopedic surgery, surgeons often rely on intraoperative digital radiography (DR) images for lesion localization and intraoperative navigation. Common image navigation systems include CT navigation and CBCT navigation, which depend on intraoperative 3D reconstruction, registration, and navigation system support to achieve real-time instrument positioning relative to the patient's anatomical structures. However, these surgical navigation systems require 3D CT imaging equipment, increasing equipment costs.

[0024] To address the aforementioned issues, this application provides a navigation method for surgical instruments that can, without relying on 3D CT imaging equipment, present the relationship between the actual position of the surgical instruments and the pre-operative planned path in the patient's DR image in real time, thereby improving surgical accuracy and ease of operation for doctors.

[0025] The navigation method for surgical instruments according to embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0026] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the navigation method for surgical instruments according to this application. In this embodiment, the navigation method for surgical instruments includes steps S10 to S40:

[0027] Step S10: Obtain the DR image and optical image of the calibration module.

[0028] Among them, the calibration module DR image is the DR image of the calibration module captured by the DR imaging device, the calibration module optical image is the image of the calibration module captured by the optical tracking system, and the calibration module is provided with at least one marker point.

[0029] In some implementations, the DR imaging device can be controlled to capture DR images of the calibration module from both an orthogonal view and a lateral view, so as to form an orthogonal calibration module DR image on the imaging plane of the orthogonal DR imaging light source of the DR imaging device, and a lateral calibration module DR image on the imaging plane of the lateral DR imaging light source.

[0030] In some embodiments, the calibration module includes a first calibration module and a second calibration module. The first calibration module is provided with at least three first marker points, and the second calibration module is provided with at least three second marker points. The calibration module DR image includes the first calibration module DR image and the second calibration module DR image, and the calibration module optical image includes the second calibration module optical image.

[0031] Furthermore, the first calibration module DR image includes an orthogonal first calibration module DR image formed on the imaging plane of the orthogonal DR imaging light source and a lateral first calibration module DR image formed on the imaging plane of the lateral DR imaging light source. The second calibration module DR image includes an orthogonal second calibration module DR image formed on the imaging plane of the orthogonal DR imaging light source and a lateral second calibration module DR image formed on the imaging plane of the lateral DR imaging light source.

[0032] In some implementations, the DR imaging device can be a C-arm. A C-arm, also known as a C-arm X-ray machine, generates X-rays that penetrate the target object through X-ray emitters and receivers at both ends of a C-shaped frame, and converts the signals into real-time DR images for intraoperative navigation.

[0033] Preferably, the first calibration module is an image calibrator with multiple metal balls as first markers, and the second calibration module is a calibration plate with multiple metal balls as second markers.

[0034] like Figure 2 As shown, Figure 2 The spatial relationship between the C-arm, the image calibrator, and the calibration plate is shown. The C-arm includes a C-shaped frame 1, with an X-ray emitter 2 mounted at one end and a flat panel sensor 3 (i.e., an X-ray receiver) mounted at the other end (near the bottom). The image calibrator 4 is placed on the flat panel sensor 3, and the calibration plate 5 is located between the X-ray emitter 2 and the image calibrator 4. A first marker 6 is placed on the image calibrator 4, and a second marker 7 is placed on the calibration plate. The first marker 6 and the second marker 7 can be metal spheres.

[0035] Step S20: Based on the imaging position of the marker point in the DR image of the calibration module and the imaging position of the marker point in the optical image of the calibration module, determine the transformation matrix between the coordinate system of the optical tracking system and the coordinate system of the DR imaging plane, and denot it as the first transformation matrix.

[0036] Specifically, the principle of DR imaging device can be regarded as the perspective projection process of monocular camera. Therefore, it is necessary to take pictures and calibrate through marker points to establish the mapping relationship between the world coordinate system (the optical tracking system coordinate system is used as the world coordinate system in this step) and the DR imaging plane coordinate system, that is, to complete the calibration of the internal and external parameters of DR imaging device.

[0037] The imaging position of the marker point in the DR image of the calibration module refers to the coordinate position of the marker point in the DR imaging plane coordinate system. The DR imaging plane coordinate system is a two-dimensional coordinate system, which is the projection of the DR imaging light source of the DR imaging device onto the physical imaging plane.

[0038] In some implementations, step S20 may include:

[0039] Based on the imaging position of each first marker point in the DR image of the first calibration module, the intrinsic parameters of the DR imaging device are determined.

[0040] Based on the imaging positions of each second marker point in the DR image of the second calibration module and the imaging positions of each second marker point in the optical image of the second calibration module, the external parameters of the DR imaging device are determined. The external parameters are used to characterize the transformation relationship between the coordinate system of the optical tracking system and the coordinate system of the DR imaging device.

[0041] Based on intrinsic and extrinsic parameters, the transformation matrix between the coordinate system of the optical tracking system and the image coordinates of the DR imaging device is determined.

[0042] Step S30: The surgical instruments are tracked by an optical tracking system, and the instrument feature points of the surgical instruments are projected onto the patient's DR image based on the first transformation matrix.

[0043] In some implementations, the above-mentioned S30 may include:

[0044] S301, based on the first transformation matrix and the predetermined transformation matrix between the optical tracking system coordinate system and the patient reference coordinate system, determine the position of the patient DR image captured by the DR imaging device in the patient reference coordinate system and the position of the DR imaging light source of the DR imaging device in the patient reference coordinate system.

[0045] Specifically, a patient reference array is installed on the patient's bones, and a patient reference coordinate system, denoted as PatientRF, is established based on the patient reference array. Then, the patient reference coordinate system PatientRF is tracked in real time using an optical tracking system (e.g., a binocular camera) to obtain the transformation relationship between the patient reference coordinate system PatientRF and the optical tracking system coordinate system, i.e., the transformation matrix between the optical tracking system coordinate system and the patient reference coordinate system.

[0046] In some embodiments, the DR images of the patient captured by the DR imaging device include anteroposterior (AP) DR images and lateral DR images. In a specific implementation, the AP DR images, lateral DR images, AP DR imaging light sources, and lateral DR imaging light sources are projected onto the patient reference coordinate system to obtain the positions of the AP DR images, lateral DR images, AP DR imaging light sources, and lateral DR imaging light sources in the patient reference coordinate system.

[0047] Specifically, the anteroposterior view of the patient's DR image / lateral view of the patient's DR image / anteroposterior view of the DR imaging light source / lateral view of the DR imaging light source is projected onto the patient reference coordinate system through the following steps to obtain the position of the anteroposterior view of the patient's DR image / lateral view of the patient's DR image / anteroposterior view of the DR imaging light source / lateral view of the DR imaging light source projected onto the patient reference coordinate system:

[0048] First, if the first transformation matrix is ​​the transformation matrix from the optical tracking system coordinate system to the DR imaging plane coordinate system, then the first transformation matrix is ​​multiplied by the transformation matrix from the patient reference coordinate system to the optical tracking system coordinate system to obtain the transformation matrix from the patient reference coordinate system to the DR imaging plane coordinate system.

[0049] Then, based on the transformation matrix from the patient reference coordinate system to the DR imaging plane coordinate system, the anteroposterior patient DR image / lateral patient DR image / anteroposterior DR imaging light source / lateral DR imaging light source in the DR imaging plane coordinate system are transformed to the patient reference coordinate system, so as to obtain the position of the anteroposterior patient DR image / lateral patient DR image / anteroposterior DR imaging light source / lateral DR imaging light source in the patient reference coordinate system.

[0050] S302, based on the predetermined transformation matrix between the instrument reference coordinate system and the patient reference coordinate system, calculate the projection points of the instrument feature points of the surgical instrument in the patient reference coordinate system, and record them as the first instrument projection point.

[0051] Specifically, surgical instruments (such as guide sleeves and drills) are equipped with optical reflection arrays. An instrument reference coordinate system, denoted as ToolRF, is established based on this optical reflection array. Then, the coordinates of instrument feature points (such as the drill tip position or the direction of the sleeve's central axis) in the ToolRF are predefined. Figure 4 As shown.

[0052] Then, the real-time spatial transformation relationship between the instrument reference coordinate system ToolRF and the patient reference coordinate system PatientRF is obtained with the help of an optical tracking system. The instrument feature points in the instrument reference coordinate system ToolRF are then converted into three-dimensional positions in the patient reference coordinate system PatientRF in real time to obtain the first instrument projection point.

[0053] S303, based on the position of the DR imaging light source in the patient reference coordinate system, projects the first instrument projection point onto the patient DR image located in the patient reference coordinate system.

[0054] In some implementations, S303 may include:

[0055] In the patient reference coordinate system, a ray is constructed with the position point of the DR image light source as the starting point and radiating towards the first instrument projection point;

[0056] Calculate the intersection of the ray and the patient DR image plane in the patient reference coordinate system to obtain the projection point of the first instrument projection point on the patient DR image, and record it as the second instrument projection point;

[0057] The second instrument projection point is rendered onto the patient's DR image, which is located in the patient's reference coordinate system.

[0058] Specifically, in the PatientRF coordinate system, let the position of the anteroposterior DR imaging light source be S, the instrument feature point of the surgical instrument be P, and the imaging plane of the anteroposterior DR imaging light source be denoted as II, with its unit normal vector being... The presence of known points in the DR image of the patient in the frontal view is denoted as... Then, the following steps are performed to obtain the projection point of the first instrument projection point on the patient's DR image:

[0059] First, establish a projection line passing through the positive DR imaging light source S and the instrument feature point P, with its direction vector being... :

[0060] ,

[0061] The projected straight line The parametric equation is:

[0062] , t For coefficients;

[0063] Imaging plane of a positive DR imaging light source The general form is:

[0064] ,

[0065] in, Represents the imaging plane A general point on the line, in this embodiment, corresponds to the projected line. With the imaging plane The intersection point vector is the coordinate of the projection point to be determined;

[0066] Then, Substitute into the plane equation and solve for the intersection parameters. :

[0067] ,

[0068] Final intersection Represented as:

[0069] ,

[0070] The intersection point P′ is the projection point of the first instrument projection point on the patient's DR image.

[0071] Furthermore, the principle of projecting the first instrument projection point onto the lateral view DR image of the patient is the same as that of projecting the first instrument projection point onto the anterior view DR image of the patient, and will not be repeated here.

[0072] For example, please see Figure 3 , Figure 3 This is a schematic diagram of the first instrument projection point projected onto the anteroposterior and lateral DR images of the patient, as provided in this embodiment. Figure 3 In the diagram, O1 represents the lateral DR imaging light source, O2 represents the frontal DR imaging light source, and F... APIImage F represents the imaging plane coordinate system of the positive DR imaging light source. LTImage The coordinate system represents the imaging plane of the lateral DR imaging light source. C1 and C2 represent the first instrument projection points corresponding to the two instrument feature points, respectively. API1 and API2 represent the projection points of the first instrument projection point C1 and the second instrument projection point C2 on the imaging plane coordinate system of the anteroposterior DR imaging light source, respectively. LT1 and LT2 represent the projection points of the first instrument projection point C1 and the second instrument projection point C2 on the imaging plane coordinate system of the lateral DR imaging light source, respectively.

[0073] In some implementations, the above steps, based on the imaging positions of each first marker point in the DR image of the first calibration module, to determine the intrinsic parameters of the DR imaging device, may include steps a1 to a5:

[0074] Step a1: Based on the predetermined coordinates of each type of first marker point in the first calibration module coordinate system and the imaging position of each type of first marker point in the DR imaging device coordinate system, determine the transformation matrix between the first calibration module coordinate system and the DR imaging plane coordinate system, and denote it as the second transformation matrix.

[0075] Step a2: Based on the second transformation matrix, transform each type of first marker point in the first calibration module coordinate system to the DR imaging plane coordinate system to obtain the projection points of each type of first marker point in the DR imaging plane coordinate system.

[0076] Step a3: Connect each type II first marker point with its corresponding projection point in the DR imaging plane coordinate system to obtain multiple sets of connections.

[0077] Step a4: Calculate the coordinates of the intersection points of multiple sets of lines to obtain the coordinates of the DR imaging light source of the DR imaging device in the DR imaging plane coordinate system;

[0078] Step a5: Determine the intrinsic parameters of the DR imaging device based on the coordinates of the DR imaging light source in the DR imaging plane coordinate system.

[0079] Specifically, the first marker point includes a first type of first marker point and a second type of first marker point. The first type of first marker point is used for DR image registration, and the second type of first marker point is used for calculating the position of the DR imaging light source, that is, calculating the intrinsic parameters of the DR imaging device.

[0080] Taking the first calibration module using an image calibration device as an example, please refer to [link to relevant documentation]. Figure 2 The image calibration device 4 has two layers, with the lower layer closely attached to the flat panel sensor. Both the upper and lower layers contain several metal balls. The metal balls in the upper layer (i.e., the layer on the side away from the flat panel sensor) serve as first-class markers 62, and the metal balls in the lower layer serve as second-class markers 61.

[0081] The calculation process for the intrinsic parameters of a DR imaging device is as follows:

[0082] First, obtain the coordinates of each type of first marker point in the first calibration module coordinate system (the first type of first marker point has been pre-calibrated in the first calibration module coordinate system, so the coordinates are known values). Then, based on the coordinates of each type of first marker point in the first calibration module coordinate system and the coordinates in the imaging plane coordinate system of the orthogonal DR imaging light source (hereinafter referred to as the orthogonal DR imaging plane coordinate system), calculate the transformation matrix from the first calibration module coordinate system to the orthogonal DR image coordinate system, and denote it as the orthogonal second transformation matrix. Also, based on the coordinates of each type of first marker point in the first calibration module coordinate system and the coordinates in the imaging plane coordinate system of the lateral DR imaging light source (hereinafter referred to as the lateral DR imaging plane coordinate system), calculate the transformation matrix from the first calibration module coordinate system to the lateral DR image coordinate system, and denote it as the lateral second transformation matrix.

[0083] Then, using the aforementioned orthogonal second transformation matrix, the two types of first marker points in the first calibration module coordinate system are transformed to the orthogonal DR imaging plane coordinate system, obtaining the projection points of each type of first marker point in the orthogonal DR image coordinate system. In the orthogonal DR imaging plane coordinate system, each type of first marker point is connected to its corresponding projection point, resulting in multiple sets of lines. The intersection of these multiple sets of lines represents the coordinates of the orthogonal DR imaging light source in the orthogonal DR imaging plane coordinate system. Similarly, using the aforementioned lateral second transformation matrix, the two types of first marker points in the first calibration module coordinate system are transformed to the lateral DR imaging plane coordinate system, obtaining the projection points of each type of first marker point in the lateral DR imaging plane coordinate system. In the lateral DR imaging plane coordinate system, each type of first marker point is connected to its corresponding projection point, resulting in multiple sets of lines. The intersection of these multiple sets of lines represents the coordinates of the lateral DR imaging light source in the lateral DR imaging plane coordinate system.

[0084] Subsequently, based on the coordinates of the orthogonal DR imaging source in the orthogonal DR imaging plane coordinate system and the vertical distance between the X-ray emitter and receiver in the DR imaging device, the imaging geometry relationship from the orthogonal DR imaging source to the orthogonal DR imaging plane is calculated; and based on the coordinates of the lateral DR imaging source in the lateral DR imaging plane coordinate system and the vertical distance between the X-ray emitter and receiver in the DR imaging device, the imaging geometry relationship from the lateral DR imaging source to the lateral DR imaging plane is calculated, thereby completing the intrinsic parameter calibration of the DR imaging device.

[0085] In some embodiments, the surgical instrument navigation method further includes:

[0086] The movement path of the projection points of the instrument feature points on the patient's DR image;

[0087] The movement path is compared with the preset planned path to obtain the positional and directional deviations of the surgical instruments;

[0088] Visualize positional and orientation deviations on the patient's DR images.

[0089] Specifically, an optical tracking system tracks surgical instruments and projects the real-time positions of the instrument feature points onto projection points on the patient's DR image, thus obtaining the movement path of the projection points of the instrument feature points on the patient's DR image. Let P be the movement path of the projection points of the surgical instrument feature points on the DR image. 实际={(u1, v1), (u2, v2), ..., (un, vn)}, where u1, u2, and un represent the positions of the 1st, 2nd, and nth projection points, respectively, and v1, v2, and vn represent the angles of the 1st, 2nd, and nth projection points, respectively. Assume the preset planning path of the surgical instruments is P. 规划 ={(u1′,v1′),(u2′,v2′),…,(un′,vn′)}, where u1′, u2′ and un′ represent the positions of the 1st planning point, the 2nd planning point and the nth planning point, respectively, and v1′, v2′ and vn′ represent the angles of the 1st planning point, the 2nd planning point and the nth planning point, respectively.

[0090] The coordinates of the i-th projection point are denoted as (xi, yi), and the coordinates of the i-th planning point are denoted as (xi′, yi′). The position deviation di of the i-th point is calculated using the following formula:

[0091] ,

[0092] Then, calculate the direction vector of every two adjacent projection points in the movement path. And the direction vectors of every two adjacent planning points in the preset planning path. Then, the directional deviation is determined by the following formula. :

[0093] .

[0094] Finally, the difference information, including positional and directional deviations, is visualized on the patient's DR image using methods such as arrows, color coding, and numerical labels to achieve a visual navigation feedback mechanism.

[0095] In the above implementation plan, by providing feedback on the difference between the planned path and the actual movement path of the instrument in the image, doctors can promptly correct the direction and depth, avoiding surgical risks such as mis-implantation, misalignment, or perforation, and ensuring the accuracy of the implant's position.

[0096] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the navigation method of the surgical instruments of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0097] This application provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the surgical instrument navigation method of the above embodiment 1.

[0098] The following is for reference. Figure 5 The diagram illustrates a structural schematic of an electronic device suitable for implementing embodiments of this application. The electronic devices in these embodiments may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 5 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0099] like Figure 5 As shown, the electronic device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the electronic device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. While electronic devices with various systems are shown in the figures, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0100] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0101] The electronic device provided in this application employs the surgical instrument navigation method described in the above embodiments. Compared with the prior art, the beneficial effects of the electronic device provided in this application are the same as those of the surgical instrument navigation method provided in the above embodiments, and other technical features of the electronic device are the same as those disclosed in the method of the previous embodiment, and will not be repeated here.

[0102] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0103] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0104] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to perform the navigation method of the surgical instruments in the above embodiments.

[0105] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0106] The aforementioned computer-readable storage medium may be included in an electronic device or may exist independently without being assembled into an electronic device.

[0107] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0108] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0109] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0110] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., a computer program) for executing the navigation method of the surgical instruments described above. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the navigation method of the surgical instruments provided in the above embodiments, and will not be repeated here.

[0111] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the surgical instrument navigation method described above.

[0112] Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as the beneficial effects of the navigation method for surgical instruments provided in the above embodiments, and will not be repeated here.

[0113] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. An electronic device, characterized in that, The electronic device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement steps of a navigation method for surgical instruments, including: Acquire calibration module DR image and calibration module optical image, wherein the calibration module DR image is a DR image of the calibration module captured by a DR imaging device, and the calibration module optical image is an image of the calibration module captured by an optical tracking system. The calibration module includes a first calibration module and a second calibration module. The first calibration module is provided with at least one first marker point, and the second calibration module is provided with at least one second marker point. The first marker point includes a first type of first marker point and a second type of first marker point. The calibration module DR image includes the first calibration module DR image and the second calibration module DR image, and the calibration module optical image includes the second calibration module optical image. Based on the predetermined coordinates of each of the first type of first marker points in the first calibration module coordinate system and the imaging position of each of the first type of first marker points in the DR image of the first calibration module, the transformation matrix between the first calibration module coordinate system and the DR imaging plane coordinate system is determined and denoted as the second transformation matrix. Based on the second transformation matrix, each of the two types of first marker points located in the first calibration module coordinate system is transformed to the DR imaging plane coordinate system to obtain the projection points of each of the two types of first marker points in the DR imaging plane coordinate system; In the DR imaging plane coordinate system, each of the two types of first marker points is connected to the corresponding projection point to obtain multiple sets of connections; Calculate the coordinates of the intersection points of the multiple sets of connecting lines to obtain the coordinates of the DR imaging light source of the DR imaging device in the DR imaging plane coordinate system; The intrinsic parameters of the DR imaging device are determined based on the coordinates of the DR imaging light source in the DR imaging plane coordinate system. Based on the imaging positions of each of the second marker points in the DR image of the second calibration module and the imaging positions of each of the second marker points in the optical image of the second calibration module, the extrinsic parameters of the DR imaging device are determined. The extrinsic parameters are used to characterize the transformation relationship between the coordinate system of the optical tracking system and the coordinate system of the DR imaging device. Based on the intrinsic and extrinsic parameters, the transformation matrix between the optical tracking system coordinate system and the DR imaging plane coordinate system is determined and denoted as the first transformation matrix; The optical tracking system tracks the surgical instruments and projects the instrument feature points onto the patient's DR image based on the first transformation matrix.

2. The electronic device as claimed in claim 1, characterized in that, The step of projecting the instrument feature points of the surgical instrument onto the patient's DR image based on the first transformation matrix includes: Based on the first transformation matrix and the predetermined transformation matrix between the optical tracking system coordinate system and the patient reference coordinate system, the position of the patient DR image captured by the DR imaging device in the patient reference coordinate system and the position of the DR imaging light source of the DR imaging device in the patient reference coordinate system are determined. Based on the predetermined transformation matrix between the instrument reference coordinate system and the patient reference coordinate system, the projection points of the instrument feature points of the surgical instrument in the patient reference coordinate system are calculated and denoted as the first instrument projection point. Based on the position of the DR imaging light source in the patient reference coordinate system, the first instrument projection point is projected onto the patient DR image located in the patient reference coordinate system.

3. The electronic device as described in claim 2, characterized in that, The step of projecting the first instrument projection point onto the patient's DR image located in the patient reference coordinate system based on the position of the DR imaging light source in the patient reference coordinate system includes: In the patient reference coordinate system, a ray is constructed with the position point of the DR image light source as the starting point and radiating towards the first instrument projection point; Calculate the intersection point of the ray and the patient's DR image in the patient reference coordinate system to obtain the projection point of the first instrument projection point on the patient's DR image, and record it as the second instrument projection point; The second instrument projection point is rendered onto the patient's DR image located in the patient reference coordinate system.

4. The electronic device as claimed in claim 1, characterized in that, The navigation method for the surgical instruments also includes: Calculate the movement path of the projection point of the instrument feature point on the patient's DR image; The movement path is compared with the preset planned path to obtain the positional and directional deviations of the surgical instrument; The positional deviation and the orientation deviation are visualized on the patient's DR image.

5. The electronic device as claimed in claim 1, characterized in that, The DR imaging device is a C-arm, the first calibration module is an image calibrator with multiple metal balls as the first marker points, and the second calibration module is a calibration plate with multiple metal balls as the second marker points.

6. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, implements the steps of the navigation method for surgical instruments as described in any one of claims 1 to 5.

7. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the navigation method for surgical instruments as described in any one of claims 1 to 5.

Images