Real-time display method and device of surgical instrument in medical image
The magnetic field information is sensed through the probe sensor, combined with the conversion relationship between the magnetic field and the image coordinate system, and the position of the surgical instrument is displayed in real time, solving the problem of low accuracy in traditional surgical navigation systems in complex environments, and improving the accuracy and safety of the surgery.
Patent Information
- Application Number
- CN202410934622.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-12
- Publication Date
- 2024-11-08
AI Technical Summary
In complex surgical environments, traditional surgical navigation systems have inaccurate identification of surgical instruments due to factors such as light changes, tissue deformation and occlusion, resulting in low accuracy of surgical navigation systems, which in turn reduces the accuracy and safety of the surgery.
The probe sensor perceives real-time magnetic field information, determines the conversion relationship between the probe coordinate system and the magnetic field coordinate system, and combines the target conversion relationship to display the position of the surgical instrument in medical images in real time, avoiding relying on image shooting, and improving position information accuracy.
It realizes the accurate display of surgical instruments in medical images, improves the accuracy of the surgical navigation system, enhances the accuracy and safety of the surgery, and reduces the dependence on light and tissue deformation.
Smart Images

Figure CN118902610A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of medical equipment, and in particular to a method and device for real-time display of surgical instruments in medical images. Background Art
[0002] The surgical navigation system is used to accurately locate surgical instruments and display them in medical images to assist doctors in precise operations during surgery and provide real-time navigation guidance to help doctors more accurately locate and treat diseased tissues, reduce surgical risks, and improve surgical success rates. Traditional surgical navigation systems use a combination of imaging, computer vision, and positioning technologies. By using medical imaging equipment to capture images of patients, the system can obtain three-dimensional visual medical images. During surgery, the system takes real-time photos of the surgical scene and uses image recognition technology to determine the position of the surgical instruments. It then uses feature matching to determine the exact display position of the surgical instruments in the medical image, so that the surgical instruments can be updated and displayed in real time in the form of virtual probes on the medical image.
[0003] However, in traditional surgical navigation, the surgical scene needs to be photographed. In complex surgical environments, such as changes in light during surgery, possible tissue deformation, occlusion, etc., the quality of the captured images is not high. Therefore, the position recognition of surgical instruments will be inaccurate or the features cannot be accurately matched, resulting in low accuracy of the surgical navigation system, leading to low accuracy and safety of the surgery. Summary of the invention
[0004] The problem solved by the present invention is that the accuracy and safety of surgery are low due to the low precision of the surgical navigation system.
[0005] In order to solve the above problems, the present invention provides a method and device for real-time display of surgical instruments in medical images.
[0006] In a first aspect, the present invention provides a method for real-time display of surgical instruments in medical images, comprising: sensing real-time magnetic field information through a probe sensor, and determining a real-time conversion relationship between a probe coordinate system corresponding to the probe sensor and a magnetic field coordinate system corresponding to a magnetic field generator according to the real-time magnetic field information, wherein the relative position of the probe sensor and the surgical instrument remains unchanged;
[0007] Determine the real-time position of the surgical instrument in the medical image according to the real-time conversion relationship and the target conversion relationship to obtain the target position, wherein the target conversion relationship is used to represent the conversion relationship between the magnetic field coordinate system and the image coordinate system corresponding to the medical image, and the medical image is obtained by performing an electronic computer tomography scan on the target area of the patient;
[0008] The surgical instrument is displayed in real time at the target position in the medical image.
[0009] Optionally, before determining the real-time position of the surgical instrument in the medical image based on the real-time conversion relationship and the target conversion relationship, the method also includes: acquiring the coordinates of multiple markers in the image coordinate system to obtain multiple first coordinates, wherein the multiple markers are pasted on the target area; determining the coordinates of the multiple markers in the magnetic field coordinate system to obtain multiple second coordinates; and determining the target conversion relationship based on the multiple first coordinates and the multiple second coordinates.
[0010] Optionally, determining the coordinates of the multiple markers in the magnetic field coordinate system to obtain the multiple second coordinates includes: performing the following operations for each of the multiple markers, and the marker when performing the following operations is the current marker: when the registration sensor is rigidly connected to the current marker, obtaining the offset of the current marker relative to the registration sensor to obtain the current offset; sensing the current magnetic field information through the registration sensor, and determining the current conversion relationship when the registration sensor is rigidly connected to the current marker based on the current magnetic field information, wherein the current conversion relationship is used to represent the conversion relationship between the registration coordinate system corresponding to the registration sensor and the magnetic field coordinate system; determining the current second coordinate based on the current offset and the current conversion relationship.
[0011] Optionally, the method also includes: when the position of the patient moves or deflects, sensing first magnetic field information through a registration sensor, and determining a first conversion relationship based on the first magnetic field information, wherein the first conversion relationship is used to represent the conversion relationship between the registration coordinate system after the patient moves and the registration coordinate system before the patient moves; updating the target conversion relationship based on the first conversion relationship.
[0012] Optionally, determining the real-time position of the surgical instrument in the medical image according to the real-time conversion relationship and the target conversion relationship to obtain the target position includes: respectively determining the coordinates of the first part and the second part of the surgical instrument in the probe coordinate system corresponding to the probe sensor to obtain a third coordinate and a fourth coordinate; determining the coordinates of the first part in the image coordinate system according to the real-time conversion relationship, the third coordinate and the target conversion relationship to obtain a fifth coordinate; determining the coordinates of the second part in the image coordinate system according to the real-time conversion relationship and the fourth coordinate to obtain a sixth coordinate, wherein the target position includes the fifth coordinate and the sixth coordinate.
[0013] Optionally, the target position in the medical image displays the surgical instrument in real time, including: in the medical image, connecting the position corresponding to the fifth coordinate and the position corresponding to the sixth coordinate to obtain a target line segment; and displaying the target line segment in the medical image, wherein the target line segment is used to represent the surgical instrument.
[0014] Optionally, before determining the real-time position of the surgical instrument in the medical image according to the real-time conversion relationship and the target conversion relationship, the method also includes: sensing second magnetic field information through a registration sensor, and determining a second conversion relationship according to the second magnetic field information, wherein the second conversion relationship is used to represent the conversion relationship between the registration coordinate system corresponding to the registration sensor and the magnetic field coordinate system; obtaining the coordinates of multiple markers in the image coordinate system to obtain multiple first coordinates, wherein multiple markers are pasted on the target area; obtaining the offsets of multiple markers relative to the registration sensor to obtain multiple offsets, wherein multiple markers are rigidly connected to the registration sensor; determining a third conversion relationship according to multiple first coordinates and multiple offsets, wherein the third conversion relationship is used to represent the conversion relationship between the registration coordinate system corresponding to the registration sensor and the image coordinate system; determining the target conversion relationship according to the second conversion relationship and the third conversion relationship.
[0015] In a second aspect, the present invention provides a real-time display device for surgical instruments in medical images, comprising: a first determination module, for sensing real-time magnetic field information through a probe sensor, and determining a real-time conversion relationship between a probe coordinate system corresponding to the probe sensor and a magnetic field coordinate system corresponding to a magnetic field generator according to the real-time magnetic field information, wherein the relative position of the probe sensor and the surgical instrument remains unchanged;
[0016] A second determination module is used to determine the real-time position of the surgical instrument in the medical image according to the real-time conversion relationship and the target conversion relationship to obtain the target position, wherein the target conversion relationship is used to represent the conversion relationship between the magnetic field coordinate system and the image coordinate system corresponding to the medical image, and the medical image is obtained by performing an electronic computer tomography scan on the target area of the patient;
[0017] A display module is used to display the surgical instrument in real time at the target position in the medical image.
[0018] In a third aspect, the present invention provides an electronic device, including a memory and a processor;
[0019] The memory is used to store computer programs;
[0020] The processor is used to implement the real-time display method of surgical instruments in medical images as described in the first aspect when executing the computer program.
[0021] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for real-time display of surgical instruments in medical images as described in the first aspect is implemented.
[0022] The beneficial effects of the real-time display method and device of surgical instruments in medical images of the present invention are as follows: a magnetic field is generated by a magnetic field generator, and a probe sensor senses magnetic field information, i.e., real-time magnetic field information; a real-time conversion relationship between a probe coordinate system and a magnetic field coordinate system is determined by the real-time magnetic field information; and the accurate display position of the surgical instrument in the medical image, i.e., the target position, can be obtained by combining the real-time conversion relationship and the target conversion relationship. Therefore, the accurate display position of the surgical instrument in the medical image can be determined without relying on image capture during the operation. The magnetic field information sensed by the probe sensor provides stable position information, which is not affected by external factors such as light, occlusion, and tissue deformation, thereby realizing three-dimensional position positioning of the surgical instrument and improving the accuracy of the position information of the surgical instrument. This solves the problem in the related art that the accuracy of the surgical navigation system is not high, resulting in low surgical accuracy and safety, thereby improving the accuracy of the surgical navigation system and thereby improving the accuracy and safety of the operation. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] Figure 1 A schematic flow chart of a method for real-time display of surgical instruments in medical images according to an embodiment of the present invention;
[0024] Figure 2 A schematic diagram of a conversion relationship between coordinate systems according to an embodiment of the present invention;
[0025] Figure 3 A schematic diagram of the conversion relationship between another coordinate system according to an embodiment of the present invention;
[0026] Figure 4 It is a structural schematic diagram of a real-time display device of a surgical instrument in a medical image according to an embodiment of the present invention;
[0027] Figure 5 The figure is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION
[0028] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. Although certain embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be interpreted as being limited to the embodiments described herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present invention. It should be understood that the drawings and embodiments of the present invention are only for exemplary purposes and are not intended to limit the scope of protection of the present invention.
[0029] It should be understood that the various steps described in the method embodiments of the present invention may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this respect.
[0030] The term "including" and its variations used in this document are open inclusions, that is, "including but not limited to"; the term "based on" means "based at least in part on"; the term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments"; the term "optionally" means "optional embodiments". The relevant definitions of other terms will be given in the following description. It should be noted that the concepts of "first", "second", etc. mentioned in the present invention are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.
[0031] It should be noted that the modifications of "one" and "plurality" mentioned in the present invention are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise clearly indicated in the context, it should be understood as "one or more".
[0032] The names of the messages or information exchanged between multiple devices in the embodiments of the present invention are only used for illustrative purposes, and are not used to limit the scope of these messages or information.
[0033] In view of the problems existing in the above-mentioned related technologies, this embodiment provides a method and device for real-time display of surgical instruments in medical images.
[0034] like Figure 1 As shown, an embodiment of the present invention provides a method for real-time display of surgical instruments in medical images, comprising:
[0035] S100, sensing first magnetic field information through a probe sensor, and determining a real-time conversion relationship between a probe coordinate system corresponding to the probe sensor and a magnetic field coordinate system corresponding to a magnetic field generator according to the first magnetic field information, wherein the relative position of the probe sensor and the surgical instrument remains unchanged.
[0036] Specifically, the above method is applied to a surgical navigation system. In this embodiment, the surgical navigation system includes a magnetic field generator, a probe consumable, and a display screen, wherein the magnetic field generator is used to generate a magnetic field, the probe consumable is clamped with a surgical instrument by a clip, and a probe sensor, i.e., a probe sensor corresponding to the surgical instrument, is fixed on the probe consumable or the surgical instrument, and the display screen is used to display the patient's medical image. During the operation, the magnetic field generator in the surgical navigation system is fixed in position, generates a magnetic field at a fixed position, and forms a magnetic field coordinate system; the probe sensor is a sensor with spatial positioning capability, and the probe sensor senses the magnetic field information to obtain the position and angle of the probe sensor's own coordinate system relative to the magnetic field coordinate system, and the position and angle can be combined into a transformation relationship between the two coordinate systems; the display screen is used to display the patient's corresponding medical image.
[0037] It should be noted that the relative position between the surgical instrument and the probe sensor remains unchanged. The doctor uses the surgical instrument to perform medical treatment on the patient's lesion area. During the treatment process, the position of the surgical instrument is constantly changing, and the position of the probe sensor is also constantly changing. Therefore, the positional relationship between the probe sensor and the magnetic field generator is also constantly changing, that is, the change relationship between the probe coordinate system corresponding to the probe sensor and the magnetic field coordinate system corresponding to the magnetic field generator is also changing in real time. The probe sensor can sense the magnetic field information of its own location in real time, that is, the real-time magnetic field information, and determine the real-time conversion relationship between the probe coordinate system corresponding to the probe sensor and the magnetic field coordinate system corresponding to the magnetic field generator based on the perceived real-time magnetic field information. The coordinates of any position in the probe coordinate system can be converted into the coordinates of the position in the magnetic field coordinate system through this real-time conversion relationship.
[0038] S200, determining the real-time position of the surgical instrument in the medical image according to the real-time conversion relationship and the target conversion relationship to obtain the target position, wherein the target conversion relationship is used to represent the conversion relationship between the magnetic field coordinate system and the image coordinate system corresponding to the medical image, and the medical image is obtained by performing an electronic computed tomography scan on the target area of the patient.
[0039] Specifically, when the surgical instrument is displayed in real time in the medical image, it is necessary to determine the real-time position of the surgical instrument in the medical image, such as Figure 2As shown, the relationship between the probe coordinate system and the registered coordinate system can be determined according to the real-time conversion relationship and the target conversion relationship, that is, the real-time position of the surgical instrument in the medical image can be determined according to the real-time conversion relationship and the target conversion relationship, wherein the real-time conversion relationship is the conversion relationship between the real-time probe coordinate system and the magnetic field coordinate system, and the target conversion relationship is the conversion relationship between the magnetic field coordinate system and the image coordinate system. For the surgical instrument, its coordinates in the probe coordinate system can be converted into its coordinates in the magnetic field coordinate system through the real-time conversion relationship, and its coordinates in the magnetic field coordinate system can be converted into its coordinates in the image coordinate system through the target conversion relationship, and its coordinates in the image coordinate system can represent the real-time position of the surgical instrument in the medical image.
[0040] The target area of the patient refers to the lesion area of the patient, that is, a specific area where abnormal changes or abnormal structures occur in human tissues or organs. Medical imaging of the patient can obtain a three-dimensional visual medical image, which has a corresponding three-dimensional coordinate system, namely, an image coordinate system. For example, before surgery, a computer tomography scan is performed on the target area of the patient to obtain a medical image. Computer tomography (CT scan) is a medical imaging examination technology that generates cross-sectional images using X-rays and computer technology to provide detailed information on the human anatomical structure. CT scans can display the internal tissue structure, organ location, and pathological conditions of the body.
[0041] It should be noted that the target conversion relationship is obtained by calibration during surgery. When the position of the patient and the position of the magnetic field generator do not move, the target conversion relationship is determined to remain unchanged.
[0042] S300, displaying the surgical instrument in real time at the target position in the medical image.
[0043] Specifically, after determining the corresponding position of the surgical instrument in the medical image, i.e., the target position, the surgical instrument is displayed at the target position in the medical image. The doctor can observe the position of the surgical instrument in the patient's body on the medical image displayed on the display screen to understand the position, direction and depth of the surgical instrument to ensure the accuracy of the surgical operation.
[0044] In this embodiment, a magnetic field is generated by a magnetic field generator, and the probe sensor senses the magnetic field information, i.e., real-time magnetic field information. The real-time conversion relationship between the probe coordinate system and the magnetic field coordinate system is determined by the real-time magnetic field information. The accurate display position of the surgical instrument in the medical image, i.e., the target position, can be obtained by combining the real-time conversion relationship and the target conversion relationship. Therefore, the accurate display position of the surgical instrument in the medical image can be determined without relying on image capture during the operation. The magnetic field information sensed by the probe sensor provides stable position information, which is not affected by external factors such as light, occlusion, and tissue deformation. The three-dimensional position positioning of the surgical instrument is realized, and the accuracy of the position information of the surgical instrument is improved. The problem of low accuracy of the surgical navigation system in the related technology, resulting in low surgical accuracy and safety, is solved, the accuracy of the surgical navigation system is improved, and the accuracy and safety of the operation are improved.
[0045] Optionally, before determining the real-time position of the surgical instrument in the medical image based on the real-time conversion relationship and the target conversion relationship, the method also includes: acquiring the coordinates of multiple markers in the image coordinate system to obtain multiple first coordinates, wherein the multiple markers are pasted on the target area; determining the coordinates of the multiple markers in the magnetic field coordinate system to obtain multiple second coordinates; and determining the target conversion relationship based on the multiple first coordinates and the multiple second coordinates.
[0046] Specifically, before starting the operation, the surgical navigation system needs to be calibrated to determine the target transformation relationship so that the target transformation relationship can be used during the operation to transform the coordinates of the same spatial position in the magnetic field coordinate system into the coordinates in the image coordinate system.
[0047] During calibration, multiple markers are pasted on the target area of the patient, wherein the number of markers should be three or more, and the number of markers can be three, four, etc. In addition, the markers are pasted in the target area of the patient, wherein the target area includes the lesion area. For example, in neurosurgery, multiple markers are pasted on the patient's head.
[0048] After a plurality of markers are affixed to the target area of the patient, a computerized tomography scan, i.e. a CT scan, is performed on the target area of the patient, and a medical image of the target area of the patient is acquired by X-ray scanning. These markers can be developed in the medical image, and each marker is displayed in the medical image in the form of a developing ball. The positions of these developing balls in the medical image can be acquired, i.e., the coordinates of the developing balls in the image coordinate system, i.e., a plurality of first coordinates, are acquired.
[0049] Determine the coordinates of multiple markers in the magnetic field coordinate system, that is, multiple second coordinates. By matching the coordinates of multiple markers in the two coordinate systems, a correspondence between the markers is established, and their correspondence in the two coordinate systems is determined. The conversion relationship between the two coordinate systems can be calculated by mathematical methods, that is, the target conversion relationship is determined based on multiple first coordinates and multiple second coordinates.
[0050] Optionally, determining the coordinates of multiple markers in the magnetic field coordinate system to obtain multiple second coordinates includes: performing the following operations for each of the multiple markers, and the marker when performing the following operations is the current marker: when the registration sensor is rigidly connected to the current marker, obtaining the offset of the current marker relative to the registration sensor to obtain the current offset; sensing the current magnetic field information through the registration sensor, and determining the current conversion relationship when the registration sensor is rigidly connected to the current marker based on the current magnetic field information, wherein the current conversion relationship is used to represent the conversion relationship between the registration coordinate system corresponding to the registration sensor and the magnetic field coordinate system; determining multiple second coordinates based on the current offset and the current conversion relationship.
[0051] Specifically, when there is only one registration sensor and the registration sensor can only be connected to one marker at a time, the coordinates of multiple markers in the magnetic field coordinate system are determined separately, that is, the registration sensor is first rigidly connected to a marker to obtain the coordinates of the marker in the magnetic field coordinate system, and then the connection between the marker and the registration sensor is disconnected, and then the registration sensor is moved to be rigidly connected to the next marker to determine the coordinates of the next marker in the magnetic field coordinate system. The above operation is repeated until the coordinates of all markers in the magnetic field coordinate system are determined to obtain multiple second coordinates.
[0052] When determining the coordinates of any one of the multiple markers (i.e., the current marker) in the magnetic field coordinate system, the current marker is rigidly connected to the registration sensor, and the offset of the current marker relative to the registration sensor, i.e., the current offset, can be obtained. Based on this, the coordinates of the multiple markers in the registration coordinate system corresponding to the registration sensor can be determined, wherein the current offset can be a known quantity measured in advance, which is measured when the registration sensor leaves the factory and pre-stored in the system. The registration sensor is connected to any marker in a fixed rigid connection manner. When the marker is connected to the registration sensor, the offset pre-stored in the system is obtained, i.e., the offset of the current marker relative to the registration sensor; then, when the registration sensor is rigidly connected to the current marker, the conversion relationship between the registration coordinate system and the magnetic field coordinate system, i.e., the current conversion relationship, is obtained. According to the current conversion relationship and the current offset, the coordinates of the current marker in the magnetic field coordinate system, the current second coordinate, can be determined.
[0053] Rigid connection means that there is no deformable or adjustable connection part between the marker and the registration sensor, maintaining a fixed relative position relationship. The rigid connection design ensures the correspondence between the position and posture of the marker and the registration sensor.
[0054] Optionally, the method further includes: when the position of the patient moves or deflects, sensing first magnetic field information through a registration sensor, and determining a first conversion relationship based on the first magnetic field information, wherein the first conversion relationship is used to represent the conversion relationship between the registration coordinate system after the patient moves and the registration coordinate system before the patient moves; updating the target conversion relationship based on the first conversion relationship
[0055] Specifically, during surgery, when the patient's position may move or deflect, the actual conversion relationship between the magnetic field coordinate system and the image coordinate system will change accordingly. Therefore, the target conversion relationship needs to be updated to ensure accurate determination of the position of the surgical instrument in the medical image.
[0056] The target conversion relationship can be updated through the change of the magnetic field information sensed by the registration sensor. Specifically, when the patient moves or deflects, the relative position of the registration sensor and the magnetic field sensor changes. The magnetic field information of the registration sensor after the position change is sensed by the registration sensor, that is, the first magnetic field information. According to the first magnetic field information, the conversion relationship of the registration coordinate system before and after the patient moves, that is, the first conversion relationship, can be determined.
[0057] The target transformation relationship is updated according to the first transformation relationship, that is, a new transformation relationship is obtained according to the product of the first transformation relationship and the target transformation relationship before movement to compensate for the change in the target transformation relationship after the patient moves. The target transformation relationship is updated to the new transformation relationship, and when the coordinate position of the surgical instrument in the image coordinate system is subsequently determined during the operation, the coordinate position of the surgical instrument in the magnetic field coordinate system is converted into its coordinate position in the image coordinate system through the updated target transformation relationship.
[0058] In this embodiment, when the patient moves and deflects, determining the conversion relationship of the registered coordinate system before and after the patient's position changes can determine the conversion relationship between the real-time magnetic field coordinate system and the image coordinate system, avoiding the need for recalibration after the patient moves and deflects.
[0059] Optionally, before determining the real-time position of the surgical instrument in the medical image according to the real-time conversion relationship and the target conversion relationship, the method also includes: before determining the real-time position of the surgical instrument in the medical image according to the real-time conversion relationship and the target conversion relationship, the method also includes: sensing the second magnetic field information through the registration sensor, and determining the second conversion relationship according to the second magnetic field information, wherein the second conversion relationship is used to represent the conversion relationship between the registration coordinate system corresponding to the registration sensor and the magnetic field coordinate system; obtaining the coordinates of multiple markers in the image coordinate system to obtain multiple first coordinates, wherein the multiple markers are pasted on the target area; obtaining the offsets of the multiple markers relative to the registration sensor to obtain multiple offsets, wherein the multiple markers are rigidly connected to the registration sensor; determining the third conversion relationship according to the multiple first coordinates and the multiple offsets, wherein the third conversion relationship is used to represent the conversion relationship between the registration coordinate system corresponding to the registration sensor and the image coordinate system; determining the target conversion relationship according to the second conversion relationship and the third conversion relationship.
[0060] Specifically, before starting the operation, the surgical navigation system needs to be calibrated to determine the target transformation relationship so that the target transformation relationship can be used during the operation to transform the coordinates of the same spatial position in the magnetic field coordinate system into the coordinates in the image coordinate system.
[0061] During calibration, multiple markers are pasted on the target area of the patient, wherein the number of markers should be three or more, and the number of markers can be three, four, etc. In addition, the markers are pasted in the target area of the patient, wherein the target area includes the lesion area. For example, in neurosurgery, multiple markers are pasted on the patient's head.
[0062] Multiple markers are rigidly connected to the registration sensor respectively. Here, multiple markers are rigidly connected to the same registration sensor at the same time. The fixed relative position relationship between any marker and the registration sensor can be represented by the offset of the marker relative to the registration sensor, and multiple markers correspond to multiple offsets one by one. Rigid connection means that there is no deformable or adjustable connection part between the marker and the registration sensor, and a fixed relative position relationship is maintained. The design of rigid connection ensures that the position and posture of multiple markers and the offset between the registration sensor remain unchanged. The fixed relative position relationship between any marker and the registration sensor can be represented by the offset of the marker relative to the registration sensor, and multiple markers correspond to multiple offsets one by one.
[0063] After a plurality of markers are affixed to the target area of the patient, a computerized tomography scan, i.e. a CT scan, is performed on the target area of the patient, and a medical image of the target area of the patient is acquired by X-ray scanning. These markers can be developed in the medical image, and each marker is displayed in the medical image in the form of a developing ball. The positions of these developing balls in the medical image can be acquired, i.e., the coordinates of the developing balls in the image coordinate system, i.e., a plurality of first coordinates, are acquired.
[0064] Multiple first coordinates represent the coordinate positions of multiple markers in the image coordinate system, and multiple offsets represent the positions of multiple markers relative to the registration sensor. Based on the multiple offsets, the coordinate positions of the multiple markers in the registration coordinate system corresponding to the registration sensor can be determined. Based on the coordinate positions of the multiple markers in two different coordinate systems, the conversion relationship between the above two coordinate systems (registration coordinate system and image coordinate system), that is, the third conversion relationship, can be determined.
[0065] In addition, the registration sensor rigidly connected to multiple markers can sense their magnetic field information in the magnetic field generated by the magnetic field generator, that is, the second magnetic field information. According to the second magnetic field information, the conversion relationship between the registration coordinate system of the registration sensor and the magnetic field coordinate system, that is, the second conversion relationship, can be determined.
[0066] like Figure 3 As shown, after determining the conversion relationship between the registered coordinate system and the magnetic field coordinate system and the conversion relationship between the registered coordinate system and the image coordinate system, the conversion relationship between the magnetic field coordinate system and the image coordinate system can be determined based on the above two conversion relationships (the second conversion relationship and the third conversion relationship), that is, the target conversion relationship is determined based on the second conversion relationship and the third conversion relationship.
[0067] For example, the above target conversion relationship can be determined by the following formula:
[0068] C=A*B
[0069]
[0070]
[0071] Among them, A represents the second conversion relationship, B represents the third conversion relationship, and C represents the target conversion relationship. Represents the coordinate position of the i-th position in the actual space in the image coordinate system, Represents the coordinate position of the i-th position in the registered coordinate system, Represents the coordinate position of the i-th position in the magnetic field coordinate system.
[0072] In this embodiment, through multiple markers and registration sensors rigidly connected to the multiple markers at the same time, the conversion relationship between the registration coordinate system corresponding to the registration sensor and the image coordinate system and the magnetic field coordinate system is determined, and the registration sensor is used as an intermediate link to establish the connection between the magnetic field coordinate system and the image coordinate system to obtain the target conversion relationship.
[0073] Through the rigid connection between the marker and the registration sensor, the exact offset of the marker relative to the registration sensor is accurately obtained to ensure the accuracy of the conversion between various coordinate systems.
[0074] Optionally, the method also includes: when the position of the patient moves or deflects, sensing magnetic field information through the registration sensor for the third time, and determining a fourth conversion relationship based on the third magnetic field information, wherein the fourth conversion relationship is used to represent the conversion relationship between the registration coordinate system corresponding to the registration sensor and the magnetic field coordinate system after the patient moves; and updating the target conversion relationship based on the third conversion relationship and the fourth conversion relationship.
[0075] Specifically, during surgery, the patient's position may move or deflect and when multiple markers are rigidly connected to the same registration sensor at the same time, the actual transformation relationship between the magnetic field coordinate system and the image coordinate system will change accordingly. Therefore, the target transformation relationship needs to be updated to ensure the accurate determination of the position of the surgical instrument in the medical image.
[0076] The target conversion relationship can be updated by the change of the magnetic field information sensed by the registration sensor. Specifically, when the patient's position moves or deflects, the relative position of the registration sensor and the magnetic field sensor changes. The magnetic field information of the registration sensor after the position change is sensed by the registration sensor, that is, the third magnetic field information. According to the third magnetic field information, the conversion relationship between the current registration sensor and the magnetic field sensor, that is, the fourth conversion information, can be determined.
[0077] Since multiple markers are pasted on the patient and are rigidly connected to the registration sensor, when the patient's position moves or deflects, the relative position of the marker and the registration sensor remains unchanged, that is, the position of the marker in the registration coordinate system remains unchanged, and the position of the marker in the target area of the patient also remains unchanged, that is, the position of the marker in the image coordinate system remains unchanged. It can be seen that the movement or deflection of the patient's position will not cause the conversion relationship between the registration coordinate system and the image coordinate system to change, that is, the conversion relationship between the registration coordinate system and the image coordinate system remains unchanged as the third conversion relationship.
[0078] The target transformation relationship is updated according to the third transformation relationship and the fourth transformation relationship, that is, a new transformation relationship is obtained according to the product of the third transformation relationship and the fourth transformation, and the target transformation relationship is updated to the new transformation relationship. When the coordinate position of the surgical instrument in the image coordinate system is subsequently determined during the operation, the coordinate position of the surgical instrument in the magnetic field coordinate system is converted into its coordinate position in the image coordinate system through the updated target transformation relationship.
[0079] In this embodiment, when the patient moves and deflects, multiple markers are affixed to the patient, and the multiple markers are rigidly connected to the registration sensor. It is only necessary to determine the coordinate positions of the registration coordinate system and the magnetic field coordinate system after the patient's position changes to determine the conversion relationship between the real-time magnetic field coordinate system and the image coordinate system, thereby avoiding the need for recalibration after the patient moves and deflects.
[0080] Optionally, determining the real-time position of the surgical instrument in the medical image according to the real-time conversion relationship and the target conversion relationship to obtain the target position includes: respectively determining the coordinates of the first part and the second part of the surgical instrument in the probe coordinate system corresponding to the probe sensor to obtain a third coordinate and a fourth coordinate; determining the coordinates of the first part in the image coordinate system according to the real-time conversion relationship, the third coordinate and the target conversion relationship to obtain a fifth coordinate; determining the coordinates of the second part in the image coordinate system according to the real-time conversion relationship and the fourth coordinate to obtain a sixth coordinate, wherein the target position includes the fifth coordinate and the sixth coordinate.
[0081] Optionally, the target position in the medical image displays the surgical instrument in real time, including: in the medical image, connecting the position corresponding to the fifth coordinate and the position corresponding to the sixth coordinate to obtain a target line segment; and displaying the target line segment in the medical image, wherein the target line segment is used to represent the surgical instrument.
[0082] Specifically, the first part may be the head end of the surgical instrument, and the second part may be the tail end of the surgical instrument. When the surgical instrument is displayed in a medical image, the positions of the head end and the tail end in the medical image are determined, and the two positions corresponding to the head end and the tail end are connected with a line segment in the medical image, and the line segment, i.e., the target line segment, is displayed in the medical image.
[0083] When determining the real-time position of the surgical instrument in the medical image based on the real-time conversion relationship and the target conversion relationship, determine the coordinate positions of the head end and the tip of the surgical instrument in the image coordinate system, that is, the fifth coordinate and the sixth coordinate. First, determine the coordinates of the head end and the tip of the surgical instrument in the probe coordinate system to obtain the third coordinate and the fourth coordinate.
[0084] According to the real-time conversion relationship, the coordinates of the head end of the surgical instrument in the probe coordinate system, i.e., the third coordinate, are converted to obtain the coordinates of the head end in the magnetic field coordinate system. According to the target conversion relationship, the coordinates of the head end in the magnetic field coordinate system are converted to obtain the coordinates of the head end in the image coordinate system, i.e., the fifth coordinate. According to the real-time conversion relationship, the coordinates of the end of the surgical instrument in the probe coordinate system, i.e., the fourth coordinate, are converted to obtain the coordinates of the end in the magnetic field coordinate system. According to the target conversion relationship, the coordinates calculated in the magnetic field coordinate system are converted to obtain the coordinates of the end in the image coordinate system, i.e., the sixth coordinate.
[0085] In this embodiment, the display position of the surgical instrument in the medical image is determined by the head end and the tail end, i.e., the first part and the second part, providing the doctor with intuitive instrument position information, helping the doctor to better understand the position and direction of the surgical instrument in the patient's body, thereby guiding the doctor's surgical operation and improving the accuracy and safety of the operation.
[0086] like Figure 4 As shown, an embodiment of the present invention provides a real-time display device 400 of a surgical instrument in a medical image, comprising:
[0087] A first determination module 410 is used to sense real-time magnetic field information through a probe sensor, and determine a real-time conversion relationship between a probe coordinate system corresponding to the probe sensor and a magnetic field coordinate system corresponding to a magnetic field generator according to the real-time magnetic field information, wherein the relative position between the probe sensor and the surgical instrument remains unchanged;
[0088] A second determination module 420 is used to determine the real-time position of the surgical instrument in the medical image according to the real-time conversion relationship and the target conversion relationship to obtain the target position, wherein the target conversion relationship is used to represent the conversion relationship between the magnetic field coordinate system and the image coordinate system corresponding to the medical image, and the medical image is obtained by performing an electronic computer tomography scan on the target area of the patient;
[0089] The display module 430 is used to display the surgical instrument in real time at the target position in the medical image.
[0090] The real-time display device of surgical instruments in medical images of this embodiment is used to implement the real-time display method of surgical instruments in medical images as described above. Its advantages over the prior art are the same as the advantages of the real-time display method of surgical instruments in medical images as described above over the prior art, and will not be repeated here.
[0091] Optionally, the above-mentioned device is also used to obtain the coordinates of multiple markers in the image coordinate system to obtain multiple first coordinates before determining the real-time position of the surgical instrument in the medical image according to the real-time conversion relationship and the target conversion relationship, wherein the multiple markers are pasted on the target area; determine the coordinates of the multiple markers in the magnetic field coordinate system to obtain multiple second coordinates; and determine the target conversion relationship based on the multiple first coordinates and the multiple second coordinates.
[0092] Optionally, the above-mentioned device is also used to determine the coordinates of multiple markers in the magnetic field coordinate system in the following manner to obtain multiple second coordinates: perform the following operations for each of the multiple markers, and the marker when performing the following operations is the current marker: when the registration sensor is rigidly connected to the current marker, obtain the offset of the current marker relative to the registration sensor to obtain the current offset; sense the current magnetic field information through the registration sensor, and determine the current conversion relationship when the registration sensor is rigidly connected to the current marker based on the current magnetic field information, wherein the current conversion relationship is used to represent the conversion relationship between the registration coordinate system corresponding to the registration sensor and the magnetic field coordinate system; determine the current second coordinate based on the current offset and the current conversion relationship.
[0093] Optionally, the above-mentioned device is also used to sense the first magnetic field information through the registration sensor when the position of the patient moves or deflects, and determine a first conversion relationship based on the first magnetic field information, wherein the first conversion relationship is used to represent the conversion relationship between the registration coordinate system after the patient moves and the registration coordinate system before the patient moves; update the target conversion relationship based on the first conversion relationship
[0094] Optionally, the above-mentioned device is also used to determine the real-time position of the surgical instrument in the medical image according to the real-time conversion relationship and the target conversion relationship in the following manner, and obtaining the target position includes: respectively determining the coordinates of the first part and the second part of the surgical instrument in the probe coordinate system corresponding to the probe sensor to obtain a third coordinate and a fourth coordinate; determining the coordinates of the first part in the image coordinate system according to the real-time conversion relationship, the third coordinate and the target conversion relationship to obtain a fifth coordinate; determining the coordinates of the second part in the image coordinate system according to the real-time conversion relationship and the fourth coordinate to obtain a sixth coordinate, wherein the target position includes the fifth coordinate and the sixth coordinate.
[0095] Optionally, the above-mentioned device is also used to display the surgical instrument in real time at the target position in the medical image in the following manner, including: in the medical image, connecting the position corresponding to the fifth coordinate and the position corresponding to the sixth coordinate to obtain a target line segment; displaying the target line segment in the medical image, wherein the target line segment is used to represent the surgical instrument.
[0096] Optionally, the above-mentioned device is also used to, before determining the real-time position of the surgical instrument in the medical image according to the real-time conversion relationship and the target conversion relationship: sense the second magnetic field information through the registration sensor, and determine the second conversion relationship according to the second magnetic field information, wherein the second conversion relationship is used to represent the conversion relationship between the registration coordinate system corresponding to the registration sensor and the magnetic field coordinate system; obtain the coordinates of multiple markers in the image coordinate system to obtain multiple first coordinates, wherein multiple markers are pasted on the target area; obtain the offsets of multiple markers relative to the registration sensor to obtain multiple offsets, wherein multiple markers are rigidly connected to the registration sensor; determine the third conversion relationship according to the multiple first coordinates and the multiple offsets, wherein the third conversion relationship is used to represent the conversion relationship between the registration coordinate system corresponding to the registration sensor and the image coordinate system; determine the target conversion relationship according to the second conversion relationship and the third conversion relationship.
[0097] like Figure 5 As shown, an electronic device 500 provided by an embodiment of the present invention includes a memory 510 and a processor 520; the memory 510 is used to store a computer program; the processor 520 is used to implement the real-time display method of surgical instruments in medical images as described above when executing the computer program.
[0098] In other words, an electronic device 500 includes a memory 510 and a processor 520 coupled to the memory 510; the memory 510 is configured to store a computer program; and the processor 520 is configured to perform the following operations when executing the computer program:
[0099] The probe sensor senses real-time magnetic field information, and determines a real-time conversion relationship between a probe coordinate system corresponding to the probe sensor and a magnetic field coordinate system corresponding to a magnetic field generator according to the real-time magnetic field information, wherein the relative position between the probe sensor and the surgical instrument remains unchanged;
[0100] Determine the real-time position of the surgical instrument in the medical image according to the real-time conversion relationship and the target conversion relationship to obtain the target position, wherein the target conversion relationship is used to represent the conversion relationship between the magnetic field coordinate system and the image coordinate system corresponding to the medical image, and the medical image is obtained by performing an electronic computer tomography scan on the target area of the patient;
[0101] The surgical instrument is displayed in real time at the target position in the medical image.
[0102] An embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method for real-time display of surgical instruments in medical images as described above is implemented.
[0103] In other words, a non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor performs the following operations:
[0104] The probe sensor senses real-time magnetic field information, and determines a real-time conversion relationship between a probe coordinate system corresponding to the probe sensor and a magnetic field coordinate system corresponding to a magnetic field generator according to the real-time magnetic field information, wherein the relative position between the probe sensor and the surgical instrument remains unchanged;
[0105] Determine the real-time position of the surgical instrument in the medical image according to the real-time conversion relationship and the target conversion relationship to obtain the target position, wherein the target conversion relationship is used to represent the conversion relationship between the magnetic field coordinate system and the image coordinate system corresponding to the medical image, and the medical image is obtained by performing an electronic computer tomography scan on the target area of the patient;
[0106] The surgical instrument is displayed in real time at the target position in the medical image.
[0107] An electronic device 500 that can be used as a server or client of the present invention will now be described, which is an example of a hardware device that can be applied to various aspects of the present invention. The electronic device 500 is intended to represent various forms of digital electronic computer equipment, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device 500 can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples, and are not intended to limit the implementation of the present invention described and / or required herein.
[0108] The electronic device 500 includes a computing unit, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) or a computer program loaded from a storage unit into a random access memory (RAM). In the RAM, various programs and data required for the operation of the device can also be stored. The computing unit, ROM, and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.
[0109] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc. In the present application, the unit described as a separate component may or may not be physically separated, and the component displayed as a unit may or may not be a physical unit, that is, it may be located in one place, or it may be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment of the present invention. In addition, each functional unit in each embodiment of the present invention can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit.
[0110] Although the present invention is disclosed as above, the protection scope of the present invention is not limited thereto. Those skilled in the art may make various changes and modifications without departing from the spirit and scope of the present invention, and these changes and modifications will fall within the protection scope of the present invention.
Claims
1. A method for real-time display of surgical instruments in medical images, characterized in that: include: The probe sensor senses real-time magnetic field information, and determines a real-time conversion relationship between a probe coordinate system corresponding to the probe sensor and a magnetic field coordinate system corresponding to a magnetic field generator according to the real-time magnetic field information, wherein the relative position between the probe sensor and the surgical instrument remains unchanged; Determine the real-time position of the surgical instrument in the medical image according to the real-time conversion relationship and the target conversion relationship to obtain the target position, wherein the target conversion relationship is used to represent the conversion relationship between the magnetic field coordinate system and the image coordinate system corresponding to the medical image, and the medical image is obtained by performing an electronic computer tomography scan on the target area of the patient; The surgical instrument is displayed in real time at the target position in the medical image.
2. The method for real-time display of surgical instruments in medical images according to claim 1, characterized in that: Before determining the real-time position of the surgical instrument in the medical image according to the real-time conversion relationship and the target conversion relationship, the method further includes: Acquire coordinates of a plurality of markers in the image coordinate system to obtain a plurality of first coordinates, wherein the plurality of markers are attached to the target area; Determine the coordinates of a plurality of the markers in the magnetic field coordinate system to obtain a plurality of second coordinates; The target conversion relationship is determined according to a plurality of the first coordinates and a plurality of the second coordinates.
3. The method for real-time display of surgical instruments in medical images according to claim 2, characterized in that: The step of determining the coordinates of the plurality of markers in the magnetic field coordinate system to obtain a plurality of second coordinates comprises: For each of the multiple markers, the following operations are performed, and the marker when performing the following operations is the current marker: In the case where the registration sensor is rigidly connected to the current marker, obtaining an offset of the current marker relative to the registration sensor to obtain a current offset; The current magnetic field information is sensed by the registration sensor, and a current conversion relationship when the registration sensor is rigidly connected to the current marker is determined according to the current magnetic field information, wherein the current conversion relationship is used to represent the conversion relationship between the registration coordinate system corresponding to the registration sensor and the magnetic field coordinate system; The current second coordinate is determined according to the current offset and the current conversion relationship.
4. The method for real-time display of surgical instruments in medical images according to claim 2, characterized in that: The method further comprises: In the case where the position of the patient moves or deflects, first magnetic field information is sensed by a registration sensor, and a first conversion relationship is determined according to the first magnetic field information, wherein the first conversion relationship is used to represent a conversion relationship between the registration coordinate system after the patient moves and the registration coordinate system before the patient moves; The target conversion relationship is updated according to the first conversion relationship.
5. The method for real-time display of surgical instruments in medical images according to claim 1, characterized in that: Determining the real-time position of the surgical instrument in the medical image according to the real-time conversion relationship and the target conversion relationship to obtain the target position includes: Respectively determining the coordinates of the first part and the second part of the surgical instrument in the probe coordinate system corresponding to the probe sensor to obtain a third coordinate and a fourth coordinate; Determine the coordinates of the first part in the image coordinate system according to the real-time conversion relationship, the third coordinate and the target conversion relationship to obtain a fifth coordinate; The coordinates of the second part in the image coordinate system are determined according to the real-time conversion relationship and the fourth coordinate to obtain a sixth coordinate, wherein the target position includes the fifth coordinate and the sixth coordinate.
6. The method for real-time display of surgical instruments in medical images according to claim 5, characterized in that: The real-time display of the surgical instrument at the target position in the medical image includes: In the medical image, connecting the position corresponding to the fifth coordinate and the position corresponding to the sixth coordinate to obtain a target line segment; The target line segment is displayed in the medical image, wherein the target line segment is used to represent the surgical instrument.
7. The method for real-time display of surgical instruments in medical images according to claim 1, characterized in that: Before determining the real-time position of the surgical instrument in the medical image according to the real-time conversion relationship and the target conversion relationship, the method further includes: Sensing second magnetic field information through a registration sensor, and determining a second conversion relationship according to the second magnetic field information, wherein the second conversion relationship is used to represent a conversion relationship between a registration coordinate system corresponding to the registration sensor and the magnetic field coordinate system; Acquire coordinates of a plurality of markers in the image coordinate system to obtain a plurality of first coordinates, wherein the plurality of markers are attached to the target area; Acquire the offsets of the plurality of markers relative to the registration sensor to obtain a plurality of offsets, wherein the plurality of markers are rigidly connected to the registration sensor; Determine a third conversion relationship according to the plurality of the first coordinates and the plurality of the offsets, wherein the third conversion relationship is used to represent a conversion relationship between a registration coordinate system corresponding to the registration sensor and the image coordinate system; The target conversion relationship is determined according to the second conversion relationship and the third conversion relationship.
8. A real-time display device for surgical instruments in medical images, characterized in that: include: A first determination module is used to sense real-time magnetic field information through a probe sensor, and determine a real-time conversion relationship between a probe coordinate system corresponding to the probe sensor and a magnetic field coordinate system corresponding to a magnetic field generator according to the real-time magnetic field information, wherein the relative position of the probe sensor and the surgical instrument remains unchanged; A second determination module is used to determine the real-time position of the surgical instrument in the medical image according to the real-time conversion relationship and the target conversion relationship to obtain the target position, wherein the target conversion relationship is used to represent the conversion relationship between the magnetic field coordinate system and the image coordinate system corresponding to the medical image, and the medical image is obtained by performing an electronic computer tomography scan on the target area of the patient; A display module is used to display the surgical instrument in real time at the target position in the medical image.
9. An electronic device, characterized in that: including memory and processor; The memory is used to store computer programs; The processor is used to implement the real-time display method of the surgical instrument in the medical image as described in any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the real-time display method of the surgical instrument in the medical image as described in any one of claims 1 to 7 is implemented.
Citation Information
Cited By
Image registration method based on magnetic navigation
CN121421686A
