Calibration method, device control method, apparatus, electronic device and storage medium
By using the coordinate transformation relationship between liquid and solid markers in MRI equipment, the optical positioning system was effectively calibrated in MRI equipment, solving the problem of soft tissue positioning in MRI systems and providing accurate coordinate transformation and positioning capabilities.
Patent Information
- Application Number
- CN202211160190.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-09-22
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2042-09-22
AI Technical Summary
Existing optical positioning systems struggle to achieve precise soft tissue localization in MRI systems, especially in MRI imaging technology. Effectively calibrating the optical positioning system with the coordinate system of the MRI equipment is a challenge.
By setting liquid and solid markers on the marking tool, the liquid markers are scanned by the MRI equipment to obtain their first coordinates in the coordinate system of the MRI equipment, and the solid markers are photographed by the optical positioning system to obtain their second coordinates in the world coordinate system. Based on the position data of the two, the third coordinates of the solid markers in the coordinate system of the MRI equipment are determined. Finally, the MRI equipment is calibrated through coordinate transformation.
It enables effective calibration of the optical positioning system in MRI equipment, allowing the coordinates detected by the MRI equipment to be transformed into the world coordinate system, providing precise positioning capability for soft tissues and simplifying the calibration process.
Smart Images

Figure CN115399880B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of surgical navigation technology, and in particular to a calibration method, an instrument control method, a device, an electronic device and a storage medium. Background Art
[0002] Current optical positioning systems can be used in the medical field to assist in locating the position of operating objects, track the position of surgical instruments in real time, and provide precise surgical navigation.
[0003] At present, medical robots usually use CT imaging technology combined with optical positioning systems to achieve surgical navigation and positioning in bone treatment. The optical positioning system mainly identifies solid components such as bones. When the optical positioning system is used in the CT system, the markers are easier to identify.
[0004] There is also a demand for performing surgery on soft tissues (such as the brain, muscles, heart, etc.) through medical robots. Imaging of soft tissues of the body requires MRI (magnetic resonance imaging) technology. Therefore, how to achieve optical positioning in the MRI system is very important. Summary of the invention
[0005] The present invention provides a calibration method, an instrument control method, a device, an electronic device and a storage medium to achieve optical positioning in an MRI system.
[0006] In a first aspect, the present invention provides a calibration method for calibrating an MRI device by using a marking tool, wherein the marking tool is provided with a liquid marker and a solid marker, comprising:
[0007] Controlling the MRI device to scan the liquid marker to obtain a first coordinate of the liquid marker in the coordinate system of the MRI device, and photographing the solid marker through an optical positioning system to obtain a second coordinate of the solid marker in the world coordinate system;
[0008] Determine a third coordinate of the solid marker in the coordinate system of the MRI device according to the position data of the liquid marker and the solid marker on the marking tool and / or the first coordinate;
[0009] The MRI device is calibrated according to the second coordinate and the third coordinate.
[0010] In a second aspect, the present invention provides an apparatus control method, comprising:
[0011] Obtain the coordinates of the device in the world coordinate system;
[0012] Acquire the calibration coordinates of the operation object in the coordinate system of the MRI device from the magnetic resonance image containing the operation object;
[0013] Using the calibration coordinates as the operation object coordinates in the world coordinate system;
[0014] controlling the instrument according to the operation object coordinates and the instrument coordinates;
[0015] Wherein, the MRI device is calibrated by the calibration method described in the first aspect.
[0016] In a third aspect, the present invention provides a calibration device, comprising:
[0017] A marker coordinate acquisition module, used to control the MRI device to scan the liquid marker to obtain a first coordinate of the liquid marker in the coordinate system of the MRI device, and to photograph the solid marker through the optical positioning system to obtain a second coordinate of the solid marker in the world coordinate system;
[0018] A third coordinate acquisition module, used to determine a third coordinate of the solid marker in the coordinate system of the MRI device according to the first coordinate;
[0019] An MRI device calibration module is used to calibrate the MRI device based on the second coordinate and the third coordinate.
[0020] In a fourth aspect, the present invention provides an apparatus control device, comprising:
[0021] An instrument coordinate acquisition module is used to acquire the instrument coordinates of the instrument in the world coordinate system;
[0022] An MRI calibration coordinate acquisition module is used to acquire the calibration coordinates of the operation object in the coordinate system of the MRI device from the magnetic resonance image containing the operation object;
[0023] A world coordinate acquisition module, used to use the calibration coordinates as the operation object coordinates in the world coordinate system;
[0024] An instrument control module, used for controlling the instrument according to the operation object coordinates and the instrument coordinates;
[0025] Wherein, the MRI device is calibrated by the calibration method described in the first aspect.
[0026] In a fifth aspect, the present invention provides an electronic device, the electronic device comprising:
[0027] at least one processor; and
[0028] a memory communicatively connected to the at least one processor; wherein,
[0029] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the calibration method described in the first aspect of the present invention, and / or the instrument control method described in the second aspect.
[0030] In a sixth aspect, the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the calibration method described in the first aspect of the present invention, and / or the instrument control method described in the second aspect when executed.
[0031] A calibration method provided by an embodiment of the present invention is characterized in that it is used to calibrate an MRI device through a marking tool, on which a liquid marker and a solid marker are arranged, and includes: controlling the MRI device to scan the liquid marker to obtain a first coordinate of the liquid marker in the coordinate system of the MRI device, and photographing the solid marker through an optical positioning system to obtain a second coordinate of the solid marker in the world coordinate system; determining a third coordinate of the solid marker in the coordinate system of the MRI device based on position data of the liquid marker and the solid marker on the marking tool, and / or the first coordinate; and calibrating the MRI device based on the second coordinate and the third coordinate. According to the position data of the liquid marker and the solid marker on the marking tool, and / or the first coordinate, by comparing the coordinates of the same liquid marker in different coordinate systems, the conversion relationship between the coordinate system of the marking tool and the coordinate system of the MRI device can be analyzed, and then the position data of the solid marker on the marking tool can be converted to the coordinate system of the MRI device according to the conversion relationship to obtain the third coordinate, and finally, according to the third coordinate of the solid marker in the MRI device and the second coordinate of the solid marker in the world coordinate system, the final conversion relationship between the coordinate system of the MRI device and the world coordinate system can be obtained, and the calibration of the MRI device can be realized based on the final conversion relationship. The whole calibration process is simple, making it feasible to implement the optical positioning system in the MRI device.
[0032] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0034] Figure 1 is a flow chart of a calibration method provided according to Embodiment 1 of the present invention;
[0035] Figure 2 is a schematic diagram of the coordinates of a marker collected by an MRI device and an optical positioning system according to Embodiment 1 of the present invention;
[0036] Figure 3 is a flow chart of a calibration method provided according to Embodiment 2 of the present invention;
[0037] Figure 4 is a schematic diagram of a marking tool provided according to Embodiment 2 of the present invention;
[0038] Figure 5 is a schematic diagram of a coordinate system of a marking tool provided according to Embodiment 2 of the present invention;
[0039] Figure 6 is a schematic diagram of another marking tool provided according to Embodiment 2 of the present invention;
[0040] Figure 7 is a flow chart of a device control method provided according to Embodiment 3 of the present invention;
[0041] Figure 8 is a structural schematic diagram of a calibration device provided according to a fourth embodiment of the present invention;
[0042] Fig. 9 is a structural schematic diagram of an apparatus control device provided according to Embodiment 5 of the present invention;
[0043] Fig.10 It is a schematic diagram of the structure of an electronic device provided in Embodiment 6 of the present invention. DETAILED DESCRIPTION
[0044] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0045] Embodiment 1
[0046] Figure 1This is a flow chart of a calibration method provided in the first embodiment of the present invention. This embodiment is applicable to the case where an MRI (magnetic resonance imaging) device is calibrated by a marking tool. The method can be executed by a calibration device, which can be implemented in the form of hardware and / or software. The calibration device can be configured in an electronic device, for example, in a computer used to calibrate the MRI device. Figure 1 As shown, the calibration method includes:
[0047] S101, controlling the MRI device to scan the liquid marker to obtain a first coordinate of the liquid marker in the coordinate system of the MRI device, and photographing the solid marker through the optical positioning system to obtain a second coordinate of the solid marker in the world coordinate system.
[0048] The optical positioning system can identify solid components such as bones. For example, using the optical positioning system in the CT system, the positioning points in the bones can be easily observed. The optical positioning system can be a binocular vision positioning system, which is based on the principle of parallax. Two cameras are fixed on the same rigid body at a certain distance and angle. When working, the two cameras respectively obtain the mapping image of the same feature point in the field of view, and then according to the placement of the two cameras and the position of the feature point obtained in the image, the specific position of the feature point in the three-dimensional space is calculated through the triangulation principle, and the position of multiple feature points can be dynamically obtained.
[0049] MRI mainly identifies fluid components. For example, by using MRI to scan arteries and veins, clear cross-sectional and sagittal images can be obtained.
[0050] The MRI device, the marking tool, and the optical positioning system have different coordinate systems. The coordinate system of the optical positioning system is the world coordinate system, which is the absolute coordinate system. Therefore, the MRI device can be calibrated to convert the coordinates detected by the MRI device into coordinates in the world coordinate system, so that the coordinates of the two are placed in the same coordinate system, so as to facilitate the determination of the position of the operating object in the world coordinate system.
[0051] Figure 2 Schematic diagram of obtaining coordinates of different markers on the marking tool for the MRI device and the optical positioning system, such as Figure 2As shown, the marking tool 2 is generally a rigid tool, and the marking tool 2 includes a plurality of rigid rods, each of which is provided with a liquid marker and a solid marker. The liquid marker can be a soft container filled with liquid and sealed, or a soft substance, and the liquid marker can be detected by the MRI device 1 to obtain the first coordinate. The solid marker can be a reflective marker ball, and the solid marker can be identified by the optical positioning system 3 to obtain the second coordinate. Among them, the optical positioning system 3 is movable, and the shooting position and angle can be adjusted according to the position of the marking tool 2.
[0052] Specifically, the optical positioning system can form a binocular vision system through two near-infrared cameras. Infrared LEDs are fixed around the optical axes of the two cameras for illumination. The infrared light emitted by the infrared LEDs is reflected back to the camera photosensitive chip through the reflective balls on the solid markers on the marking tool. The images of the marking points are collected by the camera, and the marking points are identified and located through digital image processing technology with the assistance of a computer. The coordinates and direction of the needle tip of the marking tool can be calibrated according to the three-dimensional coordinates of the marking points, thereby achieving precise navigation of the surgery.
[0053] S102: Determine a third coordinate of the solid marker in the coordinate system of the MRI device according to the position data of the liquid marker and the solid marker on the marking tool and / or the first coordinate.
[0054] The position data of the liquid marker and the solid marker on the marking tool are the coordinates of the liquid marker and the solid marker on the marking tool.
[0055] Based on the position data of the liquid marker and the solid marker on the marking tool, and / or the first coordinate, the conversion relationship between the coordinate system of the marking tool and the coordinate system of the MRI device can be analyzed by comparing the liquid and liquid coordinates. Then, based on the conversion relationship, the position data of the solid marker on the marking tool can be converted to the coordinate system of the MRI device to obtain the third coordinate.
[0056] S103: Calibrate the MRI device according to the second coordinate and the third coordinate.
[0057] After determining the third coordinate of the solid marker in the coordinate system of the MRI device, the final conversion relationship from the coordinate system of the MRI device to the world coordinate system can be obtained according to the third coordinate of the solid marker in the MRI device and the second coordinate of the solid marker in the world coordinate system, that is, the conversion relationship of the two coordinate systems can be obtained for the coordinates of the same solid marker in two different coordinate systems, and finally the calibration of the MRI device can be realized based on the final conversion relationship. Then the coordinate points detected in the MRI device can be directly converted to the world coordinate system. The operator can intuitively see the coordinates of the soft or liquid object collected by the MRI device in the world coordinate system. In addition, a display screen can be set in the optical positioning system to display the coordinates of the operating object collected by the MRI device in the world coordinate system. The display screen can also display the position of the marking tool that can be detected by the optical positioning system in the world coordinate system, which is convenient for guiding the operator or manipulator to control the marking tool to operate the operating object.
[0058] A calibration method provided by an embodiment of the present invention is characterized in that it is used to calibrate an MRI device through a marking tool, on which a liquid marker and a solid marker are arranged, and includes: controlling the MRI device to scan the liquid marker to obtain a first coordinate of the liquid marker in the coordinate system of the MRI device, and photographing the solid marker through an optical positioning system to obtain a second coordinate of the solid marker in the world coordinate system; determining a third coordinate of the solid marker in the coordinate system of the MRI device based on position data of the liquid marker and the solid marker on the marking tool, and / or the first coordinate; and calibrating the MRI device based on the second coordinate and the third coordinate. According to the position data of the liquid marker and the solid marker on the marking tool, and / or the first coordinate, by comparing the coordinates of the same liquid marker in different coordinate systems, the conversion relationship between the coordinate system of the marking tool and the coordinate system of the MRI device can be analyzed, and then the position data of the solid marker on the marking tool can be converted to the coordinate system of the MRI device according to the conversion relationship to obtain the third coordinate, and finally, according to the third coordinate of the solid marker in the MRI device and the second coordinate of the solid marker in the world coordinate system, the final conversion relationship between the coordinate system of the MRI device and the world coordinate system can be obtained, and the calibration of the MRI device can be realized based on the final conversion relationship. The whole calibration process is simple, making it feasible to implement the optical positioning system in the MRI device.
[0059] Embodiment 2
[0060] Figure 3 This is a flow chart of a calibration method provided in the second embodiment of the present invention. The embodiment of the present invention is optimized based on the above-mentioned first embodiment. Figure 3 As shown, the calibration method includes:
[0061] S301, controlling the MRI device to scan the liquid marker to obtain a first coordinate of the liquid marker in the coordinate system of the MRI device, and photographing the solid marker through the optical positioning system to obtain a second coordinate of the solid marker in the world coordinate system.
[0062] S301 is the same as S101 in the first embodiment, and reference may be made to S101. Figure 4 As shown, the liquid marker 21 is arranged outside the solid marker 22, that is, for the same pair of liquid marker and solid marker, the coordinates of the two on the marking tool are different.
[0063] S302, obtaining a fourth coordinate of the liquid marker in the coordinate system of the marking tool, and obtaining a fifth coordinate of the solid marker in the coordinate system of the marking tool.
[0064] The marking tool itself has a set of coordinate systems, such as Figure 5 The coordinate system of the marking tool is shown in the figure. The coordinate system can be established with the plane where the rigid rods of the marking tool are located as the horizontal plane and the intersection point of the rigid rods as the origin. If the rigid rods are not in the same horizontal plane at the same time or there are multiple intersection points of the rigid rods, the plane where some of the rigid rods are located can be used as the horizontal plane, or, the intersection points of some of the rigid rods can be used as the origin. The present invention does not limit the setting method of the coordinate system of the marking tool.
[0065] It should be noted that when there are multiple liquid markers and multiple solid markers on the marking tool, the solid / liquid markers described by the MRI device, optical system, and marking tool should be the same solid / liquid marker. Taking the liquid marker as an example, Figure 5 As shown, in the marking tool, there are multiple liquid markers A1, A2, A3, and A4 at the same time. If the coordinate of the liquid marker in the coordinate system of the MRI device is the first coordinate for the liquid marker A1, then the coordinate of the liquid marker in the coordinate system of the marking tool is the position data and is also the coordinate corresponding to the liquid marker A1.
[0066] The fourth coordinate and the fifth coordinate are the coordinates in the coordinate system of the marking tool. When the coordinate system of the marking tool is established, the fourth coordinate and the fifth coordinate are determined. It should be noted that a set of monitoring equipment for the marking tool can be set up to specifically establish the coordinate system of the marking tool.
[0067] S303. Calculate an intermediate transformation matrix from the coordinate system of the marking tool to the coordinate system of the MRI device according to the first coordinate and the fourth coordinate.
[0068] The first coordinate is the coordinate of the liquid marker in the coordinate system of the MRI device, and the fourth coordinate is the coordinate of the liquid marker in the coordinate system of the marking tool. According to the relationship between the fourth coordinate and the first coordinate, the transformation relationship from the coordinate system of the marking tool to the coordinate system of the MRI device can be analyzed to obtain the intermediate transformation matrix.
[0069] Specifically, an intermediate transformation matrix from the coordinate system of the marking tool to the coordinate system of the MRI device can be calculated according to the first coordinate and the fourth coordinate, including:
[0070] Construct the first objective function:
[0071]
[0072] Among them, Ma i is the first coordinate of the liquid marker in the coordinate system of the MRI device, Ta i is the fourth coordinate of the liquid marker in the coordinate system of the marking tool, Rtm is the rotation matrix, Ttm is the translation matrix, and N is the number of liquid markers.
[0073] After constructing the first objective function, Rtm and Ttm, which minimize the function value of the first objective function, can be solved as the intermediate transformation matrix from the coordinate system of the marking tool to the coordinate system of the MRI device. Generally, ICP (iterative closest point) matching is used to solve it. For example, SVD (matrix singular value decomposition) and nonlinear optimization methods can be used to solve it.
[0074] S304 , using an intermediate transformation matrix to transform the fifth coordinate into a third coordinate of the solid marker in the coordinate system of the MRI device.
[0075] The fifth coordinate is the coordinate of the solid marker in the coordinate system of the marking tool. After solving the intermediate transformation matrix from the coordinate system of the marking tool to the coordinate system of the MRI device, the intermediate transformation matrix can be used to convert the fifth coordinate into the third coordinate of the solid marker in the coordinate system of the MRI device.
[0076] Specifically, the fifth coordinate is converted into the third coordinate of the solid marker in the coordinate system of the MRI device using an intermediate transformation matrix, including:
[0077] The third coordinate of the solid marker in the coordinate system of the MRI device is calculated by the following formula:
[0078] Mb i =Rtm×Tb i +Ttm
[0079] Among them, Mb i is the third coordinate of the solid marker in the coordinate system of the MRI device, Tb iis the fifth coordinate of the solid marker in the coordinate system of the marking tool, and Rtm and Ttm serve as the intermediate transformation matrix from the coordinate system of the marking tool to the coordinate system of the MRI device.
[0080] S305 , determining a target transformation matrix from the coordinate system of the MRI device to the world coordinate system according to the second coordinate and the third coordinate.
[0081] The second coordinate is the coordinate of the solid marker in the world coordinate system, and the third coordinate is the coordinate of the solid marker in the coordinate system of the MRI device. The conversion relationship from the coordinate system of the MRI device to the world coordinate system can be obtained based on the third coordinate and the second coordinate.
[0082] Specifically, determining a target transformation matrix from the coordinate system of the MRI device to the world coordinate system according to the second coordinate and the third coordinate includes:
[0083] Construct the second objective function:
[0084]
[0085] Among them, Mob i is the second coordinate of the solid marker in the world coordinate system, Mb i is the third coordinate of the solid marker in the coordinate system of the MRI device, and Rmo and Tmo are the target transformation matrices from the coordinate system of the MRI device to the world coordinate system.
[0086] The Rmo and Tmo that minimize the function value of the second objective function are solved as the target transformation matrix from the coordinate system of the MRI device to the world coordinate system.
[0087] S306: Calibrate the MRI device based on the target transformation matrix.
[0088] The target transformation matrix represents the conversion relationship from the coordinate system of the MRI device to the world coordinate system. Therefore, the MRI device can be calibrated based on the target transformation matrix. The coordinate points detected in the MRI device can be directly converted to the world coordinate system. The operator can intuitively see the coordinates of the soft or liquid object collected by the MRI device in the world coordinate system. In addition, a display screen can be set in the optical positioning system to display the coordinates of the operating object collected by the MRI device in the world coordinate system. The display screen can also display the position of the marking tool that can be detected by the optical positioning system in the world coordinate system, which is convenient for guiding the operator or manipulator to control the marking tool to operate the operating object.
[0089] A calibration method provided by an embodiment of the present invention is used to calibrate an MRI device through a marking tool, on which a liquid marker and a solid marker are arranged, comprising: controlling the MRI device to scan the liquid marker to obtain a first coordinate of the liquid marker in the coordinate system of the MRI device, and photographing the solid marker through an optical positioning system to obtain a second coordinate of the solid marker in the world coordinate system; obtaining a fourth coordinate of the liquid marker in the coordinate system of the marking tool and a fifth coordinate of the solid marker in the coordinate system of the marking tool; calculating an intermediate transformation matrix from the coordinate system of the marking tool to the coordinate system of the MRI device according to the first coordinate and the fourth coordinate; using the intermediate transformation matrix to convert the fifth coordinate into a third coordinate of the solid marker in the coordinate system of the MRI device, and then determining a target transformation matrix from the coordinate system of the MRI device to the world coordinate system according to the third coordinate and the second coordinate MB, and calibrating the MRI device based on the transformation matrix. According to the relationship between the fourth coordinate and the first coordinate, the intermediate transformation matrix from the coordinate system of the marking tool to the coordinate system of the MRI device can be analyzed. Then, based on the intermediate transformation matrix, the fifth coordinate of the solid marker in the coordinate system of the marking tool can be converted to the coordinate system of the MRI device to obtain the third coordinate. Finally, according to the third coordinate of the solid marker in the MRI device and the second coordinate of the solid marker in the world coordinate system, the target transformation matrix from the coordinate system of the MRI device to the world coordinate system can be obtained. Then, the calibration of the MRI device can be realized based on the target transformation matrix. The whole calibration process is simple, making it feasible to implement the optical positioning system in the MRI device.
[0090] In an optional embodiment of the present invention, Figure 6 As shown, the liquid marker 21 is inside the solid marker 22 , and the center of the liquid marker 21 coincides with the center of the solid marker 22 .
[0091] Determining a third coordinate of the solid marker in a coordinate system of the MRI device according to the position data of the liquid marker and the solid marker on the marking tool and / or the first coordinate includes:
[0092] The first coordinate is determined as a third coordinate of the solid marker in the coordinate system of the MRI device.
[0093] The centers of the liquid marker and the solid marker on the marking tool are set to coincide, and their coordinates on the marking tool are the same. Further, when the MRI device detects the first coordinate of the liquid marker, the first coordinate can also be regarded as the coordinate of the solid marker. Therefore, the first coordinate can be directly determined as the third coordinate of the solid marker in the coordinate system of the MRI device. Further, the target transformation matrix can be obtained according to the third coordinate of the solid marker in the coordinate system of the MRI device and the second coordinate of the solid marker in the world coordinate system. There is no need to solve the intermediate transformation matrix, and there is no need to transform the coordinates of the solid marker on the marking tool to the coordinate system of the MRI device, which saves computing resources and further simplifies the calibration process of the MRI device.
[0094] Embodiment 3
[0095] Figure 7 This is a flowchart of an instrument control method provided in Embodiment 3 of the present invention. This embodiment is applicable to the case where an instrument is controlled to operate an operation object. The method can be executed by an instrument control device. The instrument control device can be implemented in the form of hardware and / or software. The instrument control device can be configured in a surgical navigator for controlling an instrument, such as Figure 7 As shown, the device control method includes:
[0096] S701. Obtain the coordinates of the device in the world coordinate system.
[0097] The instrument in this embodiment may be a surgical instrument, such as a surgical probe, a scalpel, a manipulator of a medical robot, etc.
[0098] The world coordinate system is the coordinate system of the optical positioning system. The optical positioning system is mainly used to detect solids. The instrument itself is generally solid, or the instrument may be provided with a solid marker. Therefore, the optical positioning system can identify the fixed marker on the instrument and obtain the instrument coordinates. It should be noted that the solid marker on the instrument may also be a probe, a puncture needle, or other tool used to perform a surgical operation on the operating object. In this case, the coordinates of the probe, the puncture needle, etc. in the world coordinate system can be used as the instrument coordinates.
[0099] S702: Acquire the calibration coordinates of the operation object in the coordinate system of the MRI device from the magnetic resonance image containing the operation object.
[0100] The operation object is a soft or liquid tissue area, so the MRI device can detect the coordinates of the operation object, wherein the MRI device is calibrated by the calibration method of any of the above embodiments, that is, the MRI device can also convert the coordinates of the detected liquid substance or soft substance into the world coordinate system, that is, obtain the calibrated coordinates.
[0101] S703: Use the calibrated coordinates as the coordinates of the operation object in the world coordinate system.
[0102] The calibrated coordinates are coordinates in the world coordinate system. Therefore, the calibrated coordinates of the operation object in the coordinate system of the MRI device can be directly used as the coordinates of the operation object in the world coordinate system.
[0103] S704: Control the instrument according to the coordinates of the operation object and the coordinates of the instrument.
[0104] The coordinates of the operation object and the instrument coordinates are both coordinates in the world coordinate system. The positional relationship between the two can be displayed in the surgical navigator, so that the operator or the robot can move the instrument to the position of the operation object based on the positional relationship between the two to perform precise operations on the operation object.
[0105] This embodiment obtains the instrument coordinates of the instrument in the world coordinate system; obtains the calibration coordinates of the operating object in the coordinate system of the MRI device from the magnetic resonance image containing the operating object; uses the calibration coordinates as the operating object coordinates in the world coordinate system; and controls the instrument according to the operating object coordinates and the instrument coordinates. Wherein, the MRI device is calibrated by the calibration method of any of the above embodiments, and the MRI device can also convert the coordinates of the detected liquid substance or soft substance into the world coordinate system, that is, obtain the calibration coordinates, which are the coordinates of the operating object in the world coordinate system. Furthermore, the positional relationship between the operating object and the instrument in the world coordinate system can be displayed in the surgical navigator, so that the operator or the manipulator can move the instrument to the position of the operating object according to the positional relationship between the two, so as to perform precise operations on the operating object.
[0106] Embodiment 4
[0107] Figure 8 This is a schematic diagram of the structure of a calibration device provided in Embodiment 4 of the present invention. Figure 8 As shown, the calibration device comprises:
[0108] The marker coordinate acquisition module 801 is used to control the MRI device to scan the liquid marker to obtain the first coordinate of the liquid marker in the coordinate system of the MRI device, and to photograph the solid marker through the optical positioning system to obtain the second coordinate of the solid marker in the world coordinate system;
[0109] A third coordinate acquisition module 802 is used to determine the third coordinate of the solid marker in the coordinate system of the MRI device according to the first coordinate;
[0110] The MRI device calibration module 803 is used to calibrate the MRI device based on the second coordinate and the third coordinate.
[0111] The calibration device provided in the embodiment of the present invention can execute the calibration method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.
[0112] Embodiment 5
[0113] Fig. 9 This is a schematic diagram of the structure of an instrument control device provided in Embodiment 4 of the present invention. Fig. 9 As shown, the device control device includes:
[0114] The device coordinate acquisition module 901 is used to acquire the device coordinates of the device in the world coordinate system;
[0115] The MRI calibration coordinate acquisition module 902 is used to acquire the calibration coordinates of the operation object in the coordinate system of the MRI device from the magnetic resonance image containing the operation object;
[0116] A world coordinate acquisition module 903 is used to use the calibration coordinates as the operation object coordinates in the world coordinate system;
[0117] An instrument control module 904, used to control the instrument according to the coordinates of the operation object and the coordinates of the instrument;
[0118] The MRI device is calibrated by the calibration method of any of the above embodiments.
[0119] The instrument control device provided in the embodiment of the present invention can execute the instrument control method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.
[0120] Embodiment 6
[0121] Fig.10 A schematic diagram of an electronic device 70 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.
[0122] like Fig.10As shown, the electronic device 70 includes at least one processor 71, and a memory connected to the at least one processor 71, such as a read-only memory (ROM) 72, a random access memory (RAM) 73, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 71 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 72 or the computer program loaded from the storage unit 78 to the random access memory (RAM) 73. In the RAM 73, various programs and data required for the operation of the electronic device 70 can also be stored. The processor 71, the ROM 72, and the RAM 73 are connected to each other via a bus 74. An input / output (I / O) interface 75 is also connected to the bus 74.
[0123] A number of components in the electronic device 70 are connected to the I / O interface 75, including: an input unit 76, such as a keyboard, a mouse, etc.; an output unit 74, such as various types of displays, speakers, etc.; a storage unit 78, such as a disk, an optical disk, etc.; and a communication unit 79, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 79 allows the electronic device 70 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0124] The processor 71 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 71 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 71 performs the various methods and processes described above, such as calibration methods, and / or instrument control methods.
[0125] In some embodiments, the calibration method, and / or the instrument control method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 78. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 70 via the ROM 72 and / or the communication unit 79. When the computer program is loaded into the RAM 73 and executed by the processor 71, one or more steps of the calibration method, and / or the instrument control method described above may be performed. Alternatively, in other embodiments, the processor 71 may be configured to perform the calibration method, and / or the instrument control method in any other appropriate manner (e.g., by means of firmware).
[0126] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0127] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0128] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0129] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).
[0130] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.
[0131] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.
[0132] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.
[0133] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A calibration method, characterized in that: Used to calibrate MRI equipment by a marking tool, the marking tool is provided with a liquid marker and a solid marker, the marking tool is a rigid tool, the marking tool comprises a plurality of rigid rods, each rigid rod is provided with a liquid marker and a solid marker, the liquid marker is a soft substance or a soft container filled with liquid and sealed, the solid marker comprises a reflective marking ball, the liquid marker is provided outside the solid marker, or the liquid marker is inside the solid marker, and the center of the liquid marker coincides with the center of the solid marker; The calibration method comprises: Controlling the MRI device to scan the liquid marker to obtain a first coordinate of the liquid marker in the coordinate system of the MRI device, and photographing the solid marker through an optical positioning system to obtain a second coordinate of the solid marker in the world coordinate system; Determine a third coordinate of the solid marker in the coordinate system of the MRI device according to the position data of the liquid marker and the solid marker on the marking tool and / or the first coordinate; The MRI device is calibrated according to the second coordinate and the third coordinate.
2. The calibration method according to claim 1, characterized in that: The liquid marker is disposed outside the solid marker, and the determining of the third coordinate of the solid marker in the coordinate system of the MRI device according to the position data of the liquid marker and the solid marker on the marking tool and / or the first coordinate includes: Acquire a fourth coordinate of the liquid marker in the coordinate system of the marking tool, and acquire a fifth coordinate of the solid marker in the coordinate system of the marking tool; Calculate an intermediate transformation matrix from the coordinate system of the marking tool to the coordinate system of the MRI device according to the first coordinate and the fourth coordinate; The fifth coordinate is converted into a third coordinate of the solid marker in the coordinate system of the MRI device using the intermediate transformation matrix.
3. The calibration method according to claim 2, characterized in that: The calculating, according to the first coordinate and the fourth coordinate, an intermediate transformation matrix from the coordinate system of the marking tool to the coordinate system of the MRI device comprises: Construct the first objective function: ; in, is the first coordinate of the liquid marker in the coordinate system of the MRI device, is the fourth coordinate of the liquid marker in the coordinate system of the marking tool, is the rotation matrix, is the translation matrix, N is the number of liquid markers; Find the solution that minimizes the function value of the first objective function. and As an intermediate transformation matrix from the coordinate system of the marking tool to the coordinate system of the MRI device.
4. The calibration method according to claim 2, characterized in that: The step of using the intermediate transformation matrix to transform the fifth coordinate into a third coordinate of the solid marker in the coordinate system of the MRI device comprises: The third coordinate of the solid marker in the coordinate system of the MRI device is calculated by the following formula: ; in, is the third coordinate of the solid marker in the coordinate system of the MRI device, is the fifth coordinate of the solid marker in the coordinate system of the marking tool, and As an intermediate transformation matrix from the coordinate system of the marking tool to the coordinate system of the MRI device.
5. The calibration method according to claim 1, characterized in that: The liquid marker is inside the solid marker, and the center of the liquid marker coincides with the center of the solid marker, and determining the third coordinate of the solid marker in the coordinate system of the MRI device according to the position data of the liquid marker and the solid marker on the marking tool and / or the first coordinate includes: The first coordinate is determined as a third coordinate of the solid marker in a coordinate system of the MRI device.
6. The calibration method according to any one of claims 1 to 5, characterized in that: The calibrating the MRI device according to the second coordinate and the third coordinate includes: Determine a target transformation matrix from the coordinate system of the MRI device to the world coordinate system according to the second coordinate and the third coordinate; The MRI device is calibrated based on the target transformation matrix.
7. The calibration method according to claim 6, characterized in that: The step of determining a target transformation matrix from the coordinate system of the MRI device to the world coordinate system according to the second coordinate and the third coordinate includes: Construct the second objective function: ; in, is the second coordinate of the solid marker in the world coordinate system, is the third coordinate of the solid marker in the coordinate system of the MRI device, and is the target transformation matrix from the coordinate system of the MRI device to the world coordinate system; Find the solution that minimizes the function value of the second objective function. and As the target transformation matrix from the coordinate system of the MRI device to the world coordinate system.
8. A device control method, characterized in that: include: Obtain the coordinates of the device in the world coordinate system; Acquire the calibration coordinates of the operation object in the coordinate system of the MRI device from the magnetic resonance image containing the operation object; Using the calibration coordinates as the operation object coordinates in the world coordinate system; controlling the instrument according to the operation object coordinates and the instrument coordinates; Wherein, the MRI device is calibrated by the calibration method described in any one of claims 1-7.
9. A calibration device, characterized in that: include: A marker coordinate acquisition module, used for controlling the MRI device to scan the liquid marker to obtain the first coordinate of the liquid marker in the coordinate system of the MRI device, and photographing the solid marker through the optical positioning system to obtain the second coordinate of the solid marker in the world coordinate system, wherein the liquid marker and the solid marker are arranged on a marking tool, and the marking tool is a rigid tool, and the marking tool comprises a plurality of rigid rods, and each rigid rod is provided with a liquid marker and a solid marker, and the liquid marker is a soft substance or a soft container filled with liquid and sealed, and the solid marker comprises a reflective marker ball, and the liquid marker is arranged outside the solid marker, or the liquid marker is inside the solid marker, and the center of the liquid marker coincides with the center of the solid marker; A third coordinate acquisition module, used to determine a third coordinate of the solid marker in the coordinate system of the MRI device according to the first coordinate; An MRI device calibration module is used to calibrate the MRI device based on the second coordinate and the third coordinate.
10. An apparatus control device, characterized in that: include: An instrument coordinate acquisition module is used to acquire the instrument coordinates of the instrument in the world coordinate system; An MRI calibration coordinate acquisition module is used to acquire the calibration coordinates of the operation object in the coordinate system of the MRI device from the magnetic resonance image containing the operation object; A world coordinate acquisition module, used to use the calibration coordinates as the operation object coordinates in the world coordinate system; An instrument control module, used for controlling the instrument according to the operation object coordinates and the instrument coordinates; Wherein, the MRI device is calibrated by the calibration method described in any one of claims 1-7.
11. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the calibration method described in any one of claims 1-7, and or the instrument control method described in claim 8.
12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the calibration method described in any one of claims 1 to 7, and or the instrument control method described in claim 8 when executed.
