Tracking system, method and device based on tracking system and storage medium

Abstract

The invention relates to the technical field of tracking processing of medical robot systems, provides a tracking system, a method and device based on the tracking system and a storage medium, and aims to perform positioning tracking by virtue of a magnetic field and reduce the requirement on a placement position. The tracking system comprises a tracking device and multiple types of electromagnetic markers. The tracking device comprises a magnetic field generator for generating a magnetic field; the plurality of types of electromagnetic markers at least comprise a first type of electromagnetic marker arranged on a fixed part and a second type of electromagnetic marker arranged on a target part, and can comprise at least one of a third type of electromagnetic marker arranged on a medical instrument and a fourth type of electromagnetic marker arranged at the tail end of a mechanical arm; the fixing component is used for fixing a target part; the tail end of the mechanical arm is used for clamping a medical instrument; the electromagnetic marker is used for generating an electric signal under the action of a magnetic field; the electric signal is used for determining the coordinate of the electromagnetic marker in the tracking equipment coordinate system.

Description

Technical Field

[0001] The present application relates to the technical field of tracking and processing of medical robot systems, and in particular to a tracking system and a method, device, storage medium and computer program product based on the tracking system. Background Art

[0002] Tracking technology can be used in medical robotic systems. The location of a specific target can be tracked through the tracking device. Based on the tracked location, the movement of the robotic arm can be controlled to transport the medical device to the designated location.

[0003] Existing tracking devices are generally implemented based on optical methods. If the optical path used for tracking is blocked, it will be difficult to locate. This type of tracking device has high requirements for placement. Summary of the invention

[0004] Based on this, it is necessary to provide a tracking system and a method, device, storage medium and computer program product based on the tracking system to address the above technical problems.

[0005] The present application provides a tracking system, the tracking system comprising:

[0006] A tracking device comprising a magnetic field generator for generating a magnetic field;

[0007] The multiple types of electromagnetic markers include at least a first type of electromagnetic marker and a second type of electromagnetic marker, and the multiple types of electromagnetic markers also include at least one of a third type of electromagnetic marker and a fourth type of electromagnetic marker; wherein the first type of electromagnetic marker is used to be arranged on a fixed component, the second type of electromagnetic marker is used to be arranged on a target part, the third type of electromagnetic marker is used to be arranged on a medical device, and the fourth type of electromagnetic marker is used to be arranged on the end of a robotic arm; the fixed component is used to fix the target part; and the end of the robotic arm is used to clamp the medical device;

[0008] The electromagnetic marker is used to generate an electrical signal under the action of the magnetic field; the electrical signal is used to determine the coordinates of the electromagnetic marker in the tracking device coordinate system.

[0009] In one embodiment, the tracking device further comprises a light emitting component and a light receiving component;

[0010] The light emitting component is used to emit structured light to the fixed component;

[0011] The light receiving component is used to receive the reflected light reflected by the fixed component based on the structured light; the reflected light is used to form a point cloud of the fixed component.

[0012] The present application provides a medical device navigation method, which is applied to the tracking system described in the above embodiment, and the method includes:

[0013] Acquire a first coordinate set; the first coordinate set includes the coordinates of the first type of electromagnetic marker in the tracking device coordinate system and the fixed component coordinate system, the coordinates of the second type of electromagnetic marker in the tracking device coordinate system and the image coordinate system, and the coordinates of the third type of electromagnetic marker in the tracking device coordinate system and the medical device coordinate system; the image coordinate system is a coordinate system that describes the image of the target part;

[0014] Based on the first coordinate set, obtaining a conversion relationship between the image coordinate system and the medical device coordinate system;

[0015] The medical device is navigated based on the conversion relationship between the image coordinate system and the medical device coordinate system.

[0016] In one embodiment, obtaining the conversion relationship between the image coordinate system and the medical device coordinate system based on the first coordinate set includes:

[0017] Based on the coordinates of the first type of electromagnetic marker in the tracking device coordinate system and the fixed component coordinate system, converting the coordinates of the second type of electromagnetic marker and the third type of electromagnetic marker in the tracking device coordinate system to the fixed component coordinate system;

[0018] Based on the coordinates of the second type of electromagnetic marker in the image coordinate system and the fixed component coordinate system, and the coordinates of the third type of electromagnetic marker in the medical device coordinate system and the fixed component coordinate system, the conversion relationship between the image coordinate system and the medical device coordinate system is obtained.

[0019] In one embodiment, obtaining the conversion relationship between the image coordinate system and the medical device coordinate system based on the coordinates of the second type of electromagnetic marker in the image coordinate system and the fixed component coordinate system, and the coordinates of the third type of electromagnetic marker in the medical device coordinate system and the fixed component coordinate system, includes:

[0020] Based on the coordinates of the second type of electromagnetic marker in the image coordinate system and the fixed component coordinate system, obtaining a conversion relationship between the fixed component coordinate system and the image coordinate system;

[0021] Based on the coordinates of the third type electromagnetic marker in the medical device coordinate system and the fixed component coordinate system, obtaining a conversion relationship between the fixed component coordinate system and the medical device coordinate system;

[0022] Based on the conversion relationship between the fixed component coordinate system and the image coordinate system, and the conversion relationship between the fixed component coordinate system and the medical device coordinate system, the conversion relationship between the image coordinate system and the medical device coordinate system is obtained.

[0023] The present application provides a medical device navigation device, which is applied to the tracking system described in the above embodiment, and the device includes:

[0024] A coordinate acquisition module, used for acquiring a first coordinate set; the first coordinate set includes the coordinates of the first type of electromagnetic marker in the tracking device coordinate system and the fixed component coordinate system, the coordinates of the second type of electromagnetic marker in the tracking device coordinate system and the image coordinate system, and the coordinates of the third type of electromagnetic marker in the tracking device coordinate system and the medical device coordinate system; the image coordinate system is a coordinate system describing the image of the target part;

[0025] A conversion relationship acquisition module, used for obtaining a conversion relationship between the image coordinate system and the medical device coordinate system based on the first coordinate set;

[0026] A navigation module is used to navigate the medical device based on the conversion relationship between the image coordinate system and the medical device coordinate system.

[0027] The present application provides a method for controlling the movement of a robot end, which is applied to the tracking system described in the above embodiment, and the method includes:

[0028] Acquire a second coordinate set; the second coordinate set includes the coordinates of the first type of electromagnetic marker in the tracking device coordinate system and the fixed component coordinate system, the coordinates of the second type of electromagnetic marker in the tracking device coordinate system and the image coordinate system, and the coordinates of the fourth type of electromagnetic marker in the tracking device coordinate system and the robot arm end coordinate system;

[0029] Based on the second coordinate set and the conversion relationship between the robot end coordinate system and the robot base coordinate system, a conversion relationship between the robot base coordinate system and the image coordinate system is obtained;

[0030] The selected point on the image of the target part is obtained, and the movement of the end of the robotic arm is controlled according to the conversion relationship between the robotic arm base coordinate system and the image coordinate system.

[0031] In one embodiment, obtaining the conversion relationship between the robot base coordinate system and the image coordinate system based on the second coordinate set and the conversion relationship between the robot end coordinate system and the robot base coordinate system includes:

[0032] Based on the coordinates of the fourth type of electromagnetic marker in the tracking device coordinate system and the robot arm end coordinate system, the coordinates of the first type of electromagnetic marker in the tracking device coordinate system and the fixed component coordinate system, and the conversion relationship between the robot arm end coordinate system and the robot arm base coordinate system, the conversion relationship between the fixed component coordinate system and the robot arm base coordinate system is obtained;

[0033] Based on the coordinates of the second type of electromagnetic marker in the tracking device coordinate system and the image coordinate system, and the coordinates of the first type of electromagnetic marker in the tracking device coordinate system and the fixed component coordinate system, a conversion relationship between the fixed component coordinate system and the image coordinate system is obtained;

[0034] Based on the conversion relationship between the fixed component coordinate system and the robot base coordinate system, and the conversion relationship between the fixed component coordinate system and the image coordinate system, the conversion relationship between the robot base coordinate system and the image coordinate system is obtained.

[0035] In one embodiment, obtaining the conversion relationship between the fixed component coordinate system and the image coordinate system based on the coordinates of the second type of electromagnetic marker in the tracking device coordinate system and the image coordinate system, and the coordinates of the first type of electromagnetic marker in the tracking device coordinate system and the fixed component coordinate system, comprises:

[0036] Based on the coordinates of the first type of electromagnetic marker in the tracking device coordinate system and the fixed component coordinate system, obtaining a conversion relationship between the tracking device coordinate system and the fixed component coordinate system;

[0037] Based on the conversion relationship between the tracking device coordinate system and the fixed component coordinate system, and the coordinates of the second type of electromagnetic marker in the tracking device coordinate system, obtain the coordinates of the second type of electromagnetic marker in the fixed component coordinate system;

[0038] Based on the coordinates of the second-type electromagnetic marker in the fixed component coordinate system and the image coordinate system, a conversion relationship between the fixed component coordinate system and the image coordinate system is obtained.

[0039] In one embodiment, the conversion relationship between the fixed component coordinate system and the robotic arm base coordinate system is obtained based on the coordinates of the fourth type of electromagnetic marker in the tracking device coordinate system and the robotic arm end coordinate system, the coordinates of the first type of electromagnetic marker in the tracking device coordinate system and the fixed component coordinate system, and the conversion relationship between the robotic arm end coordinate system and the robotic arm base coordinate system, including:

[0040] Based on the coordinates of the first type of electromagnetic marker in the tracking device coordinate system and the fixed component coordinate system, obtaining a conversion relationship between the tracking device coordinate system and the fixed component coordinate system;

[0041] Based on the conversion relationship between the tracking device coordinate system and the fixed component coordinate system, and the coordinates of the fourth type of electromagnetic marker in the tracking device coordinate system, the coordinates of the fourth type of electromagnetic marker in the fixed component coordinate system are obtained;

[0042] Based on the coordinates of the fourth type electromagnetic marker in the fixed component coordinate system and the robot arm end coordinate system, a conversion relationship between the robot arm end coordinate system and the fixed component coordinate system is obtained;

[0043] Based on the conversion relationship between the robot end coordinate system and the fixed component coordinate system, and the conversion relationship between the robot end coordinate system and the robot base coordinate system, the conversion relationship between the fixed component coordinate system and the robot base coordinate system is obtained.

[0044] The present application provides a device for controlling the movement of a robot end, which is applied to the tracking system described in the above embodiment, and the device includes:

[0045] A coordinate acquisition module, used to acquire a second coordinate set; the second coordinate set includes the coordinates of the first type of electromagnetic marker in the tracking device coordinate system and the fixed component coordinate system, the coordinates of the second type of electromagnetic marker in the tracking device coordinate system and the image coordinate system, and the coordinates of the fourth type of electromagnetic marker in the tracking device coordinate system and the robot end coordinate system;

[0046] A conversion relationship acquisition module, used for obtaining a conversion relationship between the robot arm base coordinate system and the image coordinate system based on the second coordinate set and the conversion relationship between the robot arm end coordinate system and the robot arm base coordinate system;

[0047] The mobile control module is used to obtain the selected point on the image of the target part and control the movement of the end of the robotic arm according to the conversion relationship between the robotic arm base coordinate system and the image coordinate system.

[0048] The present application provides a path planning method, which is applied to the tracking system described in the above embodiment, and the method includes:

[0049] Acquiring a point cloud of the fixed component by means of the tracking device;

[0050] Based on the coordinates of the second type of electromagnetic marker in the tracking device coordinate system and the image coordinate system, obtaining a conversion relationship between the tracking device coordinate system and the image coordinate system;

[0051] Based on the conversion relationship between the tracking device coordinate system and the image coordinate system, the point cloud is converted from the tracking device coordinate system to the image coordinate system to perform path planning for the image of the target part.

[0052] In one embodiment, the method further comprises:

[0053] Based on the coordinates of the fourth type of electromagnetic marker in the tracking device coordinate system and the robot arm end coordinate system, and the conversion relationship between the robot arm end coordinate system and the robot arm base coordinate system, a conversion relationship between the tracking device coordinate system and the robot arm base coordinate system is obtained;

[0054] Based on the conversion relationship between the tracking device coordinate system and the robot arm base coordinate system, the point cloud is converted from the tracking device coordinate system to the robot arm base coordinate system to perform path planning for the robot arm.

[0055] The present application provides a path planning device, which is applied to the tracking system described in the above embodiment, and the device includes:

[0056] A point cloud acquisition module, used to acquire a point cloud of the fixed component through the tracking device;

[0057] A conversion relationship acquisition module, used for obtaining a conversion relationship between the tracking device coordinate system and the image coordinate system based on the coordinates of the second type of electromagnetic marker in the tracking device coordinate system and the image coordinate system;

[0058] The obstacle avoidance processing module is used to convert the point cloud from the tracking device coordinate system to the image coordinate system based on the conversion relationship between the tracking device coordinate system and the image coordinate system, so as to plan the path of the image of the target part.

[0059] The present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to execute the above method.

[0060] The present application provides a computer program product, on which a computer program is stored, and the computer program is executed by a processor to perform the above method.

[0061] The tracking system provided by the present application uses a magnetic field to perform positioning and tracking, reducing the requirements for placement; the tracking system includes a magnetic field generator and multiple types of electromagnetic markers; the magnetic field generator is used to generate a magnetic field; the electromagnetic marker is used to generate an electrical signal under the action of the magnetic field to determine the coordinates of the electromagnetic marker in the coordinate system of the tracking device. The tracking system can also have multiple functions such as navigation, positioning and obstacle avoidance. When the tracking system has obstacle avoidance and navigation functions, the multiple types of electromagnetic markers include a first type of electromagnetic marker for being set on a fixed component, a second type of electromagnetic marker for being set on a target part, and a third type of electromagnetic marker for being set on a medical device; when the tracking system has obstacle avoidance and positioning functions, the multiple types of electromagnetic markers include a first type of electromagnetic marker, a second type of electromagnetic marker, and a fourth type of electromagnetic marker for being set at the end of a robotic arm; thus, the multiple types of electromagnetic markers include at least a first type of electromagnetic marker and a second type of electromagnetic marker, and may include at least one of a third type of electromagnetic marker and a fourth type of electromagnetic marker. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 A schematic diagram of a scenario of a surgical robot system in one embodiment;

[0063] Figure 2 A schematic diagram of setting up a first type of electromagnetic marker on the head in one embodiment;

[0064] Figure 3 A schematic diagram of providing a third type of electromagnetic marker on a medical device in one embodiment;

[0065] Figure 4 A schematic diagram of providing a fourth type of electromagnetic marker in a robotic arm module in one embodiment;

[0066] Figure 5 is a front view of a tracking device in one embodiment;

[0067] Figure 6 is a schematic diagram of the back side of a tracking device in one embodiment;

[0068] Figure 7 A schematic diagram of the internal structure of a tracking device in one embodiment;

[0069] Figure 8 A schematic diagram of the internal structure of an electromagnetic marker in one embodiment;

[0070] Fig. 9 is a flowchart of a medical device navigation method in one embodiment;

[0071] Fig.10 A schematic diagram of a flow chart of a method for controlling movement of a robot end in one embodiment;

[0072] Fig.11 A schematic diagram of a process flow of a path planning method in one embodiment;

[0073] Fig.12 is a schematic diagram of point clouds at multiple angles in one embodiment;

[0074] Fig.13 is a schematic diagram of fusion of multi-angle point clouds in one embodiment;

[0075] Fig.14 is a structural block diagram of a medical device navigation device in one embodiment;

[0076] Fig.15 A structural block diagram of a device for controlling movement of a robot end in one embodiment;

[0077] Fig.16 is a structural block diagram of a path planning device in one embodiment;

[0078] Fig.17 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0079] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0080] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0081] The tracking system provided in the present application includes a tracking device and an electromagnetic marker. The tracking device includes a magnetic field generator, which can generate a magnetic field. The electromagnetic marker can generate an electrical signal under the action of the magnetic field generated by the magnetic field generator, and the electrical signal can be used to determine the coordinates of the electromagnetic marker in the coordinate system of the tracking device.

[0082] According to the location where the electromagnetic markers are set, the electromagnetic markers in the tracking system can be divided into several categories. For example, electromagnetic markers set on fixed parts are called first-category electromagnetic markers, electromagnetic markers set on target parts are called second-category electromagnetic markers, electromagnetic markers set on medical devices are called third-category electromagnetic markers, and electromagnetic markers set at the end of the robotic arm are called fourth-category electromagnetic markers.

[0083] The target part can be the part where the medical device acts. In a stereotactic surgical robot system for neurosurgery, the head can be used as the target part. The fixing component is used to fix the target part. When the target part is the head, the fixing component can be called a head frame. The end of the robotic arm is used to clamp the medical device.

[0084] Take the stereotactic surgical robot system for neurosurgery as an example, refer to Figure 1 In the surgical robot system, the surgical cart 110 can be used to place the robotic arm 120, the end of the robotic arm 120 (hereinafter referred to as the robotic arm end) can clamp medical equipment, the patient's head can be fixed with a head frame 130, the head frame 130 can be supported by a support arm 1101 of the surgical cart 110, and the display 140 can display images. The tracking system provided by the present application is applied to the surgical robot system, and accordingly, electromagnetic markers can be set in places such as the head frame 130, the head, the medical equipment and the end of the robotic arm. The number of electromagnetic markers set in the same place can be no less than 3. Figure 2 The first type of electromagnetic marker 1601 is shown disposed on the head frame 130. Figure 3 The third type of electromagnetic marker 1603 is shown disposed on the medical device 170. Figure 4 A fourth type of electromagnetic marker 1604 is shown disposed at the end of the robotic arm 1201 .

[0085] Under the action of the magnetic field generated by the magnetic field generator of the tracking device 150, the electromagnetic marker can generate an electrical signal, based on which the coordinates of the corresponding electromagnetic marker in the coordinate system of the tracking device can be determined.

[0086] When the tracking system provided in the present application has navigation and obstacle avoidance functions, electromagnetic markers are arranged on fixed parts, target parts and medical devices, that is, the multiple types of electromagnetic markers include first-type electromagnetic markers, second-type electromagnetic markers and third-type electromagnetic markers.

[0087] When the tracking system provided in the present application has positioning and obstacle avoidance functions, electromagnetic markers are arranged at fixed parts, target parts and the end of the robotic arm, that is, at this time the multiple types of electromagnetic markers include first type electromagnetic markers, second type electromagnetic markers and fourth type electromagnetic markers.

[0088] Therefore, according to the different functional combinations of the tracking system, the categories of electromagnetic markers can have corresponding combinations, and the category combination of electromagnetic markers can be: multiple categories of electromagnetic markers include at least first category electromagnetic markers and second category electromagnetic markers, and can include at least one of third category electromagnetic markers and fourth category electromagnetic markers.

[0089] The tracking system provided by the present application uses a magnetic field to perform positioning and tracking, reducing the requirements for placement; the tracking system includes a magnetic field generator and multiple types of electromagnetic markers; the magnetic field generator is used to generate a magnetic field; the electromagnetic marker is used to generate an electrical signal under the action of the magnetic field to determine the coordinates of the electromagnetic marker in the coordinate system of the tracking device. The tracking system can also have multiple functions such as navigation, positioning and obstacle avoidance. When the tracking system has obstacle avoidance and navigation functions, the multiple types of electromagnetic markers include a first type of electromagnetic marker for being set on a fixed component, a second type of electromagnetic marker for being set on a target part, and a third type of electromagnetic marker for being set on a medical device; when the tracking system has obstacle avoidance and positioning functions, the multiple types of electromagnetic markers include a first type of electromagnetic marker, a second type of electromagnetic marker, and a fourth type of electromagnetic marker for being set at the end of a robotic arm; thus, the multiple types of electromagnetic markers include at least a first type of electromagnetic marker and a second type of electromagnetic marker, and may include at least one of a third type of electromagnetic marker and a fourth type of electromagnetic marker.

[0090] In one embodiment, the tracking device 150 further includes Figure 5 The light emitting component 1501 and the light receiving component 1502 are shown. The light emitting component is used to emit structured light to the fixed component. The light receiving component is used to receive reflected light reflected by the fixed component based on the structured light; the reflected light is used to form a point cloud of the fixed component. When the fixed component is a head frame, the point cloud formed is the point cloud of the head frame.

[0091] Furthermore, the front structure of the tracking device can be as follows Figure 5 As shown. A laser indicator light 1503 can be set in the middle of the front of the tracking device, which can be turned on or off by the user to indicate the field of view of the tracking device. A light emitting component 1501 and a light receiving component 1502 can be set on the front of the tracking device. The number of light emitting components 1501 can be 1, and the number of light receiving components 1502 can be 2. The light emitting component 1501 can emit structured light to fixed components such as the head frame, and the light receiving component 1502 is used to receive reflected light reflected from fixed components such as the head frame; according to the principle of structured light imaging, a point cloud of fixed components such as the head frame can be formed according to the reflection point. In addition, an RGB camera 1504 can be set on the front of the tracking device to obtain color information of objects in the shooting picture. According to the color information, fixed components such as the head frame can be distinguished from other objects such as patients. In combination with the light emitting component and the light receiving component, the point cloud of fixed components such as the head frame can be obtained. In addition, armrests 1505 can be set on the left and right sides of the tracking device to facilitate users to take it.

[0092] Furthermore, the back structure of the tracking device 150 can be as follows: Figure 6As shown, a power button 1506 may be provided at the bottom of the back of the tracking device to turn on or off the tracking device. The tracking device may also be provided with a power on / off status indicator 1507 to display the power on / off status of the tracking device. The tracking device may also be provided with a power interface 1508 to connect a power source to power the tracking device. The tracking device may also be provided with a battery slot 1509 to install a battery for power supply. The tracking device may also be provided with a communication interface 1510 (e.g., Ethernet, USB, etc.) to communicate with a workstation host.

[0093] The tracking device may be provided with a wireless communication module to support wireless communication (wireless communication methods such as Bluetooth, WiFi, etc.), and the tracking device may also be provided with a communication status indicator 1511 to display the current communication status. A laser light button 1512 may also be provided on the top of the back of the tracking device to control the on or off of the laser indicator light.

[0094] When the tracking device is battery-powered and wirelessly communicated at the same time, the tracking device does not need to be equipped with cables, and the user can easily pick it up and use it. In the operating room environment, there are often higher requirements for sterile protection. At this time, the tracking device can be placed in a sterile cover (usually a sterile transparent plastic film shipped from the factory), which is completely sealed, meeting the sterility requirements and not affecting its use.

[0095] Further, the internal schematic diagram of the tracking device is as follows Figure 7 As shown, the tracking device may include a control module and a camera module. The control module may be connected to the following modules: a magnetic field generator, a wireless communication module, an external communication interface, a power button, an external power input module, a battery power input module, a laser light button, a laser indicator light, a power-on status indicator light, and a communication indicator light. The camera module mainly includes: a light transmitting component, a light receiving component, and an RGB camera.

[0096] The tracking device generates a magnetic field through a magnetic field generator, and the electromagnetic marker generates an electrical signal under the action of the magnetic field. The electrical signal can be received by the tracking device, so that the tracking device can obtain the coordinates of the electromagnetic marker in the coordinate system of the tracking device.

[0097] Electromagnetic markers can be placed in the corresponding places by gluing or mechanical connection. The internal structure of the electromagnetic marker is as follows Figure 8 As shown, it can include an electromagnetic sensor, a battery and a control board. The electromagnetic sensor can generate a corresponding electrical signal under the action of a magnetic field. The power of the electromagnetic marker is relatively small, generally in the milliwatt level, and the volume requirement is also relatively small, so the battery used for power supply can be a button battery. The control board is provided with a wireless communication module, which can be used to transmit electrical signals.

[0098] Based on the tracking system introduced above, the present application provides a medical device navigation method. At this time, the electromagnetic markers of the tracking system are set on the fixed parts, the target parts and the medical devices, that is, the multiple types of electromagnetic markers of the tracking system include the first type of electromagnetic markers, the second type of electromagnetic markers and the third type of electromagnetic markers. The medical device navigation method may include Fig. 9 The steps shown are:

[0099] Step S901, obtaining a first coordinate set.

[0100] The first coordinate set includes the coordinates of the first type of electromagnetic marker in the tracking device coordinate system and the fixed component coordinate system, the coordinates of the second type of electromagnetic marker in the tracking device coordinate system and the image coordinate system, and the coordinates of the third type of electromagnetic marker in the tracking device coordinate system and the medical device coordinate system; wherein the image coordinate system is a coordinate system that describes the image of the target part, such as a CT image coordinate system.

[0101] Step S902: Based on the first coordinate set, a conversion relationship between the image coordinate system and the medical device coordinate system is obtained.

[0102] Step S903: Navigate the medical device based on the conversion relationship between the image coordinate system and the medical device coordinate system.

[0103] Specifically, based on the conversion relationship between the image coordinate system and the medical device coordinate system, the position of the medical device can be displayed in the CT image to achieve navigation of the medical device.

[0104] This embodiment uses the tracking system introduced above to perform navigation, which reduces the requirements for the placement of components in the tracking system and improves navigation efficiency.

[0105] Furthermore, based on the first coordinate set, a conversion relationship between the image coordinate system and the medical device coordinate system is obtained, which may specifically include: based on the coordinates of the first type of electromagnetic marker in the tracking device coordinate system and the fixed component coordinate system, converting the coordinates of the second type of electromagnetic marker and the third type of electromagnetic marker in the tracking device coordinate system to the fixed component coordinate system; based on the coordinates of the second type of electromagnetic marker in the image coordinate system and the fixed component coordinate system, and the coordinates of the third type of electromagnetic marker in the medical device coordinate system and the fixed component coordinate system, obtaining the conversion relationship between the image coordinate system and the medical device coordinate system.

[0106] Furthermore, based on the coordinates of the second type of electromagnetic marker in the image coordinate system and the fixed component coordinate system, and the coordinates of the third type of electromagnetic marker in the medical device coordinate system and the fixed component coordinate system, a conversion relationship between the image coordinate system and the medical device coordinate system is obtained, which may specifically include: based on the coordinates of the second type of electromagnetic marker in the image coordinate system and the fixed component coordinate system, a conversion relationship between the fixed component coordinate system and the image coordinate system is obtained; based on the coordinates of the third type of electromagnetic marker in the medical device coordinate system and the fixed component coordinate system, a conversion relationship between the fixed component coordinate system and the medical device coordinate system is obtained; based on the conversion relationship between the fixed component coordinate system and the image coordinate system, and the conversion relationship between the fixed component coordinate system and the medical device coordinate system, a conversion relationship between the image coordinate system and the medical device coordinate system is obtained.

[0107] Take the target part as the head and the fixed part as the head frame as an example. In addition, in this example, the tracking device can be called an electromagnetic optical positioning and tracking device (Electromagnetic Optical Positioning and Tracking Equipment, which can be abbreviated as EOPT):

[0108] The coordinates of the second type of electromagnetic marker on the head in the EOPT coordinate system are: The coordinates of the first type of electromagnetic markers set on the head frame in the EOPT coordinate system are: (hf stands for head frame, the abbreviation of head frame), the coordinates of the third type electromagnetic marker set in the medical device in the EOPT coordinate system are The coordinates in the EOPT coordinate system are known quantities. In this application, uppercase bold letters (such as T) represent matrices, and lowercase bold letters (such as p and q) represent vectors.

[0109] The homogeneous transformation matrix of the EOPT coordinate system relative to the head frame coordinate system is: have:

[0110]

[0111] (remember Represents the homogeneous transformation matrix of the B coordinate system relative to the A coordinate system, where )

[0112] It represents the coordinates of the first type of electromagnetic marker in the head frame coordinate system. This coordinate is related to the mechanical dimensions of the first type of electromagnetic marker and is a known quantity. is also a known quantity, so combined with It can be calculated

[0113] Since the doctor may move the EOPT during navigation, the coordinates of the EOPT coordinate system can be Transform to the head frame coordinate system:

[0114]

[0115]

[0116] Similarly:

[0117]

[0118]

[0119] It can be calculated and have:

[0120]

[0121] Calculate After that, you can put the medical device on any point Convert to the corresponding point in the CT image coordinate system

[0122]

[0123] In this way, the position of the medical device can be displayed in real time in the CT image, realizing the navigation function of the medical device.

[0124] In addition, in the navigation function, It changes with the movement of the medical device, so EOPT can always participate in the calculation. During the navigation process, the position of the medical device may change. When it changes, it can be recalculated.

[0125] If the position of the medical device does not change, but the position of the EOPT changes, There will be no change and no need to recalculate. During navigation, due to the magnetic field interference of EOPT, it may be lost. At this time, you may need to move EOPT and recalculate For example, if there is a CT machine at the operating site, the EOPT may be affected if it is too close to the CT machine. In this case, the position of the EOPT can be moved.

[0126] Based on the tracking system introduced above, the present application provides a method for controlling the movement of the end of a robotic arm. The method belongs to the positioning function. At this time, the electromagnetic markers of the tracking system are set at the fixed part, the target part and the end of the robotic arm, that is, the multiple types of electromagnetic markers of the tracking system include the first type of electromagnetic markers, the second type of electromagnetic markers and the fourth type of electromagnetic markers. The method for controlling the movement of the end of a robotic arm may include Fig.10 The steps shown are:

[0127] Step S1001, obtaining a second coordinate set.

[0128] The second coordinate set includes the coordinates of the first type of electromagnetic marker in the tracking device coordinate system and the fixed component coordinate system, the coordinates of the second type of electromagnetic marker in the tracking device coordinate system and the image coordinate system, and the coordinates of the fourth type of electromagnetic marker in the tracking device coordinate system and the robotic arm end coordinate system.

[0129] Step S1002, based on the second coordinate set and the conversion relationship between the robot end coordinate system and the robot base coordinate system, obtain the conversion relationship between the robot base coordinate system and the image coordinate system;

[0130] Step S1003, obtaining the selected point on the image of the target part, and controlling the movement of the end of the robot arm according to the conversion relationship between the robot arm base coordinate system and the image coordinate system.

[0131] Specifically, after obtaining the conversion relationship between the robot arm base coordinate system and the image coordinate system, a point is selected on the CT image, and the end of the robot arm can be controlled to move to the point to realize the positioning function.

[0132] This embodiment uses the tracking system introduced above to perform navigation, which reduces the requirements for the placement of components in the tracking system and improves positioning efficiency.

[0133] In one embodiment, based on the second coordinate set and the conversion relationship between the robot arm end coordinate system and the robot arm base coordinate system, the conversion relationship between the robot arm base coordinate system and the image coordinate system is obtained, which may specifically include: based on the coordinates of the fourth type of electromagnetic marker in the tracking device coordinate system and the robot arm end coordinate system, the coordinates of the first type of electromagnetic marker in the tracking device coordinate system and the fixed component coordinate system, and the conversion relationship between the robot arm end coordinate system and the robot arm base coordinate system, the conversion relationship between the fixed component coordinate system and the robot arm base coordinate system is obtained; based on the coordinates of the second type of electromagnetic marker in the tracking device coordinate system and the image coordinate system, and the coordinates of the first type of electromagnetic marker in the tracking device coordinate system and the fixed component coordinate system, the conversion relationship between the fixed component coordinate system and the image coordinate system is obtained; based on the conversion relationship between the fixed component coordinate system and the robot arm base coordinate system, and the conversion relationship between the fixed component coordinate system and the image coordinate system, the conversion relationship between the robot arm base coordinate system and the image coordinate system is obtained.

[0134] Furthermore, based on the coordinates of the second type of electromagnetic marker in the tracking device coordinate system and the image coordinate system, and the coordinates of the first type of electromagnetic marker in the tracking device coordinate system and the fixed component coordinate system, a conversion relationship between the fixed component coordinate system and the image coordinate system is obtained, including: based on the coordinates of the first type of electromagnetic marker in the tracking device coordinate system and the fixed component coordinate system, the conversion relationship between the tracking device coordinate system and the fixed component coordinate system is obtained; based on the conversion relationship between the tracking device coordinate system and the fixed component coordinate system, and the coordinates of the second type of electromagnetic marker in the tracking device coordinate system, the coordinates of the second type of electromagnetic marker in the fixed component coordinate system are obtained; based on the coordinates of the second type of electromagnetic marker in the fixed component coordinate system and the image coordinate system, the conversion relationship between the fixed component coordinate system and the image coordinate system is obtained.

[0135] Furthermore, based on the coordinates of the fourth type of electromagnetic marker in the tracking device coordinate system and the robot arm end coordinate system, the coordinates of the first type of electromagnetic marker in the tracking device coordinate system and the fixed component coordinate system, and the conversion relationship between the robot arm end coordinate system and the robot arm base coordinate system, the conversion relationship between the fixed component coordinate system and the robot arm base coordinate system is obtained, including: based on the coordinates of the first type of electromagnetic marker in the tracking device coordinate system and the fixed component coordinate system, the conversion relationship between the tracking device coordinate system and the fixed component coordinate system is obtained; based on the conversion relationship between the tracking device coordinate system and the fixed component coordinate system, and the coordinates of the fourth type of electromagnetic marker in the tracking device coordinate system, the coordinates of the fourth type of electromagnetic marker in the fixed component coordinate system are obtained; based on the coordinates of the fourth type of electromagnetic marker in the fixed component coordinate system and the robot arm end coordinate system, the conversion relationship between the robot arm end coordinate system and the fixed component coordinate system is obtained; based on the conversion relationship between the robot arm end coordinate system and the fixed component coordinate system, and the conversion relationship between the robot arm end coordinate system and the robot arm base coordinate system, the conversion relationship between the fixed component coordinate system and the robot arm base coordinate system is obtained.

[0136] The target part is the head, and the fixed part is the head frame. In this example, the tracking device can be EOPT:

[0137] The coordinates of the fourth type of electromagnetic marker at the end of the robot arm in the coordinate system of the robot arm end are: (TCP is the abbreviation of tool centerpoint, which means the end tool center point in Chinese). The coordinates of the fourth type of electromagnetic marker in the robot arm base coordinate system are as follows: The coordinates of the fourth type of electromagnetic marker in the EOPT coordinate system are:

[0138] The following formula can be obtained:

[0139]

[0140]

[0141] Substituting the coordinates of the points, we can calculate

[0142]

[0143] It is the second transformation matrix of the robot end coordinate system relative to the robot base coordinate system. It belongs to the conversion relationship between the robot end coordinate system and the robot base coordinate system. This conversion relationship is related to the mechanical size of the robot and the shutdown angle of each axis. When the robot posture is determined, is a known quantity.

[0144] is the second transformation matrix from the head frame coordinate system to the robot arm base coordinate system, which belongs to the conversion relationship between the head frame coordinate system and the robot arm base coordinate system; since the head frame and the robot arm base are fixed, It is also fixed and has nothing to do with the EOPT position.

[0145] Depend on It can be calculated Since the base and head of the robot arm are fixed, It is fixed and has nothing to do with the posture of the robot arm or the position of the EOPT.

[0146] Calculate Later, with the help of You can put any point on the CT image Convert to the robot base coordinate system Thus, the posture of the robot arm can be controlled so that the end of the robot arm reaches this point. In this way, by selecting a point in the CT image, the end of the robot arm can reach this point, thus realizing the positioning function.

[0147] because is fixed, so we can calculate After that, you don't need to calculate again.

[0148] In addition, if the surgery requires higher positioning accuracy, the position of EOPT can be moved repeatedly, and then the calculation is performed each time. and In theory, and It should be constant, but EOPT itself has point selection errors. and There will be slight differences. Through some algorithm, such as taking the average value to calculate and based on and calculate and improve accuracy.

[0149] Based on the tracking system introduced above, this application provides a path planning method, which belongs to the obstacle avoidance function. At this time, the electromagnetic marker of the tracking system is set at the target location, that is, the multiple types of electromagnetic markers of the tracking system include the second type of electromagnetic markers. The path planning method may include Fig.11 The steps shown are:

[0150] Step S1101, obtaining a point cloud of a fixed component by tracking a device;

[0151] Step S1102: Based on the coordinates of the second type of electromagnetic marker in the tracking device coordinate system and the imaging coordinate system, obtain the conversion relationship between the tracking device coordinate system and the imaging coordinate system;

[0152] Step S1103: Based on the conversion relationship between the tracking device coordinate system and the imaging coordinate system, convert the point cloud from the tracking device coordinate system to the imaging coordinate system to perform path planning for the image of the target part.

[0153] Taking the fixed component as the head frame and the tracking device as the EPOT as an example for introduction: Using the light emission component and the light reception component of the tracking device, obtain the point cloud of the captured image, and distinguish the head frame from the background through the RGB camera, then the point cloud of the head frame can be extracted from the point cloud of the captured image.

[0154] Denote the point cloud of the head frame as in the EOPT coordinate system as Denote it as in the CT imaging coordinate system as (PC is the abbreviation of point cloud, and the Chinese of point cloud is point cloud).

[0155]

[0156] According to the above formula, the point cloud of the head frame can be converted from the EOPT coordinate system to the CT imaging coordinate system. Thus, when planning the surgical path, the head frame model can be visually seen, so that the planned surgical path can avoid the head frame.

[0157] Furthermore, when obtaining the point cloud of the head frame, the point cloud obtained from only one angle may not be comprehensive and the point cloud of the back surface cannot be obtained. Therefore, the point cloud at different angles can be obtained by taking pictures from multiple angles. Refer to Fig.12 , Fig.12 where (a), (b), (c) and (d) are the point clouds at different angles, and then they are fused into a complete point cloud as shown in Fig.13 shown.

[0158] After converting the point cloud of the head frame to the imaging coordinate system, the image of the target part and the point cloud of the head frame will both be displayed based on the imaging coordinate system. Thus, the doctor can perform path planning such as the puncture path on the image of the target part so that the planned path can avoid the head frame.

[0159] This embodiment uses the above-introduced tracking system for obstacle avoidance, reduces the requirements for the placement positions of the components in the tracking system, and improves the obstacle avoidance efficiency.

[0160] In one embodiment, the path planning method also includes: obtaining the conversion relationship between the tracking device coordinate system and the robotic arm base coordinate system based on the coordinates of the fourth type of electromagnetic marker in the tracking device coordinate system and the robotic arm end coordinate system, and the conversion relationship between the robotic arm end coordinate system and the robotic arm base coordinate system; based on the conversion relationship between the tracking device coordinate system and the robotic arm base coordinate system, converting the point cloud from the tracking device coordinate system to the robotic arm base coordinate system to perform path planning for the robotic arm.

[0161] pass The head frame point cloud can be transformed from the EOPT coordinate system to the robot arm coordinate system, thereby planning the joint paths of the robot arm joints so that the robot arm joints do not touch the head frame during the transformation process.

[0162] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0163] The present application provides a medical device navigation device, which is based on the tracking system introduced above and includes Fig.14 Modules shown:

[0164] The coordinate acquisition module 1401 is used to acquire a first coordinate set; the first coordinate set includes the coordinates of the first type of electromagnetic marker in the tracking device coordinate system and the fixed component coordinate system, the coordinates of the second type of electromagnetic marker in the tracking device coordinate system and the image coordinate system, and the coordinates of the third type of electromagnetic marker in the tracking device coordinate system and the medical device coordinate system; the image coordinate system is a coordinate system describing the image of the target part;

[0165] A conversion relationship acquisition module 1402, used for obtaining a conversion relationship between the image coordinate system and the medical device coordinate system based on the first coordinate set;

[0166] The navigation module 1403 is used to navigate the medical device based on the conversion relationship between the image coordinate system and the medical device coordinate system.

[0167] In one embodiment, the conversion relationship acquisition module 1402 is also used to convert the coordinates of the second type of electromagnetic marker and the third type of electromagnetic marker in the tracking device coordinate system to the fixed component coordinate system based on the coordinates of the first type of electromagnetic marker in the tracking device coordinate system and the fixed component coordinate system; based on the coordinates of the second type of electromagnetic marker in the image coordinate system and the fixed component coordinate system, and the coordinates of the third type of electromagnetic marker in the medical device coordinate system and the fixed component coordinate system, obtain the conversion relationship between the image coordinate system and the medical device coordinate system.

[0168] In one embodiment, the conversion relationship acquisition module 1402 is also used to obtain the conversion relationship between the fixed component coordinate system and the image coordinate system based on the coordinates of the second type of electromagnetic marker in the image coordinate system and the fixed component coordinate system; to obtain the conversion relationship between the fixed component coordinate system and the medical device coordinate system based on the coordinates of the third type of electromagnetic marker in the medical device coordinate system and the fixed component coordinate system; and to obtain the conversion relationship between the image coordinate system and the medical device coordinate system based on the conversion relationship between the fixed component coordinate system and the image coordinate system and the conversion relationship between the fixed component coordinate system and the medical device coordinate system.

[0169] The present application provides a device for controlling the movement of the end of a robotic arm. The device is based on the tracking system introduced above. The device includes Fig.15 Modules shown:

[0170] The coordinate acquisition module 1501 is used to acquire a second coordinate set; the second coordinate set includes the coordinates of the first type of electromagnetic marker in the tracking device coordinate system and the fixed component coordinate system, the coordinates of the second type of electromagnetic marker in the tracking device coordinate system and the image coordinate system, and the coordinates of the fourth type of electromagnetic marker in the tracking device coordinate system and the robot end coordinate system;

[0171] A conversion relationship acquisition module 1502 is used to obtain a conversion relationship between the robot arm base coordinate system and the image coordinate system based on the second coordinate set and the conversion relationship between the robot arm end coordinate system and the robot arm base coordinate system;

[0172] The movement control module 1503 is used to obtain the selected point on the image of the target part, and control the movement of the end of the robot arm according to the conversion relationship between the robot arm base coordinate system and the image coordinate system.

[0173] In one embodiment, the conversion relationship acquisition module 1502 is also used to obtain the conversion relationship between the fixed component coordinate system and the robot arm base coordinate system based on the coordinates of the fourth type of electromagnetic marker in the tracking device coordinate system and the robot arm end coordinate system, the coordinates of the first type of electromagnetic marker in the tracking device coordinate system and the fixed component coordinate system, and the conversion relationship between the robot arm end coordinate system and the robot arm base coordinate system; obtain the conversion relationship between the fixed component coordinate system and the image coordinate system based on the coordinates of the second type of electromagnetic marker in the tracking device coordinate system and the image coordinate system, and the coordinates of the first type of electromagnetic marker in the tracking device coordinate system and the fixed component coordinate system; obtain the conversion relationship between the robot arm base coordinate system and the image coordinate system based on the conversion relationship between the fixed component coordinate system and the robot arm base coordinate system, and the conversion relationship between the fixed component coordinate system and the image coordinate system.

[0174] In one embodiment, the conversion relationship acquisition module 1502 is also used to obtain the conversion relationship between the tracking device coordinate system and the fixed component coordinate system based on the coordinates of the first type of electromagnetic marker in the tracking device coordinate system and the fixed component coordinate system; to obtain the coordinates of the second type of electromagnetic marker in the fixed component coordinate system based on the conversion relationship between the tracking device coordinate system and the fixed component coordinate system and the coordinates of the second type of electromagnetic marker in the tracking device coordinate system; and to obtain the conversion relationship between the fixed component coordinate system and the image coordinate system based on the coordinates of the second type of electromagnetic marker in the fixed component coordinate system and the image coordinate system.

[0175] In one embodiment, the conversion relationship acquisition module 1502 is also used to obtain the conversion relationship between the tracking device coordinate system and the fixed component coordinate system based on the coordinates of the first type of electromagnetic marker in the tracking device coordinate system and the fixed component coordinate system; obtain the coordinates of the fourth type of electromagnetic marker in the fixed component coordinate system based on the conversion relationship between the tracking device coordinate system and the fixed component coordinate system, and the coordinates of the fourth type of electromagnetic marker in the tracking device coordinate system; obtain the conversion relationship between the robot arm end coordinate system and the fixed component coordinate system based on the coordinates of the fourth type of electromagnetic marker in the fixed component coordinate system and the robot arm end coordinate system; obtain the conversion relationship between the fixed component coordinate system and the robot arm base coordinate system based on the conversion relationship between the robot arm end coordinate system and the fixed component coordinate system, and the conversion relationship between the robot arm end coordinate system and the robot arm base coordinate system.

[0176] The present application provides a path planning device, which is based on the tracking system introduced above, and includes Fig.16 Modules shown:

[0177] The point cloud acquisition module 1601 is used to acquire the point cloud of the fixed component through the tracking device;

[0178] A conversion relationship acquisition module 1602 is used to obtain a conversion relationship between the tracking device coordinate system and the image coordinate system based on the coordinates of the second type of electromagnetic marker in the tracking device coordinate system and the image coordinate system;

[0179] The path planning processing module 1603 is used to transform the point cloud from the tracking device coordinate system to the image coordinate system based on the conversion relationship between the tracking device coordinate system and the image coordinate system, so as to perform path planning for the image of the target part.

[0180] In one embodiment, the transformation relationship acquisition module 1602 is also used to obtain the transformation relationship between the tracking device coordinate system and the robotic arm base coordinate system based on the coordinates of the fourth type of electromagnetic marker in the tracking device coordinate system and the robotic arm end coordinate system, and the transformation relationship between the robotic arm end coordinate system and the robotic arm base coordinate system; the path planning processing module 1603 is also used to transform the point cloud from the tracking device coordinate system to the robotic arm base coordinate system based on the transformation relationship between the tracking device coordinate system and the robotic arm base coordinate system, so as to perform path planning for the robotic arm.

[0181] For the specific definition of each device, please refer to the definition of the corresponding method above, which will not be repeated here. Each module in the above-mentioned devices can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0182] In one embodiment, a computer device is provided, whose internal structure diagram can be as follows: Fig.17As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data related to the above method. The network interface of the computer device is used to communicate with an external terminal through a network connection. The computer device also includes an input and output interface, which is a connection circuit for exchanging information between the processor and the external device. They are connected to the processor through a bus, referred to as an I / O interface. When the computer program is executed by the processor, one of the above methods is implemented.

[0183] Those skilled in the art will understand that Fig.17 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0184] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps in the above-mentioned various method embodiments when executing the computer program.

[0185] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0186] In one embodiment, a computer program product is provided, on which a computer program is stored, and the computer program is used by a processor to execute the steps in the above-mentioned various method embodiments.

[0187] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the above-mentioned computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0188] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0189] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A tracking system, characterized in that: The tracking system includes: A tracking device comprising a magnetic field generator for generating a magnetic field; The multiple types of electromagnetic markers include at least a first type of electromagnetic marker and a second type of electromagnetic marker, and the multiple types of electromagnetic markers also include at least one of a third type of electromagnetic marker and a fourth type of electromagnetic marker; wherein the first type of electromagnetic marker is used to be arranged on a fixed component, the second type of electromagnetic marker is used to be arranged on a target part, the third type of electromagnetic marker is used to be arranged on a medical device, and the fourth type of electromagnetic marker is used to be arranged on the end of a robotic arm; the fixed component is used to fix the target part; and the end of the robotic arm is used to clamp the medical device; The electromagnetic marker is used to generate an electrical signal under the action of the magnetic field; the electrical signal is used to determine the coordinates of the electromagnetic marker in the tracking device coordinate system.

2. The tracking system according to claim 1, characterized in that: The tracking device also includes a light emitting component and a light receiving component; The light emitting component is used to emit structured light to the fixed component; The light receiving component is used to receive the reflected light reflected by the fixing component based on the structured light; The reflected light is used to form a point cloud of the fixed component.

3. A medical device navigation method, characterized in that: Applied to the tracking system of claim 1 or 2, the method comprises: Acquire a first coordinate set; the first coordinate set includes the coordinates of the first type of electromagnetic marker in the tracking device coordinate system and the fixed component coordinate system, the coordinates of the second type of electromagnetic marker in the tracking device coordinate system and the image coordinate system, and the coordinates of the third type of electromagnetic marker in the tracking device coordinate system and the medical device coordinate system; the image coordinate system is a coordinate system that describes the image of the target part; Based on the first coordinate set, obtaining a conversion relationship between the image coordinate system and the medical device coordinate system; The medical device is navigated based on the conversion relationship between the image coordinate system and the medical device coordinate system.

4. The method according to claim 3, characterized in that The obtaining, based on the first coordinate set, a conversion relationship between the image coordinate system and the medical device coordinate system includes: Based on the coordinates of the first type of electromagnetic marker in the tracking device coordinate system and the fixed component coordinate system, converting the coordinates of the second type of electromagnetic marker and the third type of electromagnetic marker in the tracking device coordinate system to the fixed component coordinate system; Based on the coordinates of the second type of electromagnetic marker in the image coordinate system and the fixed component coordinate system, and the coordinates of the third type of electromagnetic marker in the medical device coordinate system and the fixed component coordinate system, the conversion relationship between the image coordinate system and the medical device coordinate system is obtained.

5. The method according to claim 3, characterized in that: The method of obtaining a conversion relationship between the image coordinate system and the medical device coordinate system based on the coordinates of the second type of electromagnetic marker in the image coordinate system and the fixed component coordinate system, and the coordinates of the third type of electromagnetic marker in the medical device coordinate system and the fixed component coordinate system, comprises: Based on the coordinates of the second-type electromagnetic marker in the image coordinate system and the fixed component coordinate system, obtaining a conversion relationship between the fixed component coordinate system and the image coordinate system; Based on the coordinates of the third type electromagnetic marker in the medical device coordinate system and the fixed component coordinate system, obtaining a conversion relationship between the fixed component coordinate system and the medical device coordinate system; Based on the conversion relationship between the fixed component coordinate system and the image coordinate system, and the conversion relationship between the fixed component coordinate system and the medical device coordinate system, the conversion relationship between the image coordinate system and the medical device coordinate system is obtained.

6. A method for controlling the movement of a robot end, characterized in that: Applied to the tracking system of claim 1 or 2, the method comprises: Acquire a second coordinate set; the second coordinate set includes the coordinates of the first type of electromagnetic marker in the tracking device coordinate system and the fixed component coordinate system, the coordinates of the second type of electromagnetic marker in the tracking device coordinate system and the image coordinate system, and the coordinates of the fourth type of electromagnetic marker in the tracking device coordinate system and the robot arm end coordinate system; Based on the second coordinate set and the conversion relationship between the robot end coordinate system and the robot base coordinate system, a conversion relationship between the robot base coordinate system and the image coordinate system is obtained; The selected point on the image of the target part is obtained, and the movement of the end of the robotic arm is controlled according to the conversion relationship between the robotic arm base coordinate system and the image coordinate system.

7. The method according to claim 6, characterized in that The step of obtaining the conversion relationship between the robot base coordinate system and the image coordinate system based on the second coordinate set and the conversion relationship between the robot end coordinate system and the robot base coordinate system comprises: Based on the coordinates of the fourth type of electromagnetic marker in the tracking device coordinate system and the robot arm end coordinate system, the coordinates of the first type of electromagnetic marker in the tracking device coordinate system and the fixed component coordinate system, and the conversion relationship between the robot arm end coordinate system and the robot arm base coordinate system, the conversion relationship between the fixed component coordinate system and the robot arm base coordinate system is obtained; Based on the coordinates of the second type of electromagnetic marker in the tracking device coordinate system and the image coordinate system, and the coordinates of the first type of electromagnetic marker in the tracking device coordinate system and the fixed component coordinate system, a conversion relationship between the fixed component coordinate system and the image coordinate system is obtained; Based on the conversion relationship between the fixed component coordinate system and the robot base coordinate system, and the conversion relationship between the fixed component coordinate system and the image coordinate system, the conversion relationship between the robot base coordinate system and the image coordinate system is obtained.

8. The method according to claim 7, characterized in that The method of obtaining a conversion relationship between the fixed component coordinate system and the image coordinate system based on the coordinates of the second type of electromagnetic marker in the tracking device coordinate system and the image coordinate system, and the coordinates of the first type of electromagnetic marker in the tracking device coordinate system and the fixed component coordinate system, comprises: Based on the coordinates of the first type of electromagnetic marker in the tracking device coordinate system and the fixed component coordinate system, obtaining a conversion relationship between the tracking device coordinate system and the fixed component coordinate system; Based on the conversion relationship between the tracking device coordinate system and the fixed component coordinate system, and the coordinates of the second type of electromagnetic marker in the tracking device coordinate system, obtain the coordinates of the second type of electromagnetic marker in the fixed component coordinate system; Based on the coordinates of the second-type electromagnetic marker in the fixed component coordinate system and the image coordinate system, a conversion relationship between the fixed component coordinate system and the image coordinate system is obtained.

9. The method according to claim 7, characterized in that: The conversion relationship between the fixed component coordinate system and the robotic arm base coordinate system is obtained based on the coordinates of the fourth type of electromagnetic marker in the tracking device coordinate system and the robotic arm end coordinate system, the coordinates of the first type of electromagnetic marker in the tracking device coordinate system and the fixed component coordinate system, and the conversion relationship between the robotic arm end coordinate system and the robotic arm base coordinate system, including: Based on the coordinates of the first type of electromagnetic marker in the tracking device coordinate system and the fixed component coordinate system, obtaining a conversion relationship between the tracking device coordinate system and the fixed component coordinate system; Based on the conversion relationship between the tracking device coordinate system and the fixed component coordinate system, and the coordinates of the fourth type of electromagnetic marker in the tracking device coordinate system, the coordinates of the fourth type of electromagnetic marker in the fixed component coordinate system are obtained; Based on the coordinates of the fourth type electromagnetic marker in the fixed component coordinate system and the robot arm end coordinate system, a conversion relationship between the robot arm end coordinate system and the fixed component coordinate system is obtained; Based on the conversion relationship between the robot end coordinate system and the fixed component coordinate system, and the conversion relationship between the robot end coordinate system and the robot base coordinate system, the conversion relationship between the fixed component coordinate system and the robot base coordinate system is obtained.

10. A path planning method, characterized in that: Applied to the tracking system of claim 2, the method comprising: Acquiring a point cloud of the fixed component by means of the tracking device; Based on the coordinates of the second type of electromagnetic marker in the tracking device coordinate system and the image coordinate system, obtaining a conversion relationship between the tracking device coordinate system and the image coordinate system; Based on the conversion relationship between the tracking device coordinate system and the image coordinate system, the point cloud is converted from the tracking device coordinate system to the image coordinate system to perform path planning for the image of the target part.

11. The method according to claim 10, characterized in that The method comprises: Based on the coordinates of the fourth type of electromagnetic marker in the tracking device coordinate system and the robot arm end coordinate system, and the conversion relationship between the robot arm end coordinate system and the robot arm base coordinate system, a conversion relationship between the tracking device coordinate system and the robot arm base coordinate system is obtained; Based on the conversion relationship between the tracking device coordinate system and the robot arm base coordinate system, the point cloud is converted from the tracking device coordinate system to the robot arm base coordinate system to perform path planning for the robot arm.

12. A medical device navigation device, characterized in that: Applicable to the tracking system of claim 1 or 2, the device comprising: A coordinate acquisition module, used for acquiring a first coordinate set; the first coordinate set includes the coordinates of the first type of electromagnetic marker in the tracking device coordinate system and the fixed component coordinate system, the coordinates of the second type of electromagnetic marker in the tracking device coordinate system and the image coordinate system, and the coordinates of the third type of electromagnetic marker in the tracking device coordinate system and the medical device coordinate system; the image coordinate system is a coordinate system describing the image of the target part; A conversion relationship acquisition module, used for obtaining a conversion relationship between the image coordinate system and the medical device coordinate system based on the first coordinate set; A navigation module is used to navigate the medical device based on the conversion relationship between the image coordinate system and the medical device coordinate system.

13. A device for controlling the movement of a robot end, characterized in that: Applicable to the tracking system of claim 1 or 2, the device comprising: A coordinate acquisition module, used to acquire a second coordinate set; the second coordinate set includes the coordinates of the first type of electromagnetic marker in the tracking device coordinate system and the fixed component coordinate system, the coordinates of the second type of electromagnetic marker in the tracking device coordinate system and the image coordinate system, and the coordinates of the fourth type of electromagnetic marker in the tracking device coordinate system and the robot end coordinate system; A conversion relationship acquisition module, used for obtaining a conversion relationship between the robot arm base coordinate system and the image coordinate system based on the second coordinate set and the conversion relationship between the robot arm end coordinate system and the robot arm base coordinate system; The mobile control module is used to obtain the selected point on the image of the target part and control the movement of the end of the robot arm according to the conversion relationship between the robot arm base coordinate system and the image coordinate system.

14. A path planning device, characterized in that: Applied to the tracking system of claim 2, the device comprises: A point cloud acquisition module, used to acquire a point cloud of the fixed component through the tracking device; A conversion relationship acquisition module, used for obtaining a conversion relationship between the tracking device coordinate system and the image coordinate system based on the coordinates of the second type of electromagnetic marker in the tracking device coordinate system and the image coordinate system; The obstacle avoidance processing module is used to convert the point cloud from the tracking device coordinate system to the image coordinate system based on the conversion relationship between the tracking device coordinate system and the image coordinate system, so as to plan the path of the image of the target part.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 3 to 11 is implemented.

Images