A robot positioning tracking and guiding method for transcranial magnetic stimulation and related device

Abstract

The application discloses a robot positioning tracking guiding method for transcranial magnetic stimulation, which comprises the following steps: firstly confirming the positional relationship between a first camera and a second camera relative to a multi-axis robot, then determining the positional relationship between the first camera and the second camera and corresponding markers, guiding based on the positional relationship of the first camera, and then correcting the guiding process based on the positional relationship of the second camera, so as to improve the accuracy of robot positioning tracking guiding for transcranial magnetic stimulation and reduce the cost of manual operation. The application also discloses a robot positioning tracking guiding device for transcranial magnetic stimulation, a computing device and a computer readable storage medium, which have the above beneficial effects.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automation, in particular to a robot positioning tracking guiding method for transcranial magnetic stimulation, a robot positioning tracking guiding device, a computing device and a computer readable storage medium. BACKGROUND

[0002] Transcranial magnetic stimulation (TMS) is a technology that uses a pulsed magnetic field to stimulate the central nervous system. Transcranial magnetic stimulation requires a TMS operation to be aimed at the operation target point of the head of the operation object and maintained for a given time.

[0003] In related technologies, the face of the operation object is spatially positioned according to RGBD (three primary color information, depth information) image data of the face of the operation object, and the pose of the face of the operation object relative to the camera coordinate system is obtained. The head of the operation object is modeled, the head model of the operation object is matched with the actual head, the pose of the target point of the head of the operation object in space is obtained, and then the target point pose is sent to the robot, and the robot holds the TMS operation to operate the operation object. This scheme can reduce the physical and time of the operator, and can align the TMS operation to the target point of the head of the operation object. However, it mainly has the shortcomings of low precision and complicated operation. When there is a position offset, it cannot be corrected well.

[0004] Therefore, how to improve the precision of robot positioning tracking guiding for transcranial magnetic stimulation and reduce the cost of manual operation is an important problem for those skilled in the art. SUMMARY

[0005] The purpose of the present application is to provide a robot positioning tracking guiding method for transcranial magnetic stimulation, a robot positioning tracking guiding device, a computing device and a computer readable storage medium to improve the precision of robot positioning tracking guiding for transcranial magnetic stimulation and reduce the cost of manual operation.

[0006] To solve the above technical problems, the present application provides a robot positioning tracking guiding method for transcranial magnetic stimulation, comprising:

[0007] Hand-eye calibration is performed on a multi-axis robot relative to a first camera and a second camera to obtain a first pose of the first camera relative to the multi-axis robot and a second pose of the second camera relative to the multi-axis robot;

[0008] calculate a third pose of the first camera relative to the first marker, and calculate a seventh pose of the second camera relative to a coordinate system of the second marker group, and an eighth pose of the first marker relative to the coordinate system of the second marker group; wherein the coordinate system of the second marker group is calculated from the fourth pose, the fifth pose, and the sixth pose of the second marker group;

[0009] send a ninth pose of the first marker relative to the multi-axis robot to the multi-axis robot so that an end of the multi-axis robot is close to a position of the first marker; wherein the ninth pose is calculated from the first pose and the third pose;

[0010] When the first marker is blocked, update the calculation of a new seventh pose, calculate a new ninth pose based on the new seventh pose and the eighth pose, and send the new ninth pose to the multi-axis robot.

[0011] Optionally, hand-eye calibration is performed on the multi-axis robot relative to the first camera and the second camera to obtain a first pose of the first camera relative to the multi-axis robot and a second pose of the second camera relative to the multi-axis robot, comprising:

[0012] move the multi-axis robot provided with a calibration board to ten positions under the field of view of the first camera and the second camera, and perform hand-eye calibration calculation based on the flange pose of the multi-axis robot and the calibration board image data collected by the first camera and the second camera;

[0013] perform hand-eye calibration calculation based on the flange pose of the multi-axis robot and the calibration board image data collected by the first camera and the second camera to obtain the first pose of the first camera relative to the multi-axis robot and the second pose of the second camera relative to the multi-axis robot.

[0014] Optionally, the calculation of the third pose of the first camera relative to the first marker comprises:

[0015] the first camera captures an image of the first marker; wherein the first marker is a preset two-dimensional code image;

[0016] pose calculation based on the image of the first marker to obtain the third pose.

[0017] Optionally, the calculation of the seventh pose of the second camera relative to the coordinate system of the second marker group, and the eighth pose of the first marker relative to the coordinate system of the second marker group comprises:

[0018] the second camera captures images of multiple markers of the second marker group, and performs pose calculation to obtain the fourth pose, the fifth pose, and the sixth pose;

[0019] calculating a seventh pose of the second camera relative to a coordinate system of the second marker group based on the fourth pose, the fifth pose and the sixth pose;

[0020] calculating an eighth pose of the first marker relative to the coordinate system of the second marker group based on the first pose, the second pose, the third pose and the seventh pose.

[0021] Optionally, sending a ninth pose of the first marker relative to the multi-axis robot to the multi-axis robot so that an end of the multi-axis robot is close to a position of the first marker, comprising:

[0022] calculating the ninth pose of the first marker relative to the multi-axis robot based on the first pose and the third pose;

[0023] sending the ninth pose to the multi-axis robot so that an end of the multi-axis robot is close to a position of the first marker.

[0024] Optionally, when the first marker is blocked, calculating a new seventh pose, calculating a new ninth pose based on the new seventh pose and the eighth pose, and sending the new ninth pose to the multi-axis robot, comprising:

[0025] When the first marker is blocked, updating the pose data through the second camera to obtain a new fourth pose, a new fifth pose and a new sixth pose;

[0026] calculating a new seventh pose based on the new fourth pose, the new fifth pose and the new sixth pose;

[0027] calculating a new ninth pose based on the new seventh pose and the eighth pose, and sending the new ninth pose to the multi-axis robot.

[0028] Optionally, the first marker and the plurality of markers of the second marker group are preset two-dimensional code images.

[0029] The application also provides a robot positioning and tracking guidance device for transcranial magnetic stimulation, comprising:

[0030] an initial calibration module configured to perform hand-eye calibration of a multi-axis robot relative to a first camera and a second camera to obtain a first pose of the first camera relative to the multi-axis robot and a second pose of the second camera relative to the multi-axis robot;

[0031] a marker calibration module, configured to calculate a third pose of the first camera relative to a first marker, and calculate a seventh pose of the second camera relative to a coordinate system of a second marker set, and an eighth pose of the first marker relative to the coordinate system of the second marker set, wherein the coordinate system of the second marker set is calculated based on a fourth pose, a fifth pose, and a sixth pose of the second marker set;

[0032] a first guiding module, configured to send a ninth pose of the first marker relative to the multi-axis robot to the multi-axis robot, so that an end of the multi-axis robot approaches a position of the first marker, wherein the ninth pose is calculated based on the first pose and the third pose;

[0033] a second guiding module, configured to, when the first marker is blocked, update to calculate a new seventh pose, calculate a new ninth pose based on the new seventh pose and the eighth pose, and send the new ninth pose to the multi-axis robot.

[0034] The application further provides a computing device, comprising:

[0035] a memory, configured to store a computer program;

[0036] a processor, configured to execute the computer program to implement the steps of the robot positioning, tracking and guiding method.

[0037] The application further provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the robot positioning, tracking and guiding method.

[0038] The robot positioning tracking guidance method of transcranial magnetic stimulation provided in the application comprises: performing hand-eye calibration of a multi-axis robot relative to a first camera and a second camera to obtain a first pose of the first camera relative to the multi-axis robot and a second pose of the second camera relative to the multi-axis robot; calculating a third pose of the first camera relative to a first marker, and calculating a seventh pose of the second camera relative to a coordinate system of a second marker group and an eighth pose of the first marker relative to the coordinate system of the second marker group; wherein the coordinate system of the second marker group is calculated by a fourth pose, a fifth pose and a sixth pose of the second marker group; a ninth pose of the first marker relative to the multi-axis robot is sent to the multi-axis robot so that an end of the multi-axis robot is close to a position of the first marker; wherein the ninth pose is calculated by the first pose and the third pose; when the first marker is blocked, a new seventh pose is calculated, a new ninth pose is calculated based on the new seventh pose and the eighth pose, and the new ninth pose is sent to the multi-axis robot.

[0039] By first confirming the positional relationship between the first camera and the second camera relative to the multi-axis robot, and then determining the positional relationship between the first camera and the second camera and the corresponding markers, the guidance is first performed based on the positional relationship of the first camera, and then the guidance process is corrected based on the positional relationship of the second camera, so as to improve the accuracy of the robot positioning tracking guidance of transcranial magnetic stimulation and reduce the cost of manual operation.

[0040] The application also provides a robot positioning tracking guidance device of transcranial magnetic stimulation, a computing device and a computer readable storage medium, which have the above beneficial effects, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description only belong to the embodiments of the application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of the provided drawings.

[0042] Figure 1 A flowchart of a robot positioning tracking guidance method of transcranial magnetic stimulation provided by an embodiment of the application;

[0043] Figure 2 An operation schematic diagram of a robot positioning tracking guidance method of transcranial magnetic stimulation provided by an embodiment of the application;

[0044] Figure 3An operation schematic diagram of another robot positioning tracking guidance method of transcranial magnetic stimulation provided by an embodiment of the present application;

[0045] Figure 4 A structural schematic diagram of a robot positioning tracking guidance device of transcranial magnetic stimulation provided by an embodiment of the present application;

[0046] Figure 5 A structural schematic diagram of a computing device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0047] The core of the present application is to provide a robot positioning tracking guidance method of transcranial magnetic stimulation, a robot positioning tracking guidance device of transcranial magnetic stimulation, a computing device and a computer readable storage medium, so as to improve the accuracy of robot positioning tracking guidance of transcranial magnetic stimulation and reduce the cost of manual operation.

[0048] To make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0049] In the related art, the face of an operating object is spatially positioned according to RGBD image data of the face of the operating object, and a pose of the face of the operating object relative to a camera coordinate system is obtained. The head of the operating object is modeled, the operating object head model is matched with the actual head, a pose of a target point of the head of the operating object in space is obtained, and then the target point pose is sent to a robot. The robot holds a TMS operating probe to operate on the operating object. This scheme can reduce the physical labor and time of the operator, and can align the TMS operating probe to the target point of the head of the operating object. However, the main disadvantages are low accuracy and complicated operation. When there is a position offset, it cannot be corrected well.

[0050] Therefore, the present application provides a robot positioning tracking guidance method of transcranial magnetic stimulation. The position relationship between a first camera and a second camera relative to a multi-axis robot is first confirmed, and then the position relationship between the first camera and the second camera and corresponding markers is determined. The guidance is first performed based on the position relationship of the first camera, and then the guidance process is corrected based on the position relationship of the second camera, so as to improve the accuracy of robot positioning tracking guidance of transcranial magnetic stimulation and reduce the cost of manual operation.

[0051] The robot positioning tracking guidance method of transcranial magnetic stimulation provided by the present application will be described below through an embodiment.

[0052] Reference is made to Figure 1 , Figure 1 A flowchart of a robot positioning tracking and guiding method of transcranial magnetic stimulation provided by an embodiment of the present application.

[0053] In this embodiment, the method can include:

[0054] S101, hand-eye calibration is performed on the multi-axis robot relative to the first camera and the second camera to obtain a first pose of the first camera relative to the multi-axis robot and a second pose of the second camera relative to the multi-axis robot;

[0055] This step aims to perform hand-eye calibration on the multi-axis robot relative to the first camera and the second camera to obtain a first pose of the first camera relative to the multi-axis robot and a second pose of the second camera relative to the multi-axis robot. That is, the initial calibration operation is completed to determine the positional relationship between the first camera and the second camera and the multi-axis robot respectively.

[0056] Further, this step can include:

[0057] Step 1, the multi-axis robot provided with a calibration board is moved at ten points in the field of view of the first camera and the second camera, and based on the flange pose of the multi-axis robot and the calibration board image data collected by the first camera and the second camera;

[0058] Step 2, based on the flange pose and the calibration board image data collected by the first camera and the second camera, hand-eye calibration calculation is performed to obtain a first pose of the first camera relative to the multi-axis robot and a second pose of the second camera relative to the multi-axis robot.

[0059] As can be seen, in this optional solution, how to perform hand-eye calibration is mainly explained. In this optional solution, the multi-axis robot provided with a calibration board is moved at ten points in the field of view of the first camera and the second camera, and based on the flange pose of the multi-axis robot and the calibration board image data collected by the first camera and the second camera; based on the flange pose and the calibration board image data collected by the first camera and the second camera, hand-eye calibration calculation is performed to obtain a first pose of the first camera relative to the multi-axis robot and a second pose of the second camera relative to the multi-axis robot.

[0060] S102, a third pose of the first camera relative to the first marker is calculated, and a seventh pose of the second camera relative to the coordinate system of the second marker group and an eighth pose of the first marker relative to the coordinate system of the second marker group are calculated; wherein the coordinate system of the second marker group is calculated from the fourth pose, the fifth pose and the sixth pose of the second marker group;

[0061] On the basis of S101, the step is to calculate the third pose of the first camera relative to the first marker, and calculate the seventh pose of the second camera relative to the coordinate system of the second marker group, and the eighth pose of the first marker relative to the coordinate system of the second marker group; wherein the coordinate system of the second marker group is calculated by the fourth pose, the fifth pose, and the sixth pose of the second marker group. That is, the positional relationship of the first camera and the second camera relative to the corresponding markers is determined.

[0062] Wherein, the first marker and the second marker group are markers arranged on the surface of the operation object. And the first marker and the second marker group are respectively on the two side surfaces of the operation object, avoiding the problem that the first marker and the second marker group are both blocked, so as to correct the positions of each other.

[0063] Wherein, the plurality of markers of the first marker and the second marker group are preset two-dimensional code images.

[0064] Further, the calculation of the third pose of the first camera relative to the first marker in this step can include:

[0065] Step 1, the first camera shoots the image of the first marker; wherein the first marker is a preset two-dimensional code image;

[0066] Step 2, the third pose is obtained by performing pose calculation based on the image of the first marker.

[0067] It can be seen that in this optional solution, how to calculate the pose between the marker is mainly explained. In this optional solution, the first camera shoots the image of the first marker; wherein the first marker is a preset two-dimensional code image; the third pose is obtained by performing pose calculation based on the image of the first marker.

[0068] Further, the calculation of the seventh pose of the second camera relative to the coordinate system of the second marker group, and the eighth pose of the first marker relative to the coordinate system of the second marker group in this step can include:

[0069] Step 1, the second camera shoots the image of the plurality of markers of the second marker group, and performs pose calculation to obtain the fourth pose, the fifth pose, and the sixth pose;

[0070] Step 2, the seventh pose of the second camera relative to the coordinate system of the second marker group is calculated based on the fourth pose, the fifth pose, and the sixth pose;

[0071] Step 3, the eighth pose of the first marker relative to the coordinate system of the second marker group is calculated based on the first pose, the second pose, the third pose, and the seventh pose.

[0072] It can be seen that the present alternative mainly illustrates how to determine the seventh pose and the eighth pose. In the present alternative, the second camera captures images of the plurality of markers of the second marker group, and performs pose calculation to obtain the fourth pose, the fifth pose, and the sixth pose; the seventh pose of the second camera relative to the coordinate system of the second marker group is calculated based on the fourth pose, the fifth pose, and the sixth pose; and the eighth pose of the first marker relative to the coordinate system of the second marker group is calculated based on the first pose, the second pose, the third pose, and the seventh pose.

[0073] S103, sending the ninth pose of the first marker relative to the multi-axis robot to the multi-axis robot, so that the end of the multi-axis robot is close to the position of the first marker; wherein the ninth pose is calculated from the first pose and the third pose;

[0074] On the basis of S102, the present step aims to send the ninth pose of the first marker relative to the multi-axis robot to the multi-axis robot, so that the end of the multi-axis robot is close to the position of the first marker; wherein the ninth pose is calculated from the first pose and the third pose. That is, without blocking the first marker, the multi-axis robot is guided to operate based on the positional relationship of the first marker.

[0075] Further, the present step can include:

[0076] Step 1, calculating the ninth pose of the first marker relative to the multi-axis robot based on the first pose and the third pose;

[0077] Step 2, sending the ninth pose to the multi-axis robot, so that the end of the multi-axis robot is close to the position of the first marker.

[0078] It can be seen that the present alternative mainly illustrates how to guide the multi-axis robot. In the present alternative, the ninth pose of the first marker relative to the multi-axis robot is calculated based on the first pose and the third pose; and the ninth pose is sent to the multi-axis robot, so that the end of the multi-axis robot is close to the position of the first marker.

[0079] S104, when the first marker is blocked, a new seventh pose is calculated, a new ninth pose is calculated based on the new seventh pose and the eighth pose, and the new ninth pose is sent to the multi-axis robot.

[0080] On the basis of S102, the present step aims to update the calculation of a new seventh pose when the first marker is blocked, calculate a new ninth pose based on the new seventh pose and the eighth pose, and send the new ninth pose to the multi-axis robot.

[0081] Further, the present step can include:

[0082] Step 1, when the first marker is blocked, the pose data is updated by the second camera to obtain a new fourth pose, a new fifth pose, and a new sixth pose;

[0083] Step 2, a new seventh pose is calculated based on the new fourth pose, the new fifth pose, and the new sixth pose;

[0084] Step 3, a new ninth pose is calculated based on the new seventh pose and the eighth pose, and the new ninth pose is sent to the multi-axis robot.

[0085] It can be seen that the present optional solution mainly illustrates how to guide the multi-axis robot in the case of occlusion. In the present optional solution, when the first marker is blocked, the pose data is updated by the second camera to obtain a new fourth pose, a new fifth pose, and a new sixth pose; a new seventh pose is calculated based on the new fourth pose, the new fifth pose, and the new sixth pose; a new ninth pose is calculated based on the new seventh pose and the eighth pose, and the new ninth pose is sent to the multi-axis robot.

[0086] In summary, the present embodiment first confirms the positional relationship between the first camera and the second camera relative to the multi-axis robot, and then determines the positional relationship between the first camera and the second camera and the corresponding markers, guides based on the positional relationship of the first camera, and then corrects the guiding process based on the positional relationship of the second camera, so as to improve the accuracy of robot positioning and tracking guidance for transcranial magnetic stimulation, and reduce the cost of manual operation.

[0087] The present application provides a robot positioning and tracking guidance method for transcranial magnetic stimulation.

[0088] Please refer to Figure 2 , Figure 2 for the operation schematic diagram of the robot positioning and tracking guidance method for transcranial magnetic stimulation provided by the present embodiment.

[0089] In the present embodiment, the corresponding typical system structure is as follows Figure 2 , which comprises a multi-axis robot (1) for holding a TMS operation pad (2), not less than two cameras (3), (5), and an operation bed (4).

[0090] Among them, the multi-axis robot (1) is installed at one end of the operation bed (4) and is fixed relative to the operation bed (4); the cameras (3) are arranged around the operation bed (4) near one end and are fixed relative to the operation bed (4); when the operation object lies on the operation bed (4), the cameras (3), (5) can collect the head data of the operation object.

[0091] Further, the use of the system is divided into two stages:

[0092] (a) System calibration stage.

[0093] In the system calibration stage, the hand-eye calibration of the multi-axis robot (1) relative to the cameras (3) (5) is mainly carried out, and the pose of the camera (3) (5) coordinate system relative to the multi-axis robot (1) base coordinate system is obtained.

[0094] This stage usually involves the multi-axis robot (1) holding a calibration board at the end, moving ten or so points in the field of view of the camera (3) (5), and recording the flange pose of the multi-axis robot (1) and the image data of the calibration board collected by the camera (3) (5). Finally, the hand-eye calibration result is obtained according to the above collected data. Hand-eye calibration is a very mature algorithm in the industry, and will not be described here. The poses of the camera (3) and the camera (5) relative to the multi-axis robot (1) base coordinate system obtained by hand-eye calibration are T1 and T2, respectively.

[0095] (b) System usage stage.

[0096] Please refer to Figure 3 , Figure 3 Another operation schematic diagram of the transcranial magnetic stimulation robot positioning and tracking guidance method provided in the embodiments of the present application.

[0097] Reference Figure 3 In the system usage stage, the system is mainly used to operate the operation object, and the operation process can include:

[0098] Step 1: The operator determines the target point to be operated and the operation parameters of the operation object; the operation object wears a pathological cap, and the operator determines the head target point to be operated of the operation object, and pastes a mark (6) on the pathological cap. It is assumed that the mark (6) can be observed by the camera (3).

[0099] Step 2: The operator pastes marks (7) (8) (9) on the other side of the head region where the mark (6) is located, and needs to ensure that the marks (7), (8), and (9) can be observed by the camera (5). In short, if the mark (6) is on the left side of the head, paste the marks (7) (8) (9) on the right side of the head; if the mark (6) is on the right side of the head, paste the marks (7) (8) (9) on the left side of the head.

[0100] Step 3: The camera (3) captures the image of the mark (6). The mark (6) and the marks (7) (8) (9) can be ordinary two-dimensional codes or other marks such as AprilTag. The characteristic of such marks is that if the actual physical size of the mark is known, the position and attitude of the mark relative to the camera can be calculated through affine transformation. Here, the pose of the mark (6) relative to the camera (3) is T3.

[0101] Step 4: At the same time of step 3, camera (5) takes pictures of the markers (7)(8)(9), and gets the pose of the three markers relative to camera (5), which are recorded as T4, T5, T6 respectively. The three center points of the markers (7)(8)(9) can form a plane P. A new coordinate system M is created, where M is located at the centroid of the markers (7)(8)(9), the X axis of M points to the center of marker (7), the Z axis of M is perpendicular to the plane P and points to the direction of camera (5), and the Y axis of M can be obtained according to the right-hand rule. According to the position data of T4, T5, T6, the pose of coordinate system M relative to camera (5) can be easily obtained, which is recorded as T7.

[0102] Step 5: Calculate the pose of marker (6) relative to coordinate system M, T8 = T7.inv() * T2.inv() * T1 * T3, where inv() represents matrix inversion.

[0103] Step 6: At this time, the operation begins, the pose of marker (6) relative to the base coordinate system of multi-axis robot (1) is sent to multi-axis robot (1), T9 = T1 * T3, and multi-axis robot (1) holds TMS operation paddle (2) close to marker (6).

[0104] Step 7: When TMS operation paddle (2) is close to marker (6), marker (6) will be blocked by operation paddle (2) and camera (3) cannot see marker (6). At this time, markers (7)(8)(9) can be observed by camera (5), and the current pose of space coordinate system M relative to camera (5) can be calculated in real time, T7_new. At this time, because the pose of marker (6) relative to space coordinate system M is fixed, T8, the current pose of marker (6) relative to multi-axis robot (1) is T9_new = T7_new * T8, which is sent to multi-axis robot (1), and multi-axis robot (1) can hold TMS operation paddle (2) close to marker (6) at all times. At this time, by continuously observing markers (7)(8)(9) through camera (5), the pose of marker (6) relative to the base of multi-axis robot (5) can be calculated in real time, guiding multi-axis robot (5) to hold TMS operation paddle (2) and keep close to marker (6). Even if the head of the operation object moves, it can still ensure good operation effect.

[0105] Obviously, in this embodiment, multiple cameras are used to observe and locate the markers on the patient's head, which can determine the pose of the target point to be operated on the patient relative to the base coordinate system of the multi-axis robot, guide the multi-axis robot to hold the TMS operation paddle, and keep close to the target point on the patient's head. During the operation, the displacement of the patient's head is calculated in real time, and the latest pose data of the target point on the patient's head is calculated. By using multiple cameras combined with multiple markers, the influence of the TMS operation paddle on the operation target point can be avoided.

[0106] It can be seen that, in the embodiment, the position relationship between the first camera and the second camera relative to the multi-axis robot is first confirmed, and then the position relationship between the first camera and the second camera and the corresponding markers is determined, the guiding is first performed based on the position relationship of the first camera, and then the guiding process is corrected based on the position relationship of the second camera, so that the accuracy of the robot positioning and tracking guiding of the transcranial magnetic stimulation is improved, and the cost of manual operation is reduced.

[0107] The robot positioning and tracking guiding device provided in the embodiment of the application will be introduced below, and the robot positioning and tracking guiding device described below can be correspondingly referred to the transcranial magnetic stimulation robot positioning and tracking guiding method described above.

[0108] Please refer to Figure 4 , Figure 4 A structure schematic diagram of a transcranial magnetic stimulation robot positioning and tracking guiding device provided in the embodiment of the application.

[0109] In the embodiment, the device can include:

[0110] An initial calibration module 100 is configured to perform hand-eye calibration on the multi-axis robot relative to the first camera and the second camera, to obtain a first pose of the first camera relative to the multi-axis robot and a second pose of the second camera relative to the multi-axis robot;

[0111] A marker calibration module 200 is configured to calculate a third pose of the first camera relative to the first marker, and calculate a seventh pose of the second camera relative to a coordinate system of the second marker group and an eighth pose of the first marker relative to the coordinate system of the second marker group, wherein the coordinate system of the second marker group is calculated from the fourth pose, the fifth pose and the sixth pose of the second marker group;

[0112] A first guiding module 300 is configured to send a ninth pose of the first marker relative to the multi-axis robot to the multi-axis robot, so that the end of the multi-axis robot approaches the position of the first marker, wherein the ninth pose is calculated from the first pose and the third pose;

[0113] A second guiding module 400 is configured to update the calculation of a new seventh pose when the first marker is blocked, calculate a new ninth pose based on the new seventh pose and the eighth pose, and send the new ninth pose to the multi-axis robot.

[0114] The application also provides a computing device, please refer to Figure 5 , Figure 5 A structure schematic diagram of a computing device provided in the embodiment of the application, the computing device can include:

[0115] A memory is configured to store a computer program;

[0116] The processor, when executing a computer program, can implement the steps of any of the transcranial magnetic stimulation robot positioning and tracking guidance methods described above.

[0117] like Figure 5 The diagram shows the structural composition of a computing device, which may include a processor 10, a memory 11, a communication interface 12, and a communication bus 13. The processor 10, memory 11, and communication interface 12 all communicate with each other through the communication bus 13.

[0118] In this embodiment, the processor 10 may be a central processing unit (CPU), an application-specific integrated circuit, a digital signal processor, a field-programmable gate array, or other programmable logic devices.

[0119] The processor 10 can call the program stored in the memory 11. Specifically, the processor 10 can execute the operations in the embodiment of the abnormal IP identification method.

[0120] The memory 11 is used to store one or more programs. The programs may include program code, which includes computer operation instructions. In this embodiment, the memory 11 stores at least a program for implementing the following functions:

[0121] Hand-eye calibration is performed on the multi-axis robot relative to the first camera and the second camera to obtain the first pose of the first camera relative to the multi-axis robot and the second pose of the second camera relative to the multi-axis robot.

[0122] Calculate the third pose of the first camera relative to the first marker, and calculate the seventh pose of the second camera relative to the coordinate system of the second marker group, and the eighth pose of the first marker relative to the coordinate system of the second marker group; wherein the coordinate system of the second marker group is calculated from the fourth, fifth, and sixth poses of the second marker group;

[0123] A ninth pose of a first marker relative to the multi-axis robot is sent to the multi-axis robot so that the end effector of the multi-axis robot approaches the position of the first marker; wherein the ninth pose is calculated from the first pose and the third pose;

[0124] When the first marker is occluded, the new seventh pose is updated and calculated. Based on the new seventh pose and the eighth pose, the new ninth pose is calculated and sent to the multi-axis robot.

[0125] In a possible implementation, the memory 11 can include a program storage area and a data storage area, where the program storage area can store an operating system and at least one application required by a function, etc., and the data storage area can store data created during use.

[0126] In addition, the memory 11 can include a high-speed random access memory, and can also include a non-volatile memory, for example, at least one magnetic disk storage device or other volatile solid-state storage device.

[0127] The communication interface 12 can be an interface of a communication module, used for connecting with other devices or systems.

[0128] Of course, it needs to be explained that, Figure 5 The structure shown does not constitute a limitation on the computing device in the embodiments of the present application, and in actual applications, the computing device can include more or fewer components than Figure 5 those shown, or combine certain components.

[0129] The present application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of any one of the transcranial magnetic stimulation robot positioning tracking guidance methods described above.

[0130] The computer readable storage medium can include a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0131] For the computer readable storage medium provided by the present application, refer to the above method embodiments, and the present application will not be repeated here.

[0132] The embodiments in the specification are described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts of each embodiment can be referred to each other. For the device disclosed by the embodiments, since it corresponds to the method disclosed by the embodiments, the description is relatively simple, and the related parts can be referred to the method part.

[0133] Those skilled in the art will further appreciate that the functionality of the various examples illustrated in the figures can be carried out internally or by an external device or devices and are not limited to the architecture shown. Accordingly, the examples described herein can be embodied and / or implemented and / or stored in any tangible medium for use independently or in combination with other devices, in any customised, universal, or proprietary computer hardware, software, firmware, firmware module, or circuitry. In addition, any connections (or ports) to the external devices can be wired or wireless. Furthermore, the present application can be implemented by using an integrated circuit, a logic chip, a microprocessor, or a microprocessor chip to execute the various examples described herein.

[0134] The steps of a method or algorithm described in connection with the embodiments disclosed herein can be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module can reside in RAM, flash memory, ROM, electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. The present application can be also be implemented by using a combination of both software and hardware.

[0135] The above provides a robot positioning tracking guidance method of transcranial magnetic stimulation, a robot positioning tracking guidance device, a computing device, and a computer readable storage medium. The principles and implementation manners of the present application are described by using specific examples. The above example is only used to help understand the method of the present application and its core idea. It should be pointed out that, for those skilled in the art, without departing from the principles of the present application, some improvements and modifications can be made to the present application, and these improvements and modifications also fall within the protection scope of the claims of the present application.

Claims

1. A robot positioning and tracking guidance method for transcranial magnetic stimulation, characterized by, The method comprises the following steps: hand-eye calibration of a multi-axis robot relative to a first camera and a second camera, to obtain a first pose of the first camera relative to the multi-axis robot and a second pose of the second camera relative to the multi-axis robot; calculating a third pose of the first camera relative to a first marker, and calculating a seventh pose of the second camera relative to a coordinate system of a second marker group, and an eighth pose of the first marker relative to the coordinate system of the second marker group; wherein the coordinate system of the second marker group is calculated from a fourth pose, a fifth pose, and a sixth pose of the second marker group; sending a ninth pose of the first marker relative to the multi-axis robot to the multi-axis robot, so that the end of the multi-axis robot is close to the position of the first marker; wherein the ninth pose is calculated from the first pose and the third pose; when the first marker is blocked, updating the calculation of a new seventh pose, calculating a new ninth pose based on the new seventh pose and the eighth pose, and sending the new ninth pose to the multi-axis robot.

2. The robotic positioning, tracking, and guidance method of claim 1, wherein, The method comprises the following steps: hand-eye calibration of a multi-axis robot relative to a first camera and a second camera, to obtain a first pose of the first camera relative to the multi-axis robot and a second pose of the second camera relative to the multi-axis robot, comprising: moving the multi-axis robot provided with a calibration board to ten points in the field of view of the first camera and the second camera, and based on the flange pose of the multi-axis robot and the image data of the calibration board collected by the first camera and the second camera; 3. The robotic positioning, tracking, and guidance method of claim 1, wherein, based on the flange pose and the image data of the calibration board collected by the first camera and the second camera, performing hand-eye calibration calculation to obtain the first pose of the first camera relative to the multi-axis robot and the second pose of the second camera relative to the multi-axis robot. calculating a third pose of the first camera relative to a first marker, comprising: the first camera shoots an image of the first marker; wherein the first marker is a preset two-dimensional code image; 4. The robotic positioning, tracking, and guidance method of claim 1, wherein, based on the image of the first marker, pose calculation is performed to obtain the third pose. calculating a seventh pose of the second camera relative to a coordinate system of a second marker group, and an eighth pose of the first marker relative to the coordinate system of the second marker group; comprising: the second camera shoots an image of a plurality of markers of the second marker group, and performs pose calculation to obtain the fourth pose, the fifth pose, and the sixth pose; based on the fourth pose, the fifth pose, and the sixth pose, the seventh pose of the second camera relative to the coordinate system of the second marker group is calculated; 5. The robotic positioning, tracking, and guiding method of claim 1, wherein, based on the first pose, the second pose, the third pose, and the seventh pose, the eighth pose of the first marker relative to the coordinate system of the second marker group is calculated. sending a ninth pose of the first marker relative to the multi-axis robot to the multi-axis robot, so that the end of the multi-axis robot is close to the position of the first marker, comprising: calculating, based on the first pose and the third pose, a ninth pose of the first marker relative to the multi-axis robot; sending the ninth pose to the multi-axis robot so that an end of the multi-axis robot is close to a position of the first marker.

6. The robotic positioning, tracking, and guiding method of claim 1, wherein, When the first marker is blocked, updating the seventh pose, calculating a new ninth pose based on the new seventh pose and the eighth pose, and sending the new ninth pose to the multi-axis robot, comprising: When the first marker is blocked, updating the pose data through the second camera to obtain a new fourth pose, a new fifth pose, and a new sixth pose; calculating a new seventh pose based on the new fourth pose, the new fifth pose, and the new sixth pose; calculating a new ninth pose based on the new seventh pose and the eighth pose, and sending the new ninth pose to the multi-axis robot.

7. The robotic positioning, tracking, and guiding method of claim 1, wherein, The first marker and the plurality of markers of the second marker group are preset two-dimensional code images.

8. A robotically positioned and tracked transcranial magnetic stimulation apparatus, comprising: Comprising: an initial calibration module configured to calibrate a multi-axis robot relative to a first camera and a second camera to obtain a first pose of the first camera relative to the multi-axis robot and a second pose of the second camera relative to the multi-axis robot; a marker calibration module configured to calculate a third pose of the first camera relative to a first marker, and calculate a seventh pose of the second camera relative to a coordinate system of a second marker group, and an eighth pose of the first marker relative to the coordinate system of the second marker group; wherein the coordinate system of the second marker group is calculated based on a fourth pose, a fifth pose, and a sixth pose of the second marker group; a first guiding module configured to send a ninth pose of the first marker relative to the multi-axis robot to the multi-axis robot so that an end of the multi-axis robot is close to a position of the first marker; wherein the ninth pose is calculated based on the first pose and the third pose; a second guiding module configured to update the seventh pose when the first marker is blocked, calculate a new ninth pose based on the new seventh pose and the eighth pose, and send the new ninth pose to the multi-axis robot.

9. A computing device, comprising: Comprising: a memory configured to store a computer program; a processor configured to execute the computer program to implement the steps of the robot positioning, tracking, and guiding method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer program is stored on the computer readable storage medium, and when executed by the processor, the computer program implements the steps of the robot positioning, tracking, and guiding method according to any one of claims 1 to 7.

Images