Robot system and control method

Abstract

The present disclosure relates to the field of robotics, and discloses a robotic system and a control method. The robotic system includes a visual positioning device, a surgical tool base, and a surgical tool, wherein the surgical tool includes a surgical tool end. The control method includes: obtaining information obtained by the visual positioning device, the information including information about an end positioning mark set on the end of the surgical tool, determining the relative posture relationship between the surgical tool end and the visual positioning device based on the information about the end positioning mark, determining the relative posture relationship between the surgical tool end and the surgical tool base based on the relative posture relationship between the surgical tool end and the visual positioning device and the relative posture relationship between the surgical tool base and the visual positioning device, and determining the current posture of the surgical tool end based on the relative posture relationship between the surgical tool end and the surgical tool base. Based on visual feedback, the surgical tool is controlled to move precisely to the target position.

Description

Technical Field

[0001] The present disclosure relates to the field of robotics technology, and in particular to a robotic system and a control method. Background Art

[0002] A surgical robot is a device that requires very high precision and human-computer interaction experience. Existing surgical robots mainly achieve remote motion control of the main control arm through mapping conversion between mechanical movements.

[0003] The existing master-slave mapping control to achieve remote operation may cause a certain difference between the target position and the ideal position due to irresistible factors of the hardware, such as screw clearance, deformation of the drive wire, and differences between the mathematical model and the actual situation. This difference is difficult to predict, especially for flexible multi-segment arms, whose positioning is difficult to confirm by checking the positioning of the joints. Therefore, the closed-loop control of such products has great technical difficulties. Summary of the Invention

[0004] In some embodiments, the present disclosure provides a control method for a robotic system, wherein the robotic system includes a visual positioning device, a surgical tool base and a surgical tool, the surgical tool including a surgical tool end, and the control method includes: obtaining information obtained by the visual positioning device, the information including information about an end positioning mark set on the surgical tool end; determining a relative posture relationship between the surgical tool end and the visual positioning device based on the relative posture relationship between the surgical tool end and the visual positioning device and the relative posture relationship between the surgical tool base and the visual positioning device; and determining a current posture of the surgical tool end based on the relative posture relationship between the surgical tool end and the surgical tool base.

[0005] In some embodiments, the present disclosure provides a robotic system comprising: a visual positioning device for capturing images; at least one surgical tool base; at least one surgical tool comprising a surgical tool end and an end positioning marker disposed on the surgical tool end; and a control device configured to execute the control method as described in the above embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] To more clearly illustrate the technical solutions in the embodiments of the present disclosure, the following briefly describes the drawings required for describing the embodiments of the present disclosure. The drawings described below only illustrate some embodiments of the present disclosure. Those skilled in the art can, without inventive effort, derive other embodiments based on the contents of the embodiments of the present disclosure and these drawings.

[0007] Figure 1 A schematic structural diagram of a robot system according to some embodiments of the present disclosure is shown;

[0008] Figure 2 A flowchart illustrating a control method for a robot system according to some embodiments of the present disclosure is shown;

[0009] Figure 3 A flowchart illustrating a control method for determining a relative posture relationship between a surgical tool base and a visual positioning device according to some embodiments of the present disclosure is shown;

[0010] Figure 4 A schematic diagram showing the auxiliary tool base positioning identification structure of the auxiliary tool base according to some embodiments of the present disclosure is shown;

[0011] Figure 5 A schematic structural diagram of each coordinate system of a robot system according to some embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0012] In order to make the technical problems solved by the present disclosure, the technical solutions adopted and the technical effects achieved more clear, the technical solutions of the embodiments of the present disclosure will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only exemplary embodiments of the present disclosure, rather than all embodiments.

[0013] In the description of the present disclosure, it should be noted that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present disclosure and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present disclosure. In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In the description of the present disclosure, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected" and "coupled" should be understood in a broad sense, for example, it can be a fixed connection or a detachable connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be a communication between the two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present disclosure can be understood according to the specific circumstances. In this disclosure, the end closest to the operator (e.g., doctor) is defined as the proximal end, near portion, or rear end, and the end closest to the patient being operated on is defined as the distal end, far end, or front end. Those skilled in the art will appreciate that the embodiments of this disclosure can be used in medical devices or surgical robots, as well as other non-medical devices.

[0014] In this disclosure, the term "position" refers to the positioning of an object or a portion of an object in three-dimensional space (e.g., three translational degrees of freedom can be described using changes in Cartesian X, Y, and Z coordinates, such as three translational degrees of freedom along the Cartesian X, Y, and Z axes, respectively). In this disclosure, the term "pose" refers to the rotational setting of an object or a portion of an object (e.g., three rotational degrees of freedom, which can be described using roll, pitch, and yaw). In this disclosure, the term "pose" refers to the combination of the position and pose of an object or a portion of an object, such as six of the six degrees of freedom mentioned above.

[0015] For ease of description, spatially relative terms may be used herein to describe the relationship of one element or feature relative to another element or feature as shown in the figures, such as "inside," "outside," "inner side," "outer side," "lower," "upper," etc. Such spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the figures.

[0016] Figure 1 FIG. 1 shows a schematic diagram of the structure of a robot system 100 according to some embodiments of the present disclosure. Figure 1 As shown, the robotic system 100 includes a vision positioning device 10, a surgical tool base 20, and at least one surgical tool 30, wherein the surgical tool 30 includes a surgical tool tip 31. In some embodiments, the robotic system 100 may further include a control device 40. The control device 40 may capture visual information of the surgical tool 30 or the surgical tool base 20 through the vision positioning device 10 and control the movement of the surgical tool tip 31 based on the captured visual information.

[0017] In some embodiments, the surgical tool base 20 may include a position at the distal end of the surgical tool, a sheath, a drive transmission mechanism, or a drive module. It should be understood that the position at the distal end of the surgical tool may include a portion located at the distal end of the surgical tool arm and having an unchanged posture. In some embodiments, the surgical tool 30 may include a flexible surgical tool arm 32 and a surgical tool end 31 disposed at the distal end of the flexible surgical tool arm 32. It should be understood that the surgical tool end 31 may include an end instrument. In some embodiments, the flexible surgical tool arm 32 may include a flexible continuum structure, for example, a flexible multi-segment arm. In some embodiments, an end positioning mark is provided on the surgical tool end 31, which can be used to position the surgical tool end 31.

[0018] In some embodiments, the visual positioning device 10 includes, but is not limited to, a camera, and can be used to capture images. For example, it can capture an image of a distal end positioning marker located on the distal end 31 of a surgical tool. In some embodiments, the robotic system 100 can determine the positional relationship between the visual positioning device 10 and the object where the distal end positioning marker is located by detecting and identifying the distal end positioning marker in the captured image. For example, the positional relationship between the visual positioning device 10 and the object where the distal end positioning marker is located can be determined by, for example, detecting the distal end positioning marker located on the distal end 31 of a surgical tool.

[0019] The present disclosure provides a control method that can be used for a robot system. Figure 2 A flow chart of a control method 200 for a robotic system (e.g., robotic system 100) according to some embodiments of the present disclosure is shown. Method 200 can be performed by a control device (e.g., control device 40) of robotic system 100. Control device 40 is configured on a computing device. Method 200 can be implemented using software and / or hardware.

[0020] like Figure 2 As shown, in step 201, information obtained by the vision positioning device is obtained. In some embodiments, the information may include information about the end positioning mark provided on the end of the surgical tool. In some embodiments, an image captured by the vision positioning device 10 may be obtained, and the image may include an image of the end positioning mark provided on the end of the surgical tool 31. For example, the image captured by the vision positioning device 10 is obtained by the control device 40 of the robotic system 100.

[0021] In step 203, the relative position relationship between the surgical tool tip and the vision positioning device is determined based on the information about the end positioning mark. For example, the relative position relationship between the end positioning mark and the vision positioning device 10 can be determined by identifying the end positioning mark in the image, thereby determining the relative position relationship between the surgical tool tip 31 and the vision positioning device 10.

[0022] In step 205, the relative position relationship between the surgical tool tip 31 and the surgical tool base is determined based on the relative position relationship between the surgical tool tip and the vision positioning device, as well as the relative position relationship between the surgical tool base and the vision positioning device. It should be understood that the position of the surgical tool base 20 remains unchanged, while the position of the surgical tool tip 31 or the vision positioning device 10 may change. The vision positioning device 10 serves as an intermediate conversion to determine the relative position relationship between the surgical tool tip 31 and the surgical tool base 20.

[0023] In some embodiments, a predetermined relative positional relationship between the surgical tool base and the vision positioning device can be obtained to determine the relative positional relationship between the surgical tool base and the vision positioning device. For example, the robotic system 100 may further include a vision positioning device base, with the vision positioning device 10 disposed on the vision positioning device base. The positional relationship between the surgical tool base 20 and the vision positioning device base remains unchanged, and the relative positional relationship between them can be predetermined. In some embodiments, in response to the relative positional relationship between the vision positioning device 10 and the vision positioning device base remaining unchanged, the relative positional relationship between the surgical tool base 20 and the vision positioning device 10 is determined based on the relative positional relationship between the vision positioning device base and the surgical tool base 20. In some embodiments, in response to a change in the relative positional relationship between the vision positioning device 10 and the vision positioning device base, the relative positional relationship between the surgical tool base 20 and the vision positioning device 10 is determined based on the relative positional relationship between the vision positioning device 10 and the vision positioning device base and the relative positional relationship between the vision positioning device base and the surgical tool base 20.

[0024] In some embodiments, the relative position relationship between the surgical tool base and the visual positioning device can be determined based on the tool base positioning mark set on the surgical tool base. Figure 3 The method shown is described in detail.

[0025] In step 207, the current posture of the surgical tool tip is determined based on the relative posture relationship between the surgical tool tip and the surgical tool base. In some embodiments, since the posture of the surgical tool base 20 remains unchanged, the current posture of the surgical tool tip 31 can be determined based on the relative posture relationship between the surgical tool tip 31 and the surgical tool base 20.

[0026] In some embodiments, the robotic system 100 may further include a visual positioning device base. The positional relationship between the surgical tool end and the visual positioning device may be calculated using the following formula:

[0027] toolbase p tip = toolbase R Trackerbase Tracker R Trackerbase T ( Tracker p tip - Tracker p toolbase )

[0028] Wherein, the coordinate systems tip and Trackerbase are the coordinate system of the end of the surgical tool and the base coordinate system of the visual positioning device, respectively. toolbase R Trackerbase and Tracker R Trackerbaseare the relative posture relationships between the visual positioning device base, the surgical tool base, and the visual positioning device base, respectively. Tracker p tip and Tracker p toolbase The relative positional relationships among the surgical tool end 31, the surgical tool base 20 and the visual positioning device 10 are shown respectively.

[0029] Tracker R toolbase T = toolbase R Trackerbase Tracker R Trackerbase T

[0030] Among them, the relationship between the visual positioning device 10 and the visual positioning device base can be obtained based on the kinematic control algorithm data of the intermediate motion arm. There is a set of established kinematic control algorithms for the motion control of any motion arm. This algorithm is the control algorithm that existed at the beginning of the design of the motion arm. Different controllable motion arms have their own unique control algorithms to achieve effective control of the motion arm. This is common knowledge in the field of robotics. Therefore, the mapping of the relative position relationship between the visual positioning device and the visual positioning device base must also be obtained based on the existing kinematic control algorithm. There are many ways to represent and implement the kinematic control algorithm of this type of motion arm. Commonly used methods include DH parameter method and exponential product representation method. This is taken as an example, but is not limited to this.

[0031] It should be understood that determining the positional relationship between the surgical tool end and the visual positioning device, and the positional relationship between the surgical tool end and the surgical tool base is exemplary and not restrictive, and the posture relationship between the surgical tool end and the visual positioning device, and the posture relationship between the surgical tool end and the surgical tool base can also be determined similarly.

[0032] In some embodiments, for example, the positional relationship between the surgical tool tip 31 and the surgical tool base 20 may be calculated using the following formula:

[0033] toolbase p tip = Tracker R toolbase T ( Tracker p tip - Tracker p toolbase )

[0034] Where, Tracker p toolbase and Tracker R toolbaseis the relative position and posture relationship between the surgical tool base and the visual positioning device (or the expression of the surgical tool base coordinate system toolbase in the visual positioning device coordinate system Trcaker), Tracker p tip The relative position relationship between the surgical tool end 31 and the visual positioning device 10 (or the expression of the surgical tool end coordinate system tip in the visual positioning device coordinate system Trcaker), Tracker is the visual positioning device coordinate system, tip is the surgical tool end coordinate system, toolbase is the surgical tool base coordinate system, which can be referred to Figure 5 shown.

[0035] In some embodiments, method 200 further includes step 209. In step 209, it is determined whether the difference between the current posture of the end of the surgical tool and the target posture is less than a threshold value. In some embodiments, the target posture of the end of the surgical tool is determined based on the operation command. For example, the operation command may include a command input by a user through a user interface or an instruction stored on a computer. For example, the user interface may include but is not limited to devices such as buttons, keyboards, touch screens, and microphones. For example, the target posture can also be obtained from the operating hand or input from the control interface. In some embodiments, the instructions stored on the computer can be determined based on a preset computer model. For example, the preset computer model can calculate one or more instructions based on the relative posture relationship between the visual positioning device, the visual positioning device base, the surgical tool base, or the end of the surgical tool.

[0036] In some embodiments, method 200 further includes step 211. In step 211, the surgical tool is controlled to move toward a target pose. For example, in response to a difference between the current pose and the target pose being no less than a threshold, the surgical tool is controlled to move toward the target pose. It should be understood that the threshold can be a pre-set value. Based on the difference being no less than the threshold, the robotic system 100 can instruct the drive module to drive the surgical tool tip 31 toward the target pose based on the current pose and the target pose of the surgical tool tip 31. For example, the drive module can include one or more motors. Target motor drive values for the surgical tool 30 can be calculated based on an inverse kinematics algorithm to generate motor control instructions, which are then sent to the drive module of the surgical tool 30 via a network. The inverse kinematics algorithm can determine the pose of other unknown parts of the motion arm based on the known fixed pose of the motion arm. The drive module receives the drive value data via a network data packet and sends it to the corresponding motor controller via a data transmission bus to drive the corresponding motor. The above method 200 is repeatedly executed until the surgical tool 30 reaches the target pose, thereby causing the surgical tool tip 31 to move to the target pose. In some embodiments, the data transmission bus may include but is not limited to a CAN protocol bus. In some embodiments, the driving module of the surgical tool 30 may include a motor controller, and the surgical tool 30 is coupled to the driving module. It should be understood that the motor controller may be an EPOS controller.

[0037] In some embodiments, method 200 further includes step 213. In step 213, the surgical tool 30 is controlled to stop moving. For example, the surgical tool 30 may be controlled to stop moving in response to a difference between the current posture and the target posture being less than a threshold. It should be understood that a difference between the current posture and the target posture being less than the threshold indicates that the surgical tool 30 has moved to the target posture.

[0038] The control method provided by some embodiments of the present disclosure can achieve closed-loop control of the movement of surgical tools based on the visual feedback of the visual positioning device, so that the surgical tools can be accurately moved to the target position, especially surgical tools that cannot detect the position of the joints, such as flexible multi-segment arms and surgical tools with similar structures.

[0039] Existing surgical robot control methods suffer from structural backlash and model errors, which are difficult to predict, resulting in inaccurate movement of surgical tools to their target positions. The control methods provided by some embodiments of the present disclosure, by comparing the current position of the surgical tool tip with the target position and performing an inverse calculation, enable intuitive, immediate, and repeatable secondary adjustments to the target position, significantly improving movement accuracy.

[0040] Figure 3A flow chart of a control method 300 for determining the relative pose relationship between a surgical tool base and a vision positioning device according to some embodiments of the present disclosure is shown. Method 300 can be executed by a control device (e.g., control device 40) of robotic system 100. Control device 40 can be configured on a computing device. Method 300 can be implemented using software and / or hardware.

[0041] like Figure 3 As shown, in step 301, an image captured by a visual positioning device is acquired.

[0042] In step 303, a determination is made as to whether the acquired information includes information regarding the tool base positioning marker provided on the surgical tool base. It should be understood that the surgical tool base 20 may be located at the distal end of the flexible surgical tool arm 31, and the tool base positioning marker is provided on the surgical tool base 20. For example, the control device 40 of the robotic system 100 acquires an image captured by the visual positioning device 10 to determine whether the image includes the base positioning marker provided on the surgical tool base 20.

[0043] In step 305, the relative positional relationship between the surgical tool base and the vision positioning device is determined based on the information regarding the tool base positioning mark. For example, the tool base positioning mark can be identified by the vision positioning device 10. In response to the acquired image including the tool base positioning mark, the relative positional relationship between the tool base positioning mark and the vision positioning device 10 is determined based on the tool base positioning mark, thereby determining the relative positional relationship between the surgical tool base 20 and the vision positioning device 10.

[0044] In step 307, the relative positional relationship between the auxiliary tool base and the vision positioning device is determined based on information about the auxiliary tool base positioning mark on the auxiliary tool base. For example, in response to the image not including the tool base positioning mark, the relative positional relationship between the auxiliary tool base and the vision positioning device is determined based on the auxiliary tool base positioning mark. Figure 4 A schematic diagram showing the auxiliary tool base positioning identification structure of the auxiliary tool base according to some embodiments of the present disclosure is shown. Figure 5 Schematic diagram showing the structure of each coordinate system of the robot system according to some embodiments of the present disclosure. Figure 4 and Figure 5 As shown, the robot system 100 further includes an auxiliary tool base 50. It should be understood that the position and posture of the auxiliary tool base 50 remain unchanged, and it has a certain relative position and posture relationship with the surgical tool base 20, for example, a known corresponding conversion relationship. In some embodiments, the relative position relationship and relative posture relationship between the surgical tool base 20 and the auxiliary tool base 50 are respectively M p toolbase and M R toolbase(or the expression of the surgical tool base coordinate system toolbase in the auxiliary tool base coordinate system M), the relative position relationship between the auxiliary tool base 50 and the visual positioning device 10 (or the expression of the auxiliary tool base coordinate system M in the visual positioning device coordinate system Trcaker) is Trcaker p M The image captured by the visual positioning device 10 can include the auxiliary tool base positioning mark provided on the auxiliary tool base 50 .

[0045] In step 309, the relative position relationship between the surgical tool base and the visual positioning device is determined based on the determined relative position relationship between the auxiliary tool base and the surgical tool base, as well as the relative position relationship between the auxiliary tool base and the visual positioning device. For example, in response to the image not including the tool base positioning mark, the relative position relationship between the auxiliary tool base 50 and the visual positioning device 10 is determined based on the auxiliary tool base positioning mark. Based on the determined relative position relationship between the auxiliary tool base 50 and the surgical tool base 20, as well as the relative position relationship between the auxiliary tool base 50 and the visual positioning device 10, the relative position relationship between the surgical tool base 20 and the visual positioning device 10 is determined. The auxiliary tool base 50 is used as an intermediate conversion to determine the relative position relationship between the surgical tool base 20 and the visual positioning device 10.

[0046] In some embodiments, the end positioning mark, the tool base positioning mark or the auxiliary tool base positioning mark may include multiple marking points, and the multiple marking points form at least a first line segment and a second line segment that intersect with each other.

[0047] In some embodiments, the relative positional relationship between the plurality of marking points and the visual positioning device is determined based on the distances between the plurality of marking points or the distances between the plurality of marking points and the intersection of the first line segment and the second line segment. The relative positional relationship between the end positioning marker, the tool base positioning marker, or the auxiliary tool base positioning marker and the visual positioning device is determined based on the relative positional relationship between the plurality of marking points and the visual positioning device. And the relative posture relationship between the end positioning marker, the tool base positioning marker, or the auxiliary tool base positioning marker and the visual positioning device is determined based on the relative posture relationship between the first line segment and the second line segment and the visual positioning device.

[0048] In some embodiments, the auxiliary tool base positioning mark may include multiple marking points. Figure 4 As shown, the plurality of marking points may include a first marking point 1, a second marking point 2, a third marking point 3, and a fourth marking point 4. In some embodiments, the first marking point and the third marking point form a first line segment 13, and the second marking point and the fourth marking point form a second line segment 24. The intersection of the first line segment and the second line segment may be the origin O of the auxiliary tool base coordinate system M.

[0049] In some embodiments, the relative position relationship between the auxiliary tool base and the visual positioning device is calculated by the following formula:

[0050]

[0051] Where, Trcaker p i (i = 1, 2, 3, 4) is the relative position relationship between each marker point and the visual positioning device (e.g. Trcaker p1 is the relative position relationship between the first marker and the visual positioning device (or the representation of the first marker in the Trcaker coordinate system of the visual positioning device), which is a 3*1 matrix. The positions of other markers can be analogous. 13 is the distance between the first marking point and the third marking point, L 1O is the distance between the first marking point and the intersection O of the first line segment and the second line segment, L 13 and L 1O This is known information.

[0052] In some embodiments, the relative posture relationship between the auxiliary tool base and the visual positioning device is calculated by the following formula:

[0053]

[0054] In formula (1), are the vectors of the x-axis, y-axis and z-axis in the M coordinate system relative to the Tracker coordinate system.

[0055]

[0056]

[0057]

[0058] In formula (2), The vector normalization is recorded as and It should be understood that Trcaker R M according to Trcaker p i (i=1,2,3,4) coordinate information, identify the long side and the short side, and record the long side vector as The short side vector is The vector normalization is recorded as and The relative posture relationship between the auxiliary tool base and the visual positioning device is determined by calculating the three axes of the auxiliary tool base.

[0059] In some embodiments, the positional relationship between the surgical tool tip and the auxiliary tool base can be calculated using the following formula:

[0060] M p tip = Trcaker R M T ( Trcaker p tip - Trcaker p M )

[0061] Where, M p tip is the positional relationship between the surgical tool tip and the auxiliary tool base (or the expression of the surgical tool tip coordinate system tip in the auxiliary tool base coordinate system M), Tracker p tip is the positional relationship between the end of the surgical tool and the visual positioning device (or the expression of the surgical tool end coordinate system tip in the visual positioning device coordinate system Trcaker), Trcaker p M and Trcaker R M They are respectively the relative position relationship and relative posture relationship between the auxiliary tool base M and the visual positioning device (or the position and posture relationship of the auxiliary tool base coordinate system M in the visual positioning device coordinate system Trcaker).

[0062] In some embodiments, based on the relationship between the auxiliary tool base coordinate system M and the surgical tool base coordinate system toolbase, the current position of the surgical tool end can be calculated using the following formula:

[0063] toolbase p tip = M R toolbase T ( M p tip - M p toolbase ).

[0064] Where, toolbase p tip is the positional relationship between the end of the surgical tool and the base of the surgical tool, M p tip and M p toolbase are the relative positional relationships among the surgical tool end, the surgical tool base, and the auxiliary tool base, M R toolbase It is the relative posture relationship between the surgical tool base and the auxiliary tool base.

[0065] Those skilled in the art will understand that Figure 4The marking points shown and the above formulas can be applied to the end position mark or the tool base position mark.In addition, although in the above description, the position is used as an example, the posture or position of the surgical tool end can be determined similarly.

[0066] In some embodiments, the visual positioning device can utilize various existing spatial information acquisition devices. For example, the visual positioning device can utilize a monocular camera lens, such as a conventional endoscope. In some embodiments, the end positioning mark on the end of the surgical tool, the tool base positioning mark on the surgical tool base, or the auxiliary tool base positioning mark can utilize visual positioning marks with inherent spatial relationships, such as checkerboard marks. It should be understood that the monocular camera lens can also be used in conjunction with other visual positioning marks with inherent spatial relationships besides checkerboard marks. The data with positioning marks collected by the monocular camera lens can be used to calculate the corresponding spatial coordinate system and their mutual relationships through the control computer.

[0067] In some embodiments, the visual positioning device may also utilize a binocular camera lens. In some embodiments, the end positioning mark on the surgical tool end, the tool base positioning mark on the surgical tool base, or the auxiliary tool base positioning mark may utilize a visual positioning mark without a spatial relationship, such as an ordinary small ball. The binocular camera lens can capture the visual positioning mark of the ordinary small ball. The ordinary small ball is a visual positioning mark without a spatial relationship. The binocular camera lens can capture the center of the ball and then obtain its spatial position and establish a three-dimensional spatial coordinate system information. This establishment process can be implemented using the program algorithm provided by the binocular camera lens or by a control computer. This is prior art and is not described in detail here. In some embodiments, the positioning recognition method used by the visual positioning device includes but is not limited to image recognition, and also includes infrared recognition, X-ray fluoroscopy, or magnetic resonance imaging. It should be understood that the visual positioning device may include an infrared recognition device, such as an infrared sensor, to enable recognition of the positioning mark in poor lighting conditions. Alternatively, the visual positioning device may also include an X-ray fluoroscopy device or a magnetic resonance imaging device. For example, X-rays, magnetic resonance imaging, etc. can penetrate obstructions to detect and locate the visual positioning mark, thereby enabling recognition of the positioning mark in the presence of obstructions. In addition, the visual positioning device can also detect auxiliary markers composed of multiple visual positioning markers to define a spatial coordinate system.

[0068] Note that the above are only exemplary embodiments of the present disclosure and the technical principles used. Those skilled in the art will understand that the present disclosure is not limited to the specific embodiments herein, and that various obvious changes, readjustments, and substitutions can be made by those skilled in the art without departing from the scope of protection of the present disclosure. Therefore, although the present disclosure has been described in more detail through the above embodiments, the present disclosure is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present disclosure, and the scope of the present disclosure is determined by the scope of the appended claims.

Claims

1. A control method for a robotic system, the robotic system comprising a visual positioning device, a surgical tool base, and a surgical tool, the surgical tool comprising a surgical tool tip, the control method comprising: obtaining information obtained by the visual positioning device, wherein the information includes information about a terminal positioning mark provided on a terminal end of the surgical tool; Determining a relative position relationship between the surgical tool end and the visual positioning device based on information about the end positioning mark; Determining the relative position relationship between the surgical tool tip and the surgical tool base based on the relative position relationship between the surgical tool tip and the visual positioning device and the relative position relationship between the surgical tool base and the visual positioning device; as well as Determining a current posture of the surgical tool tip based on a relative posture relationship between the surgical tool tip and the surgical tool base; The surgical tool comprises a flexible surgical tool arm and a surgical tool tip located at the distal end of the flexible surgical tool arm, and the surgical tool base comprises a sheath; The control method further includes: Obtaining a predetermined relative posture relationship between the surgical tool base and the vision positioning device, the robotic system further comprising a vision positioning device base, the vision positioning device being arranged on the vision positioning device base, the postures of the surgical tool base and the vision positioning device base being unchanged, and the relative posture relationship between each other being predetermined, and determining the relative posture relationship between the surgical tool base and the vision positioning device based on the relative posture relationship between the vision positioning device and the vision positioning device base and the relative posture relationship between the vision positioning device base and the surgical tool base, wherein the relative posture relationship between the vision positioning device and the vision positioning device base is fixed; or Determining the relative position relationship between the surgical tool base and the visual positioning device based on a tool base positioning mark provided on the surgical tool base or an auxiliary tool base positioning mark provided at a position having a determined relative position relationship with the surgical tool base; The control method further includes: Determining whether a difference between a current posture of the surgical tool end and a target posture is less than a threshold; In response to a difference between the current posture and the target posture being not less than a threshold, controlling the surgical tool tip to move toward the target posture; The control method is executed repeatedly in a loop until the distal end of the surgical tool moves to the target posture.

2. The control method according to claim 1, characterized in that: Also includes: determining whether the information obtained by the visual positioning device includes information about the tool base positioning mark provided on the surgical tool base; In response to the information obtained by the vision positioning device including information about the tool base positioning mark, the relative position relationship between the surgical tool base and the vision positioning device is determined based on the tool base positioning mark.

3. The control method according to claim 2, characterized in that: Also includes: In response to the information obtained by the visual positioning device not including information about the tool base positioning mark, the relative posture relationship between the surgical tool base and the visual positioning device is determined based on the auxiliary tool base positioning mark and the determined relative posture relationship.

4. The control method according to any one of claims 1 to 3, characterized in that: The end positioning mark, the tool base positioning mark or the auxiliary tool base positioning mark includes a plurality of marking points, and the plurality of marking points form at least a first line segment and a second line segment intersecting with each other.

5. The control method according to claim 4, characterized in that: Also includes: Determining a relative positional relationship between the plurality of marking points and the vision positioning device based on distances between the plurality of marking points or distances between the plurality of marking points and intersections of the first line segment and the second line segment; Determine the relative positional relationship between the end positioning mark, the tool base positioning mark or the auxiliary tool base positioning mark and the visual positioning device based on the relative positional relationship between the multiple marking points and the visual positioning device; as well as Based on the relative posture relationship between the first line segment, the second line segment and the visual positioning device, the relative posture relationship between the end positioning mark, the tool base positioning mark or the auxiliary tool base positioning mark and the visual positioning device is determined.

6. The control method according to claim 4, characterized in that: The multiple marking points include a first marking point, a second marking point, a third marking point and a fourth marking point, the first marking point and the third marking point form the first line segment, the second marking point and the fourth marking point form the second line segment, and the relative position relationship between the auxiliary tool base and the visual positioning device is: Trcaker p M , Trcaker p M The calculation formula is as follows: Where, Trcaker p i (i=1,2,3,4) is the relative position relationship between the i-th mark point and the visual positioning device, L 13 is the distance between the first marking point and the third marking point, L 1O is the distance between the first marking point and the intersection of the first line segment and the second line segment.

7. The control method according to claim 6, characterized in that: The relative posture relationship between the auxiliary tool base and the visual positioning device is as follows: In formula (1), are the vectors of the x-axis, y-axis and z-axis in the auxiliary tool base coordinate system M coordinate system relative to the visual positioning device coordinate system Trcaker coordinate system; In formula (2), Denote as the long side vector, Denoted as the short side vector, where the long side and short side are Trcaker R M according to Trcaker p i (i=1,2,3,4) coordinate information, identified long and short sides, and After vector normalization, they are recorded as and 8. The control method according to claim 1, characterized in that: Also includes: The target posture of the surgical tool tip is determined based on an operation command, wherein the operation command includes a command input by a user through a user interface or an instruction stored on a computer.

9. The control method according to any one of claims 1 to 3, characterized in that: The end positioning mark, the tool base positioning mark or the auxiliary tool base positioning mark includes a visual positioning mark with a spatial relationship or a visual positioning mark without a spatial relationship.

10. A robotic system comprising: Visual positioning equipment for capturing images; at least one surgical tool base; At least one surgical tool, comprising a surgical tool end and an end positioning mark provided on the surgical tool end; as well as A control device, wherein the control device is configured to execute the control method according to any one of claims 1 to 9.

11. The robot system according to claim 10, wherein: The surgical tool includes a terminal instrument disposed at a surgical tool terminal located at a distal end of the flexible surgical tool arm.

12. The robot system according to claim 10, wherein: The visual positioning device includes at least one of a monocular camera lens, a binocular camera lens, an infrared recognition device, an X-ray fluoroscopy device or a nuclear magnetic resonance scanning device.

Images