Surgical robot system for simultaneous operation of multiple persons
By designing a collaborative control system with multiple slave tools and main operators, the problem that the existing surgical robot system cannot take into account multiple surgical procedures is solved, and complex surgical operations with multiple people collaborative operation are realized, improving surgical efficiency and flexibility sex.
Patent Information
- Application Number
- CN202311430147.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-05-12
- Filing Date
- 2023-10-31
- Publication Date
- 2024-11-12
AI Technical Summary
The existing surgical robot system cannot take into account multiple surgical procedures at the same time, and in complex operations, multiple medical staff need to operate multiple surgical tools at the same time, resulting in challenges in system design.
A surgical robot system is designed, including multiple slave tools and a main operator, connected to the main operator through a control device to realize the synchronous movement of multiple slave tools, supporting at least four main operators and four slaves. The tool collaborative operation is compatible with single-hole and multi-hole surgical operations, and it adapts to different surgical needs through modular design.
It realizes complex surgical operations with multiple people simultaneously, improves surgical efficiency and flexibility, adapts to the needs of multiple surgical procedures, and enhances the applicability and operation convenience of the system.
Smart Images

Figure CN118924436A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of robot control, and in particular to a surgical robot system for simultaneous operation by multiple persons. Background Art
[0002] With the development of science and technology, the use of surgical robots to assist medical staff in performing surgeries has developed rapidly. Surgical robots can not only help medical staff perform a series of medical diagnoses and auxiliary treatments, but also effectively alleviate the problem of tight medical resources.
[0003] Current surgical robot systems are usually designed for a specific type of surgical operation, such as laparoscopic surgical robots, orthopedic surgical robots, neurosurgery surgical robots, etc., and cannot take into account multiple surgical procedures at the same time. In addition, in some complex surgical operations, multiple medical staff are sometimes required to operate multiple surgical tools at the same time. These pose challenges to the design of surgical robot systems. Summary of the invention
[0004] In some embodiments, the present disclosure provides a surgical robot system, comprising: a plurality of slave tools, the plurality of slave tools including at least a first slave tool, a second slave tool, a third slave tool and a fourth slave tool; a plurality of master manipulators, the plurality of master manipulators including at least a first left-hand master manipulator, a first right-hand master manipulator, a second left-hand master manipulator and a second right-hand master manipulator; and a control device, connected to the plurality of slave tools and the plurality of master manipulators, and configured to control the slave instrument movement of the first slave tool, the second slave tool, the third slave tool and the fourth slave tool based on the movement of the handles of the first left-hand master manipulator, the first right-hand master manipulator, the second left-hand master manipulator and the second right-hand master manipulator. BRIEF DESCRIPTION OF THE DRAWINGS
[0005] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the description of the embodiments of the present disclosure. The drawings described below only illustrate some embodiments of the present disclosure. For ordinary technicians in this field, other embodiments can be obtained based on the contents of the embodiments of the present disclosure and these drawings without creative work.
[0006] Figure 1 A structural block diagram of a surgical robot system according to some embodiments of the present disclosure is shown;
[0007] Figure 2 A schematic diagram showing the structure of a surgical robot system according to some embodiments of the present disclosure is shown;
[0008] Figure 3 A schematic structural diagram of a first main control vehicle according to some embodiments of the present disclosure is shown;
[0009] Figure 4 A schematic diagram showing the structure of a first left-hand master operator according to some embodiments of the present disclosure is shown;
[0010] Figure 5 A schematic diagram showing a first surgical trolley according to some embodiments of the present disclosure;
[0011] Figure 6 A schematic diagram showing a second operating table according to some embodiments of the present disclosure;
[0012] Figure 7 A schematic diagram showing observation of an operating area by a dynamic visual device according to some embodiments of the present disclosure;
[0013] Figure 8 A schematic diagram showing an equipment trolley according to some embodiments of the present disclosure;
[0014] Fig. 9 A schematic diagram showing a dispensing state of a driven tool according to some embodiments of the present disclosure;
[0015] Fig.10 A schematic diagram showing a coordinate system in remote teleoperation according to some embodiments of the present disclosure;
[0016] Fig.11 A schematic diagram showing the allocation status of slave visual devices according to some embodiments of the present disclosure. DETAILED DESCRIPTION
[0017] In order to make the technical problems solved by the present disclosure, the technical solutions adopted and the technical effects achieved more clearly, the technical solutions of the embodiments of the present disclosure will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only exemplary embodiments of the present disclosure, rather than all embodiments.
[0018] In the description of the present disclosure, it should be noted that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present disclosure and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present disclosure. In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In the description of the present disclosure, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", and "coupled" should be understood in a broad sense, for example, it can be a fixed connection or a detachable connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be the internal connection of two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present disclosure can be understood according to the specific circumstances. In the present disclosure, the end away from the operation object (such as a patient) is defined as the proximal end, the near end, the rear end, or the rear end, and the end close to the operation object is defined as the distal end, the far end, the front end, or the front end. Those skilled in the art will appreciate that the embodiments of the present disclosure can be used for medical instruments or surgical robots, and can also be used for other non-medical devices.
[0019] In the present disclosure, the term "position" refers to the positioning of an object or a part of an object in a three-dimensional space (for example, three translational degrees of freedom can be described using changes in Cartesian X, Y and Z coordinates, such as three translational degrees of freedom along the Cartesian X axis, Y axis and Z axis, respectively). In the present disclosure, the term "posture" refers to the rotational setting of an object or a part of an object (for example, three rotational degrees of freedom, which can be described using roll, pitch and yaw). In the present disclosure, the term "pose" refers to a combination of the position and posture of an object or a part of an object, for example, it can be described using six parameters of the six degrees of freedom mentioned above. In the present disclosure, the pose of the handle of the master manipulator can be represented by a set of joint information of the master manipulator joints (for example, a one-dimensional matrix composed of these joint information). The pose of the slave instrument of the slave tool can be determined by the drive information of the slave tool (for example, the drive information of the tool arm of the slave tool). In the present disclosure, the joint information of the joint may include the angle of rotation of the corresponding joint relative to the corresponding joint axis or the distance moved relative to the initial position.
[0020] In the present disclosure, the reference coordinate system can be understood as a coordinate system that can describe the position and posture of an object. According to the actual positioning requirements, the reference coordinate system can choose to use the origin of the virtual reference object or the origin of the physical reference object as the origin of the coordinate system. In some embodiments, the reference coordinate system can be a world coordinate system or a coordinate system of the space where the master operator, the slave tool, the slave visual device, the image acquisition device is located, or the operator's own perception coordinate system, etc.
[0021] In the present disclosure, an object may be understood as an object or target to be positioned, such as a tool arm of a driven tool or the end of a tool arm. The posture of a tool arm or a part thereof (such as the end) may refer to the posture of a coordinate system defined by the tool arm or a part thereof relative to a reference coordinate system.
[0022] Figure 1 FIG. 1 is a block diagram showing a structure of a surgical robot system 100 according to some embodiments of the present disclosure. Figure 1 As shown, the surgical robot system 100 may include a plurality of master manipulators, a plurality of slave tools, and a control device 150. The plurality of master manipulators (e.g., Figure 1 The first left-hand main operator 111a, the first right-hand main operator 111b, the second left-hand main operator 112a, the second right-hand main operator 112b, Figure 2 The first left-hand main operator 211a, the first right-hand main operator 211b, the second left-hand main operator 221a, the second right-hand main operator 221b, Figure 3 The first left-hand main operator 301a, the first right-hand main operator 301b, Figure 4 The first left-hand main operator 400, Fig. 9 The first left-hand main operator 911a, the first right-hand main operator 911b, the second left-hand main operator 912a, the second right-hand main operator 912b or Fig.11 The first left-hand master manipulator 1111a, the first right-hand master manipulator 1111b, the second left-hand master manipulator 1112a, and the second right-hand master manipulator 1112b shown in the figure function as the operation end of the surgical robot system 100, so that multiple operators (for example, two operators) can perform remote operation at the same time. Figure 1 The first driven tool 131, the second driven tool 132, the third driven tool 133, the fourth driven tool 134, Figure 2 The driven tool 231, the fourth driven tool 241, Figure 5 The first driven tool 530, the second driven tool 540, the third driven tool 550, Figure 6 The fourth driven tool 630 shown, Figure 7The first driven tool 710, the second driven tool 720, the third driven tool 730, the fourth driven tool 740, Fig. 9 The first driven tool 931, the second driven tool 932, the third driven tool 933, the fourth driven tool 934, the fifth driven tool 935 or Fig.11 The first slave tool 1131, the second slave tool 1132, the third slave tool 1133, and the fourth slave tool 1134 shown in the figure function as the working end of the surgical robot system 100 to perform surgical operations simultaneously under remote operation. The control device 150 can be connected to multiple master manipulators and multiple slave tools, and based on the movement of the master manipulator, control the movement of the slave tools associated with the master manipulator to achieve surgical operations.
[0023] In some embodiments, the plurality of master operators may include at least four master operators, such as a first left-hand master operator 111a, a first right-hand master operator 111b, a second left-hand master operator 112a, and a second right-hand master operator 112b, so that at least two operators can perform remote operation simultaneously. The plurality of slave tools may include at least four slave tools, such as a first slave tool 131, a second slave tool 132, a third slave tool 133, and a fourth slave tool 134, so as to perform surgical operations simultaneously under remote operation of at least two operators. Those skilled in the art should understand that the number of master operators and slave tools is not limited to the above, for example, the number of master operators and slave tools can be greater than four respectively, and the number of master operators and slave tools can also be inconsistent, and the number of master operators and slave tools of the surgical robot system can be adjusted according to actual needs.
[0024] The plurality of slave tools can be assigned to the first operator who controls the first left-hand master operator and the first right-hand master operator and the second operator who controls the second left-hand master operator and the second right-hand master operator according to the needs. For example, the first slave tool and the second slave tool in the plurality of slave tools can be assigned to the first left-hand master operator and the first right-hand master operator operated by the first operator, and the third slave tool and the fourth slave tool can be assigned to the second left-hand master operator and the second right-hand master operator respectively operated by the second master operator, such as Fig. 9 or Fig.11 As shown. Those skilled in the art should understand that there are multiple allocation methods for multiple slave tools of the surgical robot system to meet different surgical operation requirements. For example, the first slave tool, the second slave tool, the third slave tool and the fourth slave tool among the multiple slave tools can be reallocated according to the allocation request and allocation conditions when they are respectively allocated to the first left-hand master operator, the first right-hand master operator, the second left-hand master operator and the second right-hand master operator. The specific content will be described later.
[0025] In some embodiments, the master manipulator may include a robotic arm (e.g. Figure 4 The robot arm 401 shown in FIG. 401 and a handle (eg, Figure 4 In some embodiments, the robot arm may be a robot arm with multiple degrees of freedom composed of multiple joints, for example, a robot arm that can achieve 4 to 7 degrees of freedom, which will be described in detail later.
[0026] In some embodiments, the slave tool may include a tool arm and a slave instrument disposed at the end of the tool arm. In some embodiments, the tool arm may be a flexible arm, for example, a continuous deformable arm that can be controllably bent. In some embodiments, the slave instrument may be a surgical instrument such as surgical forceps, electric cutters, electric hooks, and electric coagulation. It should be understood that the first to fourth slave tools are not limited to the above-mentioned structures, for example, they may also be straight rod-type slave tools, or passive flexible slave tools. In the surgical robot system, the plurality of slave tools may be a combination of at least one flexible slave tool and one other type of slave tool, for example, a combination of three flexible slave tools and one straight rod-type slave tool. In some embodiments, at least one of the plurality of slave tools (for example, Figure 1 The fourth driven tool 134 shown, Figure 2 The fourth driven tool 241 shown, Figure 6 The fourth driven tool 630 shown, Figure 7 The fourth driven tool 740 is shown Fig. 9 The fourth driven tool 934 or Fig.11 The fourth driven tool 1134) shown may include at least one of a laparoscopic system surgery tool, an orthopedic surgery tool, a neurosurgery tool, a puncture surgery tool, an interventional surgery tool, and a transurethral bladder surgery tool, the details of which will be described later.
[0027] In some embodiments, the control device 150 can be connected to multiple master operators and multiple slave tools, and based on the movement of the handle of at least one of the multiple master operators, control the movement of the slave instrument of at least one of the multiple slave tools to achieve surgical operations. In some embodiments, the control device 150 can generate a control signal for a slave tool associated with the master operator based on the movement of the handles of the multiple master operators, and the control device 150 can be connected to the drive device (such as a servo motor) of the slave tool in communication and send a control signal to the drive device, so that the drive device controls the movement of the slave instruments of the multiple slave tools based on the control signal. For example, the control device 150 can send a control signal to the drive device of the slave tool via the CAN bus to control the slave instrument of the slave tool to move to the corresponding target posture or perform the corresponding surgical operation to achieve surgical operations. It can be understood that the control device 150 can be a single controller with a centralized setting, or can include multiple controllers with a distributed setting.
[0028] In the present disclosure, the surgical robot system 100 may allow a first operator to operate a first left-hand master manipulator 111a and a first right-hand master manipulator 111b to control a slave tool (e.g., Figure 1 The first slave tool 131 and the second slave tool 132 shown in the figure can allow the second operator to operate the second left-hand master operator 112a and the second right-hand master operator 112b to control the slave tools (for example, Figure 1 The third slave tool 133 or the fourth slave tool 134 is shown, thereby realizing complex surgical operations in which at least four slave tools are remotely operated by two operators at the same time, and the details will be described later.
[0029] In the present disclosure, the surgical robot system 100 can be compatible with single-hole surgery, single-multiple-hole combined surgery, or multi-hole surgery. In some embodiments, at least a portion of the tool arms of the first to fourth slave tools 131-134 and the slave instruments disposed at the ends of the tool arms are configured to be able to pass through a surgical connection device (e.g., an incision or a natural opening) formed in the patient's body. Figure 2 The surgical connection device 270 or Figure 5The surgical connection device 570 shown in the figure enters the operating area to perform the surgical operation simultaneously under the remote operation of the first operator and the second operator, realizing a single-hole multi-tool surgical operation. Alternatively, the first to fourth driven tools 131-134 can be configured to enter the operating area through multiple surgical connection devices inserted into multiple openings formed on the patient's body according to surgical needs, so as to perform the surgical operation simultaneously under the remote operation of the first operator and the second operator, realizing a multi-hole multi-tool surgical operation. In some embodiments, the surgical connection device can be, for example, a tube sleeve, a sheath sleeve, etc. Those skilled in the art should understand that the surgical robot system 100 can be applied to special or general robot systems in multiple medical fields, such as a laparoscopic surgical robot system.
[0030] In the present disclosure, the surgical robot system may be a modular surgical robot system. In some embodiments, the surgical robot system may include a main control module for carrying multiple master manipulators and a surgical module for carrying multiple slave tools. The main control module may be, for example, a main control trolley (e.g., Figure 2 The first main control trolley 210, the second main control trolley 220 or Figure 3 The first main control vehicle 300 is used to carry a plurality of main operators (e.g. Figure 1 The surgical module may be, for example, a surgical trolley (e.g., Figure 2 The first operating table 230, the second operating table 240, Figure 5 The first operating table 500 or Figure 6 The second operating table 600 is used to carry a plurality of driven tools (e.g., Figure 1 The first driven tool 131, the second driven tool 132, the third driven tool 133, the fourth driven tool 134, etc. are shown in the figure). In some embodiments, the operating trolley may be provided with a driving device for driving multiple driven tools, and the multiple driven tools are installed on the operating trolley and connected to the driving device, and the specific content is described later. The main control trolley is connected to the operating trolley for controlling the multiple driven tools to perform surgical operations. In some embodiments, the main control trolley and the operating trolley can be connected by wired transmission or wireless transmission.
[0031] In some embodiments, the surgical robot system may include a combination of at least one main control module and at least one surgical module. Figure 1As shown, the surgical robot system may include a main control trolley and an operating trolley. The main control trolley is used to carry a first left-hand main operator, a first right-hand main operator, a second left-hand main operator and a second right-hand main operator for simultaneous operation by the first operator and the second operator. The operating trolley can be used to centrally carry the first to fourth slave tools to perform surgical operations simultaneously under the remote operation of the first operator and the second operator. Alternatively, the surgical robot system may also include two main control trolleys as main control modules, which are respectively operated by the first operator and the second operator, as described later. The surgical robot system may also include multiple (for example, two) operating trolleys as surgical modules, and at least four surgical tools can be dispersed and carried on multiple operating trolleys based on actual needs, as described later. In some embodiments, the surgical robot system may also include an equipment trolley as an auxiliary module, which is used to carry a display device and a positioning sensing unit, etc., as described later.
[0032] Figure 2 FIG. 2 is a schematic diagram showing a structure of a surgical robot system 200 according to some embodiments of the present disclosure. Figure 2 As shown, the surgical robot system 200 may include at least one main control trolley as a main control module, such as a first main control trolley 210 and a second main control trolley 220. In some embodiments, the first main control trolley 210 may be used to carry a first main operator, which may include a first left-hand main operator 211a and a first right-hand main operator 211b, and the second main control trolley 220 may be used to carry a second main operator, which may include a second left-hand main operator 221a and a second right-hand main operator 221b.
[0033] Figure 3 FIG. 3 is a schematic diagram showing the structure of the first main control vehicle 300 of some embodiments of the present disclosure. Figure 3 As shown, the first master control trolley 300 includes a first master operator (e.g., a first left-hand master operator 301a and a first right-hand master operator 301b), a master control trolley display (e.g., displays 302-304) and pedals (e.g., pedals 305-307). Figure 1 The control device 150 shown is respectively connected to the first main operator, the main control trolley display and the pedal for communication, and is used for signal interaction with the first main operator, the main control trolley display and the pedal, and generates corresponding control instructions based on the collected control information.
[0034] In some embodiments, the master control trolley display may include a stereoscopic display 302, a master external display 303, and a master touch display 304. The stereoscopic display 302 is used to display a head image (e.g., a head stereoscopic image) and a system status prompt to a first operator operating the first master control trolley 300, the master external display 303 is used to display a head image and a system status prompt to an external user, and the touch display 304 is used to display a software user interface of the first master control trolley 300. In some embodiments, the image displayed by the stereoscopic display 302 or the master external display 303 may be based on a slave visual device (e.g., Figure 2 The slave visual device 234, Figure 7 The slave visual device 750 or Fig.11 In some embodiments, the pedals of the master console trolley can be used to collect inputs from the feet of the first operator, such as instrument pedals (such as electroresection pedal 305, electrocoagulation pedal 306), clutch pedal 307, field of view pedal (not shown in the figure), switching pedal (not shown in the figure) and other structures. Among them, the instrument pedal is used to control the surgical instrument set at the end of the slave tool, the clutch pedal 307 is used to disconnect the master-slave mapping between the first master operator and the associated surgical tool, and the field of view pedal is used to obtain the slave visual device (for example, Figure 2 The slave visual device 234, Figure 7 The slave visual device 750 or Fig.11 The switching pedal is used to quickly switch control of the surgical tools between multiple surgical tools when a master operator is assigned multiple surgical tools (for example, two surgical tools).
[0035] In some embodiments, Figure 3 As shown, the first master operator may include a first left-hand master operator 301a and a first right master operator 301b corresponding to the left-hand operation of the first operator and the right-hand operation, respectively. In an actual scenario, the first master operator (including the first left-hand master operator 301a and the first right master operator 301b) is used to collect the operation input of the first operator, and the first operator controls the associated slave tool by remotely operating the first master operator, such as the first slave tool (for example, the first left-hand master operator 301a) that establishes a master-slave mapping with the first right-hand master operator 301b. Figure 1 The first slave tool 131 shown in the figure, etc.), and a second slave tool (for example, Figure 1 The second slave tool 132 shown in the figure) or a slave visual device, for example, a slave visual device that establishes a master-slave mapping with the first left-hand master operator 301a and / or the first right master operator 301b (for example, Figure 2The slave visual device 234, Figure 7 The slave visual device 750 or Fig.11 It should be understood that the first left-hand master operator and the first right-hand master operator may have similar structures. The structure of the first master operator is described below by taking the first left-hand master operator as an example.
[0036] Figure 4 FIG. 4 is a schematic diagram showing the structure of the first left-hand master operator 400 according to some embodiments of the present disclosure. Figure 4 The definitions of the coordinate systems in the following are: The main manipulator base coordinate system {M} of the first left-hand master manipulator has its origin at the base of the manipulator arm and is vertically upward. direction, Direction Figure 4 or Fig.10 The handle coordinate system {H} of the first left-hand master manipulator has its origin at the end of the robot arm or on the handle. The axis direction of the end or the axis direction of the handle is consistent, Direction Figure 4 or Fig.10 The reference coordinate system {w} can be the coordinate system of the space where the first left-hand master manipulator is located, such as the master manipulator base coordinate system {M}, or the world coordinate system, such as Figure 4 In some embodiments, the first operator's body feeling can be used as a reference. When the first operator sits in front of the first main control vehicle, the body feeling upward is Direction, the forward direction of the body is Direction. To be understood, for clarity, Figure 4 The handle coordinate system {H} and the handle 402 are shown as separated in FIG. 4 , but they are coincident.
[0037] like Figure 4As shown, the first left-hand main manipulator 400 may include a mechanical arm 401 with multiple degrees of freedom and a handle 402 arranged on the mechanical arm 401. The mechanical arm 401 includes at least one posture joint and at least one position joint. The posture joint is used to control the posture of the handle 402 of the first left-hand main manipulator 400, and the handle 402 is controlled to reach a desired posture through one or more posture joints. The position joint is used to control the position of the handle 402 of the first left-hand main manipulator 400, and the handle 402 is controlled to reach a desired position through one or more position joints. For example, the first left-hand main manipulator 400 may include 7 joints distributed sequentially from the distal end to the proximal end, the distal end of the first left-hand main manipulator 400 may be an end close to the main control trolley (for example, an end connected to the main control trolley), and the proximal end of the first left-hand main manipulator 400 may be an end away from the main control trolley (for example, an end where the handle 402 is arranged). Among them, joint 1, joint 2, joint 5, joint 6 and joint 7 are posture joints, which are used to adjust the posture of the handle 402 of the first left-hand master manipulator 400. Joint 1, joint 2 and joint 3 are position joints, which are used to adjust the position of the handle 402 of the first left-hand master manipulator 400. Joint 4 can be a driven joint of joint 3. Joint 1 and joint 2 can be composite joints that can be used to adjust the position and posture of the handle 402 of the first left-hand master manipulator 400. In some embodiments, the handle 402 can also include a clamp 4012, which can be used to control the opening and closing angle of the driven instrument of at least one driven tool.
[0038] It should be understood that the second master control trolley can have a structure similar to that of the first master control trolley, and the second left-hand main operator and the second right-hand main operator carried on the second master control trolley can have a structure similar to that of the first left-hand main operator and the first right-hand main operator. It should be understood that the structure of the master control trolley and the main operator is not limited to the above, as long as the master control trolley and the main operator that can realize the operation of the slave tool by remote control do not deviate from the scope of the present disclosure. For example, the surgical robot system 200 can also be configured to include only one master control trolley to simultaneously carry the first left-hand main operator and the first right-hand main operator for the first operator to operate, and the second left-hand main operator and the second right-hand main operator for the second operator to operate.
[0039] In some embodiments, Figure 2 As shown, the surgical robot system 200 may include at least one surgical trolley as a surgical module, such as a first surgical trolley 230 and a second surgical trolley 240. The first surgical trolley 230 may be used to carry at least one driven tool 231, such as three driven tools (e.g., Figure 5 As shown in the figure, the first driven tool 530 , the second driven tool 540 and the third driven tool 550 , the second operating table 240 can be used to carry the fourth driven tool 241 .
[0040] In some embodiments, the first surgical trolley 230 may be a master trolley in the surgical robot system 200 , carrying a plurality of slave tools to perform surgical operations. Figure 5 Schematic diagram of a first surgical trolley 500 of some embodiments of the present disclosure is shown. In some embodiments, as Figure 5 As shown, the first surgical trolley 500 may include a trolley body 510, a moving arm 520 and a plurality of driven tools, such as a first driven tool 530, a second driven tool 540 and a third driven tool 550. In some embodiments, the trolley body 510 may include a base 511, a column 512 vertically extending from the base 511 and a crossbeam 513 installed on the top of the column 512. The crossbeam 513 may extend horizontally from the top of the column 512 perpendicular to the height direction of the base 511, and a moving arm 520 for carrying a plurality of driven tools may be rotatably provided at the end of the crossbeam 513. In some embodiments, the column 512 may be raised and lowered in the height direction to adjust the height of the first surgical trolley 500, driving the crossbeam 513, the moving arm 520 and the plurality of driven tools carried on the moving arm 520 to rise and fall as a whole.
[0041] In some embodiments, the motion arm 520 may include a motion arm with multiple degrees of freedom composed of multiple joints. For example, the motion arm 520 may be connected to the crossbeam 513 through a joint at the proximal end (e.g., the end close to the first surgical trolley 500), and may include a mounting platform 527 for carrying multiple driven tools at the distal end (e.g., the end away from the first surgical trolley 500). Figure 5 As shown, the motion arm 520 may include a first cross arm 521 and a first rotation joint. The proximal end of the first cross arm 521 is rotationally connected to the end of the cross beam 513 through the first rotation joint, so that the first cross arm 521 rotates relative to the pan / tilt head 513 around the rotation axis of the first rotation joint (e.g., an axis perpendicular to the horizontal plane). Figure 5 As shown, the motion arm 520 may further include a second cross arm 522 and a second rotation joint. The proximal end of the second cross arm 522 is rotationally connected to the distal end of the first cross arm 521 through the second rotation joint, so that the second cross arm 522 rotates relative to the first cross arm 521 around the longitudinal axis. Figure 5As shown, the motion arm 520 also includes a vertical arm 523 and a linear motion joint. The vertical arm 523 is connected to the distal end of the second horizontal arm 522 through the linear motion joint so as to be raised and lowered relative to the distal end of the second horizontal arm 522 to adjust the length of the vertical arm 523. The vertical arm 523 may also include a vertical arm rotation joint so as to rotate around the longitudinal axis relative to the distal end of the second horizontal arm 522. In some embodiments, the motion arm 520 may also include an RCM (Remote Center of Motion) mechanism. The proximal end of the RCM mechanism is rotationally connected to the distal end of the vertical arm 523, and the distal end is used to connect to the mounting platform 527. The RCM mechanism includes a plurality of arcuate arms, such as a first arcuate arm 524, a second arcuate arm 525 and a third arcuate arm 526, as shown in FIG. Figure 5 The proximal ends and distal ends of the multiple arc-shaped arms are respectively connected in rotation in sequence, and the rotation axes of the multiple arc-shaped arms intersect at the RCM, so as to allow the mounting platform 527 at the distal end to drive the driven instruments of the multiple driven tools to rotate around the RCM. In some embodiments, the vertical arm 523 can be omitted, and the RCM mechanism can be connected in rotation with the second horizontal arm 522.
[0042] In some embodiments, the moving arm 520 may further include a mounting platform 527. The mounting platform 527 is fixedly arranged at the distal end of the RCM mechanism (e.g., the third arc-shaped arm 526). The mounting platform 527 may include a plurality of mounting positions. In some embodiments, the moving arm 520 may further include a plurality of drive devices, which are respectively arranged at a plurality of mounting positions for carrying and driving a plurality of driven tools. Before the surgical operation, a plurality of driven tools may be arranged on the mounting platform 527 in a predetermined distribution. The predetermined distribution of a plurality of driven tools on the mounting platform may be predetermined based on the surgical procedure and stored in the memory of the surgical robot system. It should be understood that the plurality of drive devices on the first driven trolley 500 may have a universal drive interface, so that a plurality of driven tools have installation interchangeability on the mounting platform 527. In some embodiments, a plurality of driven tools may also be installed on the mounting platform 527 in an arbitrary distribution, and the control device may be configured to detect the type of driven tools installed in each mounting position on the mounting platform 527 in response to the driven tools being installed in place, and generate a distribution state of a plurality of driven tools on the mounting platform 527.
[0043] In some embodiments, Figure 5As shown, the plurality of slave tools may include a first slave tool 530, a second slave tool 540, and a third slave tool 550. The first slave tool 530 may include a first tool arm 531 and a first slave instrument 532 disposed at the end of the first tool arm 531, the second slave tool 540 may include a second tool arm 541 and a second slave instrument 542 disposed at the end of the second tool arm 541, and the third slave tool 550 may include a third tool arm 551 and a third slave instrument 552 disposed at the end of the third tool arm 551. In some embodiments, the first operating table may also be equipped with a slave visual device, such as Figure 5 The slave visual device 560 shown in the figure may include an imaging tool arm 561 and an endoscope 562 disposed at the end of the imaging tool arm 561, and the details are described later. In some embodiments, at least a portion of the first tool arm 531 and the first slave instrument 532, at least a portion of the second tool arm 541 and the second slave instrument 542, at least a portion of the third tool arm 551 and the third slave instrument 552, at least a portion of the imaging tool arm 561 and the endoscope 562 are configured to be able to enter the operating area through a surgical connection device 570 (e.g., a sheath), as shown in FIG. Figure 5 As shown in the enlarged area in . In some embodiments, the driven tool may further include a drive transmission device. In some embodiments, the drive transmission device may cooperate with the driving device to drive the tool arm to move. The drive transmission device is used to connect with the driving device, and the driving force of the driving device is transmitted to the tool arm of each driven tool through the drive transmission device, thereby driving the tool arm to achieve multi-degree-of-freedom movement. The driving device may also control the driven instrument of the driven tool to perform a surgical operation.
[0044] In some embodiments, the second operating table 240 can be a secondary table in the surgical robot system 200, which is used to assist the first operating table (for example, Figure 2 The first operating table 230 or Figure 5 The first operating trolley 500 shown in the figure performs the surgical operation. In some embodiments, the second operating trolley can carry a driven tool for laparoscopic surgery, or can carry driven tools of other departments to achieve cross-departmental surgical operations.
[0045] Figure 6 Schematic diagram of a second surgical trolley 600 of some embodiments of the present disclosure is shown. In some embodiments, as Figure 6 As shown, the second operating trolley 600 may be a single-arm operating trolley. The second operating trolley 600 may include a base 610, a moving arm 620, and a fourth driven tool 630. In some embodiments, the moving arm 620 is fixedly or rotatably arranged on the base 610, and the fourth driven tool 631 is detachably arranged at the distal end of the moving arm 620.
[0046] In some embodiments, the motion arm 620 may include a motion arm with multiple degrees of freedom composed of multiple joints. For example, the motion arm 620 may be connected to the base 610 through a joint at the proximal end (e.g., the end close to the second operating table 600), and a driven tool, such as the fourth driven tool 631, may be disposed at the distal end (e.g., the end away from the second operating table 600). Figure 6 As shown, the motion arm 620 may include a vertical arm 621, the proximal end of which may be fixed to the base 610, or the vertical arm 621 may be rotatably connected to the base 610 via a first rotation joint, so that the vertical arm 621 rotates relative to the base 610 around the rotation axis of the first rotation joint (e.g., an axis perpendicular to the horizontal plane). Figure 6 As shown, the motion arm 620 may further include a plurality of swing arms, such as a first swing arm 622, a second swing arm 623, and a third swing arm 624, which are respectively connected to each other in rotation at the proximal and distal ends. In some embodiments, a mounting position may be formed at the end of the third swing arm 624. The motion arm 620 may further include a driving device (not shown in the figure), which is disposed at the mounting position and is used to carry and drive the fourth driven tool 630. The fourth driven tool 630 may have the same structure as the first to third driven tools (for example, Figure 5 The fourth driven tool 630 may include a fourth tool arm 631 and a fourth driven instrument 632 disposed at the end of the fourth tool arm 631. The distal end of the fourth tool arm 631 and the fourth driven instrument 632 are configured to be able to enter the operation area through an opening formed on the patient's body through a sleeve, a sheath, etc. In some embodiments, the fourth driven tool 630 may also include a drive transmission device. The drive transmission device is used to connect with the drive device to transmit the driving force of the drive device to the fourth tool arm 631, thereby driving the fourth tool arm 631 to achieve multi-degree-of-freedom movement. The drive device can also control the fourth driven instrument 632 to perform surgical operations. It should be understood that the fourth driven tool 630 may also be other types of surgical tools suitable for cross-departmental surgical operations, such as straight-rod surgical tools such as puncture surgical tools.
[0047] In some embodiments, the surgical robot system may also include a slave visual device (e.g., Figure 2 The slave visual device 234, Figure 7 The slave visual device 750 or Fig.11 In some embodiments, the slave visual device can be used to capture images of the operating area and transmit the captured images to the master control vehicle. The images are processed by the video processing module in the master control vehicle and displayed on the display of the master control vehicle (for example, Figure 3 The operator obtains the posture of the slave device of at least one slave tool relative to the reference coordinate system in real time through the image in the display. The posture of the image of the slave device of at least one slave tool in the display relative to the reference coordinate system is the posture of the slave device perceived by the operator, and the posture of the handle of at least one master manipulator relative to the reference coordinate system is the posture of the handle perceived by the operator. The posture change felt by the operator through remote operation of the handle of at least one master manipulator and the posture change of the slave device of at least one slave tool perceived by the operator in the display meet the preset posture relationship. In this way, by remotely operating at least one master manipulator, the posture change of the handle of at least one master manipulator is converted into the posture change of the slave device of at least one slave tool based on the preset posture relationship, thereby realizing the posture control of the slave device of at least one slave tool. In this way, when the operator holds the handle of at least one main operator and moves it to operate the slave instrument of at least one associated slave tool, based on the principle of intuitive operation, the posture change of the slave instrument of at least one slave tool felt by the operator is consistent with the posture change of the handle of at least one main operator felt by the operator, which helps to improve the operator's remote operation experience and remote operation accuracy.
[0048] In some embodiments, the slave vision device may include an endoscopic camera for capturing real-time images of an operating area (e.g., a surgical area within a patient's body). The slave vision device may include, but is not limited to, a dual-lens slave vision device or a single-lens slave vision device, such as a binocular or monocular camera. In some embodiments, the slave vision device may implement at least one of visible light band imaging, infrared band imaging, CT (Computed Tomography) imaging, and acoustic wave imaging. Depending on the type of image collected, a person skilled in the art may select different slave vision devices as slave vision devices.
[0049] In some embodiments, the slave visual device can be mounted on the first operating table or the second operating table. For example, the slave visual device can be set at the distal end of the motion arm of the first operating table or the second operating table. Figure 5 As shown, the first slave tool 530, the second slave tool 540, the third slave tool 550 and the slave vision device 560 can be installed on the mounting platform 527 of the first operating trolley 500. Alternatively, the first operating trolley can be used to carry the first to fourth slave tools, and the second slave trolley can be used to carry the slave vision device to replace the fourth slave tool.
[0050] Figure 7Schematic diagram showing observation of the operating area by the slave visual device 750 in some embodiments of the present disclosure. Figure 7 As shown, the observation field of the slave vision device 750 may include at least a portion of the first slave tool 710, the second slave tool 720, the third slave tool 730, and the fourth slave tool 740 located in the operation area. The slave vision device 750 can be used to provide an image of the operation area to the first operator and / or the second operator, so as to allow the first operator and / or the second operator to intuitively operate the slave tool through the master operator to perform the surgical operation while having a field of view of the operation area where the slave tool is located.
[0051] In some embodiments, the slave vision device 750 can be fixed in position or variable in position. For example, the slave vision device 750 can maintain a specific posture after being inserted into the patient's body through a tube sleeve, a sheath sleeve, etc. to provide an image of the operation area, or the slave vision device 750 can adjust its position or posture under the remote operation of the first operator and / or the second operator to meet the field of view requirements of the first operator and / or the second operator. Figure 7 As shown, the slave visual device 750 may include an imaging tool arm 751 and an endoscopic camera 752 disposed at the end of the tool arm 751. In some embodiments, the imaging tool arm 751 may be a flexible arm, for example, a continuous deformable arm that can be controllably bent. In some embodiments, the endoscopic camera 752 may include at least one imaging unit and at least one lighting unit. The imaging unit may be disposed at the front end of the endoscopic camera 752, for example, may be a CCD camera, including a group of image sensors and an image lens. In some embodiments, at least one imaging unit may include a normal imaging unit and a wide-angle imaging unit. The normal imaging unit may be used for imaging in a normal operating mode (e.g., a surgical operating mode) to provide a high-definition image of the operating area. The wide-angle imaging unit has a field of view angle greater than that of the normal imaging unit, and may be used for imaging in a special operating mode (e.g., when switching between multiple surgical tools or looking for dropped objects). The lighting unit may be used to provide lighting so that the imaging unit can capture images.
[0052] In some embodiments, the surgical robot system 200 may further include an equipment cart 260 as an auxiliary module. Figure 8 A schematic diagram of an equipment trolley 800 of some embodiments of the present disclosure is shown. In some embodiments, as Figure 8As shown, the equipment trolley 800 may include a trolley body 810 and a display device 820. In some embodiments, the trolley body 810 may include a base 811 and a bracket 812. The bracket 812 may be, for example, a movable bracket with adjustable posture. The proximal end of the bracket 812 is set on the top of the base 811, and the distal end is used to set the display device 820. The display device 820 may include at least one display. The display may be configured to display the slave visual device (e.g., Figure 2 The slave visual device 234, Figure 7 The slave visual device 750 or Fig.11 In some embodiments, at least one display may also include a touch screen configured to display a user interface for displaying reminder information or receiving input instructions to achieve human-computer interaction with an operator. In some embodiments, the equipment trolley 800 may also be used to place various equipment required during the surgical procedure, such as driven tools (e.g., Figure 1 The first driven tool 131, the second driven tool 132, the third driven tool 133, the fourth driven tool 134, etc.) or a driven visual device (for example, Figure 2 The slave visual device 234, Figure 7 The slave visual device 750 or Fig.11 As shown, the slave visual device 1135) etc.
[0053] In some embodiments, Figure 8 As shown, the equipment trolley 800 may further include at least one positioning sensing unit 830. The positioning sensing unit 830 may be connected to a control device (eg Figure 1 The control device 150 is communicatively connected to the operating room and is used to locate the object in the operating room by sensing the signal and / or collecting the image. The positioning sensing unit may include at least one of an acoustic wave detection unit, a magnetic field detection unit, and an optical signal detection unit.
[0054] For example, the positioning sensing unit may include at least one image acquisition device for acquiring the positioning image of the mobile station. The image acquisition device may include but is not limited to a dual-lens image acquisition device or a single-lens image acquisition device, such as a monocular camera, a binocular camera, a monocular structured light camera, a binocular structured light camera, a TOF (Time offlight) camera, etc. Depending on the application environment, the image acquisition device may be a camera, an industrial camera, etc. In some embodiments, the image acquisition device may realize at least one of visible light band imaging, infrared band imaging, etc. Depending on the type of image acquired, those skilled in the art may select different image acquisition devices as image acquisition devices. In some embodiments, the equipment trolley 800 may include three positioning sensing units 830 for respectively detecting the first operating trolley (for example, Figure 2 The first operating table 230 or Figure 5 The first operating table 500 and the second operating table (eg Figure 2 The second operating table 240 or Figure 6 The second operating table trolley 600 shown) and the operating table (eg Figure 2 The surgical table 280 shown in the figure) or the surgical connection device on the surgical table (e.g. Figure 2 The surgical connection device 270 or Figure 5 The surgical connection device 570 shown in the figure is used for positioning. It will be appreciated by those skilled in the art that the positioning sensing unit is not limited to being arranged on the equipment trolley. The positioning sensing unit may also be integrated into the main control trolley or the surgical trolley, or may be arranged independently. In some embodiments, the positioning sensing unit may also be located outside the surgical robot system, for example, in an operating room environment.
[0055] Those skilled in the art should understand that the surgical robot system is not limited to the above structure and can be actually combined according to surgical needs. In some embodiments, the second operating trolley of the surgical robot can be a visual trolley, which is only used to carry a slave visual device, and a plurality of slave tools can be carried on the first operating trolley to perform surgical operations under the control of the first operator and the second operator. For example, the first operating trolley can be equipped with a plurality of slave tools, such as the first to fourth slave tools, and the second operating trolley can be equipped with a slave visual device. Alternatively, the first operating trolley can be equipped with the first to third slave tools and the slave visual device, and the second operating trolley can be equipped with a fourth slave tool, such as a slave tool of other departments, to achieve cross-departmental surgical operations. In some embodiments, the surgical robot system may also include a total of five or more slave tools to support complex surgical operations.
[0056] In the present disclosure, the surgical robot system further includes a control device (eg Figure 1The control device 150 shown in the figure is configured to control the slave instrument movement of the first slave tool, the second slave tool, the third slave tool and the fourth slave tool based on the movement of the handles of the first left-hand master operator, the first right-hand master operator, the second left-hand master operator and the second right-hand master operator to achieve the surgical operation. It can be understood by those skilled in the art that the control device can be integrated into at least one of the main control trolley, the operating trolley or the equipment trolley of the surgical robot system, or be located outside the surgical robot system.
[0057] Before or during the surgical operation, the first slave tool, the second slave tool, the third slave tool and the fourth slave tool need to be respectively assigned to the first left-hand master operator, the first right-hand master operator, the second left-hand master operator and the second right-hand master operator to establish the association between the multiple master operators and the multiple slave tools, so as to assign the surgical functions of the slave tools to the corresponding master operators. By establishing a master-slave mapping between the master operator and the slave tool with an assignment relationship, multiple slave tools can be allowed to perform surgical operations simultaneously under the master-slave teleoperation control of the first operator and the second operator.
[0058] Fig. 9 Schematic diagram showing the allocation status of slave tools in some embodiments of the present disclosure. In some embodiments, the control device can be configured to allocate the first slave tool, the second slave tool, the third slave tool, and the fourth slave tool to the first left-hand master operator, the first right-hand master operator, the second left-hand master operator, and the second right-hand master operator, respectively, in response to a preset or allocation request. For example, Fig. 9 As shown, the control device can be configured to assign the first slave tool 931, the second slave tool 932, the third slave tool 933 and the fourth slave tool 934 to the first left-hand master operator 911a, the first right-hand master operator 911b, the second left-hand master operator 912c and the second right-hand master operator 912d respectively (the solid arrow in the figure shows the assignment status of at least one master operator and at least one slave tool).
[0059] In some embodiments, the pre-settings may include a plurality of tool allocation modes corresponding to surgical procedures, so as to allow the control device to automatically allocate at least one of the plurality of slave tools to at least one of the plurality of master operators. The tool allocation mode, for example, includes an allocation relationship between at least one of the plurality of slave tools and at least one of the plurality of master operators. For example, the first slave tool, the second slave tool, the third slave tool, and the fourth slave tool may be allocated one-to-one to the first left-hand master operator, the first right-hand master operator, the second left-hand master operator, and the second right-hand master operator. Alternatively, in the case where the number of slave tools is greater than the number of master operators, the tool allocation mode may also include allocating a plurality of slave tools (e.g., two slave tools) to one master operator, and allocating the other slave tools in the plurality of slave tools to the other master operators in the plurality of master operators one-to-one, so as to allow the operator to quickly switch between a plurality of slave tools associated with a master operator, for example, the operator may switch between the plurality of slave tools by stepping on the master control trolley (e.g., Figure 3 The switching pedal on the first master control trolley 300 shown in the figure realizes the rapid switching of the slave tools. Alternatively, when the number of slave tools is greater than the number of master operators, the tool allocation mode may also include allocating the slave tools corresponding to the number of master operators among the multiple slave tools to the multiple master operators one by one, and placing the other slave tools in an unallocated state, so as to allow the first operator or the second operator to issue an allocation request for the unallocated slave tools during the operation.
[0060] In some embodiments, the tool allocation mode can be pre-set based on various surgical procedures and stored in the memory of the surgical robot system. Before the surgical operation, the control device can allocate the first to fourth slave tools to the first left-hand master manipulator, the first right-hand master manipulator, the second left-hand master manipulator, and the second right-hand master manipulator, respectively, based on the tool allocation mode corresponding to the current surgical procedure. In some embodiments, the tool allocation mode can also be pre-set based on the operator's operating habits and stored in the memory of the surgical robot system in association with the operator's identity. Before the surgical operation, the control device can select the corresponding tool allocation mode based on the operator's identity and the current surgical procedure, and allocate the first to fourth slave tools to the first left-hand master manipulator, the first right-hand master manipulator, the second left-hand master manipulator, and the second right-hand master manipulator, respectively. It should be understood that the control device can perform pre-allocation of multiple slave tools before or after the installation of the multiple slave tools. For example, the control device can be configured to pre-allocate the multiple slave tools based on the movement of the multiple slave tools on the moving arm (e.g. Figure 5The control device can also be configured to pre-assign multiple slave tools to multiple master manipulators based on the predetermined distribution on the motion arm (e.g., the first surgical robot 500) and the tool allocation mode corresponding to the current procedure. Alternatively, the control device can also be configured to, after the multiple slave tools are installed on the surgical trolley, assign multiple slave tools to multiple master manipulators based on the predetermined distribution on the motion arm (e.g., the first surgical robot 500) and the tool allocation mode corresponding to the current procedure. Figure 5 The distribution state on the motion arm 520 of the first surgical robot 500 and the tool allocation mode corresponding to the current procedure are used to pre-allocate multiple slave tools to multiple master manipulators.
[0061] In some embodiments, the allocation request may include a preoperative allocation request and an intraoperative allocation request. The preoperative allocation request may be a tool allocation request input by the first operator and / or the second operator before the surgical operation to pre-allocate the first to fourth slave tools to the first left-hand master operator, the first right-hand master operator, the second left-hand master operator, and the second right-hand master operator. The intraoperative allocation request may be a tool allocation request input by the first operator and / or the second operator during the surgical operation to allocate at least one of the first to fourth slave tools to at least one of the first left-hand master operator, the first right-hand master operator, the second left-hand master operator, and the second right-hand master operator.
[0062] In some embodiments, the control device can be configured to determine whether at least one slave tool and / or at least one master operator involved in the allocation request meets the allocation conditions based on the allocation request. In some embodiments, the allocation conditions include that at least one slave tool is not allocated and / or at least one master operator has a higher allocation priority. Those skilled in the art will understand that during a collaborative surgery between a first operator and a second operator, the first operator may be, for example, a primary surgeon who remotely operates the slave tool on the first master control trolley to perform the main surgical operation during the surgery, and the second operator may be, for example, an assistant surgeon who remotely operates the slave tool on the second master control trolley to perform secondary surgical operations to assist the primary surgeon. Therefore, during the surgery, the operating needs and field of view needs of the primary surgeon need to be met first. In the present disclosure, the allocation priority of the master operator means that among the multiple primary operators of the surgical robot system, the primary operator operated by an operator with a high priority has a higher allocation priority. In some embodiments, such as Fig. 9As shown, among the plurality of master operators, the first left-hand master operator 911a and the first right-hand master operator 911b may have a higher allocation priority than the second left-hand master operator 912a and the second right-hand master operator 912b, thereby ensuring that the allocation request of the first operator who is the surgeon in charge is met first. In some embodiments, the first left-hand master operator 911a and the first right-hand master operator 911b may have the same higher allocation priority, and the second left-hand master operator 912a and the second right-hand master operator 912b may have the same lower allocation priority.
[0063] In some embodiments, the control device may be further configured to assign the slave instrument of at least one slave tool to the handle of at least one master operator in response to the assignment condition being met. In some embodiments, the control device may be configured to assign the slave instrument of at least one slave tool to the handle of at least one master operator in response to at least one slave tool involved in the assignment request being in an unassigned state. For example, Fig. 9 As shown by the hollow line arrow in the figure, the control device can be configured to respond to the second operator's allocation request to allocate the fifth slave tool 935 in an unallocated state to the second right-hand master operator 912b, release the allocation relationship between the second right-hand master operator 912b and the fourth slave tool 934 (or disconnect the master-slave mapping), and allocate the fifth slave tool 935 to the second right-hand master operator 912b.
[0064] Before or during a surgical operation, a situation may arise in which an operator issues an allocation request to at least one slave tool that has been assigned to at least one master operator. In some embodiments, the control device may be configured to reallocate a slave tool assigned to a master operator with a lower or same priority to a master operator with a higher or same priority in response to an allocation request from a master operator with a higher or same priority to a slave tool assigned to a master operator with a lower or same priority. For example, when a first operator who is a primary surgeon issues an allocation request to at least one slave tool assigned to a second operator who is an assistant surgeon, the control device may be configured to prompt the second operator and allocate the slave tool to the master operator operated by the first operator. Fig. 9As shown by the dotted arrow in , the control device can be configured to release the allocation relationship between the second left-hand master operator 912a and the third slave tool 933 (or disconnect the master-slave mapping) and allocate the third slave tool 933 to the first left-hand master operator 911a when the first operator issues an allocation request to allocate the third slave tool 933 allocated to the second left-hand master operator 912a operated by the second operator to the first left-hand master operator 911a. Alternatively, the control device can be configured to swap the two slave tools and allocate them to the corresponding master operators when the first operator or the second operator issues an allocation request to interchange the slave tool allocated to the left-hand master operator operated by the operator and the slave tool allocated to the right-hand master operator. For example, Fig. 9 As shown by the dotted arrow, the control device may be configured to release the assignment relationship between the second left-hand master operator 912a and the third slave tool 933 (or disconnect the master-slave mapping) and the assignment relationship between the second right-hand master operator 912b and the fourth slave tool 934 (or disconnect the master-slave mapping) when the second operator issues an assignment request to exchange the third slave tool 933 assigned to the second left-hand master operator 912a and the fourth slave tool 934 assigned to the second right-hand master operator 912b, and assign the third slave tool 933 and the fourth slave tool 934 to the second right-hand master operator 912b and the second left-hand master operator 912a, respectively. In some embodiments, the control device may be configured to reallocate the slave tool assigned to the master operator with a higher priority to the master operator with a lower priority based on the assignment permission in response to the assignment request of the master operator with a lower priority to the slave tool assigned to the master operator with a higher priority. For example, the control device may be configured to send a request to the first operator when the second operator as the assistant surgeon issues an assignment request to at least one slave tool assigned to the first operator as the main surgeon, and when the first operator approves the assignment request, assign the slave tool to the main operator operated by the second operator. Fig. 9 As shown by the double-dotted arrow, the control device can be configured to send a request message to the first operator when the second operator issues an allocation request to allocate the second slave tool 932 allocated to the first right-hand master operator 911b operated by the first operator to the second left-hand master operator 912a, and when the first operator approves the allocation request, release the allocation relationship between the first right-hand master operator 911b and the second slave tool 932 (or disconnect the master-slave mapping), and allocate the second slave tool 932 to the second left-hand master operator 912a.
[0065] During the surgical operation, the operator remotely controls the position and posture of the slave instrument of at least one slave tool by controlling the handle of at least one master operator. It can be understood that when performing remote operation, if the posture (such as orientation or angle) of the handle of the master operator does not match the posture (such as orientation or angle) of the slave instrument of the slave tool, it will make it difficult for the operator to experience an intuitive sense of operation and reduce the operator's control accuracy over the slave tool. Therefore, in some embodiments, the control device can be configured to control at least one master operator to match at least one slave tool having an assignment relationship after at least one slave tool among a plurality of slave tools is assigned to at least one master operator among a plurality of master operators and before establishing a master-slave mapping between at least one master operator and at least one slave tool. In the present disclosure, the matching of the handle of at least one master operator with the slave instrument of at least one slave tool can include matching the posture of the handle of at least one master operator with the posture of the slave instrument of at least one slave tool. In some embodiments, in order to make the posture change felt by the operator through remote operation of the handle of at least one master manipulator consistent with the posture change of the slave instrument of at least one slave tool perceived by the operator in the display, the matching of the handle of at least one master manipulator with the slave instrument of at least one slave tool can include the posture of the handle of at least one master manipulator relative to the reference coordinate system being consistent with the posture of the image of the slave instrument of at least one slave tool in the display relative to the reference coordinate system.
[0066] In some embodiments, the control device can be configured to obtain an initial posture of a slave instrument of at least one slave tool; obtain an initial posture of a handle of at least one master operator that has an assigned relationship with the slave instrument of at least one slave tool; based on the initial posture of the handle of at least one master operator and the initial posture of the slave instrument of at least one slave tool, determine whether the handle of at least one master operator matches the slave instrument of at least one slave tool; and in response to the handle of at least one master operator not matching the slave instrument of at least one slave tool, generate a master operator control signal, the master operator control signal being used to adjust the posture of the handle of at least one master operator to match the posture of the slave instrument of at least one slave tool.
[0067] In some embodiments, the control device can be configured to control at least one master operator to match at least one slave tool having an allocation relationship in response to detecting a matching trigger. In some embodiments, the matching trigger can include the operation of a trigger device provided on the master operator or the master control trolley. The operation of the trigger device can be, for example, toggling a switch on the master operator, touching a sensing position on the master operator, long pressing or tapping a button on the master operator, stepping on a pedal of the master control trolley, operating a display screen of the master control trolley, etc. It should be understood that the matching trigger can be pre-set and stored in the memory of the surgical robot system. In some embodiments, the control device can also be configured to automatically control at least one master operator to match at least one slave tool having an allocation relationship after at least one slave tool among a plurality of slave tools is assigned to at least one master operator among a plurality of master operators.
[0068] Fig.10 A schematic diagram of a coordinate system in remote teleoperation according to some embodiments of the present disclosure is shown. Fig.10 The definitions of the coordinate systems in are as follows: a driven tool base coordinate system {ok} of at least one driven tool, the origin of which is located at the base of at least one driven tool (for example, the driving device of the driven tool, such as Figure 5 as shown) or into the abdominal sheath outlet, It is consistent with the extension line of the base or the axial direction of the abdominal sheath. Direction Fig.10 At least one driven tool has a driven tool coordinate system {g}, the origin of which is located at the end of the tool arm of the driven tool or on the driven tool disposed at the end of the tool arm. The axis of the end or the driven instrument is aligned with the axis of the end. Direction Fig.10 The origin of the slave visual device base coordinate system {o3} is located at the base of the slave visual device (for example, the driving device of the slave visual device) or the outlet of the abdominal sheath. It is consistent with the extension line of the base or the axial direction of the abdominal sheath. Direction Fig.10 The origin of the camera coordinate system {ge} of the slave vision device is located at the end of the tool arm of the slave vision device or is set on the endoscope (for example, on the distal end surface of the endoscope, at the midpoint of the center line of the binocular camera), and the axis of the end or the axis direction of the endoscope is Direction, the upper side is after the field of vision is adjusted The display coordinate system {D} has its origin at the center of the display and its vertical direction inward is Positive direction, the top of the screen is A driven tool image coordinate system {G} of at least one driven tool, the origin of which is located at the end of the tool arm of the driven tool or on the image of the driven tool in the display screen, The axis of the distal image or the axis of the driven instrument image is consistent with the direction of the distal image. Direction Fig.10 At least one master manipulator has a master manipulator base coordinate system {M}, with coordinate axis directions as shown in Figure 4 or Fig.10 At least one main manipulator has a handle coordinate system {H}, with the coordinate axis directions as shown in Figure 4 or Fig.10 The reference coordinate system {w} can be the coordinate system of the space where the master manipulator or the slave tool or the endoscope is located, such as the slave tool base coordinate system {ok}, or the world coordinate system, such as Fig.10 In some embodiments, the operator's body sensation may be used as a reference. When the operator sits in front of the console, the body sensation upward is Direction, the forward direction of the body is Those skilled in the art will appreciate that the coordinate system definition is not limited to the above, and other coordinate system definitions may also be used to implement remote teleoperation of the surgical robot system.
[0069] In some embodiments, the initial posture of the slave instrument of at least one slave tool may be the posture of the image of the slave instrument of at least one slave tool in the display relative to the reference coordinate system {w}, for example, the initial posture of the slave instrument image coordinate system {G} of at least one slave tool relative to the reference coordinate system {w} W R G(i) In some embodiments, the initial pose of the driven tool image coordinate system {G} of at least one driven tool relative to the reference coordinate system {w} W R G(i) The initial posture of the driven tool image coordinate system {G} relative to the display coordinate system {D} of at least one driven tool can be based on D R G(i) And the transformation relationship between the display coordinate system {D} and the reference coordinate system {w} W R D get.
[0070] In some embodiments, in a three-dimensional display imaging relationship, since the field of view of the display coordinate system {D} and the camera coordinate system {ge} are consistent, the initial posture of the driven instrument image coordinate system {G} of at least one driven tool relative to the display coordinate system {D} is D R G(i) The initial pose of the driven tool coordinate system {g} of at least one driven tool relative to the camera coordinate system {ge} of the driven vision device ge Rg(i) In some embodiments, such as Fig.10 As shown, the initial posture of the driven machine coordinate system {g} relative to the camera coordinate system {ge} ge R g(i) It can be based on the initial posture of the driven tool coordinate system {g} relative to the driven tool base coordinate system {ok} ok R g(i) , the transformation relationship between the base coordinate system {ok} of the driven tool and the base coordinate system {o3} of the driven visual device o3 R ok And the transformation relationship between the camera coordinate system {ge} and the base coordinate system {o3} of the slave visual device o3 R ge In some embodiments, the control device can be configured to determine the initial posture of the slave instrument image coordinate system {G} relative to the reference coordinate system {w} based on formula (1): W R G(i) , as the initial posture of the driven tool of at least one driven tool:
[0071] W R G(i) = W R D D R G(i) = W R D ge R g(i) = W R D ( o3 R ge ) T o3 R ok ok R g(i) (1)
[0072] In some embodiments, the control device can be configured to determine the initial posture of the driven tool coordinate system {g} of at least one driven tool relative to the driven tool base coordinate system {ok} based on the driving information of at least one driving device for driving at least one driven tool. ok R g(i) In some embodiments, the control device can be configured to obtain driving information (such as angle) of at least one driven tool through a driving device sensor, and determine the initial posture of the driven tool coordinate system {g} of at least one driven tool relative to the driven tool base coordinate system {ok} based on the driving information. ok R g(i)For example, based on the driving information of at least one driven tool and the kinematic model of the tool arm of at least one driven tool, the initial posture of the driven tool coordinate system {g} relative to the driven tool base coordinate system {ok} can be calculated by a forward kinematics algorithm. ok R g(i) In some embodiments, the drive device sensor may be disposed on the drive device, the drive device is used to drive a tool arm of at least one driven tool, the drive device sensor is used to obtain drive information, and the control device may determine the initial posture of the driven tool coordinate system {g} of at least one driven tool relative to the driven tool base coordinate system {ok} based on the drive information. ok R g(i) For example, the drive device may include at least one drive motor, and the drive device sensor is coupled to the drive motor to record and output motor data. For example, the motor data may include binary or hexadecimal numbers, which can be converted to obtain the initial posture of the driven tool coordinate system {g} relative to the driven tool base coordinate system {ok} ok R g(i) The drive device sensor may include a potentiometer or an encoder. The potentiometer or encoder is used to obtain information such as angles, thereby determining the initial posture of the driven tool coordinate system {g} relative to the driven tool base coordinate system {ok} ok R g(i) In some embodiments, the control device may also be configured to determine an initial posture of a driven tool coordinate system {g} of at least one driven tool relative to a driven tool base coordinate system {ok} based on sensor data of a posture sensor. ok R g(i) The position and posture sensor may be, for example, an optical fiber sensor, which is disposed on a tool arm of at least one driven tool and is used to sense the position and posture of the driven tool relative to a base coordinate system {ok} of the driven tool.
[0073] In some embodiments, the camera coordinate system {ge} has a predetermined transformation relationship with the base coordinate system {o3} of the slave visual device. In some embodiments, before the operator performs the operation, the slave visual device has finished moving, so that the transformation relationship between the camera coordinate system {ge} and the base coordinate system {o3} of the slave visual device is o3 R ge Similar to at least one slave tool, in some embodiments, the control device can also be configured to determine the transformation relationship between the camera coordinate system {ge} and the slave visual device base coordinate system {o3} based on the driving information of the driving device used to drive the slave visual device. o3 R geFor example, based on the driving information of the driving device of the slave vision device and the kinematic model of the tool arm of the slave vision device, the transformation relationship between the camera coordinate system {ge} and the slave vision device base coordinate system {o3} can be calculated by the forward kinematics algorithm. o3 R ge .
[0074] In some embodiments, the slave tool base coordinate system {ok} and the slave vision device base coordinate system {o3} may have a predetermined transformation relationship. Figure 5 As shown, the slave vision device 560 and at least one slave tool (such as the slave tool 530) can be respectively extended into the patient's body through the sheath on the surgical connection device 570. Therefore, the transformation relationship between the slave tool base coordinate system {ok} of at least one slave tool and the slave vision device base coordinate system {o3} can be determined based on the posture relationship between the sheath for passing through at least one slave tool (such as the slave tool 530) and the sheath for passing through the slave vision device 560. o3 R ok In some embodiments, at least one slave tool and a slave visual device may also be respectively arranged on two operating tables. The control device may be configured to determine the transformation relationship between the slave tool base coordinate system {ok} and the slave visual device base coordinate system {o3} of at least one slave tool based on the posture relationship between the two operating tables. o3 R ok , the specific content will be described later.
[0075] In some embodiments, the transformation relationship between the display coordinate system {D} and the reference coordinate system {w} may be predetermined. For example, the display may be fixedly mounted on the console vehicle, and the display coordinate system {D} and the reference coordinate system {w} have a predetermined transformation relationship.
[0076] In some embodiments, the initial posture of the handle of at least one master manipulator having an assigned relationship with the slave instrument of at least one slave tool can be the posture of the handle of at least one master manipulator relative to the reference coordinate system {w}, for example, the initial posture of the handle coordinate system {H} of at least one master manipulator relative to the reference coordinate system {w} W R H(i) In some embodiments, the initial pose of the handle coordinate system {H} of at least one master manipulator relative to the reference coordinate system {w} W R H(i) The initial posture of the handle coordinate system {H} of at least one master manipulator relative to the base coordinate system {M} of the master manipulator can be M R H(i) And the transformation relationship between the main manipulator base coordinate system {M} and the reference coordinate system {w} W R MIn some embodiments, the control device can be configured to determine the initial posture of the handle coordinate system {H} relative to the reference coordinate system {w} based on formula (2): W R H(i) , as the initial posture of the handle of at least one master manipulator having an assigned relationship with the slave instrument of at least one slave tool:
[0077] W R H(i) = W R M M R H(i) (2)
[0078] In some embodiments, the control device may also be configured to obtain initial joint information of at least one posture joint; and determine the initial posture of the handle of at least one master manipulator based on the initial joint information of at least one posture joint. Figure 4 The first left-hand main operator 400 is used as an example. Figure 4 As shown, the first left-hand main manipulator 400 can be provided with a main manipulator sensor (not shown in the figure) at each joint of the mechanical arm 401, and the initial joint information (such as initial joint angle data) is generated by the main manipulator sensor of each joint. In some embodiments, the main manipulator sensor adopts a potentiometer and / or an encoder. In some embodiments, the posture of the first left-hand main manipulator 400 can be represented by a set of joint information of the main manipulator joints (such as a one-dimensional matrix composed of these joint information). In some embodiments, joints 5, 6 and 7 of the mechanical arm 401 are posture joints for adjusting the posture of the mechanical arm. The control device can be configured to determine the initial posture of the handle coordinate system {H} relative to the main manipulator base coordinate system {M} through a forward kinematics algorithm based on the initial joint information obtained by the main manipulator sensor of one or more posture joints and the kinematic model of the mechanical arm of at least one main manipulator. M R H(i) .
[0079] In some embodiments, the transformation relationship between the master manipulator base coordinate system {M} and the reference coordinate system {w} can be predetermined. For example, the master manipulator can be set on a master control trolley. Generally, the master manipulator base coordinate system {M} can be set on a bracket or base where at least one master manipulator is located, and during the teleoperation process, the master manipulator base coordinate system {M} remains unchanged, and the master manipulator base coordinate system {M} has a predetermined transformation relationship with the reference coordinate system {w}.
[0080] In some embodiments, the control device can be configured to determine the degree of posture matching between the handle of at least one master manipulator and the slave instrument of at least one slave tool based on the initial posture of the handle of at least one master manipulator and the initial posture of the slave instrument of at least one slave tool; and based on the posture matching degree, determine whether the handle of at least one master manipulator matches the slave instrument of at least one slave tool. In the present disclosure, the posture matching degree refers to the degree of matching between the posture of the handle of at least one master manipulator and the posture of the slave instrument of at least one slave tool. In some embodiments, the control device can be configured to determine the degree of matching between the posture of the handle of at least one master manipulator and the posture of the slave instrument of at least one slave tool based on the initial posture of the handle coordinate system {H} of at least one master manipulator relative to the reference coordinate system {w}. W R H(i) The initial pose of the driven tool image coordinate system {G} of at least one driven tool relative to the reference coordinate system {w} D R G(i) , determine the posture matching degree between the handle of at least one master manipulator and the slave instrument of at least one slave tool. In some embodiments, the posture matching degree can be represented by the relationship between the posture difference and the threshold, such as the ratio or difference between the threshold and the posture difference.
[0081] The initial pose of the handle coordinate system {H} of at least one master manipulator relative to the reference coordinate system {w} W R H(i) It may include multiple attitude angles, such as the deflection angle of the handle coordinate system {H} of at least one master manipulator around the Y axis in the reference coordinate system {w}, the pitch angle around the X axis in the reference coordinate system {w}, and the roll angle around the Z axis in the reference coordinate system {w}. Similarly, the initial attitude of the slave image coordinate system {G} of at least one slave tool relative to the reference coordinate system {w} D R G(i) It may also include multiple posture angles. In some embodiments, the control device may be configured to determine the posture matching degree between the handle of at least one master operator and the slave of at least one slave tool based on the posture angle of the handle coordinate system {H} of at least one master operator relative to the reference coordinate system {w} and the posture angle of the slave image coordinate system {G} of at least one slave tool relative to the reference coordinate system {w}. In some embodiments, the control device is further configured to determine that the posture of the handle of at least one master operator does not match that of the slave of at least one slave tool in response to the posture matching degree being lower than a matching degree threshold.
[0082] In some embodiments, the control device can also be configured to determine the matching posture of the handle of at least one master operator based on the initial posture of the slave instrument of at least one slave tool in response to the handle of at least one master operator not matching with the slave instrument of at least one slave tool; and generate a master operator control signal based on the initial posture and the matching posture of the handle of at least one master operator.
[0083] In some embodiments, the matching posture of the handle of at least one master manipulator can be a posture for matching (e.g., coinciding) the initial posture of the handle of at least one master manipulator with that of the slave instrument of at least one slave tool, such as the matching posture of the handle coordinate system {H} of at least one master manipulator relative to the master manipulator base coordinate system {M}. M R H(m) In some embodiments, the matching posture of the handle coordinate system {H} of at least one master manipulator relative to the master manipulator base coordinate system {M} M R H(m) The initial posture of the driven tool image coordinate system {G} of at least one driven tool relative to the display coordinate system {D} can be D R G(i) In some embodiments, the control device may be configured to determine an initial posture of the driven tool coordinate system {g} of at least one driven tool relative to the driven tool base coordinate system {ok}. ok R g(i) , determine the matching posture of the handle coordinate system {H} of at least one master manipulator relative to the base coordinate system {M} of the master manipulator M R H(m) In some embodiments, the matching posture of the handle coordinate system {H} of at least one master manipulator relative to the master manipulator base coordinate system {M} M R H(m) It can be determined based on the following formula (13):
[0084] M R H(m) =( ok R o3 o3 R ge D R M ) Tok R g(i) (3)
[0085] In formula (3), D R MRepresents the transformation relationship between the main operator base coordinate system {M} of at least one main operator and the display coordinate system {D}. In some embodiments, the transformation relationship between the at least one main operator and the display can be predetermined, for example, the at least one main operator and the display can be respectively fixedly arranged on the main control trolley, and the main operator base coordinate system {M} of the at least one main operator and the display coordinate system {D} have a predetermined transformation relationship. Alternatively, the transformation relationship between the main operator base coordinate system {M} of the at least one main operator and the display coordinate system {D} can also be obtained based on the transformation relationship between the main operator base coordinate system {M} of the at least one main operator and the reference coordinate system {w} and the transformation relationship between the display coordinate system {D} and the reference coordinate system {w}.
[0086] It should be understood that the control device can also be configured to be based on the initial posture of the handle coordinate system {H} of at least one master operator relative to the base coordinate system {M} of the master operator. M R H(i) and a matching posture of the handle coordinate system {H} of at least one master manipulator relative to the base coordinate system {M} of the master manipulator M R H(m) The degree of matching determines whether the handle of at least one master operator matches the driven instrument of at least one driven tool.
[0087] In some embodiments, the control device is further configured to determine the initial posture of the handle coordinate system {H} of at least one master manipulator relative to the base coordinate system {M} of the master manipulator. M R H(i) The matching posture of the handle coordinate system {H} relative to the main manipulator base coordinate system {M} M R H(m) , determine the posture difference, and based on the posture difference and the inverse kinematics model of the manipulator of at least one master manipulator, generate a master manipulator control signal for controlling the manipulator of at least one master manipulator. In some embodiments, the master manipulator control signal may include a control signal for controlling at least one posture joint of the manipulator. For example, based on the difference between the initial posture and the matching posture of the handle of at least one master manipulator, the joint value of at least one posture joint included in the manipulator (or the drive value of the motor that controls the movement of at least one posture joint of the manipulator) can be determined by the inverse kinematics numerical iteration algorithm of the manipulator kinematic model. It should be understood that the kinematic model can represent a mathematical model of the motion relationship between the joint space and the task space of the manipulator. For example, the kinematic model can be established by methods such as the DH (Denavit-Hartenberg) parameter method and the exponential product representation method.
[0088] In some embodiments, the control device may be configured to determine that the handle of at least one master operator matches the slave instrument of at least one slave tool in response to the posture matching being higher than the matching threshold, and a master-slave mapping between the handle of at least one master operator and the slave instrument of at least one slave tool may be established. In some embodiments, the control device may also be configured to establish a master-slave mapping between the handle of at least one master operator and the slave instrument of at least one slave tool in response to the handle of at least one master operator matching the slave instrument of at least one slave tool. By establishing the master-slave mapping, it is ensured that the posture change of the image of the slave instrument of at least one slave tool felt by the operator and the posture change of the handle of at least one master operator felt by the operator maintain a preset transformation relationship, thereby realizing intuitive operation. In some embodiments, the control device may be configured to, after establishing the master-slave mapping between the handle of at least one master operator and the slave instrument of at least one slave tool, send the control right of the slave instrument of at least one slave tool to the handle of at least one master operator, so as to allow the operator to perform surgical operations through teleoperation.
[0089] In some embodiments, the master-slave mapping of the handle of at least one master manipulator and the slave instrument of at least one slave tool may include the posture relationship between the handle of at least one master manipulator and the slave instrument of at least one slave tool. The posture relationship between the handle of at least one master manipulator and the slave instrument of at least one slave tool may include the position relationship and posture relationship between the handle and the slave instrument. In some embodiments, the position relationship between the handle of at least one master manipulator and the slave instrument of at least one slave tool may include the relationship between the position change amount of the handle and the position change amount of the slave instrument. The posture relationship between the handle of at least one master manipulator and the slave instrument of at least one slave tool may include the relationship between the posture change amount of the handle and the posture change amount of the slave instrument.
[0090] In some embodiments, the control device can be configured to obtain the current posture of the handle of at least one master operator; obtain the previous posture of the handle of at least one master operator; obtain the starting posture of the slave instrument of at least one slave tool; determine the target posture of the slave instrument of at least one slave tool based on the previous posture of the handle of at least one master operator, the current posture of the handle of at least one master operator, the starting posture of the slave instrument of at least one slave tool, and the posture relationship between the handle of at least one master operator and the slave instrument of at least one slave tool; and generate a slave tool control signal based on the target posture of the slave instrument of at least one slave tool, and the slave tool control signal is used to control the movement of the slave instrument of at least one slave tool.
[0091] In some embodiments, the control device can be configured to obtain the current position of the handle of at least one main operator, and the current position includes the current position and the current posture. In some embodiments, the position of the handle of at least one main operator can be the position of the handle of at least one main operator relative to the reference coordinate system {w}, for example, the current position of the handle coordinate system {H} of at least one main operator relative to the reference coordinate system {w}. In some embodiments, the current position of the handle coordinate system {H} of at least one main operator relative to the reference coordinate system {w} can be obtained based on the current position of the handle coordinate system {H} of at least one main operator relative to the main operator base coordinate system {M} and the transformation relationship between the main operator base coordinate system {M} and the reference coordinate system {w}. In some embodiments, obtaining the current position of the handle of at least one main operator includes obtaining the current position of the handle coordinate system {H} of at least one main operator relative to the reference coordinate system {w}, and obtaining the current posture of the handle of at least one main operator includes obtaining the current posture of the handle coordinate system {H} of at least one main operator relative to the reference coordinate system {w}.
[0092] In some embodiments, similar to determining the initial posture of the handle of at least one main manipulator, the control device can also be configured to obtain the current joint information of at least one joint, and determine the current posture of the handle of at least one main manipulator based on the current joint information of at least one joint. In some embodiments, the control device can be configured to obtain the joint information (position or angle) of the corresponding joint based on the main manipulator sensor, and calculate the current posture of the handle of at least one main manipulator. For example, the control device can be configured to calculate the current posture of the handle of at least one main manipulator based on the joint information (such as angle) obtained by the main manipulator sensor of at least one posture joint and the forward kinematics algorithm, and calculate the current position of the handle of at least one main manipulator based on the joint information (such as position) obtained by the main manipulator sensor of at least one position joint and the forward kinematics algorithm.
[0093] In some embodiments, the control device can be configured to obtain a previous position of a handle of at least one master operator, the previous position including a previous position and a previous posture. In some embodiments, obtaining the previous position of the handle of at least one master operator includes obtaining a previous position of a handle coordinate system {H} of at least one master operator relative to a reference coordinate system {w}, and obtaining a previous posture of the handle of at least one master operator includes obtaining a previous posture of a handle coordinate system {H} of at least one master operator relative to a reference coordinate system {w}.
[0094] In some embodiments, similar to determining the initial posture of the handle of at least one main manipulator and obtaining the current posture of the handle of at least one main manipulator, the control device may receive previous joint information of at least one joint of at least one main manipulator, and determine the previous posture of the handle of at least one main manipulator based on the previous joint information of at least one joint. For example, the control device may determine the previous posture and current posture of the handle of at least one main manipulator based on the main manipulator sensor reading the joint information of at least one main manipulator at the previous time and the current time.
[0095] In some embodiments, the control device may be configured to obtain a starting posture of a slave instrument of at least one slave tool, the starting posture including a starting position and a starting attitude. In some embodiments, the starting posture of the slave instrument of at least one slave tool may be a starting posture of the slave instrument of at least one slave tool relative to the slave tool base coordinate system {ok}, for example, a starting posture of the slave instrument coordinate system {g} of at least one slave tool relative to the slave tool base coordinate system {ok}. In some embodiments, obtaining the starting position of the slave instrument of at least one slave tool includes obtaining a starting position of the slave instrument coordinate system {g} of at least one slave tool relative to the slave tool base coordinate system {ok}, and obtaining a starting attitude of the slave instrument of at least one slave tool includes obtaining a starting attitude of the slave instrument coordinate system {g} of at least one slave tool relative to the slave tool base coordinate system {ok}. In some embodiments, the control device may receive the target posture of the slave instrument of at least one slave tool obtained in the previous control cycle as the starting posture of the slave instrument of at least one slave tool in this control cycle. For the first round of control cycle, the initial posture of the driven instrument of at least one driven tool (for example, the zero position of at least one driven tool) can be used as the starting posture of the first round of control cycle.
[0096] In some embodiments, the control device can be configured to determine the amount of change in the position and posture of the handle of at least one master operator based on the previous position and current position of the handle of at least one master operator. For example, the amount of change in the position of the handle of at least one master operator in the reference coordinate system {w} can be determined based on the previous position and current position of the handle coordinate system {H} of at least one master operator relative to the reference coordinate system {w}, and the amount of change in the posture of the handle of at least one master operator in the reference coordinate system {w} can be determined based on the previous posture and current posture of the handle coordinate system {H} of at least one master operator relative to the reference coordinate system {w}.
[0097] In some embodiments, the control device may also be configured to determine the position change of the slave instrument of at least one slave tool based on the position change of the handle of at least one master operator and the position relationship between the handle of at least one master operator and the slave instrument of at least one slave tool. For example, the position change of the slave instrument of at least one slave tool may be determined based on the position change of the handle of at least one master operator and the position relationship between the handle and the slave instrument, and the position change of the slave instrument of at least one slave tool may be determined based on the position change of the handle of at least one master operator and the posture relationship between the handle and the slave instrument.
[0098] In some embodiments, the control device may also be configured to determine the target posture of the slave of at least one slave tool based on the starting posture of the slave of at least one slave tool and the posture change of the slave of at least one slave tool, the target posture including the target position and the target posture. In some embodiments, the target posture of the slave of at least one slave tool may be the target posture of the slave of at least one slave tool relative to the slave base coordinate system {ok}, for example, the target posture of the slave coordinate system {g} of at least one slave tool relative to the slave base coordinate system {ok}. For example, the target position of the slave of at least one slave tool may be determined based on the starting position of the slave of at least one slave tool and the position change of the slave of at least one slave tool, and the target posture of the slave of at least one slave tool may be determined based on the starting posture of the slave of at least one slave tool and the posture change of the slave of at least one slave tool.
[0099] In some embodiments, the posture relationship may include that the position change amount of the image of the slave instrument of at least one slave tool in the display relative to the reference coordinate system {w} is proportional to the position change amount of the handle of at least one master operator relative to the reference coordinate system {w}, and / or, the posture change amount of the image of the slave instrument of at least one slave tool in the display relative to the reference coordinate system {w} is consistent with the posture change amount of the handle of at least one master operator relative to the reference coordinate system {w}. For example, the posture relationship may include that the position change amount of the slave instrument image coordinate system {G} of at least one slave tool relative to the reference coordinate system {w} is proportional to the position change amount of the handle coordinate system {H} of at least one master operator relative to the reference coordinate system {w}, and / or, the posture change amount of the slave instrument image coordinate system {G} of at least one slave tool relative to the reference coordinate system {w} is consistent with the posture change amount of the handle coordinate system {H} of at least one master operator relative to the reference coordinate system {w}, which can be expressed as:
[0100]
[0101] In formula (4),D p G(t) , D R G(t) respectively represent a target position and a target posture of a driven tool image coordinate system {G} relative to a display coordinate system {D} of at least one driven tool, D p G(s) , D R G(s) respectively represent the starting position and starting posture of the driven tool image coordinate system {G} of at least one driven tool relative to the display coordinate system {D}, M p H , M R H Respectively represent the current position and current posture of the handle coordinate system {H} of at least one master manipulator relative to the base coordinate system {M} of the master manipulator, M p H* , M R H* respectively represent the previous position and previous posture of the handle coordinate system {H} of at least one master manipulator relative to the base coordinate system {M} of the master manipulator, W R D represents the transformation relationship between the display coordinate system {D} and the reference coordinate system {w}, W R M represents the transformation relationship between the main manipulator base coordinate system {M} and the reference coordinate system {w}, k S W The master-slave linear proportional coefficient.
[0102] In some embodiments, during the movement of the driven instrument of at least one driven tool, the driven instrument under the endoscope camera and its own image in the display always satisfy a three-dimensional display imaging relationship. The three-dimensional display imaging relationship includes that the posture of the image of the driven instrument of at least one driven tool in the display relative to the display coordinate system {D} is consistent with the posture of the driven tool relative to the camera coordinate system {ge}, and the positions maintain a proportional relationship.
[0103] In some embodiments, the three-dimensional imaging display relationship of the driven instrument of at least one driven tool in the initial position can be expressed as:
[0104]
[0105] In formula (5), ge p g(s) , ge R g(s) represent the starting position and starting posture of the driven tool of at least one driven tool relative to the camera coordinate system {ge}, respectively, ok p g(s) , ok Rg(s) respectively represent the starting position and starting posture of the driven instrument of at least one driven tool relative to the driven tool base coordinate system {ok}, o3 R ge Represents the transformation relationship between the camera coordinate system {ge} and the slave visual device base coordinate system {o3}, o3 R ok represents the transformation relationship between the slave tool base coordinate system {ok} and the slave vision device base coordinate system {o3}, k D Indicates the display imaging ratio.
[0106] In some embodiments, the three-dimensional imaging display relationship of the driven instrument of at least one driven tool in the target posture can be expressed as:
[0107]
[0108] In formula (6), ge p g(t) , ge R g(t) represent the target position and target posture of the driven tool of at least one driven tool relative to the camera coordinate system {ge}, respectively, ok p g(t) , ok R g(t) Respectively represent a target position and a target posture of a driven tool of at least one driven tool relative to a driven tool base coordinate system {ok}.
[0109] By substituting formula (5) and formula (6) into formula (4), the target position of the driven tool of at least one driven tool during the teleoperation process can be obtained: ok p g(t) and target posture ok R g(t) The expression is as follows:
[0110]
[0111] In some embodiments, the control device can be configured to generate a slave tool control signal based on the starting posture and target posture of the slave instrument of at least one slave tool, and the slave tool control signal is used to control the movement of the slave instrument of at least one slave tool. In some embodiments, the control device can also be configured to determine the posture difference based on the starting posture and target posture of the slave instrument of at least one slave tool; and determine the drive signal for controlling the movement of the slave instrument of at least one slave tool based on the posture difference and the inverse kinematics model of the tool arm of at least one slave tool. For example, based on the difference between the target posture and the starting posture of the slave instrument of at least one slave tool in the world coordinate system, the drive values of the multiple joints included in the tool arm in the current motion control cycle (or the drive values of the corresponding multiple motors that control the movement of the operating arm) can be determined by the inverse kinematics numerical iteration algorithm of the tool arm kinematic model. It should be understood that the kinematic model can represent a mathematical model of the motion relationship between the joint space and the task space of the tool arm. For example, the kinematic model can be established by methods such as the DH (Denavit-Hartenberg) parameter method and the exponential product representation method.
[0112] In some embodiments, the driven instrument of at least one driven tool may be iteratively controlled to move to a target posture through a plurality of motion control cycles having a predetermined period.
[0113] In some embodiments, the surgical robot system further includes a slave visual device (e.g., Figure 7 The slave visual device 750 or Fig.11 The slave visual device 1135 shown in the figure can be mounted on the first operating table (for example, Figure 2 The first operating table 230 or Figure 5 The first surgical trolley 500 shown) or the second surgical trolley (eg, Figure 2 The second operating table 240 or Figure 6 The second operating trolley 600 is shown.
[0114] Before or during the operation, the operator may need to remotely operate the slave visual device to adjust the surgical field of view. In some embodiments, the control device may also be configured to respond to the slave visual device allocation request, allocate the slave visual device to the handle of at least one of the multiple master operators; establish a master-slave mapping between the handle of at least one master operator and the slave visual device; disconnect the master-slave mapping between the handle of other master operators in the multiple master operators and the slave instruments of other slave tools in the multiple slave tools; and control the movement of the slave visual device based on the movement of the handle of at least one master operator to change the field of view of the slave visual device.
[0115] Fig.11Schematic diagram showing the allocation state of slave visual devices in some embodiments of the present disclosure. In some embodiments, Fig.11 As shown, the surgical robot system is in a state where the first slave tool 1131, the second slave tool 1132, the third slave tool 1133 and the fourth slave tool 1134 are respectively assigned to the first left-hand master manipulator 1111a, the first right-hand master manipulator 1111b, the second left-hand master manipulator 1112c and the second right-hand master manipulator 1112d (the assignment status of at least one master manipulator and at least one slave tool is shown by solid arrows in the figure).
[0116] In some embodiments, similar to assigning at least one slave tool, the control device can be configured to determine whether at least one master operator involved in the visual tool assignment request satisfies the visual tool assignment condition based on the visual tool assignment request, and in response to satisfying the visual tool assignment condition, assign the slave visual instrument of the slave visual device to the handle of at least one master operator. In some embodiments, the slave visual device can be operated by a master operator alone or by two master operators at the same time. In some embodiments, the visual tool assignment condition can include a higher assignment priority for at least one master operator. It should be understood that during the operation, the operation requirements and field of view requirements of the first operator as the surgeon need to be met first, so the first left-hand master operator and the first right-hand master operator operated by the first operator can have a higher assignment priority than the second left-hand master operator and the second right-hand master operator operated by the second operator.
[0117] In some embodiments, the control device may be configured to allocate the slave visual device to the master operator with a higher priority in response to the master operator with a higher priority requesting the slave visual device to be allocated. Fig.11 As shown by the dashed arrow in the figure, the control device can be configured to allocate the slave visual device 1135 to the first left-hand master operator 1111a and / or the first right-hand master operator 1111b operated by the first operator when the first operator issues a visual tool allocation request to the slave visual device 1135 that is in an unallocated state. Alternatively, the control device can be configured to prompt the second operator and allocate the slave visual device 1135 to the first left-hand master operator 1111a and / or the first right-hand master operator 1111b operated by the first operator when the first operator issues a visual tool allocation request to the slave visual device 1135 that has been allocated to the second operator. In some embodiments, the control device can be configured to allocate the slave visual device to the master operator with a lower priority based on allocation permission in response to an allocation request for the slave visual device by a master operator with a lower priority. For example, Fig.11As shown by the dotted arrow, the control device can be configured to send a request message to the first operator when the second operator issues a visual tool allocation request to the slave visual device 1135 that is in an unassigned state or has been assigned to the first operator, and when the first operator approves the allocation request, the slave visual device is allocated to the second left-hand master operator 1112c and / or the second right-hand master operator 1112d operated by the second operator.
[0118] In some embodiments, the control device may also be configured to control the movement of the slave visual device to change the field of view of the slave visual device based on the movement of the handle of at least one master manipulator. In some embodiments, the control device may be configured to generate a visual tool control signal for controlling the movement of the slave visual instrument of the slave visual device based on the posture change of the handle of at least one master manipulator and the master-slave mapping of the handle of at least one master manipulator and the slave visual device.
[0119] In some embodiments, similar to at least one slave tool, the control device may also be configured to determine whether the handle of at least one master operator matches the slave visual instrument of the slave visual device after assigning the slave visual device to the corresponding at least one master operator. In some embodiments, the control device may also be configured to generate a handle master operator control signal for controlling at least one master operator in response to the handle of at least one master operator not matching the slave visual instrument of the slave visual device; or to establish a master-slave mapping between the handle of at least one master operator and the slave visual instrument of the slave visual device in response to the handle of at least one master operator matching the slave visual device. It will be understood by those skilled in the art that the master-slave mapping between the handle of the master operator and the slave visual instrument may be established without considering the matching between the handle of the master operator and the slave visual instrument of the slave visual device.
[0120] In some embodiments, the slave vision device can be adjusted based on the position change of the handle of at least one master operator. In the case where the slave vision device is operated by two master operators at the same time, the position change of the slave vision instrument of the slave vision device can be the synthesis of the position change of the two master operators, and the posture change can be the synthesis of the posture change of the two master operators. The master-slave mapping of the handle of at least one master operator and the slave vision device can include that the position change of the slave vision instrument of the slave vision device is proportional to the position change of the handle of at least one master operator; or the posture change of the slave vision instrument of the slave vision device is consistent with or proportional to the posture change of the handle of at least one master operator.
[0121] In some embodiments, the control device can also be configured to establish a master-slave mapping in response to the handle of at least one master operator and the slave visual instrument of the slave visual device, and disconnect the master-slave mapping between the handles of other master operators among multiple master operators and the slave instruments of other slave tools among multiple slave tools, so as to prevent the operator from performing non-intuitive surgical operations due to changes in the field of vision, thereby causing surgical accidents.
[0122] After completing the remote operation of the slave visual device, the operator needs to re-operate the slave tool to perform the surgical operation. Since the movement of the slave visual device causes the surgical field of view to change, in order to re-realize intuitive operation, it is necessary to re-match at least one of the multiple slave tools and at least one of the multiple master operators. In some embodiments, the control device can also be configured to restore the allocation relationship between at least one of the multiple slave tools and at least one of the multiple master operators in response to the disconnection of the master-slave mapping of the slave visual device; and in response to the change of the field of view of the slave visual device, control the handle of at least one of the multiple master operators to match the slave instrument of at least one of the multiple slave tools with the allocation relationship. Fig.11 As shown, after the operator confirms that the remote operation of the slave visual device is completed, the control device can disconnect the master-slave mapping between the slave visual device and at least one master operator, and restore the allocation relationship between at least one of the multiple slave tools and at least one of the multiple master operators to the state before the remote operation of the slave visual device. In some embodiments, the control device can be configured to control the handle of at least one of the multiple master operators to match the slave instrument of at least one of the multiple slave tools with the allocation relationship in response to the change in the posture of the slave visual device, and establish a master-slave mapping between the handle of at least one master operator and the slave instrument of at least one slave tool in response to the handle of at least one master operator matching the slave instrument of at least one slave tool.
[0123] In some embodiments, the control device can also be configured to determine whether the slave instruments of multiple slave tools are located within the field of view after the field of view of the slave visual device changes, and in response to the slave instrument of at least one of the multiple slave tools being located outside the field of view, generate prompt information to guide the operator to control the movement of the slave tool to return the slave instrument to the field of view, or to return the slave visual device to the position before the field of view changes, so as to guide the operator to control the slave tool to reach the area after the field of view changes.
[0124] Note that the above are only exemplary embodiments of the present disclosure and the technical principles used. Those skilled in the art will understand that the present disclosure is not limited to the specific embodiments herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present disclosure. Therefore, although the present disclosure is described in more detail through the above embodiments, the present disclosure is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present disclosure, and the scope of the present disclosure is determined by the scope of the appended claims.
Claims
1. A surgical robot system, characterized in that: include: A plurality of driven tools, the plurality of driven tools comprising at least a first driven tool, a second driven tool, a third driven tool and a fourth driven tool; a plurality of master operators, the plurality of master operators comprising at least a first left-hand master operator, a first right-hand master operator, a second left-hand master operator, and a second right-hand master operator; as well as A control device is connected to the multiple slave tools and the multiple master operators, and is configured to control the slave instrument movements of the first slave tool, the second slave tool, the third slave tool and the fourth slave tool based on the movement of the handles of the first left-hand master operator, the first right-hand master operator, the second left-hand master operator and the second right-hand master operator.
2. The surgical robot system according to claim 1, characterized in that: The control device is also configured as: In response to a preset or assignment request, the first, second, third and fourth slave tools are assigned to the first left-hand master operator, the first right-hand master operator, the second left-hand master operator and the second right-hand master operator, respectively.
3. The surgical robot system according to claim 1, characterized in that: The control device is also configured to assign a slave instrument of at least one of the multiple slave tools to a handle of at least one of the multiple master operators in response to satisfying an assignment condition, wherein the assignment condition includes that the at least one slave tool is not assigned and / or the at least one master operator has a higher assignment priority.
4. The surgical robot system according to claim 3, characterized in that: The first left-hand master operator and the first right-hand master operator have higher assignment priorities than the second left-hand master operator and the second right-hand master operator.
5. The surgical robot system according to claim 3, characterized in that: The control device is also configured as: In response to an assignment request from a master operator with a higher or same priority to a slave tool assigned to a master operator with a lower or same priority, reallocating the slave tool assigned to the master operator with a lower or same priority to the master operator with a higher or same priority; or In response to an assignment request from a lower priority master operator for a slave tool assigned to a higher priority master operator, the slave tool assigned to the higher priority master operator is reallocated to the lower priority master operator based on the assignment approval.
6. The surgical robot system according to claim 1, characterized in that: The driven tool comprises a tool arm and a driven instrument disposed at the end of the tool arm, the master operator comprises a mechanical arm and a handle disposed at the end of the mechanical arm, and the control device is further configured as follows: In response to the matching of the handle of at least one master operator among the multiple master operators with the slave instrument of at least one slave tool among the multiple slave tools, a master-slave mapping between the handle of the at least one master operator and the slave instrument of the at least one slave tool is established, and the matching of the handle of the at least one master operator with the slave instrument of the at least one slave tool includes matching the posture of the handle of the at least one master operator with the posture of the slave instrument of the at least one slave tool.
7. The surgical robot system according to claim 6, characterized in that: The control device is also configured as: obtaining an initial posture of a driven tool of the at least one driven tool; obtaining an initial posture of a handle of the at least one master manipulator having an assigned relationship with a slave instrument of the at least one slave tool; Based on the initial posture of the handle of the at least one master operator and the initial posture of the slave instrument of the at least one slave tool, determining whether the handle of the at least one master operator matches the slave instrument of the at least one slave tool; and In response to the handle of the at least one master operator not matching the driven instrument of the at least one slave tool, a master operator control signal is generated, wherein the master operator control signal is used to adjust the posture of the handle of the at least one master operator to match the posture of the driven instrument of the at least one slave tool.
8. The surgical robot system according to claim 7, characterized in that: The control device is also configured as: Determining a posture matching degree between the handle of the at least one master operator and the driven instrument of the at least one driven tool based on the initial posture of the handle of the at least one master operator and the initial posture of the driven instrument of the at least one driven tool; as well as Based on the posture matching degree, it is determined whether the handle of the at least one master operator matches the driven instrument of the at least one driven tool.
9. The surgical robot system according to claim 7, characterized in that: The control device is also configured as: In response to the handle of the at least one master operator not matching the slave instrument of the at least one slave tool, determining a matching posture of the handle of the at least one master operator based on an initial posture of the slave instrument of the at least one slave tool; as well as The master operator control signal is generated based on an initial posture and a matching posture of a handle of the at least one master operator.
10. The surgical robot system according to claim 7, characterized in that: The mechanical arm of the at least one master manipulator comprises at least one posture joint for controlling the posture of a handle of the at least one master manipulator, and the master manipulator control signal comprises a control signal for controlling the at least one posture joint, The control device is also configured as: Obtaining initial joint information of the at least one posture joint; and Based on the initial joint information of the at least one posture joint, an initial posture of the handle of the at least one master manipulator is determined.
11. The surgical robot system according to claim 6, characterized in that: The master-slave mapping between the handle of the at least one master manipulator and the slave instrument of the at least one slave tool comprises a posture relationship between the handle of the at least one master manipulator and the slave instrument of the at least one slave tool, wherein the posture relationship comprises at least one of the following: The position change of the image of the driven instrument of the at least one driven tool in the display relative to the reference coordinate system is proportional to the position change of the handle of the at least one master operator relative to the reference coordinate system; or The amount of posture change of the image of the driven instrument of the at least one driven tool in the display relative to the reference coordinate system is consistent with the amount of posture change of the handle of the at least one master manipulator relative to the reference coordinate system.
12. The surgical robot system according to claim 11, characterized in that: The control device is also configured as: Obtaining a current position and posture of a handle of the at least one master manipulator; obtaining a previous pose of a handle of the at least one master manipulator; Obtaining a starting position of a driven instrument of the at least one driven tool; Determining a target posture of the slave instrument of the at least one slave tool based on a previous posture of the handle of the at least one master operator, a current posture of the handle of the at least one master operator, a starting posture of the slave instrument of the at least one slave tool, and a posture relationship between the handle of the at least one master operator and the slave instrument of the at least one slave tool; as well as Based on the target posture of the driven tool of the at least one driven tool, a driven tool control signal is generated, wherein the driven tool control signal is used to control the movement of the driven tool of the at least one driven tool.
13. The surgical robot system according to claim 1, characterized in that: Also comprising a slave visual device, the control device is further configured to: In response to an assignment request of the slave visual device, assigning the slave visual device to a handle of at least one of the plurality of master operators; Establishing a master-slave mapping between a handle of the at least one master operator and the slave visual device; Disconnecting the master-slave mapping between the handles of other master operators among the plurality of master operators and the slave instrument of at least one slave tool among the plurality of slave tools; as well as Based on the handle movement of the at least one master operator, the slave vision device is controlled to move to change the field of view of the slave vision device.
14. The surgical robot system according to claim 13, characterized in that: The control device is also configured as: In response to the master-slave mapping of the slave visual device being disconnected, restoring the assignment relationship between at least one of the plurality of slave tools and at least one of the plurality of master operators; as well as In response to a change in the field of view of the slave visual device, a handle of at least one of the plurality of master manipulators is controlled to match a slave instrument of at least one of the plurality of slave tools having an assigned relationship.
15. According to any one of claims 1 to 14, a surgical robot system, at least one of the multiple slave tools comprises at least one of a laparoscopic system surgical tool, an orthopedic surgical tool, a neurosurgery tool, a puncture surgical tool, an interventional surgical tool, and a transurethral bladder surgical tool.
Citation Information
Patent Citations
Endoscope system
CN108348134A
Multi-input robotic surgical system control scheme
CN109219413A
Master manipulator clamping mechanism, master manipulator console, surgical robot and master-slave alignment method
CN113768630A
Master-slave motion control method, robot system, equipment and storage medium
CN113876434A
Computer-readable storage medium, electronic apparatus, and surgical robot system
CN114848152A
Cited By
Surgical robot system based on gesture control
CN120859669A
Method for controlling surgical robot system based on gestures and surgical robot system
CN120859670A
Method of gesture-based control of a surgical robotic system and surgical robotic system
CN120859670B
