Surgical robot and control device and control method thereof

Abstract

The present invention relates to a surgical robot and its control device and method. The surgical robot includes a driving arm, the distal end of which is used to load and unload an end instrument. The control method comprises: obtaining changes in the morphology of an image model whose structural features and morphology are consistent with those of a first portion of the driving arm; and controlling the movement of corresponding real joint components in the first portion so that the morphology of the first portion changes in accordance with the changes in the morphology of the image model. The surgical robot of the present invention is easy and safe to operate and has a wide range of applications.

Description

[0001] This application is a divisional application submitted to the China Patent Office on December 15, 2020, with application number CN202011472821.7 and application name "Surgical robot, its control device and control method". The full text of the case is incorporated into this application by reference. Technical Field

[0002] The present invention relates to the field of medical devices, and in particular to a surgical robot and a control device and a control method thereof. Background Art

[0003] Minimally invasive surgery refers to a procedure performed inside the human body using modern medical devices such as laparoscopes and thoracoscopes. Compared to traditional surgical methods, minimally invasive surgery offers advantages such as less trauma, less pain, and faster recovery.

[0004] With technological advancements, minimally invasive surgical robotics have matured and are now widely used. A surgical robot consists of a master console and slave operating devices. The slave operating devices include multiple manipulators, including a camera arm with an imaging end-use instrument and a surgical arm with an operating end-use instrument. The master console includes a display and a handle. The surgeon manipulates the handle to control the movement of the camera arm or surgical arm, while viewing the field of view provided by the camera arm on the display.

[0005] However, there are often scenarios where it's inconvenient or even impossible for doctors to operate the slave device. These scenarios include, but are not limited to, scenarios where the shape of at least part of the slave device is required, and scenarios where at least part of the slave device needs to be operated even when a reliable field of view isn't available. In these scenarios, forcibly operating at least part of the slave device can easily lead to high risks. Summary of the Invention

[0006] Based on this, it is necessary to provide a surgical robot and its control device and control method that are easy to operate, safe, and have a wide range of applications.

[0007] On the one hand, the present invention provides a control method for a surgical robot, wherein the surgical robot includes a driving arm, and the distal end of the driving arm is used for loading and unloading end instruments. The control method includes the following steps: obtaining changes in the morphology of an image model whose structural features and morphology are consistent with the first part of the driving arm; controlling the movement of the corresponding real joint component in the first part so that the morphology of the first part changes following the changes in the morphology of the image model.

[0008] Among them, the step of obtaining the change in the morphology of the image model whose structural features and morphology are consistent with the first part of the driving arm includes: obtaining the image model whose structural features and morphology are consistent with the first part of the driving arm and displaying at least part of the image model; obtaining the change in the morphology of the image model.

[0009] The step of displaying at least a portion of the image model is specifically displaying the entire image model.

[0010] In which, the driving arm has multiple real joint components, the image model has at least a virtual joint component corresponding to the real joint component contained in the first part, and the virtual joint component has joint motion parameters consistent with the corresponding real joint component, so that the adjustability of the image model's shape can be consistent with the adjustable performance of the shape of the first part, wherein the joint motion parameters include joint motion range, joint motion speed threshold and joint motion acceleration threshold.

[0011] In the step of controlling the movement of the corresponding real joint component in the first part so that the shape of the first part changes following the change of the shape of the image model, the change of the shape of the first part completely replicates the change of the shape of the image model.

[0012] The change in the shape of the first part completely replicates the change in the shape of the image model, which specifically means that the change in the shape of the first part and the change in the shape of the image model have exactly the same motion state, and the exactly the same motion state includes the same motion trajectory and the same motion speed.

[0013] In the step of controlling the movement of the corresponding real joint component in the first part so that the shape of the first part changes following the change of the shape of the image model, the change in the shape of the first part partially replicates the change in the shape of the image model.

[0014] Among them, the change in the morphology of the first part partially replicates the change in the morphology of the image model, specifically referring to: the change in the morphology of the first part and the change in the morphology of the image model have different motion states, but at least the final forms are the same, and the different motion states include different motion trajectories or different motion speeds.

[0015] Among them, the step of controlling the movement of the corresponding real joint component in the first part so that the shape of the first part changes along with the change of the shape of the image model includes: when it is detected that the set adjustment mode is the intermittent adjustment mode, detecting whether a confirmation instruction is obtained; when the confirmation instruction is obtained, controlling the movement of the corresponding real joint component in the first part so that the shape of the first part changes along with the change of the shape of the image model.

[0016] Among them, the step of controlling the movement of the corresponding real joint component in the first part so that the shape of the first part changes along with the change of the shape of the image model includes: when it is detected that the set adjustment mode is the continuous adjustment mode, controlling the movement of the corresponding real joint component in the first part so that the shape of the first part changes along with the change of the shape of the image model.

[0017] The shape of the first portion changes substantially synchronously with the change of the shape of the image model, or the shape of the first portion changes lagging behind the change of the shape of the image model.

[0018] The change speed of the shape of the first part is lower than the change speed of the shape of the image model.

[0019] Among them, before the step of obtaining the change in the shape of the image model, it includes: receiving an input operation instruction set for changing the shape of the image model; changing the shape of the image model according to the operation instruction set, and at least displaying the final shape of the image model.

[0020] In which, the operation instruction set is associated with the task of the image model in the joint space, and the step of changing the shape of the image model according to the operation instruction set includes: parsing the operation instruction set to obtain the control nodes in the image model and their corresponding directional joint motion amounts; according to the control nodes and their corresponding joint motion amounts, making the corresponding control nodes move the corresponding joint motion amounts to change the shape of the image model.

[0021] The joint motion amount is an incremental joint motion amount.

[0022] Among them, the operation instruction set includes click operation instructions, and the step of parsing the operation instruction set to obtain the corresponding directional joint movement amount of the control node includes: parsing the click operation instruction to obtain the click position, number of clicks and / or click duration; determining the movement direction of the control node according to the click position, and determining the incremental joint movement amount of the control node in the corresponding direction according to the number of clicks and / or the click duration.

[0023] Among them, the control method also includes: generating an icon for being clicked with an adjustable direction of the motion pair of the control node, wherein the click position falling on the icon indicates the selection of the motion direction of the control node, and the number of clicks and / or the click duration indicates setting the incremental joint motion amount of the control node in the corresponding motion direction.

[0024] The joint motion amount is a target joint motion amount.

[0025] In which, the operation instruction set is associated with the task of the image model in the task space, and the step of changing the shape of the image model according to the operation instruction set includes: parsing the operation instruction set to obtain the control node in the image model, the task motion amount of the control node, the task degree of freedom of the farthest virtual joint component and the virtual joint component in the enabled state; according to the task degree of freedom of the farthest virtual joint component and the task motion amount of the control node, controlling the virtual joint component in the enabled state to link so that the control node moves corresponding to the task motion amount to change the shape of the image model.

[0026] The task motion amount is an incremental task motion amount, and the incremental task amount includes an incremental position and an incremental posture.

[0027] Among them, the operation instruction set includes click operation instructions, and the step of parsing the operation instruction set to obtain the task motion amount corresponding to the control node includes: parsing the click operation instruction to obtain the click position, number of clicks and / or click duration; determining the movement direction of the control node according to the click position, and determining the incremental task motion amount of the control node in the corresponding direction according to the number of clicks and / or the click duration.

[0028] In which, the control method includes: at least generating a coordinate image associated with the control node and including an X-coordinate axis, a Y-coordinate axis and a Z-coordinate axis, wherein the click position falling into the corresponding coordinate axis indicates the selection of the movement direction of the control node, and the number of clicks and / or the click duration indicates setting the incremental position of the control node in the corresponding movement direction.

[0029] Among them, the step of at least generating a coordinate image associated with the control node and including an X-coordinate axis, a Y-coordinate axis and a Z-coordinate axis also includes: generating an icon of an adjustable direction associated with each of the coordinate axes in the coordinate image, wherein the click position falling into the corresponding coordinate axis indicates the selection of the rotation direction of the control node, and the number of clicks and / or the click duration indicates setting the incremental posture of the control node in the corresponding motion direction.

[0030] The task motion volume is the target task motion volume.

[0031] Among them, the operation instruction set includes mode configuration instructions, and the mode configuration instructions include one or more of first mode instructions, second mode instructions, third mode instructions and fourth mode instructions. The first mode instruction is used to configure the task degree of freedom to be zero task degree of freedom, the second mode instruction is used to configure the task degree of freedom to be posture degree of freedom, the third mode instruction is used to configure the task degree of freedom to be position degree of freedom, and the fourth mode instruction is used to configure the task degree of freedom to be attitude degree of freedom.

[0032] Among them, when there is one control node and the control node is the remote virtual joint component, all the virtual joint components in the image model are used as the first segment of the virtual arm body, and then the virtual joint components in the enabled state in the first segment of the virtual arm body are controlled to link according to the task degree of freedom, the control node and its task motion amount so that the control node moves corresponding to the task motion amount to change the shape of the image model.

[0033] Among them, when there is one control node and the control node is the farthest virtual joint component, all the virtual joint components in the image model are used as the first section of the virtual arm body, and then the virtual joint components in the enabled state in the first section of the virtual arm body are controlled to link according to the task degree of freedom and the task motion amount of the control node so that the control node moves corresponding to the task motion amount to change the shape of the image model.

[0034] Among them, when there is one control node and the control node is not the farthest virtual joint component, all the virtual joint components at the proximal end of the control node are used as the first segment of the virtual arm, and all the virtual joint components at the far end of the control node are used as the second segment of the virtual arm. The control node belongs to the proximal segment of the virtual arm, and then according to the task degree of freedom and the task motion amount of the control node, the virtual joint components in the enabled state in the first segment of the virtual arm are controlled to move the control node by the task motion amount, and at the same time, the virtual joint components in the enabled state in the second segment of the virtual arm are controlled to be independent of the virtual joint components in the enabled state in the first segment of the virtual arm to realize the task degree of freedom.

[0035] Among them, when there are more than two control nodes and one of them is the farthest virtual joint component, all the virtual joint components are divided into multiple virtual arm bodies with the same number of control nodes, and each control node only belongs to a section of the virtual arm body on the proximal side of the corresponding control node, thereby controlling the virtual joint components in the enabled state in the farthest section of the virtual arm body to link and realize the task motion amount of the farthest control node under the constraint of the task degree of freedom, and at the same time controlling the virtual joint components in the enabled state in other sections of the virtual arm body to link and make the corresponding control nodes move corresponding to the task motion amount, wherein the movements of different virtual arms are relatively independent.

[0036] Among them, when there are more than two control nodes and none of them are the farthest virtual joint components, all the virtual joint components are divided into multiple virtual arm bodies, one more than the number of control nodes, and each control node only belongs to a section of the virtual arm body on the proximal side of its corresponding control node, thereby controlling the virtual joint components in the enabled state in the farthest section of the virtual arm body to jointly realize the task freedom of the farthest control node, and at the same time controlling the virtual joint components in the enabled state in other sections of the virtual arm body to move respectively so that the corresponding control nodes move the corresponding task motion amount, wherein the movements of different virtual arms are relatively independent.

[0037] The virtual joint component that can be configured to be in an enabled state and the virtual joint component that can be configured as the control node correspond to the active joint component in the first part of the driving arm.

[0038] Wherein, before the step of obtaining the change in the morphology of the image model, the method includes: zooming in and displaying at least a portion of the image model.

[0039] The control method further includes: generating a user interface having an angle switching control for switching the display angle of the image model, and then switching the display angle of the image model according to an angle switching instruction generated by triggering the angle switching control.

[0040] The portion of the image model whose shape can be changed corresponds to the portion of the first portion of the driving arm that is within the field of view of the image end instrument in the surgical robot.

[0041] Wherein, the image model is a computer image model or a projection image model.

[0042] Among them, before the step of controlling the movement of the corresponding real joint component in the first part so that the shape of the first part changes with the change of the shape of the image model, it includes: obtaining the joint variables corresponding to the virtual joint component that causes the shape of the image model to change; performing motion smoothing processing on the joint variables corresponding to the virtual joint component to obtain joint variables after motion smoothing processing; the step of controlling the movement of the corresponding real joint component in the first part so that the shape of the first part changes with the change of the shape of the image model is specifically: controlling the movement of the real joint component corresponding to the virtual joint component in the first part according to the joint variables corresponding to the virtual joint component after motion smoothing processing so that the shape of the first part changes with the change of the shape of the image model, and at the same time making the movement of the first part continuous.

[0043] Among them, the step of performing motion smoothing processing on the joint variables corresponding to the virtual joint component to obtain the joint variables after motion smoothing processing is performed before changing the shape of the image model according to the operation instruction set, so that the movement of the image model is continuous and the movement of the first part is continuous.

[0044] Among them, the step of performing motion smoothing processing on the joint variables corresponding to the virtual joint component to obtain joint variables after motion smoothing processing is performed after changing the shape of the image model according to the operation instruction set, so as to make the movement of only the first part continuous.

[0045] The motion smoothing process includes filtering and / or trajectory interpolation; and the motion continuity includes one or more of motion position continuity, motion speed continuity, and motion acceleration continuity.

[0046] On the other hand, the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program is configured to be loaded and executed by a processor to implement the steps of the control method as described in any of the above embodiments.

[0047] On the other hand, the present invention provides a control device for a surgical robot, comprising: a memory for storing a computer program; and a processor for loading and executing the computer program; wherein the computer program is configured to be loaded and executed by the processor to implement the steps of the control method described in any of the above embodiments.

[0048] On the other hand, the present invention provides a surgical robot comprising: a driving arm; a display for displaying an image model; an input device for manipulating the morphological changes of the image model; and a controller coupled to the driving arm, the display and the motion input device, and configured to execute the steps of the control method described in any of the above embodiments.

[0049] Among them, the driving arm includes a robotic arm and an operating arm, the proximal end of the operating arm is installed at the distal end of the robotic arm, the end instrument is installed at the distal end of the operating arm, the first part is the operating arm, or the first part is the robotic arm and the operating arm.

[0050] Wherein, the driving arm includes a robotic arm, an adjustment arm, a manipulator and an operating arm, the proximal end of the adjustment arm is mounted on the distal end of the robotic arm, the proximal end of the manipulator is mounted on the distal end of the adjustment arm, the proximal end of the operating arm is mounted on the distal end of the manipulator, the end instrument is mounted on the distal end of the operating arm, the first part is the operating arm, or the first part is the manipulator and the operating arm, or the first part is the robotic arm, the adjustment arm, the manipulator and the operating arm.

[0051] Wherein, the input device is one or more of a motion input device, a touch screen, and a mouse.

[0052] The surgical robot and its control device and control method of the present invention have the following beneficial effects:

[0053] By obtaining the change in the shape of the image model that is consistent with the structural characteristics and initial shape of the first part in the driving arm, the shape of the first part is controlled to change following the change in the shape of the image model. The control is convenient, safe, and has a wide range of applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 This is a schematic structural diagram of an embodiment of a surgical robot according to the present invention;

[0055] Figure 2 for Figure 1 A partial schematic diagram of an embodiment of a surgical robot is shown;

[0056] Figure 3 This is a schematic diagram of the structure of the operating arm and power unit of the surgical robot;

[0057] Figure 4 This is a schematic structural diagram of another embodiment of the surgical robot of the present invention;

[0058] Figure 5 for Figure 1 The configuration interface of an embodiment of the first configuration part of the surgical robot is shown;

[0059] Figure 6 for Figure 1 A configuration interface of another embodiment of the first configuration part of the surgical robot is shown;

[0060] Figures 7 to 11 They are respectively flow charts of a control method of a surgical robot according to an embodiment of the present invention;

[0061] Figure 12 Schematic diagram of the first portion of the driving arm of a surgical robot according to an embodiment of the present invention;

[0062] Figure 13 Schematic diagram of the image model of an embodiment of a surgical robot according to the present invention;

[0063] Figure 14 A schematic diagram of another form of the image model of an embodiment of the surgical robot of the present invention;

[0064] Figure 15 A schematic diagram of another embodiment of the first portion of the driving arm of the surgical robot according to the present invention;

[0065] Figure 16 This is a flow chart of a method for controlling a surgical robot according to an embodiment of the present invention;

[0066] Figures 17 and 18 They are schematic diagrams of the image model of an embodiment of the surgical robot of the present invention;

[0067] Figures 19 to 22 They are respectively flow charts of a control method of a surgical robot according to an embodiment of the present invention;

[0068] Figure 23 for Figure 22 A schematic diagram of a display interface of an embodiment of a control method for a surgical robot is shown;

[0069] Figure 24 2 is a schematic structural diagram of a control device for a surgical robot according to an embodiment of the present invention.

[0070] Figure 25 2 is a schematic structural diagram of a control device for a surgical robot according to an embodiment of the present invention. DETAILED DESCRIPTION

[0071] To facilitate understanding of the present invention, the present invention will be described more fully below with reference to the accompanying drawings. The accompanying drawings illustrate preferred embodiments of the present invention. However, the present invention may be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and comprehensive understanding of the disclosure of the present invention.

[0072] It should be noted that when an element is referred to as being "disposed on" another element, it may be directly on the other element or there may also be an element centered. When an element is considered to be "connected" to another element, it may be directly connected to the other element or there may be an element centered at the same time. When an element is considered to be "coupled" to another element, it may be directly coupled to the other element or there may be an element centered at the same time. The terms "vertical", "horizontal", "left", "right" and similar expressions used in the present invention are for illustrative purposes only and do not represent the only implementation method. The terms "distal end" and "proximal end" used in the present invention are used as directional words, which are commonly used terms in the field of interventional medical devices, where "distal end" refers to the end away from the operator during surgery, and "proximal end" refers to the end close to the operator during surgery. The terms "first / second" and the like used in the present invention represent a component and two or more components of a type having common characteristics.

[0073] Unless otherwise defined, all technical and scientific terms used herein have the same meanings as commonly understood by those skilled in the art to which this invention pertains. The terms used herein are for the purpose of describing specific embodiments only and are not intended to limit the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items. The term "each" as used herein includes one or more than one.

[0074] like Figures 1 to 2 As shown in FIG, they are respectively a structural schematic diagram of an embodiment of a surgical robot of the present invention and a partial schematic diagram thereof.

[0075] The surgical robot includes a master console 2 and a slave operating device 3 controlled by the master console 2. The master console 2 has a motion input device 21 and a display 22. The surgeon operates the motion input device 21 to send control commands to the slave operating device 3, causing the slave operating device 3 to perform corresponding operations based on the surgeon's control commands. The surgeon also observes the surgical area through the display 22. The slave operating device 3 includes a drive arm, which includes a robotic arm 30 and one or more manipulator arms 31 detachably mounted at the distal end of the robotic arm 30. The robotic arm 30 includes a base and a connection assembly, each of which has multiple joint assemblies. The manipulator arm 31 includes a connecting rod 32, a connection assembly 33, and an end-use instrument 34, each of which has multiple joint assemblies. The end-use instrument 34 is adjusted by adjusting the joint assemblies of the manipulator arm 31. The end-use instrument 34 includes an imaging end-use instrument 34A and a manipulation end-use instrument 34B. The imaging end-use instrument 34A is used to capture images within the field of view, and the display 22 is used to display these images. The operating end instrument 34B is used to perform surgical operations such as cutting and suturing. In this paper, the operating arm with the image end instrument 34A is referred to as the camera arm 31A, and the operating arm with the operating end instrument 34B is referred to as the surgical arm 31B.

[0076] Figure 1 The surgical robot on display is a single-port surgical robot, in which each operating arm 31 is inserted into the patient's body through the same puncture device 4 installed at the distal end of the robotic arm 30. In a single-port surgical robot, the doctor generally only controls the operating arm 31 to complete basic surgical operations. At this time, the operating arm 31 of the single-port surgical robot should have both positional freedom (i.e., positioning freedom) and posture freedom (i.e., orientation freedom) to achieve changes in posture within a certain range. For example, the operating arm 31 has horizontal movement freedom x, vertical movement freedom y, rotation freedom α, pitch freedom β, and yaw freedom γ. The operating arm 31 can also achieve forward and backward movement freedom z (i.e., feed freedom) under the drive of the distal joint assembly of the robotic arm 30, i.e., the power mechanism 301. In addition, in some embodiments, redundant degrees of freedom can be set for the operating arm 31 to achieve the possibility of more functions. For example, under the premise that the above 6 degrees of freedom can be achieved, one, two, or even more degrees of freedom can be additionally set. For example, the power mechanism 301 has a guide rail and a power unit slidingly arranged on the guide rail, and the operating arm 31 is detachably mounted on the power unit. On the one hand, the sliding of the power unit on the guide rail provides the operating arm 31 with a degree of freedom z for forward and backward movement. On the other hand, the power unit provides power to the joint assembly of the operating arm 31 to realize the remaining five degrees of freedom (i.e., [x, y, α, β, γ]).

[0077] The surgical robot also includes a controller. This controller can be integrated into the master console 2 or the slave operating device 3. Of course, the controller can also be independent of the master console 2 and the slave operating device 3. For example, the controller can be deployed locally or in the cloud. The controller can be composed of more than one processor.

[0078] The surgical robot also includes an input unit. This input unit can be integrated into the main console 2 or the slave operating device 3. Of course, the input unit can also be independent of the main console 2 and the slave operating device 3. This input unit can be, for example, a mouse, keyboard, voice input device, or touch screen. In one embodiment, a touch screen is used as the input unit, and the touch screen can be installed, for example, on the armrest of the main console 2.

[0079] The operating arm 31 also includes sensors for sensing joint variables of the joint assembly. These sensors include angle sensors for sensing rotational motion of the joint assembly and displacement sensors for sensing linear motion of the joint assembly. Specifically, appropriate sensors can be configured according to the type of joint assembly.

[0080] The controller is coupled to the sensors, and to the input and display 22 .

[0081] For example, Figure 3 As shown, the abutment surface of the driving box 310 of the manipulator arm 31 abutting the power unit 302 of the power mechanism 301 is equipped with a storage unit 311. Correspondingly, the abutment surface of the power unit 302 abutting the driving box 310 is equipped with a reading unit 303 that is compatible with the storage unit 311. The reading unit 303 is coupled to the controller. When the manipulator arm 31 is installed on the power unit 302, the reading unit 303 communicates with the storage unit 311 and reads relevant information from the storage unit 311. The storage unit 311 is, for example, a memory or an electronic tag. The storage unit stores, for example, the type of manipulator arm, the target location of the manipulator arm that can be configured, the kinematic model of the manipulator arm, etc. For example, the storage unit 311 of the camera arm 31A also stores camera parameters.

[0082] like Figure 4 As shown, it is a structural diagram of an embodiment of the surgical robot of the present invention. More specifically, Figure 4 Shown is a schematic structural diagram of an embodiment of a multi-hole surgical robot. Figure 4 The multi-port surgical robot shown is Figure 1 The difference between the single-port surgical robots shown mainly lies in the difference between their slave operating devices. Figure 4In the multi-hole surgical robot shown, the driving arm of the operating device has a robotic arm 110, an adjustment arm 120, a manipulator 130 and an operating arm 150 connected in sequence. The number of the adjustment arm 120, the manipulator 130 and the operating arm 150 is the same and there are more than two of them, for example, four. The distal end of the robotic arm 110 has an orientation platform, the proximal end of the adjustment arm 120 is connected to the orientation platform, and the proximal end of the manipulator 130 is connected to the distal end of the adjustment arm 120. The manipulator 130 is used to detachably connect the operating arm 150, and the manipulator 130 has multiple joint components. In the multi-hole surgical robot, different operating arms 150 are inserted into the patient's body through different puncture devices. The operating arm 150 of the multi-hole surgical robot generally has fewer degrees of freedom than the operating arm 31 of the single-hole surgical robot. Generally, the operating arm 150 only has posture freedom (i.e., orientation freedom). Of course, changes in its posture generally also affect the position, but because the impact is small, it can be ignored in some scenarios. The change in position of the operating arm 150 can usually be achieved with the assistance of the manipulator 130. Since the manipulator 130 and the operating arm 150 are linked to achieve posture changes, the two can be considered as a manipulator component, which is equivalent to the operating arm 31 in the single-port surgical robot.

[0083] Depending on the configuration, the motion input device 21 can input posture commands, including position commands and posture commands, to control the posture changes of the distal end of the first portion of the actuator arm. The distal end of the first portion typically refers to the end instrument. Alternatively, the distal end of the first portion may refer to a joint assembly connected to the end instrument. Changes in the posture of the end instrument typically coincide with changes in the posture of the joint assembly.

[0084] exist Figure 1 In the surgical robot shown, the driving arm includes a robotic arm and a manipulator arm. The proximal end of the manipulator arm is mounted on the distal end of the robotic arm, and the end instrument is mounted on the distal end of the manipulator arm. Depending on the configuration, the first part can be configured as the manipulator arm; alternatively, the first part can be configured as a combination of the robotic arm and the manipulator arm.

[0085] And correspondingly Figure 4 In the surgical robot shown, the driving arm includes a robotic arm, an adjustment arm, a manipulator, and an operating arm. The proximal end of the adjustment arm is mounted to the distal end of the robotic arm, the proximal end of the manipulator is mounted to the distal end of the adjustment arm, the proximal end of the operating arm is mounted to the distal end of the manipulator, and the end instrument is mounted to the distal end of the operating arm. Depending on the configuration, the first part can be configured as the operating arm; or the first part can be configured as a combination of the manipulator and the operating arm; or the first part can be configured as a combination of the robotic arm, adjustment arm, manipulator, and operating arm.

[0086] Understandable, whether Figure 1 The single-port surgical robot shown is still Figure 4In the multi-aperture surgical robot shown, the robotic arm is typically used to adjust the position of the end instrument over a wide range, while the manipulator arm is used to fine-tune the position of the end instrument. For example, the robotic arm is used to position the end instrument before surgery, while the manipulator arm is primarily used to control the surgery during surgery. Of course, in some embodiments, the robotic arm and the manipulator arm, as well as other corresponding arm structures, can also be combined to coordinate motion to achieve specific functions.

[0087] In one embodiment, the structure associated with the first part of the driving arm can be defined in the system file of the surgical robot. When the system of the surgical robot is initialized, the structure associated with the first part can be read from the system file and applied to the embodiments described below.

[0088] In one embodiment, a configuration interface for configuring the first portion can also be generated in real time based on the descriptive information of the driving arm configuration. This descriptive information includes, for example, the link parameters of all joint components in each portion of the driving arm. For example, the configuration interface includes selectable controls associated with each portion of the driving arm structure for the physician to configure. These controls can be, for example, text controls, option controls such as drop-down list controls, button controls, and other forms.

[0089] Preferably, in order to make it easier for doctors to configure the first part more intuitively through the configuration interface, an image model that is associated and contains optional controls can be generated based on the descriptive information of the configuration of the driving arm. The image model can be a computer image model that schematically illustrates a complex structure. The image model can change with changes in the state of the driving arm. Of course, the image model may not follow changes in the state of the driving arm but only reflect the configuration of the driving arm at a certain moment, such as the initial state (such as when the joint variable is zero). The controls on the image model are, for example, icon controls, and more specifically, they can be light spots, apertures, etc.

[0090] For example, Figure 1 In the surgical robot shown, the mechanical arm and the operating arm in the driving arm can each correspond to a control for selecting their entirety as the first part; Figure 4 In the surgical robot shown, the driving arm, adjustment arm, manipulator, and operating arm can each correspond to a control for selecting their entirety as the first part. Furthermore, multiple independently configured surgical robots that collaborate to perform a surgery can also be configured in the first part, provided that these multiple surgical robots do not have the same robotic arm.

[0091] For example, Figure 1 and Figure 4In the surgical robot shown, each joint component in the driving arm can correspond to a control for selecting a part or all of it as the first part. Unselected parts or components are considered non-articulated by the system, thus preventing them from moving. A closed graphic drawn by the doctor through the input unit that covers at least part of the control in the image model can be obtained, and all parts contained within (i.e., enclosed) the graphic can be used as the first part. This design can improve the configuration efficiency of the first part.

[0092] like Figure 5 and Figure 6 As shown, Figure 5 and Figure 6 They indicated Figure 1 The configuration interface of an embodiment of the first part of the configuration of the surgical robot is shown. Figure 5 and Figure 6 For example, an icon control “○” may be used to represent a portion that can be configured as at least part of the first portion, and an icon control “●” may be used to represent a portion that is configured as at least part of the first portion. Figure 5 As shown, the image model basically illustrates Figure 1 The basic structure of the single-port surgical robot shown in FIG. 1 , wherein the robotic arm and the operating arms Arm1 to Arm3 each contain a selectable control. Whether the control is selected determines whether the corresponding arm body portion is used as the first portion. For example, Figure 5 In the embodiment, only the operating arm Arm1 is configured as the first part, and the end instrument at the distal end of the operating arm Arm1 is configured as the controlled end instrument. Figure 6 As shown, the robot arm and the manipulator arms Arm1 to Arm3 in the image model each contain a plurality of selectable controls. The number of controls in the robot arm and the manipulator arms Arm1 to Arm3 is substantially the same as the number of joint components they each have. Each control can represent one of the corresponding joints, for example, Figure 5 In the figure, since all the controls of the operating arm Arm3 are selected, the operating arm Arm3 is configured as the first part as a whole.

[0093] When applied to subsequent embodiments, it is sufficient to obtain the first part configured by the doctor according to the configuration interface in advance and then use the first part to achieve the purpose of the present invention. Such a design can make it easier for the doctor to flexibly configure the desired first part to suit different application scenarios.

[0094] Depending on the configuration, one or more of the end-implements may be configured as a controlled end-implement to be controlled by a motion input device.

[0095] In one embodiment, a control method for a surgical robot is provided, which can be executed by a controller. Figure 7As shown, the control method includes the following steps:

[0096] Step S11 , obtaining a change in the shape of an image model having structural features and a shape consistent with the first part of the driving arm.

[0097] This image model is primarily used to assist in displaying the motion state of the first portion of the driving arm. This motion state includes position, attitude, velocity, acceleration, and so on. To facilitate intuitive, static observation, the image model's morphology is typically required to remain consistent with the morphology of the first portion of the driving arm. "Morphology" refers to the shape and attitude that are driven by position and attitude. Typically, from the perspective of dependent and independent variables, changes in the motion state of the first portion of the driving arm are the independent variable, and changes in the motion state of the image model are the dependent variable. The motion state of the image model changes in response to changes in the motion state of the first portion of the driving arm.

[0098] In the prior art, it may be possible to actively adjust the presentation perspective of the image model, but it is rare to be able to actively adjust the morphology of the image model to affect the morphology of the first part in the driving arm. In order to achieve the purpose of being able to adjust the morphology of the image model to affect the morphology of the first part in the driving arm, the present invention configures the image model with the same structural features as the first part in the driving arm, and these structural features include but are not limited to configuration and connecting rod parameters. For example, the driving arm has a plurality of real joint components, and the image model has a virtual joint component corresponding to the real joint component of the first part, and the virtual joint component has joint motion parameters consistent with the corresponding real joint component, so that the adjustable performance of the image model's morphology is consistent with the adjustable performance of the morphology of the first part. Among them, the joint motion parameters include joint motion range, joint motion speed threshold and joint motion acceleration threshold.

[0099] The present invention can actively adjust the shape of the image model, thereby independently acquiring changes in the shape of the image model. The active adjustment of the shape of the image model can utilize other types of input devices coupled to the controller, such as the aforementioned motion input device, a mouse, a touch screen, a voice recognition device, and / or a gesture recognition device.

[0100] At the same time, only one of the image model's shape and the shape of the first portion of the driving arm can be adjusted simultaneously to avoid confusion and ensure consistency and safety of surgical operations. In one embodiment, two switching instructions can be configured to switch the adjustable object. These two switching instructions include a first switching instruction and a second switching instruction. The first switching instruction is used to disable the active control function of the shape of the first portion of the driving arm and enable the active control function of the shape of the image model, and the second switching instruction is used to disable the active control function of the shape of the image model and enable the active control function of the shape of the first portion of the driving arm.

[0101] Typically, before step S11, the control method may include: detecting whether a first switching instruction is obtained. Then, when the first switching instruction is obtained, step S11 is executed.

[0102] Step S12: controlling the movement of the corresponding real joint components in the first part so that the shape of the first part changes along with the change of the shape of the image model.

[0103] Among them, the change in the shape of the first part is consistent with the change in the shape of the image model. The "consistency" here can allow slight differences, and the influencing factors are objective factors, such as differences caused by inaccurate transmission of the first part in the driving arm.

[0104] In one embodiment, whether the first portion changes shape in accordance with the image model requires manual confirmation by an operator before proceeding. For example, before step S12, the process may include: detecting whether a confirmation instruction has been received; if so, executing step S12; otherwise, continuing to detect whether a confirmation instruction has been received. This intervention-based adjustment mode can be understood as an intermittent adjustment mode, which facilitates first placing the image model into a suitable shape before controlling the first portion of the drive arm to change shape in accordance with the image model.

[0105] In another embodiment, whether the morphology of the first part changes with the morphology of the image model does not need to be manually confirmed by the operator. Instead, the morphology of the first part changes as long as the morphology of the image model changes. This non-intervention adjustment mode can be understood as a continuous adjustment mode. In the continuous adjustment mode, for example, the morphology of the first part can be configured to change substantially synchronously with changes in the morphology of the image model; or the morphology of the first part can be configured to change after changes in the morphology of the image model, for example, at intervals of 1 to 10 seconds. Such a setting helps provide the possibility of timely readjustment of the morphology of the image model, that is, helps to promptly interrupt the adjustment of the morphology of the first part when it is found that the adjusted morphology of the image model is not as expected, and has basically the same effect as the intermittent adjustment mode.

[0106] Whether to use the intermittent adjustment mode or the continuous adjustment mode to adjust the shape of the first portion can be pre-configured by an operator, for example.

[0107] Typically, before switching to the image control mode, since the structural features and initial shape of the image model are identical to those of the first portion of the driving arm, in some embodiments, the change in shape of the first portion can completely replicate the change in shape of the image model. "Complete replication" or "complete copying" can be defined as: the two have exactly the same motion state, including the same motion trajectory and motion speed, etc., and "complete replication" is achieved by directly sending the parameters for adjusting the morphological change of the image model to the controller without processing, and then controlling the motion of the first portion of the driving arm based on these parameters to achieve the change in shape. In other embodiments, the change in shape of the first portion can also partially replicate the change in shape of the image model. "Partial replication" or "partial copying" can be defined as: the two have non-identical motion states, but at least the same final shape, for example, different motion trajectories or different motion speeds, and "partial replication" is achieved by processing the parameters for adjusting the morphological change of the image model and then sending them to the controller, and then controlling the motion of the first portion of the driving arm based on these parameters to achieve the change in shape. For example, the method of sampling the parameters for adjusting the morphological change of the image model and then sending them to the controller, and then controlling the movement of the first part of the driving arm based on these sampled parameters to achieve the morphological change can make the movement trajectories of the two different.

[0108] For example, the movement speed of the first part of the driving arm can be adjusted by setting a movement speed coefficient so that the movement speeds of the two are different. In some embodiments, a movement speed coefficient less than 1 can be set to control the movement of the first part of the driving arm so that it has a lower change speed than the change speed of the image model's shape. This has the benefit of, for example, reducing the actual risk of collision, because even if a collision occurs, the inertia of the collision is relatively small due to the low speed. Of course, in other embodiments, a movement speed coefficient greater than 1 can also be set to control the movement of the first part of the driving arm so that it has a higher change speed than the change speed of the image model's shape. Such a usage scenario can be used, for example, under the premise of ensuring that no collision occurs.

[0109] According to the above steps S11 to S12, the change in the shape of the image model is used as the independent variable and the change in the shape of the first part in the driving arm is used as the dependent variable to control, so that the shape of the first part in the driving arm changes with the change in the shape of the image model. This can provide a new control method, which is particularly suitable for use in some scenarios where direct control of the shape of the first part in the driving arm is not suitable, such as scenarios for resolving collisions, and scenarios for placing the first part into a required shape.

[0110] In one embodiment, if Figure 8 As shown, the above step S11, i.e., the step of obtaining the change in the shape of the image model whose structural features and shape are consistent with the first part of the driving arm, includes:

[0111] Step S111 : obtaining an image model having structural features and a shape consistent with the first portion of the driving arm and displaying at least a portion of the image model.

[0112] Typically, the entire image model corresponding to the first portion is displayed. Of course, only the portion of the image model corresponding to the first portion may be displayed, for example, only the distal end of the complete image model or a portion including the distal end. Regardless of whether the entire or partial image model is displayed, the displayed portion may be modified by changing the shape of the image model and thereby the shape of the first portion using the methods described below. For example, the shape of the image model and the first portion may be modified by configuring the joint space of the image model and / or the task space of the image model.

[0113] Step S112: Acquire the change in the morphology of the image model.

[0114] In one embodiment, if Figure 9 As shown, before step S112, that is, before the step of obtaining the change in the morphology of the image model, the following steps may also be included:

[0115] Step S1121: receiving an input operation instruction set for changing the form of the image model.

[0116] The operation instruction set includes more than one instruction.

[0117] Step S1122: changing the shape of the image model according to the operation instruction set.

[0118] In the process of adjusting the image model, the final form of the image model or the final form including intermediate forms may be displayed.

[0119] The intermediate forms refer to all transitional forms before the final form. That is, the change in the image model form can directly present the final form. Of course, it can also present the intermediate forms corresponding to the entire change process and the final form at the end of the change.

[0120] Generally speaking, the shape of the image model can be changed from two aspects: first, the shape of the image model can be changed from the perspective of the joint space; second, the shape of the image model can be changed from the perspective of the task space.

[0121] In one embodiment, based on the first aspect above, the operation instruction set in step S1121 is an operation instruction set for configuring the image model in the joint space.

[0122] For example, the system defaults to each virtual joint component in the image model being in a disabled state. The operation instruction set may include an enable instruction and a move instruction. The enable instruction is used to configure one or more virtual joint components in the disabled state to be in an enabled state, and the move instruction is used to select a virtual joint component in the enabled state as a control node and configure the joint motion amount within the joint space for the control node. The motion range of the virtual joint component in the disabled state is restricted to act as a rigid body structure, while the motion range of the virtual joint component in the enabled state is not restricted and is in a free state.

[0123] Furthermore, if Figure 10 As shown, the step of changing the form of the image model according to the operation instruction set in the above step S1122 can be performed as follows:

[0124] Step S11221: parse the operation instruction set to obtain the control node and its corresponding joint motion amount.

[0125] The obtained joint motion amount is usually directional. In the present invention, the joint motion amount of the corresponding control node only changes the motion state of the control node itself without affecting other control nodes.

[0126] Step S11222: According to the control nodes and their corresponding joint movement amounts, the corresponding control nodes are made to independently move the corresponding joint movement amounts to change the shape of the image model.

[0127] Before step S11222 , it may be determined whether the joint motion amount of each control node is valid.

[0128] Among them, there are a variety of strategies for the operator to choose to use. For example, when the joint motion amounts of each control node are valid, each control node is controlled to move according to the joint motion amount in the above step S11222. For example, when the joint motion amounts of some control nodes are valid, the part of the control nodes is controlled to move according to the joint motion amount in the above step S11222. For another example, when the joint motion amounts of each control node have one invalid value, the above step S11222 is not executed, and the operator may be prompted to reconfigure a reasonable joint motion amount. Examples of the judgment criteria for effectiveness may be joint motion state parameters of the control node such as motion range, joint motion speed and / or joint motion acceleration.

[0129] In some embodiments, only one of multiple virtual joint components can be configured as a control node at a time to simplify the manipulation of the image model's morphology. This makes the effects of these changes easier to observe and understand. To adjust multiple virtual joint components, different control nodes and joint motion amounts can be configured at different times.

[0130] The joint motion amount can be an incremental joint motion amount or a target joint motion amount. For a revolute joint, the joint motion amount can be an incremental joint angle or a target joint angle; for a sliding joint, the joint motion amount can be an incremental joint offset or a target joint offset.

[0131] In one embodiment, the incremental joint motion can be configured as follows:

[0132] At least a first icon showing a kinematic pair can be generated and displayed for the control node, where the kinematic pair corresponds to a rotating joint as a rotating axis, and the kinematic pair corresponds to a moving joint as a moving axis. In addition, a second icon associated with the adjustable direction of the kinematic pair can be generated and displayed in the kinematic pair. Exemplarily, the moving instructions in the operation instruction set can be derived from a click operation instruction input by an operator with the aid of an input device such as a motion input device, a mouse, a touch screen, etc., and the click operation instruction includes a click position and a number of clicks. For example, the operator can determine the movement direction of the control node according to the click position in the triggered click operation instruction by clicking the second icon associated with a kinematic pair, and can determine the incremental joint movement amount of the control node in the corresponding movement direction according to the number of clicks. Each click corresponds to a fixed incremental joint movement. For the rotational joint, this incremental joint movement is a fixed-step rotation, such as any value between 0.1° and 1°, such as 0.1°, 0.2°, 0.3°, 0.4°, 0.5°, 0.6°, 0.7°, 0.8°, 0.9° or 1°. Of course, the fixed-step rotation can also be defined by the operator as a value other than 0.1° to 1°; for the mobile joint, this incremental joint movement is a fixed-step offset, such as any value between 1mm and 10mm, such as 1mm, 2mm, 3mm, 4mm, 5mm, 6mm, 7mm, 8mm, 9mm or 10mm. Of course, the fixed-step offset can also be defined by the operator as a value other than 1mm to 10mm.

[0133] In some embodiments, the number of clicks in a click operation instruction can be replaced by the click duration, where the incremental joint movement corresponding to the unit time t0 is s0, and the incremental joint movement corresponding to the click duration t is s = (t / t0) * s0. In addition, the click operation instruction can also be compatible with including the click duration, and the sum of the incremental joint movement corresponding to the number of clicks and the incremental joint movement corresponding to the click duration can be used as the final incremental joint movement.

[0134] Furthermore, if Figure 11 As shown, in the above step S11221, that is, the step of parsing the operation instruction set to obtain the corresponding joint movement amount of the control node may include:

[0135] Step S112211, parse the click operation instruction to obtain the click position, number of clicks and / or click duration.

[0136] The valid click position is, for example, a click on the second icon.

[0137] Step S112212: determining the movement direction of the control node according to the click position, and determining the incremental joint movement amount of the control node in the corresponding direction according to the number of clicks and / or click duration.

[0138] The present invention is Figure 1 The surgical robot shown in the figure is described as an example in which the robotic arm of the operating device is used as the first part of the driving arm. Figures 12 to 15 As shown, the first part 3 includes real joint components 3001-3005, and correspondingly, the image model 3' includes virtual joint components 3001'-3005' corresponding to the real joint components 3001-3005. Figure 12 and Figure 13 Schematic diagrams respectively illustrate the configurations of the first portion 3 and the image model 3' prior to the implementation of the control method described herein. Both components have identical configurations in their initial states. Furthermore, for example, the virtual joint component 3003' in the image model 3' is configured as a control node. To configure incremental joint motion, a first icon 61 representing the kinematic pair and a second icon 62 representing the kinematic pair's adjustable direction can be generated at a location associated with the control node 3003'. The second icon 62 can include a sub-icon 621 representing the first adjustable direction and a sub-icon 622 representing the second adjustable direction.

[0139] Here, for example, “·” represents the click position. The operator controls the image model 3′ by clicking the operation instruction. Figure 13 The shape shown is adjusted to Figure 14 For example, after confirmation by the operator, the first part 3 can be Figure 12 The shape shown is adjusted to Figure 15 The form shown ultimately makes the form of the first part 3 substantially consistent with the form of the image model 3'.

[0140] Furthermore, the first icon and the second icon can be generated and displayed for all virtual joint components in the enabled state. The click position obtained by parsing the click operation instruction can also determine the selected control node and its corresponding movement direction.

[0141] In one embodiment, the target joint motion amount can be configured as follows:

[0142] For example, at least the range of motion of the virtual joint components configured as control nodes in each enabled state is obtained. The operator can then use a motion input device to configure a target joint motion based on this range of motion, so that the control node directly moves the configured target joint motion. For example, if the control node's range of motion is -270° to 270°, the desired control node rotation angle, such as 200°, can be directly input. This will not be described in detail here.

[0143] In one embodiment, based on the second aspect above, the operation instruction set in step S1121 is an operation instruction set for configuring the image model for the task in the task space.

[0144] For example, the system defaults to enabling all virtual joint components in the image model. The operation instruction set may include mode configuration instructions and movement instructions. The mode configuration instructions are used to configure the task freedom at the far end of the image model, more specifically, to configure the task freedom at the farthest virtual joint component in the image model. The movement instructions are used to select a virtual joint component in an enabled state as a control node and configure the task motion within the task space for the control node. Furthermore, the operation instruction set may also include a disable instruction for configuring one or more virtual joint components in an enabled state to be disabled. This is equivalent to changing the configuration of the image model to suit more usage scenarios.

[0145] Furthermore, if Figure 16 As shown, the step of changing the form of the image model according to the operation instruction set in the above step S1122 can be performed as follows:

[0146] Step S11221': parse the operation instruction set to obtain the virtual joint component in the enabled state, the task degree of freedom of the farthest virtual joint component, the control node and its task motion amount.

[0147] Step S11222': controlling the enabled virtual joint components to move in a coordinated manner according to the task degree of freedom and the task motion amount of the control node so as to change the shape of the image model by moving the control node corresponding to the task motion amount.

[0148] Before step S11222', it may be determined whether the task motion amount of each control node is valid.

[0149] Wherein, there are multiple strategies for the operator to choose to use. For example, when the task motion amount of each control node is valid, each control node is controlled to move according to the task motion amount in the above-mentioned step S11222'. For example, when the task motion amount of some control nodes is valid, the above-mentioned step S11222' is controlled to move according to the task motion amount. For another example, when the task motion amount of each control node has an invalidity, the above-mentioned step S11222' is not executed, and the operator can be prompted to reconfigure a reasonable joint motion amount. For example, the judgment of the validity of the task motion amount can be converted into the motion state parameters of each virtual joint component in a section of virtual arm body associated with the control node by utilizing inverse kinematics, and then compared with the corresponding threshold value and judged, and the motion state parameters include range of motion, joint motion speed and / or joint motion acceleration. Of course, the validity of the task motion can also be judged by other methods, for example, whether a section of the virtual arm body associated with the control node will collide with other virtual arm bodies. When a collision occurs, the task motion amount of the corresponding control node is judged to be invalid. When no collision occurs, the task motion amount of the corresponding control node is judged to be valid. Usually, other virtual arms refer to other sections of virtual arms that are in parallel structure rather than series structure with the section of the virtual arm body where the control node is located, such as different virtual arms corresponding to different operating arms. The position detection method can be used to judge whether the virtual arm body collides, which will not be described in detail here. In fact, different judgment methods for the validity of the task motion amount can be used alone or in combination.

[0150] The task space is, for example, a Cartesian space. The task motion quantity may be an incremental pose or a target pose. The incremental pose includes an incremental position and / or an incremental attitude, and the target pose includes a target position and / or a target attitude.

[0151] In one embodiment, the incremental task movement amount can be configured as follows:

[0152] like Figure 17 As shown, a coordinate image 63 can be generated and displayed at least at the control node. The coordinate image 63 includes an X-axis, a Y-axis, and a Z-axis. The position of the control node can be incrementally configured based on at least these three axes. For example, the movement instructions in the operation instruction set can be derived from a click operation instruction input by an operator via an input device. The click operation instruction includes a click position and a click count.

[0153] For example, when an operator clicks on a coordinate axis, the operator can determine the movement direction of the control node based on the click position in the triggered click operation instruction, and can determine the incremental task movement amount of the control node in the corresponding movement direction along the coordinate axis based on the number of clicks. Each click corresponds to a fixed incremental task movement amount, and this incremental task movement amount is an offset with a fixed step size (i.e., an incremental position parameter), such as any value between 1mm and 10mm, such as 1mm, 2mm, 3mm, 4mm, 5mm, 6mm, 7mm, 8mm, 9mm, or 10mm. Of course, the offset with a fixed step size can also be defined by the operator as a value other than 1mm to 10mm.

[0154] In order to facilitate the control of the posture of the control node, such as Figure 18 As shown, an icon 64 of an adjustable direction associated with the coordinate axis can be further generated in the corresponding coordinate axis of the coordinate image 63. When the operator clicks the icon 64, the operator can determine the direction of rotation of the control node around the corresponding coordinate axis according to the click position in the triggered click operation instruction, and the incremental task motion amount of the control node in the direction of rotation can be determined according to the number of clicks. Each click corresponds to a fixed incremental task motion amount, wherein the incremental task motion amount is a fixed step rotation amount (i.e., incremental posture parameter), such as any value between 0.1° and 1°, such as 0.1°, 0.2°, 0.3°, 0.4°, 0.5°, 0.6°, 0.7°, 0.8°, 0.9° or 1°. Of course, the fixed step rotation amount can also be defined by the operator as a value other than 0.1° to 1°.

[0155] In some embodiments, the number of clicks in a click operation instruction can be replaced by the click duration. The incremental task motion amount corresponding to unit time t0 is s0, and the incremental task motion amount corresponding to click duration t is s = (t / t0) * s0. Furthermore, the click operation instruction can also be compatible with including click duration. The sum of the incremental task motion amount corresponding to the number of clicks and the incremental task motion amount corresponding to the click duration can be used as the final incremental task motion amount.

[0156] Furthermore, if Figure 19 As shown, in the above step S11221′, that is, the step of parsing the operation instruction set to obtain the task motion amount of the control node may include:

[0157] Step S112211': parse the click operation instruction to obtain the click position, click times and / or click duration.

[0158] The valid click position is, for example, a click on the coordinate axis and / or a click on an icon for adjusting the direction at the coordinate axis.

[0159] Step S112212': determining the movement direction of the control node according to the click position, and determining the incremental task movement amount of the control node in the corresponding direction according to the number of clicks and / or click duration.

[0160] Furthermore, the above icons can be generated and displayed for all virtual joint components in the enabled state. The click position obtained by parsing the click operation instruction can also determine the selected control node and its corresponding movement direction.

[0161] In one embodiment, the target task movement amount can be configured as follows:

[0162] Since the image model is associated with the first part of the driving arm, the reference coordinate system of the image model can be the same as the reference coordinate system of the first part of the driving arm, that is, the two can actually be controlled based on the same reference coordinate system to achieve morphological changes, and then the position of the control node can be solved based on the kinematic model of the first part and the joint variables of each real joint component contained in the first part and using forward kinematics. Among them, the parameters corresponding to the position can be numerically displayed at the control node. The operator can therefore refer to the numerically displayed position and use the input device to set the desired target task motion. For example, the current position of the control node is P0[x0,y0,z0,α0,β0,γ0], where x represents the horizontal coordinate, y represents the vertical coordinate, z represents the front-back coordinate, α represents the yaw angle, β represents the pitch angle, and γ represents the rotation angle. The operator can try to set a suitable target task motion P based on P0. m , for example, P m is [x0+x m ,y0,z0,α0,β0,γ0], which means that the operator only sets the target position for the X direction.

[0163] In some embodiments, multiple mode instructions can be set to facilitate the quick configuration of task degrees of freedom. These mode instructions include but are not limited to one or more of the first mode instructions, the second mode instructions, the third mode instructions, and the fourth mode instructions, depending on the specific needs. Among them, the first mode instruction is used to configure the task degree of freedom to be zero task degree of freedom, that is, no constraints are imposed; the second mode instruction is used to configure the task degree of freedom to be posture degree of freedom, and the posture degree of freedom includes one or more position degrees of freedom and one or more posture degrees of freedom; the third mode instruction is used to configure the task degree of freedom to be position degree of freedom, and the position degree of freedom includes one or more; the fourth mode instruction is used to configure the task degree of freedom to be posture degree of freedom, and the posture degree of freedom includes one or more. For example, the posture degree of freedom corresponding to the second mode instruction is full posture degree of freedom, that is, corresponding to all degrees of freedom related to position and posture that can be achieved by the first part; the position degree of freedom corresponding to the third mode instruction is full position degree of freedom, that is, corresponding to all degrees of freedom related to position that can be achieved by the first part; the posture degree of freedom corresponding to the fourth mode instruction is full posture degree of freedom, that is, corresponding to all degrees of freedom related to posture that can be achieved by the first part.

[0164] In the first mode instruction, that is, no constraints are imposed on the task degrees of freedom, the virtual joint component at the far end of the image model is in a slave state, and all joint components of the control node and the far end virtual joint component are configured to be disabled so as to act as a rigid body structure and follow the movement of the control node.

[0165] In the second through fourth mode instructions, the task degrees of freedom are constrained so that the virtual joint assembly at the far end of the image model is in an active state. The first objective is to cause all joint assemblies between the near-end virtual joint assembly and the control node to move according to the task motion amount, while the second objective is to simultaneously cause all joint assemblies of the control node and the far-end virtual joint assembly to move to maintain the corresponding task degrees of freedom. That is, in the second through fourth mode instructions, the image model can be divided into two or more virtual arms based on the number of control nodes, and these arms can be independently controlled to achieve their respective objectives.

[0166] For example, when there is only one control node, the entire virtual arm body between the virtual joint component at the proximal end and the control node (including the control node) in the image model is the first segment of the virtual arm body, and the entire virtual arm body between the virtual joint component at the control node (excluding the control node) and the distal end is the second segment of the virtual arm body.

[0167] For another example, when there are more than two control nodes, the entire virtual arm body between the proximal virtual joint component and the adjacent control node (including the control node) in the image model is the first segment of the virtual arm body, and the entire virtual arm body between each adjacent control node is the second segment of the virtual arm body (the number of the second segment of the virtual arm body is the same as the number of control nodes), and the entire virtual arm body between the distal control node and the distal virtual joint component is the third segment of the virtual arm body.

[0168] When there are more than two control nodes, it is necessary to configure the task motion amount for each control node separately. If the task motion amount is not configured for each control node separately, you can prompt to configure the task motion amount for the control nodes that are not configured, or you can ignore the control nodes that are not configured with task motion amount and only control the control nodes that are configured with task motion amount.

[0169] The aforementioned segmentation and subsequent control of the virtual arm in the image model is typically performed based on the assumption that the control node does not include the distal virtual joint component. Of course, if the control node does include the distal virtual joint component, the last segment of the virtual arm can be disregarded. For example, if there is only one control node, and that control node is the distal virtual joint component, then the virtual arm in the image model is considered a single segment. That is, the virtual arm from the proximal virtual joint component to the distal virtual joint component is treated as a single segment without any segmentation.

[0170] The following description is made by taking the second mode instruction corresponding to all posture degrees of freedom, the third mode instruction corresponding to all position degrees of freedom, and the fourth mode instruction corresponding to all posture degrees of freedom as examples.

[0171] A brief description is given below taking a control node as an example.

[0172] When the control node is a virtual joint component at the far end of the image model:

[0173] Among them, all the virtual joint components at the proximal end of the control node (including the virtual joint component where the control node is located) are the first section of the virtual arm body. The first section of the virtual arm body moves with the virtual joint component at its proximal end as the coordinate origin to achieve the movement amount of the corresponding control node. More specifically, for example, the virtual joint components in the enabled state in this section of the virtual arm body can be controlled to link according to inverse kinematics.

[0174] (1) Based on the first mode instruction, since the remote task freedom is not constrained, the first segment of the virtual arm can be controlled according to the configured task motion to make the control node move to achieve the task motion.

[0175] (2) Based on the second mode instruction, since the task freedom of the remote end is constrained to maintain the posture, the first segment virtual arm linkage cannot be controlled according to the configured task motion.

[0176] (3) Based on the third mode instruction, since the remote task degree of freedom is constrained to maintain position, the first segment of the virtual arm can be controlled according to the configured task motion to keep the control node in position and only change its posture. To achieve such motion, the task motion should include the posture motion.

[0177] (4) Based on the fourth mode instruction, since the remote task freedom is constrained to maintain posture, the first segment of the virtual arm can be controlled according to the configured task motion to keep the control node in posture and only change position. To achieve such motion, the task motion should include position motion.

[0178] 2. When the control node is not a remote virtual joint component in the image model:

[0179] Among them, all the virtual joint components at the proximal end of the control node (including the virtual joint component where the control node is located) are the first section of the virtual arm, and all the virtual joint components at the distal end of the control node (excluding the virtual joint component where the control node is located) are the second section of the virtual arm. Each section of the virtual arm moves independently with the virtual joint component at its proximal end as the coordinate origin to achieve the movement amount of the corresponding control node. More specifically, for example, the virtual joint components in the enabled state in each section of the virtual arm can be controlled to link according to inverse kinematics.

[0180] (1) Based on the first mode instruction, since there is no constraint on the task freedom of the remote end, all virtual joint components contained in the second section of the virtual arm are in a disabled state. Therefore, the first section of the virtual arm can be controlled to move in conjunction with the configured task motion so that the control node moves to achieve the task motion.

[0181] (2) Based on the second mode instruction, since the task freedom of the remote end is constrained to maintain the posture, the first section of the virtual arm body is usually controlled to make the control node move to achieve the task motion according to the configured task motion, and the second section of the virtual arm body is controlled to make the remote end virtual joint component maintain the posture.

[0182] (3) Based on the third mode instruction, since the task freedom of the remote end is constrained to maintain the position, the first section of the virtual arm body is usually controlled to make the control node move to achieve the task motion according to the configured task motion, and the second section of the virtual arm body is controlled to make the remote end virtual joint component maintain the position, and the posture is not constrained.

[0183] (4) Based on the fourth mode instruction, since the task freedom degree of the remote end is constrained to maintain the posture, the first section of the virtual arm body is usually controlled to make the control node move to achieve the task motion according to the configured task motion, and the second section of the virtual arm body is controlled to make the remote end virtual joint component maintain the posture, and the position is not constrained.

[0184] The above method is also applicable to the case where there are more than two control nodes. Referring to the case where there is only one control node, the only difference between the two is that the linkage of the virtual arms between adjacent control nodes is added. The linkage of the virtual arms between adjacent control nodes is relatively independent of the linkage of the proximal virtual arm of the proximal control node and the linkage of the distal virtual arm of the distal control node, so as to jointly achieve the purpose of the first aspect and the purpose of the second aspect. Among them, the reference coordinate system of each segment of the independently controlled virtual arm is different. For example, when the virtual arm in the image model is divided into the first, second and third segments of the virtual arm from the proximal segment to the distal segment, the reference coordinate system of the first segment of the virtual arm can be based on the coordinate system of the virtual joint component at the closest end of the first segment of the virtual arm relative to the base coordinate system, the reference coordinate system of the second segment of the virtual arm can be based on the coordinate system of the virtual joint component at the closest end of the second segment of the virtual arm relative to the base coordinate system, and the reference coordinate system of the third segment of the virtual arm can be based on the coordinate system of the virtual joint component at the closest end of the third segment of the virtual arm relative to the base coordinate system. That is, the first segment of the virtual arm, the second segment of the virtual arm, and the third segment of the virtual arm all move with their proximal virtual joint components as the origin. In an embodiment that is more convenient to implement, the task motion amount corresponding to each control node only affects the motion of the virtual arm segment in which it is located without affecting the motion of other segments of the virtual arm.

[0185] The virtual joint components that can be configured as enabled, disabled, or control nodes generally correspond to active joint components (i.e., independently adjustable joint components) in real joint components. That is, the virtual joint components that correspond to driven joint components (i.e., joint components that are coupled to and follow the active joint components) in real joint components generally cannot be configured as described above. For example, in a closed kinematic chain such as a four-bar linkage, only a portion, such as one, of the multiple real joint components is typically an active joint component.

[0186] In addition, when configuring the first part of the driving arm, the active joint component and the driven joint component of the real joint component can both be configured as the first part.

[0187] During surgical robot operation, collisions are often most likely to occur between two or more real arms in a parallel configuration. Therefore, in some embodiments, upon detecting a collision or potential collision with a real arm in a driving arm, the robot can, on the one hand, use the real arm in collision or potential collision as the first part, and then acquire and display an image model consistent with the structural features and morphology of the first part. Furthermore, the operator is allowed to switch from the operating mode corresponding to the second switching instruction to the operating mode corresponding to the first switching instruction. Of course, the operator's confirmation is required before the switching of operating modes is performed.

[0188] In some embodiments, in the operation mode corresponding to the first switching instruction, that is, by changing the shape of the image model to cause the driving arm to undergo the same change in shape, this can be performed under certain specific constraints. Such constraints include but are not limited to the virtual joint components in the image model whose motion state can be changed must be virtual joint components within the operation image provided by the image operation terminal device, i.e., within the field of view. Figure 19 As shown, only the virtual joint components of the image model 3' that are within the visual area can be configured to change their form. In order to facilitate the configuration of these virtual joint components within the field of view, such as control nodes and corresponding motion amounts, as described above, the virtual joint components outside the field of view in the image model can be defaulted to a disabled state. In order to further distinguish whether each virtual joint component in the image model is within the field of view, the virtual joint components within or outside the field of view can be marked to prominently prompt the operator of the subsequent operations to be performed on the image model. In more cases, the part of the image model that can change its form can be unconstrained, for example, when part or all of the immediate image model does not exist within the field of view.

[0189] In some embodiments, a user interface with an angle switching control for switching the display angle of the image model can be generated, and the display angle of the image model can be switched according to the angle switching instruction generated by triggering the angle switching control, which can facilitate the operator to selectively view the status of the image model from different angles.

[0190] In some embodiments, in an operating mode corresponding to a first switching instruction, at least a portion of the image model, such as the proximal or entire virtual joint component, can be magnified to facilitate configuration of control nodes and their corresponding motion amounts, and to facilitate observation of changes in the image model's morphology. Furthermore, when switching to an operating mode corresponding to a second switching instruction, the magnified image model can be restored to its normal size for display.

[0191] In one embodiment, since the joint variables of the real joint components used to control the movement of the first part of the driving arm may be discontinuous, and the inertia of the real joint components of the first part is relatively large, directly using these joint variables to control the first part is likely to cause jitter, that is, discontinuous movement. Therefore, in order to make the movement of the first part continuous, Figure 21 As shown, before the above step S12, i.e., the step of controlling the movement of the corresponding real joint components in the first part so that the shape of the first part changes along with the change of the shape of the image model, the following steps may be performed:

[0192] Step S121 , obtaining joint variables corresponding to the virtual joint components that cause the morphology of the image model to change.

[0193] Step S122 , performing motion smoothing processing on the joint variables corresponding to the virtual joint components to obtain joint variables after motion smoothing processing.

[0194] Furthermore, the step of controlling the movement of the corresponding real joint component in the first part so that the shape of the first part changes with the change of the shape of the image model can specifically be to control the movement of the real joint component corresponding to the virtual joint component in the first part according to the joint variables corresponding to the virtual joint component after motion smoothing so that the shape of the first part changes with the change of the shape of the image model, and at the same time make the movement of the first part continuous.

[0195] The above-mentioned step S122 can realize the motion smoothing processing of the joint variables of each real joint component in the first part respectively acquired through filtering processing and / or trajectory interpolation processing. For example, the filtering processing can adopt low-pass filtering processing, which allows low-frequency signals to pass normally while blocking or weakening high-frequency signals that exceed the set critical value. For example, the trajectory interpolation processing can perform spline curve fitting and interpolation based on the joint variables of each real joint component in the first part to plan out a smooth spline curve. The two can be implemented independently or in combination to make the motion of the first part continuous, wherein the motion continuity includes one or more of motion position continuity, motion speed continuity and motion acceleration continuity.

[0196] In one embodiment, the motion smoothing processing performed in step S122 can be performed before the shape of the image model changes, that is, the joint variables input to change the shape of the image model are subjected to motion smoothing processing. The joint variables after motion smoothing processing can be used to change the shape of the image model on the one hand, and to change the shape of the first part in the driving arm on the other hand, so that the movement of the image model and the movement of the first part are continuous.

[0197] In another embodiment, the motion smoothing processing performed in step S122 can be performed after the shape of the image model changes, that is, only the joint variables after motion smoothing can be used to change the shape of the first part of the driving arm. In this way, the movement of the image model can be discontinuous, but at least the movement of the first part can be guaranteed to be continuous.

[0198] In the above embodiment, the image model 3' can also be a projection image model that is simpler than the computer image model, such as Figure 23 Among them, Figure 22 As shown, the method for obtaining the projection image model includes the following steps:

[0199] Step S21 : obtaining a feature point sequence of the first part and a kinematic model corresponding to the first part.

[0200] Step S22: Acquire the joint variables of each joint in the first part sensed by the sensor.

[0201] Step S23: Acquire the virtual camera selected by the input unit.

[0202] The virtual camera is not a real camera and does not capture images of objects. It only reflects a viewpoint. The virtual camera can have a configurable virtual field of view and virtual depth of field.

[0203] Step S24 , determining the projection point of each feature point in the feature point sequence of the first part on the projection plane of the virtual camera according to the kinematic model of the first part and the joint variables.

[0204] Combine Figure 24 Refer to the first part, which takes a certain operating arm in a single-port surgical robot as an example to explain the projection principle.

[0205] The operating arm has a feature point sequence, which includes feature points Q1, Q2, Q3 and Q4. Under virtual imaging of a virtual camera, a projection point sequence is obtained on a projection plane, and the projection point sequence corresponds to q1, q2, q3 and q4.

[0206] For example, taking feature points Q1 and Q2 as examples, the positions of Q1 and Q2 in space are obtained as Q1(X1, Y1, Z1) and Q2(X2, Y2, Z2) respectively based on the kinematic model and joint variables. The projection points q1(x1, y1) and q2(x2, y2) of feature points Q1 and Q2 on the projection plane can be determined by combining the virtual focal length using the following formula:

[0207] x1=fx*(X1 / Z1)+cx;

[0208] y1=fy*(Y1 / Z1)+cy;

[0209] x2=fx*(X12 / Z12)+cx;

[0210] y2=fy*(Y12 / Z12)+cy;

[0211] Where fx is the horizontal focal length, fy is the vertical focal length, cx is the horizontal offset relative to the optical axis, and cy is the vertical offset relative to the optical axis. The values ​​of fx and fy can be equal or different.

[0212] Step S25 , sequentially fitting and connecting the projection points to generate the projection image model of the first part.

[0213] Step S26: display the projection image model.

[0214] According to the above steps S21 to S26 , a projection image model can be obtained.

[0215] In one embodiment, a computer-readable storage medium is provided, which stores a computer program, and the computer program is configured to be loaded and executed by a processor to implement the following steps: obtaining changes in the morphology of an image model whose structural features and morphology are consistent with the first part of the driving arm; controlling the movement of the corresponding real joint component in the first part so that the morphology of the first part changes following the changes in the morphology of the image model.

[0216] In one embodiment, a control device for a surgical robot is provided. Figure 25 As shown, the control device may include: a processor (processor) 501 , a communication interface (Communications Interface) 502 , a memory (memory) 503 , and a communication bus 504 .

[0217] The processor 501 , the communication interface 502 , and the memory 503 communicate with each other via the communication bus 504 .

[0218] The communication interface 502 is used to communicate with other devices such as various sensors, motors, solenoid valves, or network elements of other clients or servers.

[0219] The processor 501 is configured to execute a program 505 , and specifically may execute the relevant steps in the above method embodiment.

[0220] Specifically, the program 505 may include program codes, which include computer operation instructions.

[0221] The processor 505 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), one or more integrated circuits configured to implement an embodiment of the present invention, or a graphics processing unit (GPU). The one or more processors included in the control device may be processors of the same type, such as one or more CPUs or one or more GPUs; or they may be processors of different types, such as one or more CPUs and one or more GPUs.

[0222] The memory 503 is used to store the program 505. The memory 503 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0223] Program 505 can specifically be used to enable the processor 501 to perform the following operations: obtain the changes in the morphology of the image model whose structural features and morphology are consistent with the first part of the driving arm; control the movement of the corresponding real joint component in the first part so that the morphology of the first part changes with the changes in the morphology of the image model.

[0224] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0225] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the patent for this invention shall be determined by the appended claims.

Claims

1. A surgical robot, characterized in that: include: Drive arm; monitor; input device; and a controller coupled to the drive arm, the display, and the input device, and configured to: Acquire an image model having structural features and a morphology consistent with the first portion of the driving arm and display at least a portion of the image model on the display; receiving an operation instruction set inputted by the input device for changing the form of the image model, and parsing the operation instruction set to obtain a control node in the image model and a joint motion amount with a direction thereof, wherein the control node includes one or more virtual joint components in the image model; According to the control nodes and the corresponding joint movement amounts, the corresponding control nodes are independently moved by the corresponding joint movement amounts to change the shape of the image model, and the change in the shape of the image model is displayed on the display; Acquiring a change in the morphology of the image model; controlling the movement of corresponding real joint components in the first part according to the change in the shape, so that the shape of the first part changes following the change in the shape of the image model, and the real joint components are associated with the virtual joint components corresponding to the control nodes; Wherein, the driving arm includes more than two operating arms, and the controller is further configured to: When a real arm body that has collided or has a potential for collision is detected in the driving arm, the real arm body that has collided or has a potential for collision is used as the first part, and an image model consistent with the structural features and morphology of the first part is acquired and displayed.

2. The surgical robot according to claim 1, characterized in that: The operating arm includes a first operating arm having an image end tool and a second operating arm having an operating end tool.

3. The surgical robot according to claim 2, characterized in that: The controller is configured to: The virtual joint components in the image model that are outside a field of view are configured to be in a disabled state, wherein the field of view is an operation image provided by the image terminal instrument.

4. The surgical robot according to claim 1, wherein: The controller is further configured to: generating a user interface having an angle switching control for switching a display angle of the image model; In response to the acquired angle switching instruction generated by triggering the angle switching control, the display angle of the image model is switched according to the angle switching instruction.

5. The surgical robot according to claim 1, characterized in that: The control node includes the virtual joint component configured in an enabled state in the image model.

6. The surgical robot according to claim 1, characterized in that: The controller is configured to: After the shape of the image model changes and when the delay time is reached, the movement of the corresponding real joint component in the first part is controlled according to the change in the shape, so that the shape of the first part changes following the change in the shape of the image model.

7. The surgical robot according to claim 1, characterized in that: The controller is configured to: Acquire an input operation instruction set, where the operation instruction set is associated with a task of the image model in the joint space, and the operation instruction set includes a click operation instruction; Analyzing the click operation instruction to obtain the click position, number of clicks and / or click duration; The movement direction of the control node is determined according to the click position, and the joint movement amount of the control node in the corresponding direction is determined according to the number of clicks and / or the click duration, where the joint movement amount includes an incremental joint movement amount.

8. The surgical robot according to claim 1, characterized in that: The controller is configured to: The entire process of the change in the shape of the image model is displayed.

9. A method for controlling a surgical robot, wherein the surgical robot includes a driving arm, characterized in that: The control method comprises the following steps: Acquire an image model having structural features and a morphology consistent with the first portion of the driving arm and display at least a portion of the image model; Obtaining an input operation instruction set for changing the form of the image model, and parsing the operation instruction set to obtain control nodes in the image model and joint motion amounts with directions thereof, wherein the control nodes include one or more virtual joint components in the image model; According to the control nodes and the corresponding joint movement amounts, the corresponding control nodes are independently moved by the corresponding joint movement amounts to change the shape of the image model, and the change in the shape of the image model is displayed; Acquiring a change in the morphology of the image model; controlling the movement of corresponding real joint components in the first part according to the change in the shape, so that the shape of the first part changes following the change in the shape of the image model, and the real joint components are associated with the virtual joint components corresponding to the control nodes; Wherein, the driving arm includes more than two operating arms, and the control method further includes: When a real arm body that has collided or has a potential for collision is detected in the driving arm, the real arm body that has collided or has a potential for collision is used as the first part, and an image model consistent with the structural features and morphology of the first part is acquired and displayed.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and the computer program is configured to be loaded and executed by a processor to implement the steps of the control method according to claim 9.

Images