Surgical robot, master console and control method and control device therefor

Abstract

The application relates to a surgical robot, a master console and a control method and device thereof. The master console comprises a display part and an operation part. The display part has an imaging part for final imaging. The operation part has a base and an input part connected with the base and movable relative to the base to generate an operation instruction. The control method comprises: acquiring the posture of the base and the imaging part respectively; calculating a first deviation between the posture of the base and the posture of the imaging part; judging whether the first deviation reaches a first deviation threshold; when the first deviation reaches the first deviation threshold, compensating the first deviation to make the base and the imaging part substantially parallel; and after the base and the imaging part are substantially parallel, controlling the input part to generate an operation instruction for controlling the movement of an end instrument. The control method is beneficial to realizing intuitive control of the input part based on the imaging part.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of medical devices, in particular to a surgical robot, a master console and a control method and a control device thereof. BACKGROUND

[0002] Minimally invasive surgery refers to a surgical procedure performed inside a human body cavity using a laparoscope, a thoracoscope or other modern medical devices and related equipment. Compared with traditional surgical procedures, minimally invasive surgery has the advantages of less trauma, less pain and faster recovery.

[0003] With the progress of science and technology, minimally invasive surgical robot technology has gradually matured and is widely used. A surgical robot includes a master console and a slave operating device, and the slave operating device includes a plurality of operating arms, which include a camera arm having an image end instrument and a surgical arm having an operating end instrument. The master console includes a display and a handle. A surgeon operates the handle to control the movement of the camera arm or the surgical arm under the visual field provided by the camera arm displayed on the display.

[0004] In the prior art, the movement of the operating arm is usually presented based on the coordinate system of the display, and the control of the movement of the operating arm by the handle is presented based on another coordinate system. However, the two coordinate systems are usually not unified, which makes the control of the movement of the operating arm by the handle by the surgeon and the movement of the operating arm presented on the operating image as seen by the surgeon not unified, which is not conducive to intuitive control. SUMMARY

[0005] Therefore, it is necessary to provide a surgical robot, a master console and a control method and a control device thereof which are easy to achieve intuitive control.

[0006] In one aspect, the present application provides a control method of a master console, the master console including a display part and an operating part, the display part having an imaging part for final imaging, and the operating part having a base and an input part connected with the base and movable relative to the base to generate an operating instruction for controlling the movement of an end instrument, the control method including: acquiring the respective poses of the base and the imaging part; calculating a first deviation between the pose of the base and the pose of the imaging part; determining whether the first deviation reaches a first deviation threshold; when the first deviation reaches the first deviation threshold, compensating the first deviation to make the base and the imaging part substantially parallel; and after the base and the imaging part are substantially parallel, controlling the input part to generate the operating instruction for controlling the movement of the end instrument.

[0007] The compensating the first deviation to make the base and the imaging part substantially parallel includes: performing coordinate rotation on a first coordinate system defined on the base and / or a second coordinate system defined on the imaging part based on the first deviation, so that the first coordinate system and the second coordinate system are substantially parallel.

[0008] The main operating table includes a first adjusting mechanism for adjusting the posture of the imaging part, and the compensating the first deviation to make the base and the imaging part substantially parallel includes: controlling the first adjusting mechanism to adjust the posture of the imaging part based on the first deviation, so that a reference surface defined on the base and an imaging surface defined on the imaging part are substantially parallel; and / or, the main operating table includes a second adjusting mechanism for adjusting the posture of the base, and the compensating the first deviation to make the base and the imaging part substantially parallel includes: controlling the second adjusting mechanism to adjust the posture of the base based on the first deviation, so that the reference surface of the base and the imaging surface of the imaging part are substantially parallel.

[0009] The operation instruction generated by the input part is used to control the follow-up movement of the terminal instrument from the operating device, and the control method further includes: when the first deviation reaches the first deviation threshold, controlling the input part to be decoupled from the terminal instrument; and / or, when the first deviation does not reach the first deviation threshold, controlling the input part to be coupled with the terminal instrument.

[0010] After the compensating the first deviation to make the base and the imaging part substantially parallel, the control method further includes: acquiring the postures of the input part and the terminal instrument respectively; calculating a second deviation of the postures between the input part and the terminal instrument; judging whether the second deviation is less than a second deviation threshold; when the second deviation is less than the second deviation threshold, sending a follow-up signal to start the terminal instrument to enter a follow-up state of following the movement of the input part; and / or, when the second deviation reaches the second deviation threshold, compensating the second deviation to make the input part and the terminal instrument substantially aligned in posture.

[0011] The input part is a linkage input part having a plurality of active joints driven by motors, and the compensating the second deviation to make the input part and the terminal instrument substantially aligned in posture includes: under the condition of keeping the position of the input part, obtaining incremental joint variables of each of the active joints in the input part based on the second deviation by using inverse kinematics; and controlling the corresponding active joints to move based on each of the incremental joint variables and by using forward kinematics, so that the input part and the terminal instrument are substantially aligned in posture.

[0012] The main operating table further comprises an observation part which provides a window for observing the image formed by the imaging part, and the posture of the observation part and the imaging part is independently adjustable, and the control method comprises: acquiring a posture included angle between a line of sight axis of the observation part and the imaging part; calculating a third deviation between the posture included angle and a preset posture included angle; judging whether the third deviation reaches a third deviation threshold; when the third deviation reaches the third deviation threshold, adjusting the posture of the observation part and / or the imaging part based on the third deviation to make the posture included angle between the line of sight axis of the observation part and the imaging part substantially the same as the preset posture included angle.

[0013] The main operating table comprises two refractive adjustment assemblies for correcting the vision of left and right eyes, the refractive adjustment assembly comprises two lenses which are arranged in parallel and have an adjustable interval, and the control method further comprises: acquiring a target refractive power; matching a target interval between the two lenses from a relationship table according to the target refractive power, the relationship table has a correlation between the target refractive power and the target interval; acquiring a current interval between the two lenses; and controlling the two lenses to adjust from the current interval to the target interval based on the difference between the target interval and the current interval.

[0014] The main operating table comprises two refractive adjustment assemblies for correcting the vision of left and right eyes, the refractive adjustment assembly comprises two lenses which are arranged in parallel and have an adjustable interval, and the control method further comprises: acquiring a target refractive power; matching a target interval between the two lenses from a relationship table according to the target refractive power, the relationship table has a correlation between the target refractive power and the target interval; acquiring a current interval between the two lenses; and controlling the two lenses to adjust from the current interval to the target interval based on the difference between the target interval and the current interval.

[0015] In another aspect, the application also provides a main operating table, comprising a display part and an operating part, the display part has an imaging part for final imaging, and the operating part has a base and an input part connected with the base and movable relative to the base to generate an operation instruction, and the main operating table further comprises: a controller coupled with the display part and the operating part and configured to perform: acquiring the posture of the base and the posture of the imaging part; calculating a first deviation between the posture of the base and the posture of the imaging part; judging whether the first deviation reaches a first deviation threshold; when the first deviation reaches the first deviation threshold, compensating the first deviation to make the base and the imaging part substantially parallel; and after the base and the imaging part are substantially parallel, controlling the input part to generate an operation instruction for controlling the movement of the end instrument.

[0016] The display part comprises a display, and the imaging part is a physical display surface of the display.

[0017] The display part comprises a display and a mirror assembly, and the imaging part is a virtual display surface formed by the mirror assembly.

[0018] The mirror assembly comprises a plane mirror, and the display and the plane mirror form an included angle.

[0019] The mirror assembly comprises a convex lens, and the display and the convex lens are arranged in parallel.

[0020] The base is defined with a first coordinate system, and the imaging part is defined with a second coordinate system.

[0021] The main operating table comprises a first adjusting mechanism for adjusting the posture of the imaging part.

[0022] The operation instruction generated by the input part is used to control the end instrument to follow the movement of the operation device, and the controller is further configured to perform: when the first deviation reaches the first deviation threshold, controlling the input part to be decoupled from the end instrument; and / or, when the first deviation does not reach the first deviation threshold, controlling the input part to be coupled with the end instrument.

[0023] The controller is further configured to, after the compensation of the deviation makes the base and the imaging part substantially parallel, perform: acquiring the respective poses of the input part and the end instrument; calculating a second deviation of the poses between the input part and the end instrument; determining whether the second deviation is less than a second deviation threshold; when the second deviation is less than the second deviation threshold, sending a following signal to enable the end instrument to enter a following state of following the movement of the input part; and when the second deviation reaches the second deviation threshold, compensating the second deviation to make the input part and the end instrument substantially aligned in pose.

[0024] The input part is a linkage input part having a plurality of active joints driven by motors, and the controller is further configured to, after the compensation of the second deviation makes the input part and the end instrument substantially aligned in pose, perform: based on the second deviation, acquiring incremental joint variables of the active joints in the input part under the condition of keeping the position of the input part by using inverse kinematics; and based on the incremental joint variables and by using forward kinematics, controlling the corresponding active joints to move to make the input part and the end instrument substantially aligned in pose.

[0025] The main operation table further comprises an observation part providing a window to observe the image formed by the imaging part, and the poses of the observation part and the imaging part are independently adjustable, and the controller is further configured to perform: acquiring a pose angle between a line-of-sight axis of the observation part and the imaging part; calculating a third deviation between the pose angle and a preset pose angle; determining whether the third deviation reaches a third deviation threshold; and when the third deviation reaches the third deviation threshold, adjusting the pose of the observation part and / or the imaging part based on the third deviation to make the pose angle between the line-of-sight axis of the observation part and the imaging part substantially the same as the preset pose angle.

[0026] In another aspect, the present application also provides a computer readable storage medium storing a computer program configured to be loaded and executed by a processor to implement the steps of the control method according to any one of the above embodiments.

[0027] In another aspect, the present application also provides a control device of a master console, comprising: a memory for storing a computer program; and a processor for loading and executing the computer program; wherein the computer program is configured to be loaded and executed by the processor to implement the steps of the control method according to any one of the above embodiments.

[0028] In another aspect, the present application also provides a surgical robot comprising the master console according to any one of the above embodiments.

[0029] The surgical robot, the master console and the control method and control device thereof according to the present application have the following beneficial effects:

[0030] By compensating the posture of the base in the imaging part and the operating part in the display part with large posture deviation, the imaging part and the base are substantially parallel in posture, which is conducive to intuitive control of the input part based on the imaging part when the surgeon performs surgical operation by means of the input part, and the operation is simple and intuitive. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 FIG. 1 is a structural schematic diagram of an embodiment of the surgical robot according to the present application;

[0032] Figure 2 FIG. 2 is a partial schematic diagram of an embodiment of the surgical robot shown in FIG. 1; Figure 1

[0033] Figure 3 FIG. 3 is a flow chart of an embodiment of the control method of the surgical robot;

[0034] Figure 4 FIG. 4 is a structural schematic diagram of an embodiment of the operating arm and the power part in the surgical robot;

[0035] Figure 5 FIG. 5 is a structural schematic diagram of an embodiment of the master console according to the present application;

[0036] Figures 6-10 FIG. 6 is a structural schematic diagram of the master console in different embodiments;

[0037] Figure 11 FIG. 7 is a principle schematic diagram of the display part in the master console shown in FIG. 5; Figure 10

[0038] FIG. 8 is a structural schematic diagram of an embodiment of the adjusting mechanism in the master console; Figure 12

[0039] FIG. 9 is a structural schematic diagram of an embodiment of the adjusting mechanism in the master console; Figure 13

[0040] Figure 14 ​​Flow chart of an embodiment of the control method of the master console in the surgical robot of the present application;

[0041] Figure 15 Flow chart of an embodiment of the control method of the master console in the surgical robot of the present application;

[0042] Figure 16 Flow chart of an embodiment of the control method of the master console in the surgical robot of the present application;

[0043] Figure 17 Flow chart of an embodiment of the control method of the master console in the surgical robot of the present application;

[0044] Figure 18 Flow chart of an embodiment of the control method of the master console in the surgical robot of the present application;

[0045] Figure 19 Structure schematic diagram of the control device of the surgical robot of an embodiment of the present application. DETAILED DESCRIPTION

[0046] In order to facilitate the understanding of the present application, the present application will be described in more detail below with reference to the relevant drawings. The preferred embodiments of the present application are shown in the drawings. However, the present application can be realized in many different forms and is not limited to the embodiments described in the present application. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present application more thorough and comprehensive.

[0047] It should be noted that when an element is referred to as "provided on" another element, it can be directly on the other element or there can be a middle element. When an element is referred to as "connected" to another element, it can be directly connected to the other element or there can be a middle element. When an element is referred to as "coupled" to another element, it can be directly coupled to the other element or there can be a middle element. The terms "vertical", "horizontal", "left", "right", and similar expressions used in the present application are only for the purpose of illustration and do not represent the only embodiment. The terms "distal" and "proximal" used in the present application are used as an orientation term, which is a common term in the field of interventional medical devices, wherein "distal" means the end far from the operator during the operation, and "proximal" means the end close to the operator during the operation. The terms "first / second" and the like used in the present application represent one component and more than two components of the same type.

[0048] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description herein is for describing particular embodiments only and is not intended to be limiting of the application. The term "and / or" includes any and all combinations of one or more of the associated listed items. The term "each" means one or more than one.

[0049] As shown in FIG. 1 and FIG. 2, which are respectively a structural schematic diagram of an embodiment of the surgical robot of the present application and a partial schematic diagram thereof. Figures 1-2 As shown in FIG. 1 and FIG. 2, which are respectively a structural schematic diagram of an embodiment of the surgical robot of the present application and a partial schematic diagram thereof.

[0050] The surgical robot comprises a master console 2 and a slave operating device 3 controlled by the master console 2. The master console 2 has an operating part 21 and a display part 22. A doctor sends control commands to the slave operating device 3 by operating the operating part 21, so that the slave operating device 3 performs corresponding operations according to the control commands of the doctor operating the operating part 21, and the doctor observes the surgical area through the display part 22. The slave operating device 3 has a driving arm, which has a mechanical arm 30 and one or more operating arms 31 detachably arranged at the distal end of the mechanical arm 30. The mechanical arm 30 comprises a base and a connecting assembly connected in sequence, and the connecting assembly has a plurality of joint assemblies. The operating arm 31 comprises a connecting rod 32, a connecting assembly 33 and an end instrument 34 connected in sequence, wherein the connecting assembly 33 has a plurality of joint assemblies, the posture of the end instrument 34 is adjusted by adjusting the joint assemblies of the operating arm 31; the end instrument 34 has an image end instrument 34A and an operating end instrument 34B. The image end instrument 34A is used to collect images in the field of view, and the display part 22 is used to display the images. The operating end instrument 34B is used to perform surgical operations such as cutting and suturing. In this paper, the operating arm with the image end instrument 34A is referred to as a camera arm 31A, and the operating arm with the operating end instrument 34B is referred to as a surgical arm 31B.

[0051] Figure 1The surgical robot is a single-port surgical robot, and each operation arm 31 is inserted into the patient's body through the same puncture device 4 installed at the distal end of the mechanical arm 30. In the single-port surgical robot, the surgeon generally only controls the operation arm 31 to complete the basic surgical operation. At this time, the operation arm 31 of the single-port surgical robot should have both position freedom (i.e., positioning freedom) and attitude freedom (i.e., orientation freedom) to achieve the change of the position and attitude within a certain range, for example, the operation arm 31 has horizontal movement freedom x, vertical movement freedom y, rotation freedom a, pitch freedom β, and yaw freedom γ. The operation arm 31 can also realize the forward and backward movement freedom z (i.e., the feed freedom) under the driving of the joint assembly (i.e., the power mechanism 301) at the distal end of the mechanical arm 30. In addition, in some embodiments, the operation arm 31 can be provided with redundant freedom to realize the possibility of more functions, for example, in the above-mentioned 6 degrees of freedom, one, two or even more degrees of freedom are additionally provided. For example, the power mechanism 301 has a guide rail and a power part slidingly arranged on the guide rail, and the operation arm 31 is detachably arranged on the power part. On the one hand, the sliding of the power part on the guide rail provides the operation arm 31 with the forward and backward movement freedom z, and on the other hand, the power part provides power for the joint assembly of the operation arm 31 to realize the remaining 5 degrees of freedom (i.e., [x, y, a, β, γ]).

[0052] The surgical robot also includes a controller. The controller can be integrated into the master operating table 2 or the slave operating device 3. Of course, the controller can also be independent of the master operating table 2 and the slave operating device 3, which can be deployed locally, for example, and the controller can be deployed in the cloud, for example. Among them, the controller can be composed of more than one processor.

[0053] The surgical robot also includes an input part. The input part can be integrated into the master operating table 2. The input part can also be integrated into the slave operating device 3. Of course, the input part can also be independent of the master operating table 2 and the slave operating device 3. The input part can be a mouse, a keyboard, a voice input device, or a touch screen, for example. In an embodiment, a touch screen is used as the input part, which can be provided on the armrest of the master operating table 2, for example.

[0054] The operation arm 31 also includes sensors that sense the joint variables of the joint assembly. These sensors include angle sensors that sense the rotational motion of the joint assembly and displacement sensors that sense the linear motion of the joint assembly. The appropriate sensors can be configured according to the type of joint assembly.

[0055] The controller is coupled to these sensors.

[0056] For example, Figure 3As shown, a storage unit 311 is mounted on the contact surface of the drive box 310 of the manipulator 31 abutting against the power unit 302 of the power mechanism 301. Correspondingly, a reading unit 303, matching the storage unit 311, is mounted on the contact surface of the power unit 302 abutting against the drive box 310. The reading unit 303 is coupled to the controller. When the manipulator 31 is mounted on the power unit 302, the reading unit 303 communicates with the storage unit 311 and reads relevant information from the storage unit 311. The storage unit 311 may be, for example, a memory or an electronic tag. The storage unit may store, for example, the type of manipulator, the part of the manipulator that can be configured as a target location, and the kinematic model of the manipulator. For example, the storage unit 311 of the camera arm 31A also additionally stores camera parameters.

[0057] like Figure 4 As shown, it is a structural schematic diagram of an embodiment of the surgical robot of the present invention. More specifically, Figure 4 The diagram shown is a structural schematic of one embodiment of a multi-port surgical robot. Figure 4 The multi-hole surgical robot shown Figure 1 The main difference between the single-port surgical robots shown lies in the difference between their operating devices. Figure 4 The illustrated multi-port surgical robot comprises a robotic arm 110, an adjusting arm 120, a manipulator 130, and an operating arm 150 connected sequentially from the drive arm of the operating device. The number of adjusting arms 120, manipulators 130, and operating arms 150 is equal, and there are two or more of each, for example, four. The distal end of the robotic arm 110 has a directional platform, the proximal ends of the adjusting arms 120 are all connected to the directional platform, and the proximal end of the manipulator 130 is connected to the distal end of the adjusting arm 120. The manipulator 130 is used to detachably connect to the operating arm 150, and the manipulator 130 has multiple joint assemblies. Each manipulator 130 has a power mechanism, on which the operating arm 150 is mounted and further driven. In multi-port surgical robots, different manipulator arms 150 are inserted into the patient's body through different trocars. Compared to the manipulator arm 31 of a single-port surgical robot, the manipulator arm 150 of a multi-port surgical robot generally has fewer degrees of freedom. Typically, the manipulator arm 150 only has orientational degrees of freedom. Although changes in its orientation generally affect its position, the impact is small and can be ignored in some scenarios. Changes in the position of the manipulator arm 150 can usually be assisted by a manipulator 130. Since the manipulator 130 and the manipulator arm 150 work together to achieve changes in position, they can be considered as manipulator components, equivalent to the manipulator arm 31 in a single-port surgical robot.

[0058] According to the configuration, the operation part 21 can input a pose instruction including a position instruction and a posture instruction to control the change of the pose of the distal end of the first part. The distal end of the first part usually refers to the end instrument, and in addition, the distal end of the first part can also refer to a joint assembly connected with the end instrument, and the change of the pose of the end instrument is usually consistent with the change of the pose of the joint assembly.

[0059] In Figure 1 In the surgical robot shown in

[0060] In the surgical robot shown in Figure 4 In the surgical robot shown in

[0061] It can be understood that, whether it is the single-hole surgical robot shown in Figure 1 In the surgical robot shown in Figure 4 It can be understood that, whether it is the single-hole surgical robot shown in

[0062] In some embodiments, as shown in Figure 5 The main operating table 2 includes a controller 20, which is coupled with a display part 22 and an operation part 21. As shown in Figures 6-10As shown, the display part 22 has an imaging part 220 for final imaging, it is noted that the imaging part 220 can be a physically existing part or a virtually existing part, and refers to a part where the image finally provided for human viewing is located. The operation part 21 has a base 210 and another input part 212 connected with the base 210 and movable relative to the base 210 to generate operation instructions. The input part 212 can be a wearable input part such as a handle, a ring sleeve, etc. There is usually a wired connection between the input part 212 and the base 210, and optionally, the input part 212 and the base 210 can be flexibly connected through a cable to provide multiple degrees of freedom; alternatively, the input part 212 and the base 210 can be connected through a mechanical way, for example, a plurality of joint assemblies can be used to provide multiple degrees of freedom. Of course, the input part 212 and the base 210 can also be connected wirelessly, for example, the two are connected through electromagnetic induction, and for another example, the two are connected through 4G, 5G, Bluetooth, WiFi, etc. A reference coordinate system is defined on the base 210, and the movement of the input part 212 relative to the base 210 is mainly the movement with reference to the reference coordinate system.

[0063] The display part 22 and the operation part 21 are arranged on a support 23 of the main operating table 2. The controller 20 can be configured to perform: acquiring respective poses of the base 210 and the imaging part 220; calculating a first deviation of the poses between the base 210 and the imaging part 220; determining whether the first deviation reaches a first deviation threshold; when the first deviation reaches the first deviation threshold, compensating the first deviation to make the base 210 and the imaging part 220 substantially parallel; and after the base 210 and the imaging part 220 are substantially parallel, controlling the input part 212 to generate operation instructions for controlling the movement of the end instrument.

[0064] In the process of acquiring the respective poses of the base 210 and the imaging part 220, the controller 20 can:

[0065] In the main operating table with non-real-time adjustment of the base 210 and the imaging part 220, the pose of the base 210 and the pose of the imaging part 220 can be measured by a sensor, or can be obtained by reading the current state of the pose of the base 210 and the pose of the imaging part 220 stored in a storage device. The sensor can be a sensor arranged inside the main operating table. The sensor can also be a sensor outside the main operating table and not associated with the main operating table, which belongs to an independent measurement tool.

[0066] In some embodiments, the base 210 and / or the imaging portion 220 can be pose-adjustable in one or more of the three degrees of freedom of pitch, yaw and roll. Correspondingly, one or more sensors can be utilized to measure the pose in the corresponding degree(s) of freedom. For example, for the pose in one degree of freedom, one angle sensor can be utilized to measure the pose in the degree of freedom; for the pose in multiple degrees of freedom, multiple angle sensors can be utilized to measure the pose in the corresponding degrees of freedom, or one gyroscope can be utilized to measure the pose in multiple degrees of freedom.

[0067] wherein the acquired pose of the base 210 is usually the pose of a specified point or a specified surface defined thereon, and the acquired pose of the imaging portion 220 is usually the pose of a specified point or a specified surface defined thereon. Generally, the pose is the angle corresponding to the associated coordinate axis.

[0068] wherein the controller 20, when calculating the first deviation of the pose between the base 210 and the imaging portion 220, calculates the first deviation of the pose between the base 210 and the imaging portion 220 according to the acquired pose of the base 210 and the acquired pose of the imaging portion 220.

[0069] The first deviation usually includes the deviation of the pose in each degree of freedom, i.e. the deviation of the angle.

[0070] wherein the controller 20, when judging whether the first deviation reaches the first deviation threshold, judges whether the first deviation reaches the first deviation threshold according to the first deviation of the pose.

[0071] The first deviation threshold is set to be, for example, [-5°, 5°], of course, other larger or smaller value ranges can also be used, for example, [-8°, 8°], or for example, [-3°, 3°]. The first deviation threshold can be configured according to the different preferences of different doctors. In use, the corresponding configuration associated with the first deviation threshold can be called according to the different doctors logged in.

[0072] wherein the controller 20, when the first deviation reaches the first deviation threshold, compensates the first deviation to make the base and the imaging portion substantially parallel.

[0073] wherein the base 210 and the imaging portion 220 are considered to be substantially parallel when the first deviation is within the first deviation threshold. In particular, when the first deviation is zero, the base 210 and the imaging portion 220 are absolutely parallel. In this step S14, as long as the first deviation is within the first deviation threshold, it is not necessary that the first deviation is zero. For example, when the first deviation threshold is [-5°, 5°], the first deviation is compensated to be any value in -5°, -4°, -3°, -2°, -1°, 0°, 1°, 2°, 3°, 4°, 5° or any value between any two adjacent values.

[0074] The parallelism between the imaging portion 220 and the base 210 referred to herein can refer to the parallelism in the physical space between the imaging surface defined on the imaging portion 220 and a reference surface defined on the base 210, where the reference surface can be, for example, a certain plane in the base 210, the reference surface can be, for example, a surface on a component in the base 210, and the reference surface can also be, for example, a cross section of a component in the base 210; or can refer to the parallelism between a first coordinate system A defined on the imaging portion 220 and a second coordinate system B defined on the base 210, where the x-axis, the y-axis and the z-axis of the first coordinate system A are parallel to the x-axis, the y-axis and the z-axis of the second coordinate system B respectively when the first coordinate system A and the second coordinate system B are parallel.

[0075] For the case where neither the input portion 212 nor the imaging portion 220 is adjustable, the steps S11-S14 described above can be performed once and fixed when the main operating table is assembled.

[0076] For the case where either the input portion 212 or the imaging portion 220 is adjustable but not adjustable in real time, the steps S11-S14 described above can be performed once and triggered by the doctor when the system is started, or the steps S11-S14 described above can be performed once and automatically by the system when the system is initialized.

[0077] For the case where either the input portion 212 or the imaging portion 220 is adjustable and adjustable in real time, the steps S11-S14 described above can be performed repeatedly in real time.

[0078] Wherein, the controller 20 controls the input portion 212 to generate an operation instruction for controlling the motion of the terminal instrument after the base 210 and the imaging portion 220 are substantially parallel, and then the terminal instrument can move according to the operation instruction.

[0079] Through the above embodiments, it is beneficial to realize that the doctor operates the input portion 212 to control intuitively based on the operation image formed by the imaging portion 220. Wherein, the "intuitive control" herein refers to the fact that the motion direction of the input portion 212 is completely consistent with the motion direction of the operation terminal instrument in the operation image formed by the imaging portion 220 controlled by the input portion 212, of course, the motion direction of the input portion 212 is completely consistent with the motion direction of the operation image formed by the imaging portion 220 caused by the motion of the image terminal instrument controlled by the input portion 212 (i.e. the direction of the field of view change). Thus, the doctor can make the motion direction of the input portion 212 consistent with the motion direction of the terminal instrument in the operation image or the motion direction of the operation image itself when the doctor controls the terminal instrument to operate using the input portion 212, and then has the feeling of hand-eye consistency.

[0080] In some embodiments, as Figure 6 and Figure 7As shown, the display part 22 only includes a display, and the imaging part 220 corresponds to the display surface of the display, that is, the imaging part is physically present. In order to facilitate the measurement of the posture of the imaging part 220, a sensor can be arranged on the display, and the sensor is coupled to the controller 20.

[0081] In some embodiments, as shown in Figures 8-10 As shown, the display part 22 not only includes a display 221, but also includes a mirror assembly 222. In this embodiment, the imaging part 220 is not the display surface of the display 221, but is the display surface formed by the mirror assembly 222. Generally, the display surface formed by the mirror assembly can be considered as not physically present, but virtual, because the image formed by the mirror assembly is generally not on the corresponding mirror, for example, for both lenses and plane mirrors, the effective visible image formed by them is generally not on the mirror itself. The sensor coupled to the controller can be arranged on the display 221 or the mirror assembly 222 in the display part 22 to detect the posture of the display 221 or the mirror assembly 222, and the controller determines the posture of the imaging part 220 based on the sensed posture of the display 221 or the mirror assembly 222 and according to the relative positional relationship between the display 221 and the mirror assembly 222, which can generally be determined when the optical path is designed.

[0082] In one embodiment, as shown in Figure 8 and Figure 9 The mirror assembly 222 is more than one plane mirror, and the imaging part 220 refers to the display surface formed on one side of a plane mirror at the far end of the optical path. The support 23 is provided with a receiving cavity 24, and at least the display part 22 is arranged in the receiving cavity 24, more specifically, the display 221 and the mirror assembly 222 are accommodated in the receiving cavity 24, and the virtual imaging part can be located outside the receiving cavity 24. For example, the plane mirror 222 is one, and an included angle is formed between the display 221 and the plane mirror 222. In this embodiment, the imaging part 220 is the display surface on the side of the plane mirror 222 away from the display 221. Such an arrangement allows the operation part 21 to coincide with the imaging part 220 for more intuitive control, so that the doctor feels that his hand seems to operate on the imaging part 220 or the image formed on the imaging part 220 as real. In particular, such an arrangement can form the imaging part 220 outside the receiving cavity 24, so that the input part 212 can coincide with the imaging surface 220 for operation without being physically hindered by the receiving cavity 24.

[0083] In one embodiment, as shown in Figure 10 and Figure 11As shown, the mirror assembly 222 is a convex lens, which is arranged substantially parallel to the display 221. The imaging surface 220 can be on the same side or on the opposite side of the display 221. Preferably, the object distance between the convex lens 222 and the display 221 is set to be less than 1 focal length of the convex lens 222, so that the imaging surface 220 is on the same side of the display 221 and further away from the convex lens 222, and the image displayed on the display 221 is enlarged. In this way, the operator can intuitively control the imaging surface 220, and the doctor can feel that his hand is operating on the imaging surface 220 or the image displayed on the imaging surface 220.

[0084] In other embodiments, the mirror assembly 222 can be a combination of a plane mirror and a convex lens, and the mirror at the far end of the light path of the mirror assembly forms a virtual imaging surface.

[0085] As shown in Figure 10 and Figure 11 , in the embodiment in which the mirror assembly 222 includes a convex lens, the display 22 in the main operating console 2 can be a head-mounted display, such as a VR or AR glasses, so that the doctor can move his head flexibly. The sensor in the head-mounted display 22 is a gyroscope, which is used to directly or indirectly obtain the pose of the imaging surface 220.

[0086] In some embodiments, referring back to Figure 11 , the head-mounted display 22 further includes a refractive adjustment mechanism 250 for correcting the vision of the doctor. The refractive adjustment mechanism 250 can be a refractive adjustment mechanism for correcting myopia, which is formed by two or more concave lenses. The refractive adjustment mechanism 250 can be a refractive adjustment mechanism for correcting hyperopia, which is formed by two or more convex lenses.

[0087] Referring back to Figure 11 , the refractive adjustment mechanism 250 includes two independent refractive adjustment assemblies 260, which are respectively used to correct the vision of the left and right eyes of the doctor. Each refractive adjustment assembly 260 includes two lenses, which are arranged in parallel and have an adjustable distance. By changing the distance between the two parallel lenses, the combined focal length of the two lenses can be adjusted, and the refractive power can be changed according to the change of the combined focal length.

[0088] In an embodiment, the lenses can all be convex lenses, or all be concave lenses, or be a combination of convex lenses and concave lenses. By selecting lenses with different materials, different transmittances and / or different focal lengths, the adjustment of different ranges of refractive power can be realized.

[0089] The combined focal length of the lens can be determined according to formula (1):

[0090] f = f1 x f2 / (f1 + f2 - S) (1)

[0091] wherein f is the combined focal length, f1 is the focal length of one lens, f2 is the focal length of the other lens, and S is the distance between the two lenses. The focal length of a convex lens is usually positive, and the focal length of a concave lens is usually negative.

[0092] Further, the diopter number of the lens can be determined according to formula (2):

[0093] D = (1 / f) x 100 (2)

[0094] wherein D is the diopter number, and the unit of the combined focal length f is m.

[0095] By selecting the lenses and setting the distance between the two lenses, for example, 0-1000° of near vision diopter number can be adjusted. By selecting the lenses and setting the distance between the two lenses, for example, 0-500° of far vision diopter number can be adjusted.

[0096] In an embodiment, the above-mentioned diopter adjustment mechanism 250 can be arranged at the observation part 240 of the main operating table 2.

[0097] In an embodiment, the above-mentioned diopter adjustment mechanism 250 can also be integrated into the head-mounted device 22.

[0098] In an embodiment, a relationship table can be established between the target diopter number and the target distance between the two lenses, and then the diopter adjustment assembly 260 can be automatically adjusted according to the different diopter numbers. Accordingly, the controller 20 can also be configured to perform: obtaining a target diopter; matching a target distance from the relationship table according to the target diopter number; obtaining a current distance between the two lenses; and controlling the two lenses to adjust from the current distance to the target distance based on the difference between the target distance and the current distance.

[0099] In another embodiment, the controller 20 can also be configured to perform: obtaining a target diopter; determining the combined focal length between the two lenses according to the target diopter number; determining the target distance between the two lenses according to the determined combined focal length; obtaining the current distance between the two lenses; and controlling the two lenses to adjust from the current distance to the target distance based on the difference between the target distance and the current distance.

[0100] Wherein, when determining the combined focal length between the two lenses according to the target diopter number, the controller 20 can specifically determine the combined focal length between the two lenses according to the above-mentioned formula (1).

[0101] The controller 20 is configured to determine the target distance between the two lenses according to the determined combined focal length, specifically, the target distance between the two lenses can be determined according to the formula (2) as described above.

[0102] In some embodiments, the controller 20 can be configured to, when performing the obtaining of the target refractive power, specifically, can perform: obtaining the identity information of the doctor; obtaining the refractive power of the doctor according to the identity information of the doctor, and taking the obtained refractive power as the target refractive power. In some embodiments, the target refractive power can also be input by the doctor in real time.

[0103] In some embodiments, the controller 20 can be configured to, when performing the obtaining of the current distance between the two lenses, the current distance is obtained according to the distance sensor fixedly arranged relative to one of the two lenses. The current distance can also be obtained according to the motor encoder of the linear motor or the rotary motor.

[0104] The identity information of the doctor includes the refractive power of the doctor. The refractive power can be the refractive power input by the doctor, or the refractive power recorded when the doctor last used the refractive adjustment assembly. These information can be stored in the memory of the main operating platform, or stored in a server such as a cloud server in communication with the main operating platform.

[0105] Through such an automatic adjustment process, the controller 20 can quickly prepare an environment with clear vision for the doctor according to the different vision of the doctor, and also eliminates the annoyance of the doctor wearing glasses.

[0106] In the above embodiments, the display can be a 2D display or a 3D display.

[0107] In some embodiments, the controller 20 is configured to, when performing the compensation of the first deviation to make the base and the imaging part substantially parallel, specifically, can be: based on the first deviation, performing coordinate rotation on the first coordinate system and / or the second coordinate system to make the first coordinate system and the second coordinate system substantially parallel. Wherein, as shown in Figure 6 and Figure 8 The first coordinate system A is defined fixed on the base 210, and the second coordinate system B is defined fixed on the imaging part 220. This way realizes compensation through coordinate rotation, without changing the structure, easy to use, and this coordinate rotation in a way to make the first coordinate system A and the second coordinate system B parallel is especially suitable for the head-mounted display part.

[0108] In some embodiments, the main control panel 2 may include a first adjustment mechanism for directly or indirectly adjusting the attitude of the imaging unit. When the controller 20 performs compensation for the first deviation to make the base and the imaging unit substantially parallel, it may specifically control the first adjustment mechanism to adjust the attitude of the imaging unit based on the first deviation so that the base and the imaging unit are substantially parallel in physical space.

[0109] In some embodiments, the main control panel 2 may include a second adjustment mechanism for adjusting the posture of the base. When the controller 20 performs compensation for the first deviation to make the base substantially parallel to the imaging unit, it may specifically control the second adjustment mechanism to adjust the posture of the base based on the first deviation so that the base and the imaging unit are substantially parallel in physical space.

[0110] In other embodiments, the main control panel 2 may simultaneously include a first adjustment mechanism for directly or indirectly adjusting the attitude of the imaging unit and a second adjustment mechanism for adjusting the attitude of the base. When the controller 20 performs compensation for the first deviation to make the base and the imaging unit basically parallel, it may specifically: control the first adjustment mechanism and the second adjustment mechanism to simultaneously adjust the attitude of the imaging unit and the base based on the first deviation so that the base and the imaging unit are basically parallel in physical space.

[0111] Either the first adjustment mechanism or the second adjustment mechanism can be implemented using multiple structures. Taking the example that all three degrees of freedom of the corresponding object can be adjusted, we can illustrate this.

[0112] For example, such as Figure 12 As shown, the adjustment mechanism can be implemented using a spherical joint structure 40, primarily used in conjunction with manual adjustment of the object's posture. The spherical joint structure 40 includes a fixed portion 41 and a movable portion 42 movably disposed relative to the fixed portion 41. A support portion 43 for supporting the object is also fixedly disposed on the movable portion 42. The posture of the object supported on the support portion 43 is adjusted by moving the movable portion 42 relative to the fixed portion 41. For example, the fixed portion 41 is a spherical groove, and the movable portion 42 is a sphere. Alternatively, the fixed portion 41 is a sphere, and the movable portion 42 is a spherical groove. The component supported by the support portion 43 is typically a component with a solid structure; for example, when the imaging unit 220 is virtually present, the component supported by the support portion 43 is the display 221 and / or the mirror assembly 222. The sensor for detecting the object's posture is typically fixedly disposed with the object; it can be disposed either on the object or on the support portion 43.

[0113] Further, the ball pair structure 40 can further comprise a brake member, the control part is coupled with the ball pair structure 40 having the brake member, and more specifically, the control part is coupled with the brake member. The brake member comprises a static friction sheet and a dynamic friction sheet which are separated from each other in a power-on state and are attracted to each other in a power-off state. One of the static friction sheet and the dynamic friction sheet is arranged on the fixed part 41, and the other is arranged on the movable part 42. For example, the static friction sheet is arranged on the fixed part 41, and the dynamic friction sheet is arranged on the movable part 42. For example, the static friction sheet can be arranged on the surface of the fixed part 41 which contacts the movable part, and the dynamic friction sheet can be arranged on the surface of the movable part 42 which contacts the fixed part. Through the design of the brake member which is separated in the power-on state and is attracted in the power-off state, the movable part can be unlocked by power-on to adjust the posture of the corresponding object, and the movable part can be locked by power-off after the adjustment. This structure design is also applicable to the magnetic navigation type operation part 21, because the power-on to generate the magnetic force is triggered only when the posture of the base 210 of this type of operation part 21 is adjusted, and this adjustment process does not affect the normal use because the input part 212 is not used or is not allowed to be used to control the end instrument.

[0114] The above brake member can be replaced. For example, one of the groove and the ball in the ball pair structure 40 is a component which can generate a magnetic force by power-on, and the other is a component which can be attracted by the magnetic force. The control part is coupled with the component which can generate a magnetic force by power-on. Here, the component which can generate a magnetic force by power-on refers to the component itself which can generate a magnetic force by power-on, or refers to the component which is provided with a film layer which can generate a magnetic force by power-on. The component which can be attracted by the magnetic force refers to the component itself which can be attracted by the magnetic force, or refers to the component which is provided with a film layer which can be attracted by the magnetic force. For example, when the fixed part 41 is the groove and the movable part 42 is the ball, the groove is the groove which can generate a magnetic force by power-on, and correspondingly, the ball is the ball which can be attracted by the magnetic force generated by the groove when the groove is powered on. For example, the ball is made of a material such as iron or iron-nickel alloy. When the posture of the corresponding object needs to be adjusted, the control part controls the power-off to lock the movable part, and when the posture of the corresponding object does not need to be adjusted, the control part controls the power-on to unlock the movable part.

[0115] Because the groove and the ball in the ball pair structure 40 can be unlocked or locked by power-on, and correspondingly, can be locked or unlocked by power-off, it is easy to adjust the posture of the attached object supported on the support part connected with one of the groove and the ball to meet the individual needs of different doctors for different postures of the attached object. When the posture of the attached object needs to be adjusted, the ball pair structure 40 is unlocked to adjust the posture of the attached object, and when the posture of the attached object does not need to be adjusted, the ball pair structure 40 is locked to maintain the posture of the attached object, which is simple and easy to use. The attached object here can be the base 210 of the operation part 21 or the imaging part 220 of the display part 22.

[0116] For example, as shown in FIG. 6, the ball pair structure 40 can be replaced by a ball pair structure 40' which comprises a fixed part 41' and a movable part 42'. The fixed part 41' is provided with a groove 43', and the movable part 42' is provided with a ball 44'. The ball 44' is arranged in the groove 43' and can move in the groove 43'. The ball 44' is provided with a magnetic material, and the groove 43' is provided with a magnetic film layer which can attract the ball 44' when the magnetic film layer is powered on. The control part is coupled with the magnetic film layer. When the posture of the corresponding object needs to be adjusted, the control part controls the power-off of the magnetic film layer to lock the movable part 42', and when the posture of the corresponding object does not need to be adjusted, the control part controls the power-on of the magnetic film layer to unlock the movable part 42'. Figure 13As shown, the adjusting mechanism can also be implemented by a multi-attitude freedom mechanical arm 50, which can adjust the attitude of the corresponding object in a manual manner and / or an automatic manner. The mechanical arm 50 at least includes a base 51 and a first connecting rod 52, a second connecting rod 53 and a third connecting rod 54. The first connecting rod 52 is movably connected with the base 51 and adjustable in a first attitude. The second connecting rod 53 is movably connected with the first connecting rod 52 and adjustable in a second attitude. The third connecting rod 54 is movably connected with the second connecting rod 53 and adjustable in a third attitude. The movable connection between adjacent components is achieved by a rotating joint. The third connecting rod 54 is provided with a support portion for supporting the corresponding object, so as to adjust the attitude of the corresponding object supported on the support portion. The support portion can be the third connecting rod 54 itself. For example, the first attitude is rotation, the second attitude is yaw, and the third attitude is pitch. Of course, other combinations can also be used, which will not be described here. In an embodiment, the first connecting rod 52, the second connecting rod 53 and the third connecting rod 54 can be manually driven without being driven by a rotating motor, of course, a rotating motor can also be provided to automatically drive the rotating joint. In other embodiments, when the adjustable attitude of the corresponding object is less, a simpler structure can be used, or a simple improvement can be made based on the above-mentioned spherical pair structure or multi-attitude freedom mechanical arm, which will not be described here.

[0117] In some embodiments, the operation instruction generated by the movement of the input portion 212 relative to the base 210 is used to control the slave operation device to follow the movement of the end instrument, and more specifically, the operation instruction is used to control the movement of the first portion of the proximal end configuration of the end instrument to achieve the movement of the end instrument. The controller 20 can also be configured to control the input portion 212 to decouple from the end instrument when it is determined that the first deviation reaches the first deviation threshold. The decoupling from the end instrument can be achieved by decoupling from the entire slave operation device or only from the first portion.

[0118] In some embodiments, the controller 20 can also be configured to control the input portion 212 to couple with the end instrument when it is determined that the first deviation does not reach the first deviation threshold. The coupling with the end instrument can be achieved by coupling with the entire slave operation device or only with the first portion.

[0119] According to the result of whether the first deviation reaches the first deviation threshold, whether the input portion and the end instrument are coupled or not is determined, which can avoid the occurrence of unexpected movement caused by the change of the attitude of the base relative to the imaging portion. That is, only when the first deviation does not reach the first deviation threshold, i.e., the attitude of the base relative to the imaging portion is not changed, the input portion is coupled with the end instrument, which will not cause the occurrence of unexpected movement of the end instrument.

[0120] In some embodiments, when the input portion 212 is used to manipulate the end-effector, it is generally desired that both change in the same reference coordinate system with substantially the same initial pose and consistent, and since the deviation between the pose of the base and the pose of the imaging portion is compensated, it is easy to cause the deviation between the pose of the input portion and the pose of the end-effector to become large, which is not conducive to intuitive manipulation. Therefore, the controller 20 can also be configured to, after the base and the imaging portion are substantially parallel after the first deviation is compensated, perform: obtaining the respective poses of the input portion 212 and the end-effector; calculating a second deviation in pose between the input portion 212 and the end-effector; determining whether the second deviation is less than a second deviation threshold; when the second deviation is less than the second deviation threshold, sending a following signal to start the end-effector into a following state of following the movement of the input portion 212; and when the second deviation is greater than or equal to the second deviation threshold, compensating the second deviation to substantially align the input portion 212 and the end-effector in pose.

[0121] In the process of obtaining the respective poses of the input portion 212 and the end-effector, the controller 20 can:

[0122] The pose of the end-effector can be determined based on the joint variables of the joint assemblies of the first part driving the movement of the end-effector and using forward kinematics. When the input portion is a non-link input portion such as a magnetic navigation input portion, the pose of the input portion can be obtained by a gyroscope, and when the input portion is a link input portion, the pose can also be determined based on the joint variables of the joint assemblies in the input portion and using forward kinematics.

[0123] In the process of calculating the second deviation in pose between the input portion and the end-effector, the controller 20 can:

[0124] Generally, the deviation in pose between the two needs to be calculated in the same coordinate system.

[0125] In the process of determining whether the second deviation is less than the second deviation threshold, the controller 20 can:

[0126] When the second deviation is less than the second deviation threshold, it indicates that the input portion and the end-effector are substantially aligned in pose.

[0127] When the input portion 212 is a non-link input portion such as a magnetic navigation input portion, in the process of compensating the second deviation to substantially align the input portion 212 and the end-effector in pose, the controller 20 can do so by keeping the pose of the end-effector and adjusting the pose of the input portion to align with the pose of the end-effector. In this case, a first coordinate image can be generated according to the pose of the input portion, and a second coordinate image can be generated according to the pose of the end-effector, and the two coordinate images can be set with the same origin for easy observation. The pose of the input portion can be easily adjusted to align with the pose of the end-effector according to the relationship between the two coordinate images.

[0128] When the input portion 212 is a linkage input portion having a plurality of active joints driven by rotary motors, the controller 20 is further configured to, when the second deviation is compensated so that the input portion and the end effector are substantially aligned in pose, perform: obtaining, based on the second deviation, incremental joint variables of the active joints in the input portion using inverse kinematics while maintaining the position of the input portion; and controlling the corresponding active joints to move based on the incremental joint variables using forward kinematics so that the input portion and the end effector are substantially aligned in pose.

[0129] The main console 2 further comprises a viewing portion 240, which can be provided on the accommodation cavity 24, and which provides a window for viewing the image formed by the imaging portion 220. The pose of the viewing portion 240 and the imaging portion 220 is independently adjustable, and the viewing portion 240 can be manually adjusted or automatically adjusted. The viewing portion 240 is provided with a pose sensor such as a gyroscope for sensing the pose thereof. The controller 20 can be further configured to perform: obtaining a pose angle between the line-of-sight axis of the viewing portion 240 and the imaging portion 220; calculating a third deviation between the pose angle and a preset pose angle; and when the third deviation reaches a third deviation threshold, adjusting the pose of the viewing portion 240 and / or the imaging portion 220 based on the third deviation so that the pose angle between the line-of-sight axis of the viewing portion 240 and the imaging portion 220 is substantially the same as the preset pose angle.

[0130] The pose angle can be obtained based on the pose sensed by the pose sensor provided on the viewing portion 240.

[0131] The preset pose angle is 85° to 95°, for example, 90°.

[0132] According to this embodiment, when the third deviation between the pose angle between the viewing portion 240 and the imaging portion 220 and the preset pose angle reaches the third deviation threshold, the pose of the viewing portion 240 and / or the imaging portion 220 can be adjusted to ensure that the preset pose angle is always within the preset pose angle, thereby achieving good viewing effect. In particular, in combination with the compensation of the base 210 and the imaging portion 220 to substantially parallelly arrange the base 210 and the imaging portion 220, on the one hand, the viewing effect can be improved, and on the other hand, more realistic intuitive control can be achieved.

[0133] Further, the refractive adjustment mechanism described above can be provided at the viewing portion 240 to eliminate the annoyance of myopic or hyperopic doctors wearing glasses.

[0134] The above-mentioned "electrically adjustable rotating portion" refers to a rotating portion that can be rotated by electric control to achieve pose adjustment. Such a rotating portion can be implemented, for example, by using the structure and principle of an active joint of a mechanical arm in an industrial robot, or by using a common motor-driven gear rotating structure, as long as it can be electrically controlled to rotate.

[0135] The application also provides a control method of the main console. In one embodiment, as shown in Figure 14 the control method comprises:

[0136] Step S11, acquiring the poses of the base and the imaging part respectively.

[0137] Step S12, calculating the first deviation of the poses between the base and the imaging part.

[0138] Step S13, judging whether the first deviation reaches a first deviation threshold.

[0139] Step S14, when the first deviation reaches the first deviation threshold, compensating the first deviation to make the base and the imaging part substantially parallel.

[0140] Step S15, after the base and the imaging part are substantially parallel, controlling the input part to generate an operation instruction for controlling the motion of the end instrument.

[0141] According to the above steps S11-S15, it is beneficial to realize the intuitive control of the doctor operating the input part based on the operation image formed by the imaging part.

[0142] The above step S14, i.e. compensating the first deviation to make the base and the imaging part substantially parallel, can be based on the first deviation to perform coordinate rotation on a first coordinate system defined on the base and / or a second coordinate system defined on the imaging part, so that the first coordinate system and the second coordinate system are substantially parallel.

[0143] The above step S14, i.e. compensating the first deviation to make the base and the imaging part substantially parallel, can also be based on the first deviation to control the first adjusting mechanism to adjust the pose of the imaging part and / or control the second adjusting mechanism to adjust the base, so that the reference surface defined on the base and the imaging surface defined on the imaging part are substantially parallel in the physical space.

[0144] In one embodiment, the input part can be controlled to be decoupled from the end instrument when the first deviation reaches the first deviation threshold. In addition, the input part can be controlled to be coupled with the end instrument when the first deviation does not reach the first deviation threshold.

[0145] In some embodiments, the interval between the two lenses can be formed by driving the electrically adjustable lens, as shown in Figure 15 the control method can comprise:

[0146] Step S211, acquiring a target refractive power.

[0147] Step S212, matching the target interval from the relationship table according to the target refractive power.

[0148] The relationship table is preset, and the relationship table associates the target dioptric power and the target interval between the two lenses, that is, the relationship table has an association relationship between the target dioptric power and the target interval.

[0149] In step S213, the current interval between the two lenses is obtained.

[0150] In step S214, the driving part is controlled to work based on the difference between the target interval and the current interval, so that the two lenses are adjusted from the current interval to the target interval.

[0151] In some embodiments, the interval between the two lenses can be adjusted by the electrically adjustable driving part, for example, as shown in the following figure. Figure 16 As shown in the following figure, the control method can include:

[0152] In step S221, the target dioptric power is obtained.

[0153] In step S222, the combined focal length between the two lenses is determined according to the target dioptric power.

[0154] The combined focal length between the two lenses can be determined according to the above formula (1).

[0155] In step S223, the target interval between the two lenses is determined according to the determined combined focal length.

[0156] The target interval between the two lenses can be determined according to the above formula (2).

[0157] In step S224, the current interval between the two lenses is obtained.

[0158] In step S225, the driving part is controlled to work based on the difference between the target interval and the current interval, so that the two lenses are adjusted from the current interval to the target interval.

[0159] According to the automatic adjustment process of the above steps S211-S214 or the above steps S221-S225, a clear vision environment can be quickly prepared for the doctor according to the doctor's vision, and the doctor's annoyance of wearing glasses is also avoided.

[0160] The above step S211 and the above step S221, that is, obtaining the target dioptric power, can be receiving the target dioptric power input by the doctor, in addition, the target dioptric power can also be obtained by the following steps: obtaining the identity information of the doctor; obtaining the dioptric power of the doctor according to the identity information of the doctor, and taking the obtained dioptric power as the target dioptric power.

[0161] The doctor's identity information is editable. For example, when a doctor logs into the user interface of the surgical robot, their identity information can be automatically linked. The doctor can log in by entering an account and password, by fingerprint recognition, by facial recognition, or even by voice recognition.

[0162] In some embodiments, such as Figure 17 As shown, after step S14, that is, after compensating for the first deviation to make the base substantially parallel to the imaging unit, the following steps are also included:

[0163] Step S31: Obtain the respective postures of the input unit and the end effector.

[0164] Step S32: Calculate the second deviation of the attitude between the input unit and the end effector.

[0165] Step S33: Determine whether the second deviation is less than the second deviation threshold.

[0166] When the second deviation is less than the second deviation threshold, it indicates that the input unit and the end effector are basically aligned in posture, and proceed to step S34; when the second deviation is greater than or equal to the second deviation threshold, proceed to step S35.

[0167] Step S34: Send a follow signal to start the end effector into the follow state of the follow input unit movement.

[0168] Step S35: Compensate for the second deviation to make the input unit and the end effector basically aligned in attitude, and then re-enter step S31 above.

[0169] In one embodiment, when the input unit is a linkage-type input unit having multiple active joints driven by a rotary motor, the above-mentioned step S35, namely, compensating for the second deviation to make the input unit and the end effector substantially aligned in attitude, can be performed in the following manner:

[0170] While maintaining the position of the input section, the incremental joint variables of each active joint in the input section are obtained using inverse kinematics based on the second deviation;

[0171] Based on each incremental joint variable, and using positive kinematics to control the corresponding active joint movements, the input unit and the end effector are basically aligned in posture.

[0172] In some embodiments, based on Figure 8 In simple terms, the main control panel 2 shown has an independently adjustable orientation between the observation unit 240 and the imaging unit 220. The observation unit 240 can be adjusted manually or automatically, and it is equipped with an orientation sensor, such as a gyroscope, to sense its orientation. Figure 18 As shown, the control method also includes:

[0173] Step S41, obtain the posture included angle between the line-of-sight axis of the observation part and the imaging part.

[0174] Step S42, calculate the third deviation between the posture included angle and the preset posture included angle.

[0175] The preset posture included angle is 85°-95°, for example, 90°.

[0176] Step S43, determine whether the third deviation reaches the third deviation threshold.

[0177] Step S44, when the third deviation reaches the third deviation threshold, adjust the posture of the observation part and / or the imaging part based on the third deviation to make the posture included angle between the line-of-sight axis of the observation part and the imaging part substantially the same as the preset posture included angle.

[0178] Through the above steps S41-S44, when the third deviation between the posture included angle between the observation part and the imaging part and the preset posture included angle reaches the third deviation threshold, the posture of the observation part and / or the imaging part can be adjusted to ensure that it is always within the preset posture included angle, and has a good observation effect. Especially in combination with the above steps S11-S14, on the one hand, the observation effect can be improved, and on the other hand, a more realistic intuitive control can be realized.

[0179] It is worth noting that, for the purpose of brevity, the related steps of each embodiment in the control method are not described in detail. In fact, they can be referred to in combination with the above-described embodiments of the main operating table, so that the implementation of the related steps of each embodiment in the control method can be easily understood.

[0180] On the other hand, the present application also provides a computer readable storage medium, which stores a computer program configured to be loaded and executed by a processor to implement the steps of the control method according to any one of the above embodiments.

[0181] On the other hand, the present application also provides a surgical robot comprising the main operating table according to any one of the above embodiments.

[0182] In an embodiment, a control device of a surgical robot is provided. As shown in the figure, the control device can include a processor 501, a communications interface 502, a memory 503, and a communications bus 504. Figure 19

[0183] ​The processor 501, the communication interface 502, and the memory 503 complete communication with each other through a communication bus 504.

[0184] The communication interface 502 is configured to communicate with network elements such as various sensors, rotary motors, electromagnetic valves, other clients, servers, and the like.

[0185] The processor 501 is configured to execute the program 505, and specifically can execute the related steps in the above method embodiments.

[0186] Specifically, the program 505 can include program code including computer operation instructions.

[0187] The processor 505 can be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present application, or a graphics processing unit (GPU). The one or more processors included in the control device can be processors of the same type, such as one or more CPUs, or one or more GPUs; or can be processors of different types, such as one or more CPUs and one or more GPUs.

[0188] The memory 503 is configured to store the program 505. The memory 503 can include a high-speed RAM memory, and can also include a non-volatile memory such as at least one disk memory.

[0189] The program 505 can specifically be configured to cause the processor 501 to perform the following operations: obtaining poses of the base and the imaging part respectively; calculating a first deviation of the poses between the base and the imaging part; determining whether the first deviation reaches a first deviation threshold; when the first deviation reaches the first deviation threshold, compensating the first deviation to make the base and the imaging part substantially parallel; and after the base and the imaging part are substantially parallel, controlling the input part to generate an operation instruction for controlling movement of the end instrument.

[0190] The technical features of the above-described embodiments can be combined in any manner. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described, however, as long as the combinations of the technical features do not contradict each other, they should be considered within the scope of the present disclosure.

[0191] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be noted that for ordinary skilled persons in the art, without departing from the concept of the present application, several modifications and improvements can be made, which are within the scope of protection of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. A control method of a main console, characterized by, The main console includes a display part having an imaging part for final imaging and an operation part having a base and an input part connected with the base and movable relative to the base to generate an operation instruction for controlling the motion of the end instrument, and the control method comprises: acquiring the poses of the base and the imaging part respectively; calculating a first deviation between the pose of the base and the pose of the imaging part; the first deviation includes the pose deviation in each degree of freedom, i.e. the angle deviation; judging whether the first deviation reaches a first deviation threshold; when the first deviation reaches the first deviation threshold, compensating the first deviation to make the base and the imaging part substantially parallel; after the base and the imaging part are substantially parallel, controlling the input part to generate an operation instruction for controlling the motion of the end instrument.

2. The control method according to claim 1, characterized by, The compensation of the first deviation to make the base and the imaging part substantially parallel comprises: based on the first deviation, performing coordinate rotation on a first coordinate system defined on the base and / or a second coordinate system defined on the imaging part to make the first coordinate system and the second coordinate system substantially parallel.

3. The control method according to claim 1, characterized by, The main console includes a first adjusting mechanism for adjusting the pose of the imaging part, and the compensation of the first deviation to make the base and the imaging part substantially parallel comprises: based on the first deviation, controlling the first adjusting mechanism to adjust the pose of the imaging part to make a reference surface defined on the base and an imaging surface defined on the imaging part substantially parallel; and / or, the main console includes a second adjusting mechanism for adjusting the pose of the base, and the compensation of the first deviation to make the base and the imaging part substantially parallel comprises: based on the first deviation, controlling the second adjusting mechanism to adjust the pose of the base to make the reference surface of the base and the imaging surface of the imaging part substantially parallel.

4. The control method according to claim 1, characterized by, The operation instruction generated by the input part is used to control the end instrument to follow the motion of the operating device, and the control method further comprises: when the first deviation reaches the first deviation threshold, controlling the input part to decouple from the end instrument; and / or, when the first deviation does not reach the first deviation threshold, controlling the input part to couple with the end instrument.

5. The control method according to claim 1, characterized by, After the compensation of the first deviation to make the base and the imaging part substantially parallel, the control method further comprises: acquiring the poses of the input part and the end instrument respectively; calculating a second deviation between the poses of the input part and the end instrument; judging whether the second deviation is less than a second deviation threshold; when the second deviation is less than the second deviation threshold, sending a following signal to start the end instrument to enter a following state of following the motion of the input part; and / or, when the second deviation reaches the second deviation threshold, compensating the second deviation to make the input part and the end instrument substantially aligned in pose.

6. The control method according to claim 5, characterized by The input part is a linkage input part having a plurality of driven joints driven by motors, and the compensation of the second deviation to make the input part and the end instrument substantially aligned in pose comprises: acquiring, based on the second deviation, incremental joint variables of each of the active joints in the input portion by inverse kinematics under the condition that the position of the input portion is maintained; controlling the corresponding active joint to move based on each of the incremental joint variables and by forward kinematics so that the input portion and the end instrument are substantially aligned in pose.

7. The control method according to claim 1, characterized by, The main console further comprises an observation portion providing a window for observing an image formed by the imaging portion, the observation portion and the imaging portion being independently adjustable in pose, and the control method comprises: acquiring a pose angle between a line-of-sight axis of the observation portion and the imaging portion; calculating a third deviation between the pose angle and a preset pose angle; judging whether the third deviation reaches a third deviation threshold; when the third deviation reaches the third deviation threshold, adjusting the pose of the observation portion and / or the imaging portion based on the third deviation so that the pose angle between the line-of-sight axis of the observation portion and the imaging portion is substantially the same as the preset pose angle.

8. The control method according to claim 1, characterized by, The main console comprises two refractive adjustment assemblies for correcting the vision of left and right eyes, the refractive adjustment assembly comprising two lenses arranged in parallel and adjustable in distance, and the control method further comprises: acquiring a target refractive power; matching a target distance between the two lenses from a relationship table according to the target refractive power, the relationship table having a correlation between the target refractive power and the target distance; acquiring a current distance between the two lenses; controlling the two lenses to adjust from the current distance to the target distance based on a difference between the target distance and the current distance.

9. The control method according to claim 1, characterized by, The main console comprises two refractive adjustment assemblies for correcting the vision of left and right eyes, the refractive adjustment assembly comprising two lenses arranged in parallel and adjustable in distance, and the control method further comprises: acquiring a target refractive power; determining a combined focal length between the two lenses according to the target refractive power; determining a target distance between the two lenses according to the determined combined focal length; acquiring a current distance between the two lenses; controlling the two lenses to adjust from the current distance to the target distance based on a difference between the target distance and the current distance.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program configured to be loaded and executed by a processor to implement the steps of the control method according to any one of claims 1-9.

11. A control device of a main console, characterized by comprising: comprises: a memory for storing a computer program; and a processor for loading and executing the computer program; wherein the computer program is configured to be loaded and executed by the processor to implement the steps of the control method according to any one of claims 1-9.

12. A master console characterized by comprising: comprises a display portion having an imaging portion for final imaging and an operation portion having a base and an input portion connected with the base and movable relative to the base to generate an operation instruction, and the main console further comprises: a controller coupled with the display portion and the operation portion and configured to perform: acquiring a pose of each of the base and the imaging portion; calculating a first deviation between a pose of the base and a pose of the imaging part; the first deviation includes a pose deviation in each degree of freedom, i.e. an angle deviation; judging whether the first deviation reaches a first deviation threshold; compensating the first deviation to make the base and the imaging part substantially parallel when the first deviation reaches the first deviation threshold; controlling the input part to generate an operation instruction for controlling a motion of an end instrument after the base and the imaging part are substantially parallel.

13. The main console of claim 12, wherein, the display part includes a display, and the imaging part is a physical display surface of the display, and the display is provided with a sensor coupled with the controller for sensing a pose of the display.

14. The main console of claim 12, wherein, the display part includes a display and a mirror assembly, and the imaging part is a virtual display surface formed by the mirror assembly, and the display part has a sensor coupled with the controller for sensing a pose of the display or the mirror assembly, and the controller determines a pose of the imaging part based on the sensed pose of the display and the pose of the mirror assembly and according to a relative positional relationship between the display and the mirror assembly.

15. The main console of claim 14, wherein, the mirror assembly includes a plane mirror, and the display and the plane mirror form an included angle therebetween, and the plane mirror is located between the display and the imaging part, so that the input part can be coincided with the imaging part for intuitive control.

16. The main console of claim 14, wherein, the mirror assembly includes a convex lens, and the convex lens is arranged in parallel with the display, and the display is located between the convex lens and the imaging part, so that the input part can be coincided with the imaging part for intuitive control.

17. The master station of claim 12 wherein, the base is defined to have a first coordinate system, and the imaging part is defined to have a second coordinate system, and the compensation of the first deviation to make the base and the imaging part substantially parallel includes: performing a coordinate rotation on the first coordinate system and / or the second coordinate system based on the first deviation so that the first coordinate system and the second coordinate system are substantially parallel.

18. The master station of claim 12 wherein, the main operating table includes a first adjusting mechanism for adjusting a pose of the imaging part, and the compensation of the first deviation to make the base and the imaging part substantially parallel includes: controlling the first adjusting mechanism to adjust the pose of the imaging part based on the first deviation so that a reference surface of the base and an imaging surface of the imaging part are substantially parallel; and / or, the main operating table includes a second adjusting mechanism for adjusting a pose of the base, and the controller includes: controlling the second adjusting mechanism to adjust the pose of the base based on the first deviation so that the reference surface of the base and the imaging surface of the imaging part are substantially parallel when performing the compensation of the first deviation to make the base and the imaging part substantially parallel.

19. The master station of claim 12 wherein, the operation instruction generated by the input part is used to control a follow-up motion of an end instrument from an operating device, and the controller is further configured to perform: controlling the input part to be decoupled from the end instrument when the first deviation reaches the first deviation threshold; and / or, controlling the input part to be coupled with the end instrument when the first deviation does not reach the first deviation threshold.

20. The master station of claim 12 wherein, The controller is further configured to, after the compensating the first deviation to make the base and the imaging portion substantially parallel, perform: acquiring poses of the input portion and the end instrument respectively; calculating a second deviation of poses between the input portion and the end instrument; determining whether the second deviation is less than a second deviation threshold; when the second deviation is less than the second deviation threshold, sending a following signal to initiate the end instrument to enter a following state of following the input portion; when the second deviation reaches the second deviation threshold, compensating the second deviation to make the input portion and the end instrument substantially aligned in poses.

21. The master station of claim 20 wherein, The input portion is a linkage input portion having a plurality of active joints driven by motors, and the controller is further configured to, when the compensating the second deviation to make the input portion and the end instrument substantially aligned in poses, perform: acquiring incremental joint variables of the active joints in the input portion based on the second deviation and inverse kinematics under the condition of keeping the input portion position; controlling corresponding active joints to move based on the incremental joint variables and forward kinematics to make the input portion and the end instrument substantially aligned in poses.

22. The master station of claim 12 wherein, The main console further comprises an observation portion providing a window to observe images formed by the imaging portion, and poses of the observation portion and the imaging portion are independently adjustable, and the controller is further configured to perform: acquiring a pose angle between a line-of-sight axis of the observation portion and the imaging portion; calculating a third deviation between the pose angle and a preset pose angle; determining whether the third deviation reaches a third deviation threshold; when the third deviation reaches the third deviation threshold, adjusting poses of the observation portion and / or the imaging portion based on the third deviation to make the pose angle between the line-of-sight axis of the observation portion and the imaging portion substantially the same as the preset pose angle.

23. A surgical robot characterized by The main console comprises any one of claims 12-22.

Images