Surgical robot system and control method thereof

Abstract

An embodiment of the present invention provides a surgical robot system comprising: a main robot operable by a user; a first end effector and a second end effector that receive an input signal from the main robot and operate; an imaging unit that captures an image of at least one of the first end effector, the second end effector, and the surgical site; and a control unit that controls the position and operation of the image forming unit. The control unit is configured to be capable of extracting a first virtual point corresponding to a position of the first end effector, a second virtual point corresponding to a position of the second end effector, and a first virtual line passing through the first virtual point and the second virtual point, and to control at least one of a position and an operation of the imaging unit. The imaging section is configured so that a target virtual point located on the first virtual line is included in a photographing region of the imaging section.

Description

Technical Field

[0001] The invention relates to a surgical robot system and a control method thereof. Background Art

[0002] In medicine, surgery refers to the act of cutting, opening or manipulating the skin, mucous membrane or other tissues with medical instruments to treat diseases. In particular, laparotomy, which cuts the skin at the surgical site to treat, reshape or remove internal organs, has recently attracted attention as an alternative to surgery using robots due to problems such as bleeding, side effects, pain and scarring.

[0003] A surgical robot system is usually composed of a master robot and a slave robot for performing surgery, and the master robot and the slave robot can be implemented independently or in an integrated manner. In addition, when the surgeon (operator) operates the master robot, the surgery is performed by operating the robot arm combined with the slave robot or the surgical tool held by the robot arm (i.e., the instrument or the end-effector connected to the end of the instrument).

[0004] Typically, an operator can observe the surgical environment by inserting a surgical camera (eg, an instrument inserted into the human body through a medical trocar and an endoscope for photographing a surgical site) into the body.

[0005] However, in the case of such laparoscopic surgery, since the operator cannot directly observe the surgical site but observes it through a surgical camera, there is a problem that when the surgical instrument or end effector is out of the field of view of the surgical camera, the operator cannot perform the surgery smoothly. Summary of the invention

[0006] Technical issues to be solved

[0007] The present invention provides a surgical robot system, wherein a control unit controls the position and movement of an imaging unit by using position information of an end effector, so that the imaging unit can properly photograph a surgical environment even if the operator does not manually adjust the position of the imaging unit.

[0008] Means used to solve problems

[0009] In one aspect of the present invention, a surgical robot system is provided, comprising: a main robot capable of being operated by a user; a first end effector and a second end effector that receive input signals from the main robot and perform actions; an imaging unit that photographs at least one of the first end effector, the second end effector, and a surgical site; and a control unit that controls the position and action of the imaging unit. The control unit is configured to extract a first virtual point corresponding to the position of the first end effector, a second virtual point corresponding to the position of the second end effector, and a first virtual line passing through the first virtual point and the second virtual point, and control at least one of the position and action of the imaging unit so that a target virtual point located on the first virtual line is included in the photographing area of ​​the imaging unit.

[0010] In addition, the target virtual point may be a point in the middle of the first virtual point and the second virtual point.

[0011] In addition, the control section can extract a second virtual line passing through a third virtual point corresponding to the position of the imaging section and the target virtual point.

[0012] In addition, the control unit may control the position and movement of the imaging unit according to the distance between the first virtual point and the second virtual point so as to adjust the distance between the target virtual point and the third virtual point.

[0013] In addition, when the distance between the first virtual point and the second virtual point is less than a preset distance, the control unit may fix the position of the imaging unit.

[0014] In addition, the master robot may further include an operation unit that transmits a control command regarding the position and motion of the imaging unit to the control unit according to an operation of a user.

[0015] In addition, the control section may receive a control command from the operation section and control the imaging section to rotate with the first virtual line as a reference.

[0016] In addition, the control section may receive a control command from the operation section and control the imaging section to rotate with the second virtual line as a reference.

[0017] Another aspect of the present invention provides a control method for a surgical robot system, comprising: a step in which a control unit extracts a first virtual point corresponding to the position of a first end effector; a step in which the control unit extracts a second virtual point corresponding to the position of a second end effector; a step in which the control unit derives a target virtual point located on a first virtual line passing through the first virtual point and the second virtual point; and a step in which the control unit controls the position and movement of an imaging unit so that the target virtual point is included in a shooting area of ​​the imaging unit used to shoot a surgical site.

[0018] In addition, the target virtual point may be a point in the middle of the first virtual point and the second virtual point.

[0019] In addition, the step of controlling the position and motion of the imaging unit may include the step of controlling the position and motion of the imaging unit according to the distance between the first virtual point and the second virtual point so as to adjust the distance between the target virtual point and the imaging unit.

[0020] In addition, the step of adjusting the position and movement of the imaging unit may include the step of fixing the position of the imaging unit when the distance between the first virtual point and the second virtual point is less than a preset distance.

[0021] Effects of the Invention

[0022] The surgical robot system according to an embodiment of the present invention has the following effect: the control unit automatically controls the position and movement of the imaging unit by detecting the positions of the first end effector and the second end effector, thereby providing the user with appropriate surgical images. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is an exemplary structural diagram of a surgical robot system according to an embodiment of the present invention.

[0024] Figure 2 is an exemplary structural diagram of a slave robot according to an embodiment of the present invention.

[0025] Figure 3 This is a diagram for explaining virtual points and virtual lines that can be extracted by a control unit according to one embodiment of the present invention.

[0026] Figure 4 FIG. 1 is a diagram showing images of the first and second end effectors acquired from an imaging unit according to an embodiment of the present invention.

[0027] Figures 5 to 7 It is a diagram for explaining a usage state of a surgical robot system according to an embodiment of the present invention.

[0028] Figures 8 to 10 This is a flowchart for explaining a method in which a control unit controls at least one of a position and an operation of an imaging unit according to an embodiment of the present invention.

[0029] Description of Reference Numerals

[0030] 1: Surgical robot system IP1: First virtual point

[0031] IP2: Second virtual point IP3: Third virtual point

[0032] TP: Target virtual point IL1: First virtual line

[0033] IL2: Second virtual line 10: Main robot

[0034] 11: Operation section 12: Image section

[0035] 20: From the robot 21: Robot arm

[0036] 22: Axis 23a: First end effector

[0037] 23b: Second end effector 24: Imaging unit

[0038] 30: Control Department DETAILED DESCRIPTION

[0039] The present invention can be modified in various ways and can have various embodiments, so specific embodiments will be shown in the drawings and described in detail. Figure 1 The effects, features and methods of implementing the present invention will be clearly understood by referring to the embodiments described in detail below. However, the present invention is not limited to the embodiments disclosed below and can be implemented in various forms.

[0040] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. When describing with reference to the accompanying drawings, the same or corresponding components will be given the same reference numerals and repeated description thereof will be omitted.

[0041] In the following embodiments, unless clearly defined otherwise in the context, a singular expression shall include a plural expression.

[0042] In the following embodiments, terms such as “including” or “having” refer to the existence of features or constituent elements described in the specification, and do not preclude the possibility of the addition of one or more other features or constituent elements.

[0043] When some embodiments can be implemented differently, the specific process order can also be performed differently from the described order. For example, two processes described in succession can be performed substantially simultaneously, or in the reverse order of the described order.

[0044] In the drawings, the size of the components may be enlarged or reduced for the convenience of explanation. For example, the size and thickness of each structure shown in the drawings are arbitrarily shown for the convenience of explanation, so the following embodiments are not necessarily limited to the drawings shown.

[0045] Figure 1 is an exemplary structural diagram of a surgical robot system according to an embodiment of the present invention, Figure 2 is an exemplary structural diagram of a slave robot according to an embodiment of the present invention, Figure 3 This is a diagram for explaining virtual points and virtual lines that can be extracted by a control unit according to one embodiment of the present invention.

[0046] Figure 4 FIG. 1 is a diagram showing images of the first and second end effectors acquired from an imaging unit according to an embodiment of the present invention. Figures 5 to 7 is a diagram for explaining a use state of a surgical robot system according to an embodiment of the present invention. Figures 8 to 9 This is a flowchart for explaining a method in which a control unit controls at least one of a position and an operation of an imaging unit according to an embodiment of the present invention.

[0047] The present invention appropriately controls the position of the imaging unit 24 that captures the surgical site according to the position of the end effector 23 that performs the surgery, so that the operator U can smoothly perform the surgery using the master robot 10 .

[0048] Reference Figure 1 According to an embodiment of the present invention, a surgical robot system 1 may include: a main robot 10, which generates and transmits necessary signals according to the operation of the operator U; and a slave robot 20, which receives signals from the main robot 10 and directly performs the actions required for the operation on the patient P.

[0049] Specifically, the operator U can observe the captured images of the surgical environment provided by the imaging unit 24 through the imaging unit 12 described later, and control the drive of the end effector 23 (hereinafter, "end effector 23" is defined as the general term for the first end effector 23a and the second end effector 23b) electrically connected to the operating unit 11 by operating the operating unit 11 described later, thereby performing the operation.

[0050] Reference Figure 1 and Figure 2 In one embodiment of the present invention, a master robot 10 may include an operating unit 11 and an imaging unit 12 .

[0051] According to an embodiment of the present invention, the operating unit 11 can perform three-dimensional operations. The operator U directly holds / operates the operating unit 11 and transmits control commands to the slave robot 20, so that the end effector 23 that receives the control commands directly contacts the surgical site, enabling precise surgery.

[0052] Reference Figure 1 and Figure 2 The imaging unit 24 can control its position and movement according to the operation of the operator U, and the relevant contents will be described in detail later.

[0053] The imaging unit 12 is used to generate captured images of the surgical site, and can receive images or pictures including the surgical site from the imaging unit 24 inserted into the body of the patient P. The operator U can operate the operating unit 11 and perform surgery by observing the image transmitted and displayed on the imaging unit 12.

[0054] Reference Figures 2 to 7 The slave robot 20 according to an embodiment of the present invention may include a robot arm 21 , a shaft 22 , a first end effector 23 a , a second end effector 23 b , and an imaging unit 24 .

[0055] The slave robot 20 is equipped with one or more robot arms 21, which can receive signals from the master robot 10 and directly perform actions required for the surgery of the patient P. The robot arm 21 of the slave robot 20 can be connected to an end effector 23 that is inserted into the body of the patient P and performs the surgery, and a shaft 22 that transmits a driving force from the robot arm 21 to the end effector 23.

[0056] As one embodiment, an encoder may be provided in each robot joint (not marked in the figure) included in the robot arm 21, and the control unit 30 may use the encoder to calculate the absolute positions of the imaging unit 24, the first end effector 23a and the second end effector 23b provided at the end of the robot arm 21.

[0057] As an optional embodiment, an optical marker may be attached to one side of the first end effector 23a and the second end effector 23b. The optical marker may be a passive marker that reflects light emitted from at least one light source. In this case, the optical marker may reflect light emitted from the slave robot 20, and the imaging unit 24 may detect the absolute positions of the first end effector 23a and the second end effector 23b by detecting the light reflected from the passive marker.

[0058] As an alternative embodiment, the optical marker may be an active marker that emits light by itself. In this case, the imaging unit 24 may detect the absolute positions of the first end effector 23a and the second end effector 23b by detecting the light emitted from the active marker.

[0059] Therefore, the control unit 30 can extract a first virtual point IP1 on a virtual coordinate system corresponding to the position of the first end effector 23a, a second virtual point IP2 on a virtual coordinate system corresponding to the position of the second end effector 23b, and a third virtual point IP3 on a virtual coordinate system corresponding to the position of the third end effector 23 in the absolute coordinate system, and the related descriptions will be described in detail in the description of the control method of the control unit 30 and the surgical robot system 1.

[0060] The robot arm 21 , the shaft 22 , and the end effector 23 are applicable to existing surgical robots in various structures, and thus a detailed description of the internal structures and operating principles of the robot arm 21 , the shaft 22 , and the end effector 23 is omitted.

[0061] Reference Figure 2 and Figure 3 According to an embodiment of the present invention, the imaging unit 24 is used to photograph the surgical site or the end effector 23, which can be connected to the end of the robot arm 21 and inserted into the body of the patient P.

[0062] The imaging unit 24 may include an image converter and an objective lens optically coupled to the image converter, and the image converter may include a camera chip configured to convert an optical image projected on the lens into an electrical image signal. However, it is not limited thereto, and the imaging unit 24 may include various devices within the technical concept capable of photographing the surgical site or the end effector 23.

[0063] Reference Figures 5 to 7 The imaging unit 24 can move with 6 degrees of freedom (6-DOF) in three-dimensional space. Specifically, the imaging unit 24 or the robot arm 21 connected to the imaging unit 24 can move / rotate in three-dimensional space through a control algorithm preset in the control unit 30 described later or a signal generated by the operator U's operation.

[0064] Reference Figure 1 and Figure 2 According to an embodiment of the present invention, the control unit 30 controls the driving of the imaging unit 24, the robot arm 21 and the end effector 23, and can be implemented by using at least one of application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors and other electronic units for performing functions.

[0065] As an embodiment, the control unit 30 may generate a virtual space having three-dimensional grid coordinates. The three-dimensional grid coordinate system may be a coordinate system having mutually orthogonal x-axis, y-axis, and z-axis, but is not limited thereto and includes various coordinate systems within the technical concept capable of representing coordinates in space.

[0066] Therefore, the control unit 30 can map at least one of the pre-extracted first to third virtual points IP1, IP2, IP3 to the virtual coordinate system, and can use the mapped virtual point IP (hereinafter, "virtual point IP" is defined as the general term for the first virtual point IP1, the second virtual point IP2, the third virtual point IP3 and the target virtual point TP) to calculate at least one of the first virtual line IL1 and the second virtual line IL2.

[0067] According to one embodiment of the present invention, the control unit 30 can extract the first virtual point IP1 and the second virtual point IP2 corresponding to the absolute position of the end effector 23, the third virtual point IP3 corresponding to the absolute position of the imaging unit 24, the target virtual point TP for determining the shooting area of ​​the imaging unit 24, the first virtual line IL1 that passes through the first virtual point IP1 and the second virtual point IP2 at the same time, and the second virtual line IL2 that passes through the third virtual point IP3 and the target virtual point TP at the same time, and the position and movement of the imaging unit 24 can be controlled by using the extracted virtual point IP and virtual line IL (hereinafter, "virtual line IL" is defined as the general term for the first virtual line IL1 and the second virtual line IL2).

[0068] The description of the method in which the control unit 30 controls the position and movement of the imaging unit 24 using the virtual point IP and the virtual line IL will be described in detail in the description of the control method of the surgical robot system 1 .

[0069] The following describes the control method and effect of the surgical robot system 1 according to one embodiment of the present invention as described above. In this specification, for the sake of convenience, the control method of the surgical robot system 1 is described in chronological order, and the control sequence of the surgical robot system 1 described below is not limited to the order of description and can be performed simultaneously.

[0070] Reference Figure 3 , Figure 4 and Figure 8 , the control unit 30 may extract a first virtual point IP1 and a second virtual point IP2 corresponding to the positions of the first end effector 23a and the second end effector 23b performing the surgery, respectively.

[0071] As an embodiment, the control unit 30 may calculate the absolute positions of the first end effector 23 a and the second end effector 23 b by acquiring geometric information such as the length of the robot arm 21 and the shaft 22 , and the rotation angles of the robot joints constituting the robot arm 21 .

[0072] The control unit 30 may extract points on the virtual coordinate system corresponding to the calculated absolute positions of the first end effector 23 a and the second end effector 23 b on the absolute coordinate system as the first virtual point IP1 and the second virtual point IP2 .

[0073] As an optional embodiment, the imaging unit 24 can capture an optical marker that can be attached to one side of the end effector 23, and the control unit 30 receives the image of the optical marker from the imaging unit 24 to detect the positions of the first end effector 23a and the second end effector 23b, so that the first virtual point IP1 and the second virtual point IP2 can be extracted.

[0074] Reference Figure 3 , Figure 4 and Figure 8 , the control unit 30 may extract a first virtual line IL1 passing through the first virtual point IP1 and the second virtual point IP2. Specifically, the control unit 30 may extract a line on the virtual coordinate system passing through the first virtual point IP1 and the second virtual point IP2 on the virtual coordinate system as the first virtual line IL1.

[0075] As one embodiment, as the first end effector 23a and the second end effector 23b are driven, the control unit 30 can update the position values ​​of the first virtual point IP1 and the second virtual point IP2 in real time, and based on the real-time updated position values ​​of the first virtual point IP1 and the second virtual point IP2, the control unit 30 can update the position value mapped by the first virtual line IL1 in real time.

[0076] Reference Figure 3 and Figure 4 , the control unit 30 can extract a target virtual point TP located on the first virtual line IL1.

[0077] The target virtual point TP according to an embodiment of the present invention may be a reference point for determining the shooting area of ​​the imaging unit 24 , and the control unit 30 may control the position and movement of the imaging unit 24 by using the position value of the target virtual point TP.

[0078] As an embodiment, the control unit 30 may calculate the target virtual point TP as a point in the middle between the first virtual point IP1 and the second virtual point IP2 .

[0079] As an embodiment, the control unit 30 may adjust the ratio of the distance n between the first virtual point and the target virtual point to the distance n between the second virtual point and the target virtual point according to the ratio of the speed of the first virtual point IP1 to the speed of the first virtual point IP1.

[0080] When the speed of the first virtual point IP1 is relatively greater than the speed of the second virtual point IP2, the control unit 30 can extract the position of the target virtual point on the first virtual line IL1, wherein the distance m between the second virtual point and the target virtual point is relatively greater than the distance n between the first virtual point and the target virtual point.

[0081] Therefore, even if at least one of the first end effector 23a or the second end effector 23b moves quickly during surgery, the target virtual point TP can be set to approach the relatively slower virtual point IP, thereby preventing the photographing area from moving extremely excessively, thereby smoothly performing the surgery.

[0082] Reference Figure 3, the control unit 30 can extract the third virtual point IP3 corresponding to the position of the imaging unit 24. Specifically, the control unit 30 can calculate the absolute position of the imaging unit 24 using information such as the length of the robot arm 21 connected to the imaging unit 24, the rotation angle of the robot joints constituting the robot arm 21, and the like, and can extract the points on the virtual coordinate system corresponding to the absolute positions of the imaging unit 24 as the third virtual point IP3.

[0083] Reference Figure 3 , the control part 30 may extract the second virtual line IL2 passing through the target virtual point TP and the third virtual point IP3.

[0084] As one embodiment, when the control unit 30 controls the position and movement of the imaging unit 24 so that the second virtual line IL2 and the longitudinal center axis of the imaging unit 24 coincide, the target virtual point TP can be set at the center of the shooting area, and when the control unit 30 controls the movement of the imaging unit 24 so that the imaging unit 24 rotates by a preset angle with the second virtual line IL2 as the reference axis, the imaging unit 24 can capture the surrounding environment of the target virtual point TP.

[0085] Reference Figure 5 and Figure 8 According to an embodiment of the present invention, the control unit 30 can control the position and movement of the imaging unit 24 according to the distance D1 between the first virtual point IP1 and the second virtual point IP2, so as to adjust the distance D2 between the target virtual point TP and the third virtual point IP3.

[0086] As an embodiment, the control unit 30 may control the position and movement of the imaging unit 24 so that both the first end effector 23 a and the second end effector 23 b are included in the angle of view (AOV) of the imaging unit 24 .

[0087] As an embodiment, the control unit 30 may control the position and movement of the imaging unit 24 so that the distance D1 between the first virtual point IP1 and the second virtual point IP2 is proportional to the distance D2 between the target virtual point TP and the third virtual point IP3 .

[0088] Specifically, when the distance D1 between the first virtual point IP1 and the second virtual point IP2 decreases, the control unit 30 can make the imaging unit 24 move toward the target virtual point TP, so that the distance D2 between the target virtual point TP and the third virtual point IP3 decreases. When the distance D1 between the first virtual point IP1 and the second virtual point IP2 increases, the control unit 30 can adjust the position of the imaging unit 24 so that it is away from the target virtual point TP.

[0089] Therefore, when the first end effector 23a and the second end effector 23b are brought close to each other in order to perform surgery on a tiny part and the imaging unit 24 needs to be taken close to the surgical site, the operator U does not need to manually adjust the position of the imaging unit 24. The control unit 30 automatically moves the imaging unit 24 toward the surgical site by detecting the distance between the first end effector 23a and the second end effector 23b, thereby allowing the operator U to perform the surgery smoothly.

[0090] In addition, when the first end effector 23a and the second end effector 23b are separated from each other in order to perform surgery on a wider area, and the imaging unit 24 needs to take a picture of the surgical site from a distance so as to photograph both the first end effector 23a and the second end effector 23b performing the surgery, the operator U does not need to manually adjust the position of the imaging unit 24, and the control unit 30 automatically moves the imaging unit 24 backward, thereby achieving the effect of enabling the surgery to be performed smoothly.

[0091] In addition, when one or more of the first end effector 23a and the second end effector 23b leave the shooting area due to an operational error by the operator U, the control unit 30 controls the position and movement of the imaging unit 24 so that both the first end effector 23a and the second end effector 23b are automatically included in the shooting area, thereby keeping the operation going smoothly.

[0092] Reference Fig. 9 When the distance D1 between the first virtual point IP1 and the second virtual point IP2 is less than a preset distance, the control unit 30 according to an embodiment of the present invention may fix the position of the imaging unit 24 .

[0093] Therefore, when performing surgery on a tiny part smaller than a preset area, the photographing area of ​​the imaging unit 24 is fixed and does not move, thereby achieving an effect that the operator U can perform precise surgery.

[0094] As an embodiment, the operator U may selectively fix the position of the imaging unit 24 through the operation unit 11. In this case, the control unit 30 may fix the position of the imaging unit 24 regardless of the positions of the first virtual point IP1, the second virtual point IP2, or the distance D1 between the first virtual point IP1 and the second virtual point IP2.

[0095] In addition, the operator U can release the position of the imaging unit 24 which has been fixed by operating the operating unit 11. In this case, the control unit 30 can adjust the position of the imaging unit 24 according to the positions of the first virtual point IP1, the second virtual point IP2, or the distance D1 between the first virtual point IP1 and the second virtual point IP2.

[0096] Therefore, when performing precise work, by fixing the position of the imaging unit 24 , the operator U can perform the operation while observing the image of the fixed imaging area, thereby achieving the effect of being able to smoothly perform precise work.

[0097] Reference Figure 1 , Figure 2 , Figure 6 , Figure 7 and Fig.10 The master robot 10 according to an embodiment of the present invention may include an operation unit 11 that transmits a control command regarding the position and movement of the imaging unit 24 to the control unit 30 according to an operation of the operator U.

[0098] As an embodiment, the control unit 30 may selectively block the control command generated by the operation unit 11 according to the speed of the first virtual point IP1 and the speed of the second virtual point IP2.

[0099] As one embodiment, when the first end effector 23a and the second end effector 23b correspond to a preset speed change (when the speed of the first virtual point IP1 and the speed of the second virtual point IP2 hardly change), the control unit 30 can receive a control command generated by the operating unit 11, and when the speed change of at least one of the first end effector 23a and the second end effector 23b is relatively greater than the preset speed change, the control unit 30 can block the control command of the imaging unit 24 generated by the operating unit 11.

[0100] As one embodiment, when the first end effector 23a and the second end effector 23b correspond to within a preset position change (when the speed of the first virtual point IP1 and the position or posture of the second virtual point IP2 hardly change), the control unit 30 can receive the control command generated by the operating unit 11, and when the movement of at least one of the first end effector 23a and the second end effector 23b is faster than the preset position change, the control unit 30 can block the control command related to the imaging unit 24 generated by the operating unit 11.

[0101] As one embodiment, when the first end effector 23a and the second end effector 23b correspond to within a preset boundary area (when the first virtual point IP1 or the second virtual point IP2 is located in a preset area in space and does not change much), the control unit 30 can receive a control command generated by the operating unit 11, and when at least one of the first end effector 23a and the second end effector 23b is located outside the preset area, the control unit 30 can block the control command of the imaging unit 24 generated by the operating unit 11.

[0102] Therefore, when the end effector 23 is operated to perform a surgery, the imaging unit 24 is operated at the same time, so confusion during the surgery can be prevented.

[0103] Reference Figure 6 The control unit 30 according to an embodiment of the present invention may receive a control command from the operation unit 11 and control the imaging unit 24 to rotate with the first virtual line IL1 as a reference.

[0104] As one embodiment, the control unit 30 can extract a virtual plane VP including the second virtual line IL2 on the virtual coordinate system, and the imaging unit 24 can rotate the imaging unit 24 on the virtual plane VP with the first virtual line IL1 as the reference axis according to a control command received from the operating unit 11, according to a rotation angle set according to the control command.

[0105] Therefore, when the operator U needs to perform surgery while observing the target virtual point TP or the surgical site at different angles, the operating unit 11 can be used to control the position and movement of the imaging unit 24, so that the imaging unit 24 can capture the target virtual point TP or the surgical site at different angles, thereby improving the convenience of the surgery.

[0106] The control unit 30 according to an embodiment of the present invention may rotate the imaging unit 24 based on the first virtual line IL1 according to the angle formed between the surface of the surgical site and the axis 22 .

[0107] As an embodiment, the control unit 30 may control the position and movement of the imaging unit 24 by rotating the imaging unit 24 based on the first virtual line IL1 so that the second virtual line IL2 is located on a plane including the axis 22 and the target virtual point TP.

[0108] As an embodiment, the control unit 30 may control the position and movement of the imaging unit 24 by rotating the imaging unit 24 based on the first virtual line IL1 so that the second virtual line IL2 is configured within a preset angle with respect to a plane including the axis 22 and the target virtual point TP.

[0109] Therefore, the control unit 24 controls the position and movement of the imaging unit 24 in the relationship between the operator U and the axis 22, similar to the angle of the operator U's line of sight when the operator U directly holds the surgical tool and performs the operation, so that the foreign body sensation can be minimized even if the operator U observes the surgical site through the imaging unit 24.

[0110] Reference Figure 7 According to an embodiment of the present invention, the control unit 30 can receive a control command from the operation unit 11 and control the imaging unit 24 to rotate with the second virtual line IL2 as a reference. Specifically, the control unit 30 can control the imaging unit 24 so that the longitudinal central axis of the imaging unit 24 forms a preset angle with the second virtual line IL2.

[0111] The control unit 30 according to an embodiment of the present invention can control the position and movement of the imaging unit 24 by rotating the imaging unit 24 based on the second virtual line IL2 according to the movement of at least one of the first end effector 23a and the second end effector 23b.

[0112] As one embodiment, when the first end effector 23a performs a clamping action on the surgical site for more than a preset time, or when the first end effector 23a performs a pinching action for more than a preset time, or when the first end effector 23a is in contact with the surgical site for more than a preset time, the control unit 30 can rotate the imaging unit 24 based on the second virtual line IL2 so that the center of the angle of view (AOV) of the imaging unit 24 is consistent with the first end effector 23a or the first virtual point IP1.

[0113] As one embodiment, when the second end effector 23b performs a clamping action on the surgical site for more than a preset time, or when the second end effector 23b performs a pinching action for more than a preset time, or when the second end effector 23b is in contact with the surgical site for more than a preset time, the control unit 30 can rotate the imaging unit 24 based on the second virtual line IL2 so that the center of the angle of view (AOV) of the imaging unit 24 is consistent with the second end effector 23b or the second virtual point IP2.

[0114] Therefore, by rotating the imaging unit 24 for imaging the target virtual point TP, the operator U can image the surrounding environment of the surgical site according to the operation of the operating unit 11 .

[0115] Reference Figure 8 According to an embodiment of the present invention, a control method of a surgical robot system 1 may include: a step S100 in which the control unit 30 extracts a first virtual point IP1 corresponding to the position of a first end effector 23a; a step S200 in which the control unit 30 extracts a second virtual point IP2 corresponding to the position of a second end effector 23b; a step S300 in which the control unit 30 derives a target virtual point TP located on a first virtual line IL1 passing through the first virtual point IP1 and the second virtual point IP2; and a step S400 in which the control unit 30 controls the position and movement of the imaging unit 24 so that the target virtual point TP is included in a shooting area (AOV) of the imaging unit 24 for shooting a surgical site.

[0116] In step S100 in which the control unit 30 extracts the first virtual point IP1 corresponding to the position of the first end effector 23a and in step S200 in which the control unit extracts the second virtual point IP2 corresponding to the position of the second end effector 23b, the control unit 30 can calculate the absolute positions of the first end effector 23a and the second end effector 23b by acquiring geometric information related to the length of the robot arm 21 and the axis 22, the rotation angles of the robot joints constituting the robot arm 21, etc.

[0117] In addition, the control unit 30 may extract points on the virtual coordinate system corresponding to the absolute positions of the first end effector 23 a and the second end effector 23 b on the calculated absolute coordinate system as the first virtual point IP1 and the second virtual point IP2 .

[0118] As an optional embodiment, the imaging unit 24 can capture an optical marker that can be attached to one side of the end effector 23, and the control unit 30 receives the image of the optical marker from the imaging unit 24 and detects the positions of the first end effector 23a and the second end effector 23b, so that the first virtual point IP1 and the second virtual point IP2 can be extracted.

[0119] In step S300 in which the control unit 30 exports the target virtual point TP located on the first virtual line IL1 passing through the first virtual point IP1 and the second virtual point IP2, the control unit 30 can extract the line on the virtual coordinate system passing through the first virtual point IP1 and the second virtual point IP2 on all virtual coordinate systems as the first virtual line IL1.

[0120] In addition, as the position values ​​of the first virtual point IP1 and the second virtual point IP2 are updated, the control unit 30 may update the position value mapped by the first virtual line IL1 in real time.

[0121] Reference Fig. 9 and Fig.10 According to one embodiment of the present invention, step S400 in which the control unit 30 controls the position and movement of the imaging unit 24 so that the target virtual point TP is included in the shooting area (AOV) of the imaging unit 24 for shooting the surgical site may include: step S410 in which the control unit 30 controls the position and movement of the imaging unit 24 according to the distance between the first virtual point IP1 and the second virtual point IP2 so as to adjust the distance between the target virtual point TP and the imaging unit 24; and steps S420 and S430 in which the position of the imaging unit 24 is fixed when the distance between the first virtual point IP1 and the second virtual point IP2 is less than a preset distance.

[0122] Specifically, the control unit 30 can execute step S420 of determining whether the distance between the first virtual point IP1 and the second virtual point IP2 is less than a preset distance, and when it is determined that the distance between the first virtual point IP1 and the second virtual point IP2 is less than the preset distance, the control unit 30 can execute step S430 of fixing the position of the imaging unit 24.

[0123] In addition, according to one embodiment of the present invention, step S400 in which the control unit 30 controls the position and movement of the imaging unit 24 so that the target virtual point TP is included in the shooting area (AOV) of the imaging unit 24 for shooting the surgical site may also include: step S440 in which the control unit 30 determines whether the control unit 30 receives a control command from the operating unit 11; step S450 in which the control unit 30 determines whether the first end effector 23a and the second end effector 23b are both in a stopped state; and step S460 in which the control unit 30 controls the position and movement of the imaging unit 24 by utilizing the control command from the operating unit 11.

[0124] Specifically, when the control portion 30 determines the control command from the operation portion 11 , the control portion 30 may determine whether both the first end effector 23 a and the second end effector 23 b are in a stopped state.

[0125] In the present specification, the situation in which the first end effector 23a and the second end effector 23b are both in a stopped state may be the same as at least one of the situation in which the first end effector 23a and the second end effector 23b correspond to a preset speed change, the situation in which the first end effector 23a and the second end effector 23b correspond to a preset position change, and the situation in which the first end effector 23a and the second end effector 23b correspond to a preset boundary area.

[0126] When the control section 30 determines that both the first end effector 23 a and the second end effector 23 b are in the stopped state, the control section 30 may control the position and motion of the imaging section 24 by using a control command from the operation section 11 .

[0127] The idea of ​​the present invention should not be limited to the embodiments described above, but rather to the scope of the claims described below. All scopes that are equal to or equivalent to the scope of the claims belong to the idea of ​​the present invention.

Claims

1. A surgical robot system, characterized in that: include: The main robot can be operated by the user. The first end effector and the second end effector receive input signals from the master robot and perform actions, an imaging unit that photographs at least one of the first end effector, the second end effector, and a surgical site, and A control unit, controlling the position and movement of the imaging unit; The control unit is configured as follows: capable of extracting a first virtual point corresponding to the position of the first end effector, a second virtual point corresponding to the position of the second end effector, and a first virtual line passing through the first virtual point and the second virtual point, At least one of a position and a motion of the imaging section is controlled so that a target virtual point located on the first virtual line is included in a photographing area of ​​the imaging section.

2. The surgical robot system according to claim 1, characterized in that: The target virtual point is a point in the middle of the first virtual point and the second virtual point.

3. The surgical robot system according to claim 1, characterized in that: The control section can extract a second virtual line passing through a third virtual point corresponding to the position of the imaging section and the target virtual point.

4. The surgical robot system according to claim 3, characterized in that: The control unit controls the position and movement of the imaging unit according to the distance between the first virtual point and the second virtual point so as to adjust the distance between the target virtual point and the third virtual point.

5. The surgical robot system according to claim 4, characterized in that: When the distance between the first virtual point and the second virtual point is less than a preset distance, the control unit fixes the position of the imaging unit.

6. The surgical robot system according to claim 3, characterized in that: The main robot comprises: The operation unit transmits a control command regarding the position and movement of the imaging unit to the control unit according to a user's operation.

7. The surgical robot system according to claim 6, characterized in that: The control section receives the control command from the operation section and controls the imaging section to rotate with the first virtual line as a reference.

8. The surgical robot system according to claim 6, characterized in that: The control section receives the control command from the operation section and controls the imaging section to rotate with the second virtual line as a reference.

9. A control method for a surgical robot system, characterized in that: include: The control unit extracts a first virtual point corresponding to the position of the first end effector; The control unit extracts a second virtual point corresponding to the position of the second end effector; The control unit derives a target virtual point located on a first virtual line passing through the first virtual point and the second virtual point; as well as The control unit controls the position and movement of the imaging unit so that the target virtual point is included in a photographing area of ​​the imaging unit for photographing a surgical site.

10. The control method of the surgical robot system according to claim 9, characterized in that: The target virtual point is a point in the middle of the first virtual point and the second virtual point.

11. The control method of the surgical robot system according to claim 9, characterized in that: The step of controlling the position and movement of the imaging unit includes: The step of controlling the position and movement of the imaging unit according to the distance between the first virtual point and the second virtual point so as to adjust the distance between the target virtual point and the imaging unit.

12. The control method of the surgical robot system according to claim 9, characterized in that: The step of adjusting the position and movement of the imaging unit comprises: When the distance between the first virtual point and the second virtual point is less than a preset distance, a step of fixing the position of the imaging portion.

Images