Motion compensation device

Abstract

The motion compensation device according to the present invention may comprise: an outer cannula that is inserted into a human body and reaches a target object to be subjected to a surgery, and a compensation calculation unit that calculates a motion compensation corresponding to the motion of the target object in the human body; the outer cannula may include a surgical instrument portion that treats the target object; in response to movement of the target, a distance between the target and the surgical instrument portion or a distance between the target and the outer cannula may be maintained within a predetermined range.

Description

Technical Field

[0001] The present specification relates to a motion compensation device that calculates motion compensation for a surgical instrument corresponding to the motion of a surgical object within a human body. Background Art

[0002] In the medical field, surgery using robots including a master station and a slave station implemented independently or in one body is attracting much attention.

[0003] In the field of surgery involving insertion of a flexible endoscope without laparotomy, the use of a robot to control the movement of the flexible endoscope can improve the problems that arise when passing through internal tissues or organs and performing surgery in a confined environment.

[0004] In particular, compared to a target object such as an affected part or organ in the human body, the coordinate system or position of a flexible endoscope connected to an external slave station or a peripheral device such as a robotic arm can be fixed. On the contrary, due to human factors such as breathing and heartbeat in the body or mechanical factors such as line tension, the target object as the surgical object may move regularly or irregularly over time. At this time, there is a difficulty in that the doctor must perform the operation through the picture sent by the camera or the like provided at the front end of the endoscope during the movement of the target object. Summary of the invention

[0005] Problem that the invention aims to solve

[0006] The present specification can provide a technique for recognizing the movement of a target object as a surgical target in the human body and synchronizing a surgical instrument section provided at the distal end portion of an overtube with the movement of the target object.

[0007] Means used to solve problems

[0008] The motion compensation device of the present invention may include: an outer sleeve, which is inserted into the human body and reaches a target object as a surgical object, and a compensation calculation unit, which calculates motion compensation corresponding to the movement of the target object in the human body; the outer sleeve may include a surgical instrument unit, which processes the target object; in response to the movement of the target object, the distance between the target object and the surgical instrument unit or the distance between the target object and the outer sleeve can be maintained within a predetermined range.

[0009] Effects of the Invention

[0010] The present invention can calculate the movement of the target object including the respiratory cycle by using image data received at predetermined time intervals from an imaging unit provided at the end of an overtube.

[0011] In addition, the present invention can calculate the motion of the target object through the motion estimation modeling unit without input data or with only minimal information including a table or preset values ​​input in advance.

[0012] The target object to be operated on can move with freedom in three dimensions, and the three-dimensional movement of the target object can be compensated by the translational movement, bending movement or rotational movement of the surgical instrument part or the outer sleeve of the present invention.

[0013] Compared with a doctor coordinating with the movement of a target object to perform a processing operation such as crushing within a specific time, according to the present invention, the movement of the surgical instrument part or the outer sleeve is compensated so that the distance between the surgical instrument part or the surgical device and the stone as the target object remains constant. Therefore, the probability or intensity of hitting the stone by a crushing device such as a laser can be increased, and the operation time can be reduced or the frequency of contact with the mucosa can be reduced.

[0014] The main movement of the target object may correspond to the extension direction of the surgical instrument part of the present invention. After compensating the movement of the surgical instrument part in the translation direction, the doctor can decide through the monitoring unit whether to compensate for the remaining degrees of freedom of movement or perform a surgical operation such as crushing, or can perform the surgical operation after the movement compensation according to the present invention is completed.

[0015] When an image to which motion compensation for the target object according to the present invention is applied is displayed on a monitoring unit, the surgeon performing the operation can ensure intuitiveness of the operation. When it is determined that the motion compensation is not performed smoothly, it can be directly confirmed with the naked eye and additional correction can be performed, thereby being able to more effectively deal with unexpected situations during the operation.

[0016] Therefore, according to the present invention, when the movement of the target object according to breathing, etc. is compensated to maintain the distance between the outer tube or the surgical instrument part and the target object, the doctor does not need to directly manually perform operations to keep the distance to the target object constant or perform the operation in a state where breathing is forcibly stopped, thereby reducing dependence on the doctor's personal proficiency, reducing the risk of the operation according to the patient's condition, and reducing the burden on the doctor performing the operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is an overall explanatory diagram of the driving unit and the flexible endoscope unit of the motion compensation device of the present invention at a surgical angle.

[0018] Figure 2 It is an explanatory diagram of the flexible endoscope unit of the present invention.

[0019] Figure 3 It is a diagram showing the structure of the motion compensation method of the present invention.

[0020] Figure 4It is a diagram showing the structure of the motion compensation device of the present invention.

[0021] Figure 5 It is an explanatory diagram for explaining the movement of the overtube or the surgical instrument part according to the present invention.

[0022] Figure 6 It is an explanatory diagram of motion compensation calculation by the compensation calculation unit of the present invention.

[0023] Figure 7 The relative movement of the target object of the present invention relative to the outer sleeve is shown.

[0024] Figure 8 It is an explanatory diagram of the orientation of the present invention.

[0025] Fig. 9 This is an explanatory diagram when the target of the motion compensation of the movement of the target object according to the present invention or the driven target of the motion compensation is a surgical instrument part.

[0026] Fig.10 This is an explanatory diagram when the object of motion compensation or the driven object of motion compensation according to the present invention is an outer sleeve.

[0027] Fig.11 It is an explanatory diagram of the received first to fourth data streams according to the present invention.

[0028] Fig.12 It is an explanatory diagram for data collection or calculation of respiratory motion by the imaging unit of the present invention.

[0029] Fig.13 This is an explanatory diagram of an embodiment of the calculation of respiratory motion such as a respiratory cycle by the compensation calculation unit of the present invention.

[0030] Fig.14 This is an embodiment of acquiring the third data of the present invention from a ventilator / anesthesia workstation as an embodiment of a measuring unit.

[0031] Fig.15 This is another embodiment of acquiring the third data of the present invention from a C-arm as another embodiment of the measuring unit. DETAILED DESCRIPTION

[0032] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the technical concept of the present invention is not limited to the partial embodiments described, and can be implemented in various forms, and at least one of the constituent elements between the embodiments can be selectively combined and / or replaced within the scope of the technical concept of the present invention.

[0033] In addition, if not particularly defined, the terms of the embodiments of the present invention may be interpreted as meanings that can be generally understood by those skilled in the art, and commonly used terms may be interpreted in consideration of the contextual meanings of the related art.

[0034] In addition, the terms of the embodiments of the present invention are used to explain the embodiments rather than to limit the present invention, and unless the context otherwise specifies, a singular expression also includes a plural expression.

[0035] In addition, among the constituent elements of the embodiments of the present invention, terms such as first, second, third or A, B, C, etc. may be used, and such terms are only used to distinguish one constituent element from other constituent elements and do not limit the order or ranking, etc.

[0036] In addition, in the embodiments of the present invention, the description of a constituent element being "connected", "contacting", "combined", etc. with other constituent elements not only means that one constituent element is directly connected, contacting, or combined with other constituent elements, but also means that one constituent element is indirectly connected, contacting, or combined through another constituent element between the two constituent elements.

[0037] In addition, in the embodiments of the present invention, a component is arranged, formed, or disposed on or below other components, or above or below other components, including a component being directly or indirectly arranged, formed, or disposed on other components. The description of above or below, or above or below means not only an upper direction based on a component but also a lower direction.

[0038] Reference Figures 1 to 15 , the motion compensation device and motion compensation method of the present invention are explained.

[0039] Reference Figure 1 and Figure 2 The motion compensation device 100 of the present invention can be applied to a surgery in which the flexible endoscope portion 170 is inserted into a human body.

[0040] As an embodiment, the present invention can be applied to retrograde intrarenal surgery (RIRS) in which an overtube 200 is inserted through the urethra 27 to crush stones around the kidney 21. The present invention can be applied to crushing or removing surgery for at least one of kidney stones, renal pelvic stones and ureteral stones.

[0041] The target object 50, as a part of the surgical object, may be crushed or removed from the body, like at least a part of the affected part of an organ or a calculus. The movement of the target object 50 may include human factors or mechanical factors such as breathing and heartbeat in the body. As an embodiment, the main motion compensation of the present invention may refer to the compensation for the movement of the target object 50 caused by breathing, and the compensation for the movement of the target object 50 caused by the remaining human factors or mechanical factors other than these may be regarded as incidental.

[0042] The motion compensation device 100 may include: a monitoring unit 110 that displays images captured by the capturing unit 220; or an operating unit 130, where a doctor uses a device including a joystick, a handle, a clamp, etc. to make the driving unit 150 move.

[0043] The driving unit 150 may include at least one of a translation driving unit 151 , a bending driving unit 153 , and a rotation driving unit 155 , and the object driven by the driving unit 150 may be at least one of the outer sleeve 200 , the imaging unit 220 , and the surgical instrument unit 240 .

[0044] The imaging part 220 or the surgical instrument part 240 may be tubular and extend along the inner length of the outer sleeve 200 .

[0045] As an embodiment, the motion compensation apparatus 100 may include a master station MS or a slave station SS.

[0046] The master station MS can be used by the doctor performing the operation to monitor the operation status and can operate at least one of the flexible endoscope unit 170, the outer tube 200, the imaging unit 220, and the operation instrument unit 240. The slave station SS can receive the doctor's instruction from the master station MS to perform an operation, or can perform an operation through an additional operation unit that can be linked with other display devices.

[0047] The master station MS or the slave station SS can be implemented independently or in an integrated manner according to the surgical site or equipment layout. The monitoring unit 110 or the operating unit 130 can be included in the master station MS, and the driving unit 150, the flexible endoscope unit 170 or the robot arm 140 can be included in the slave station SS.

[0048] The flexible endoscope unit 170 may be mounted on a slave station SS including a robot arm 140 and the like, and may be adjusted to a height or angle suitable for surgery by the robot arm 140 .

[0049] The flexible endoscope unit 170 may include a flexible tube type outer sleeve 200 inserted into the human body (step S110). The outer sleeve 200 may be composed of a plurality of joints having a degree of freedom under predetermined constraints, and as a result, the end (end tip) of the outer sleeve 200 may have a degree of freedom in three dimensions and freely approach the target object 50.

[0050] As an embodiment, the outer sleeve 200 can be inserted into the urethra 27 and reach the kidney 21 or its periphery through the bladder 25 and the ureter 23. As described above, the outer sleeve 200 can be flexible so that it can extend along an irregular, curved or smooth organ and can perform three-dimensional movements through the driving unit 150.

[0051] The photographing unit 220 or the surgical instrument unit 240 may include: a line extending along the internal path of the outer sleeve 200; and an end effector protruding from the front end of the outer sleeve 200. As for the photographing unit 220, the end effector may be a photographing device such as a camera capable of collecting image data in the body. The image data collected by the photographing unit 220 includes not only two-dimensional data or three-dimensional data including RGB-D or stereo, but also data capable of obtaining body information that can be obtained by a laser radar (Lidar), infrared (IR), and sonar installed in the outer sleeve 200.

[0052] As for the surgical instrument part 240, the end effector may be a surgical device. The surgical device may include at least one of the following devices: a crushing device that can crush the target object 50 by laser or the like; a capture device such as a basket that is used to capture the target object 50 that needs to be removed from the body; and a suction device that is used to spray water onto the target object 50 or to suck the target object 50.

[0053] The surgical instrument unit 240 processes the target object 50. The term "processing" may include all actions that can be performed on the target object 50 during surgery, except for obtaining image data through the imaging unit 220. Therefore, the term "processing" may include a pulverizing device, a capturing device, or a suction device of the surgical device.

[0054] The portion of the flexible endoscope section 170 connected to the slave station SS or the driving section 150 may be referred to as a proximal portion, and the front portion of the flexible endoscope section 170 connected to a photographing device or a surgical device may be referred to as a distal portion. The proximal portion or the distal portion may be used to refer to an upstream direction or a downstream direction, respectively, on a path extending along the outer sleeve 200.

[0055] In the flexible endoscope section 170 , a handle section 180 , a channel sheath 172 , or an operation section may be provided at a proximal portion.

[0056] The handle part 180 may include: an insertion device for inserting the imaging part 220 or the surgical instrument part 240; or an operation device for manipulating at least one of the outer sleeve 200, the imaging part 220, and the surgical instrument part 240. As an example, the imaging part 220 may be provided integrally with the outer sleeve 200, and the surgical instrument part 240 may be inserted or replaced by the handle part 180.

[0057] The present invention can use the image data obtained from the imaging unit 220 to perform motion compensation, and the motion compensation includes compensation for the movement of the target object 50 caused by breathing. The imaging unit 220 or the surgical instrument unit 240 can be included in the outer sleeve 200 or set separately. The drawings of the present invention illustrate the embodiment in which the imaging unit 220 or the surgical instrument unit 240 is included in the outer sleeve 200, but it is not limited to this, and can also be applied to the case where the imaging unit 220 or the surgical instrument unit 240 and the outer sleeve 200 are separately set.

[0058] As the operation proceeds, the surgical instrument part 240 having a laser, a basket, etc. can be replaced by the insertion device of the handle part 180, or a plurality of surgical instrument parts 240 can be provided at a thickness permitted by the insertion environment.

[0059] The operating device of the handle portion 180 may be provided separately from the driving portion 150 to assist the driving portion 150 .

[0060] The access sheath 172 can be provided at the entrance of the urethra and can guide the outer tube 200 to be inserted into the human body. Without the access sheath 172, it is easy to cause damage to the organ due to friction when inserting or retracting the outer tube 200. In particular, according to the present invention, the surgical device or surgical instrument part 240 can be moved in response to the movement of the target object 50 to perform motion compensation.

[0061] Reference Figures 5 to 10 , the mechanism and structure of motion compensation such as breathing compensation and heartbeat compensation of the present invention are explained.

[0062] according to Figure 5 The surgical instrument part 240 of the outer sleeve 200 can perform translational movement, bending movement or rotational movement, and through at least a part of these movements or a combination of these movements, the surgical instrument part 240 can achieve all free movements in three dimensions. Thus, the front end of the outer sleeve 200 can pass through the inside of the curved organ and approach the target object 50 in a desired direction.

[0063] Reference Figure 5Although the surgical instrument portion 240 is shown to be translated, bent or rotated (rolled), according to the structure, as the translation (translation), bending (bending) or rotation (rotation) of the outer sleeve 200 is transmitted to the surgical instrument portion 240, the result may also include a situation where the surgical instrument portion 240 is realized as a translation, bending or rotation (or rolling).

[0064] The image or video captured by the imaging unit 220 or the imaging device can be determined according to the direction in which the front end portion of the outer tube 200 faces. The image data including the two-dimensional image of the imaging unit 220 can be realized as a coordinate system consisting of an axis (z axis) parallel to the direction in which the imaging unit 220 faces and a plane (xy plane) perpendicular to the axis. The coordinate system of the two-dimensional image of the imaging unit 220 can be a relative coordinate system that changes according to the position of the front end portion of the outer tube 200.

[0065] The main purpose of the present invention is to maintain the change in the distance between the laser, basket or other surgical instrument part 240 which can be set at the front end (i.e., the distal end) of the outer sleeve 200 and the target object 50, or the change in the distance between the outer sleeve 200 and the target object 50 below a certain level, or to maintain the distance within a predetermined range.

[0066] In most cases, the goal is to maintain the distance between the surgical instrument section 240 and the target object 50 within a predetermined range. However, when the surgical instrument section 240 is disassembled and an empty tube (or empty channel) is used instead of the suction device of the surgical instrument section 240 to suck at least a portion of the target object 50, it may be necessary to maintain the distance between the outer tube 200 or the front end of the outer tube 200 and the target object 50 within a predetermined range.

[0067] The compensation calculation unit 400 or the control unit 370 can maintain the distance between the surgical instrument unit 240 and the target object 50, or the distance between the outer tube 200 and the target object 50 within a predetermined range. When the compensation calculation unit 400 or the control unit 370 maintains the distance between the surgical instrument unit 240 and the target object 50, or the distance between the outer tube 200 and the target object 50 within a predetermined setting range, it can be determined that normal motion compensation is achieved, and when it is determined that it is out of the predetermined setting range, the motion compensation can be recalculated.

[0068] Unless otherwise specifically distinguished or mentioned in this specification, maintaining the distance between the surgical instrument portion 240 and the target object 50 within a predetermined range, and maintaining the distance between the outer sleeve 200 and the target object 50 within a predetermined range are common goals to be achieved in the present invention.

[0069] The distance between the surgical instrument part 240 and the target object 50 means that it can have all three-dimensional components in the vector concept, and the movement of the target object 50 has regularity in its characteristics and can move repeatedly in a specific direction. At this time, it can be assumed that the position of the target object 50 (stone, etc.) attached to the body will follow the body movement such as breathing. For example, the position of the target object 50 will change or show irregular movement every time you breathe, so additional motion analysis is required for compensation.

[0070] The main movement direction or the movement compensation direction of the target 50 may change according to the position of the target 50 or the position of the outer sleeve 200 .

[0071] Therefore, if motion compensation is performed in a manner that makes the direction of the front end portion of the outer sleeve 200 as consistent as possible with the specific directional movement of the target object 50, most of the movement of the target object 50 can be offset.

[0072] The present invention can be intended to provide convenience for doctors when performing surgery with a surgical robot. Even if the movement of the target object 50 is not completely offset, it can still be offset by compensating the most important movement direction, thereby offsetting most of the movement of the target object 50, so as to maintain the distance between the target object 50 and the surgical instrument part 240, the distance between the target object 50 and the outer sleeve 200, or the distance between the target object 50 and the imaging part 220 well within a predetermined range. An additional operation can be added to make the movement compensation direction of the surgical instrument part 240, the outer sleeve 200 or the imaging part 220 consistent with or parallel to the most important direction of the movement of the target object 50.

[0073] The translational motion direction is an embodiment of the main motion direction of the target 50. The motion compensation process (or the main motion direction) can be changed according to the relative field of view state or position of the outer sleeve 200 located in the organ such as the kidney.

[0074] To illustrate with the relative coordinate system of the imaging unit 220, if the surgical instrument unit 240 is compensated for the movement according to the translational movement direction (z-axis direction) (step S331) in response to the movement of the target object 50, the maximum movement in the movement of the target object 50 can be offset. The degree of freedom of the plane (xy plane) perpendicular to the remaining translational movement direction can be additionally compensated by bending movement or rotational movement (step S333). That is, when the movement of the z-axis as the translational movement direction is compensated for offset, only the movement on the plane formed by the x-axis and the y-axis can be observed.

[0075] Additional motion compensation (step S333) may include compensation for motion caused by human factors such as friction, constraint, elastic deformation of tissues caused by soft internal tissues or organs, and compensation for motion caused by mechanical factors such as tension of the wire, etc. The additional motion compensation direction may change with the position of the target 50 or the position of the outer sleeve 200.

[0076] Reference Figures 7 to 10 , the motion compensation mechanism of the present invention is explained.

[0077] Figure 7 The front end of the outer tube 200 reaches the target object 50 such as a stone (step S130). Fig. 9 and Fig.10 It is shown that motion compensation is performed to synchronize the motions between the surgical instrument portion 240 or the overtube 200 and the target object 50 with each other.

[0078] The outer sleeve 200 can be inserted into the body (step S110) and reach the target object 50 by translation, bending, or rotation (step S130). The image data can include image data such as photos and videos that doctors can monitor while performing surgery.

[0079] At this time, the photographing unit 220 may collect image data at a fixed position, where fixed may refer to being stationary relative to an absolute coordinate system associated with an extracorporeal surgical device such as a master station MS and a slave station SS that are independent of the movement of the human body.

[0080] An absolute coordinate system that can serve as a reference for the movement of the target object 50 is required to synchronize the dynamic movement of the surgical instrument section 240 with the target object 50 over time and perform motion compensation, thereby maintaining the distance between the surgical instrument section 240 and the target object 50 within a predetermined range. Unlike conventional endoscopic surgery, endoscopic surgery using a robot makes it possible to reliably fix the position of the end effector at the end of the outer tube 200, thereby having the advantage of achieving accurate motion compensation as a result.

[0081] When the front end of the surgical instrument part 240 or the outer tube 200 reaches the target point and is fixed or stationary, as shown in FIG. Figure 7 As shown, if the target object moves by α, the distance between the surgical instrument part 240 and the target object 50 will become L+α.

[0082] exist Fig. 9 or Fig.10 In the present invention, the motion compensation is used to maintain the distance change between the surgical instrument part 240 and the target object 50 below a constant value, or within a predetermined range (L-β to L+β, where β is the error).

[0083] Even within the same organ (such as the kidney), the direction of movement may be different depending on the position of the target object 50. Therefore, if the target object 50 changes, the viewing angle of the shooting unit 220 needs to be changed. Therefore, the size of the movement that needs to be compensated or the degree of movement to be observed, the direction of movement, etc. will also be different.

[0084] In terms of organ structure, if the field of view of the imaging unit 220 observed at the initial position P1 where the outer sleeve 200 approaches the target object 50 and collects data deviates to a large extent from the movement direction α of the target object 50, it may be necessary to change the orientation of the imaging direction of the imaging unit 220.

[0085] Reference Figure 8 , if the field of view direction of the shooting unit 220 at the initial position P1 is slightly different from the movement direction α of the target object 50, it can be moved to the correction position P1' to make the field of view direction consistent with the movement direction α. ​​In this case, through orientation adjustment, the relative coordinate system conversion from the coordinate system A (x1 axis, y1 axis, z1 axis) of the initial position P1 to the coordinate system A' (x2 axis, y2 axis, z2 axis) of the correction position P1' can be achieved. The z-axis direction of the corrected coordinate system A' can be parallel to the movement direction of the target object 50. The absolute coordinate system (x-axis, y-axis, z-axis) is embedded or integrated in an external device of the surgical robot or motion compensation device 100 that does not change according to movement such as breathing, or it can be a coordinate system that is converted into a coordinate system that moves with the target object 50.

[0086] The relative coordinate value with respect to the imaging unit 220 or the object 50 that moves due to breathing or the like can be accurately calculated by comparing the absolute coordinate system with the relative coordinate system (A or A′).

[0087] Fig. 9 and Fig.10 An embodiment of the motion compensation of the invention is shown. Fig. 9 In the embodiment, the motion compensation of the surgical instrument part 240 may be performed in a synchronous manner corresponding to the motion α of the target object 50. Fig.10 In the embodiment, the outer sleeve 200 may be motion compensated in a synchronous manner corresponding to the motion α of the target object 50 .

[0088] When the surgical instrument section 240 is synchronized, the risk of damage to the organ surface or the like caused by the overtube 200 can be reduced. In addition, minimizing movement on the organ surface due to the irregular internal structure of the organ helps reduce the accumulation of overtube 200 control errors.

[0089] When the overtube 200 is synchronized, during the operation after motion compensation, the control procedure of issuing subsequent control instructions to the end effector of the imaging unit 220 or the surgical instrument unit 240 can be clarified.

[0090] When the surgical instrument unit 240 is synchronized, the surgical instrument unit 240 can continue the translational movement, perform actions for additional compensation, or issue subsequent instructions such as crushing and capturing in an overlapping manner. On the contrary, when the outer sleeve 200 is synchronized, the imaging unit 220 or the surgical instrument unit 240 can be fixed in position to the outer sleeve 200. When only the outer sleeve 200 is subjected to motion compensation, the remaining imaging unit 220 or the surgical instrument unit 240 has the advantage of automatically obtaining the motion compensation effect.

[0091] The imaging unit 220 can be implemented in various configurations, such as being fixed in position to the outer tube 200, or protruding from the front end of the outer tube 200 to be able to perform actions like the surgical instrument unit 240, or being implemented in an integrated manner with the surgical instrument unit 240.

[0092] The imaging unit 220 may be a subject of motion compensation according to the motion of the object 50 , or a driven subject of motion compensation.

[0093] Depending on the object of motion compensation based on the movement of the target object 50, or the driving object of the motion compensation, the setting position of the imaging unit 220, or the configuration structure between the outer sleeve 200, the imaging unit 220 and the surgical instrument unit 240, etc., the image displayed to the doctor through the monitoring unit 110 may be shown in an almost static state due to synchronization with the target object 50 (first image), or the movement of the target object 50 may be observed (second image).

[0094] Regardless of the above two types (first image and second image) of being stationary or moving relative to each other, the distance between the surgical instrument unit 240 and the target object 50 remains constant, so one of the two types can be displayed manually or automatically according to the preference of the surgeon performing the surgery or the situation during the surgery. According to the surgeon's choice, an option to switch between the two images can be provided. According to the configuration structure between the outer sleeve 200, the imaging unit 220 or the surgical instrument unit 240, the first image and the second image can be switched relative to each other through image data compensation.

[0095] When the movement of the surgical instrument portion 240 is compensated ( Fig. 9 ), and when the position of the imaging unit 220 is fixed to the outer tube 200, the image displayed to the doctor through the monitoring unit 110 may be the movement of the target 50 according to the respiratory cycle, respiratory direction, etc. If the target 50 is to be displayed as fixed on the monitoring unit 110, additional image-based calculations may be required.

[0096] When the movement of the surgical instrument portion 240 is compensated ( Fig. 9), and when the imaging unit 220 is not fixed to the outer tube 200 and can move independently, in order to make the image displayed to the doctor through the monitoring unit 110 appear as if the target object 50 has stopped, the imaging unit 220 and the target object 50 can be synchronized through additional synchronization between the imaging unit 220 and the surgical instrument unit 240, etc.

[0097] When the movement of the surgical instrument portion 240 is compensated ( Fig. 9 ), and when the imaging unit 220 is implemented as an integrated unit with the surgical instrument unit 240, the image displayed to the doctor through the monitoring unit 110 may appear as if the target object 50 is stopped. If crushing or capturing is performed while the motion of the surgical instrument unit 240 is compensated, the target object 50 will remain stopped. However, when the surgical instrument unit 240 is additionally compensated or the doctor's operation causes additional motion, the monitored image will change.

[0098] When the motion of the outer sleeve 200 is compensated ( Fig.10 ), and when the imaging unit 220 is fixed to the outer tube 200, the image displayed to the doctor through the monitoring unit 110 may appear as if the target object 50 is stopped.

[0099] When the motion of the outer sleeve 200 is compensated ( Fig.10 ), and when the imaging unit 220 is not fixed to the outer sleeve 200 and can move independently, in order to make the image displayed to the doctor through the monitoring unit 110 appear as if the target object 50 has stopped, additional synchronization between the imaging unit 220 and the outer sleeve 200 may be required.

[0100] When the motion of the outer sleeve 200 is compensated ( Fig.10 ), and when the imaging unit 220 is implemented as an integrated unit with the surgical instrument unit 240, the image displayed to the doctor through the monitoring unit 110 may appear as if the target object 50 is stopped. If crushing or capturing is performed while the motion of the surgical instrument unit 240 is compensated, the target object 50 will remain stopped. However, when the surgical instrument unit 240 is additionally compensated or the doctor's operation causes additional motion, the monitored image will change.

[0101] In addition, the outer sleeve 200 or the surgical instrument part 240 can not only be motion-compensated individually, but both can be motion-compensated to move in conjunction with each other. That is, through the mutually-conjunctive movement of the outer sleeve 200 and the surgical instrument part 240, the distance between the target object 50 and the surgical instrument part 240, or the distance between the target object 50 and the outer sleeve 200 can be maintained within a predetermined range.

[0102] The motion compensation apparatus 100 of the present invention may include at least one of a data processing unit 310 , a navigation unit 350 , a compensation calculation unit 400 , and a motion estimation modeling unit 500 .

[0103] Reference Figure 6 and Fig.12 , represents the image captured by the imaging unit 220 in the relative coordinate system of the imaging unit 220. For example, according to a breathing cycle consisting of exhalation and inhalation, the target 50 may repeatedly approach or move away from the imaging unit 220.

[0104] Fig.12 As an embodiment of motion compensation, there is breathing compensation.

[0105] Reference Fig.12 , the target object 50 can approach the imaging unit 220 during inhalation B1, and the target object 50 can move away from the imaging unit 220 during exhalation B2. The image or video of the imaging unit 220 acquired in a time series can present a periodic pattern according to the respiratory cycle. The waveform pattern for the pressure or volume of breathing, etc. may include: the exhalation interval or exhalation peak B31, B32; the inhalation interval or inhalation peak B33, B34; and at least one of the functional residual capacity interval or peak B35, B36 as the exhaust volume that is not completely discharged. The compensation calculation unit 400 can correspond the features of this respiratory waveform pattern to each other from the image or video of the imaging unit 220, and can calculate the respiratory compensation for the target object 50 according to the breathing including the respiratory cycle, the respiratory direction (movement direction), etc.

[0106] The compensation calculation unit 400 of the present invention uses image data of a two-dimensional type or the like obtained from the shooting unit 220 disposed at the end of the outer sleeve 200 according to a predetermined time interval to calculate motion information including the distance between the surgical instrument unit 240 and the target object 50, or the movement time of the target object 50 including the respiratory cycle.

[0107] Reference Figure 6 The compensation calculation unit 400 may set a target portion TP or a feature portion FP on the image captured by the imaging unit 220. The target portion TP may be a portion of the target object 50 or the like that serves as a reference on the screen, and the feature portion FP may measure or display a position change of the target portion TP over time on the captured image.

[0108] The compensation calculation unit 400 may track the target portion TP or the feature portion FP from images captured at predetermined time intervals (eg, 100 ms) to calculate the motion of the target object 50 .

[0109] according to Fig.11 The image data captured by the capturing unit 220 may be transmitted to the data processing unit 310 or the motion estimation modeling unit 500 .

[0110] The data processing section 310 may be provided separately from the motion estimation modeling section 500 , or may be included in the motion estimation modeling section 500 and may perform the same function as when provided separately.

[0111] The data processing unit 310 may send the received data or the data whose data format has been converted to the motion estimation modeling unit 500 .

[0112] The motion estimation modeling unit 500 can use the data of the motion estimation model to perform motion compensation in a specific scene or situation even without input of image data from the photographing unit 220 or external data from the measuring unit 600 of the external medical device. In addition, the motion estimation modeling unit 500 can perform motion compensation using pre-input data.

[0113] The compensation calculation part 400 may selectively receive data from the data processing part 310 or the motion estimation modeling part 500 , or may receive data from both.

[0114] The image data captured by the capturing unit 220 may include first data D1 used for motion compensation (eg, breathing compensation, etc.) of the target object 50 or second data D2 captured continuously in real time by the capturing unit 220 .

[0115] The first data D1 may include motion information of the object 50 required for motion compensation calculation of the compensation calculation unit 400 .

[0116] The first data D1 is a motion cycle including a respiratory cycle used by the compensation calculation unit 400 to calculate the motion compensation of the overtube 200 or the surgical instrument unit 240 (step S310), or may include information of the target portion TP, the feature portion FP, and the like.

[0117] The second data D2 may be image data continuously photographed in real time by the photographing unit 220. The second data D2 may conceptually overlap at least partially with the first data D1, and after calculating motion information including a motion cycle, may be used as real-time image data collected from the photographing unit 220 for monitoring a subsequent surgical procedure.

[0118] The second data D2 can be used alone to calculate motion compensation.

[0119] The compensation calculation unit 400 may receive the second data D2 photographed by the photographing unit 220 in real time.

[0120] The compensation calculation unit 400 or the control unit 370 can use the second data D2 to calculate the images captured by the shooting unit 220 at predetermined time intervals through an image processing method or a machine learning model including depth information or optical flow, and compensate for the movement of the target object 50 in real time by using the driving unit 150 to move the outer tube 200 or the surgical instrument unit 240.

[0121] The compensation calculation unit 400 or the control unit 370 can calculate the direction that minimizes the amount of optical flow using the second data D2 observed in real time by the shooting unit 220, and adjust the joints of the outer sleeve 200 in real time along the calculated direction.

[0122] After using the first data D1 to calculate the breathing cycle, calculating the distance between the surgical instrument part 240 and the target object 50 based on the breathing cycle and compensating for the translational movement of the outer sleeve 200 or the surgical instrument part 240 (step S331), there is no need to take a series of additional measures to compensate for the remaining movement (step S333). When the second data D2 is used alone for real-time motion compensation, the outer sleeve 200 or the surgical instrument part 240 can be controlled in real time in a manner that compensates for the movement of the target object 50.

[0123] The data processing part 310 may receive third data D3 which is external data received from the measuring part 600 which is not outside the motion compensation apparatus 100 including the photographing part 220 .

[0124] Fig.14 (a) can be an artificial respiration / anesthesia workstation, Fig.14 (b) may include patient information such as respiratory pressure or respiratory volume that can be obtained from an artificial respiration / anesthesia workstation.

[0125] Reference Fig.14 and Fig.15 The third data D3 may include data D3 received from the measurement section 600 of an external medical device such as an artificial respiration / anesthesia workstation, a C-arm 610, or the like.

[0126] The third data D3 may include data received from the measuring part 600 of an external medical device other than the photographing part 220 .

[0127] The third data D3 received from the measurement section 600 of the artificial respiration / anesthesia workstation may include information on at least one of the respiratory volume per minute, the inhalation / expiration ratio, the number of respirations per minute, the exhalation / inhalation start time, the inhalation / expiration switching time point, and the functional residual capacity interval time point.

[0128] Reference Fig.14(b), an embodiment of the third data D3 obtained from the measuring unit 600 may represent pressure control per minute or volume control per minute. The third data D3 may include information of the respiratory cycle calculated based on information of the peak values ​​1230, 1240 of the pressure or volume measured by the measuring unit 600. The compensation calculation unit 400 may use the third data D3 including the respiratory cycle received from the measuring unit 600 of the external medical device to calculate the motion compensation of the target object 50 caused by breathing. The compensation calculation unit 400 may use the third data D3 as an initial value or a reference value for motion compensation, and in order to calculate the motion compensation, the image data (D1 or D2) collected by the shooting unit 220 may be correlated with the third data D3.

[0129] The third data D3 received from the measuring unit 600 of the C-shaped arm 610 may include the advancing / retreating time, the number of breaths per minute, the cycle information, or the amplitude (advancing / retreating amount) of the target 50. Fig.15 As shown, the third data D3 of the C-arm may include the shape or position of the kidney 21 or the stone 50 in the kidney. The third data D3 may be used in association with the map formation information of the navigation unit 350 .

[0130] The third data D3 received from the measurement unit 600 of the C-arm may include the degree of two-dimensional movement of the organ or target 50 on the plane. Using the third data D3, the relative movement can be estimated based on the thickness of the outer sleeve 200, and the amount of movement loss of the outer sleeve 200 due to the contacting mucosa, etc. can be calculated, and the remaining time of the surgical procedure (for example, the remaining time of the surgical procedure based on the amount of the remaining stone 50) can be predicted.

[0131] The third data D3 may partially overlap with the first data D1 or the second data D2 of the photographing unit 220 , or may be estimated from the first data D1 or the second data D2 .

[0132] The third data D3 can be provided as an initial value for respiratory cycle calculation or motion estimation modeling. As described above, when the third data D3 is used together with the first data D1 or the second data D2, a more accurate motion compensation value can be calculated.

[0133] The data processing part 310 may receive feedback data, ie, fourth data D4 , from at least one of the compensation calculation part 400 , the navigation part 350 , and the orientation part 330 .

[0134] The fourth data D4 may include feedback data of at least one of the difference between the adjustment of the front end of the outer tube 200 or the field of view direction of the shooting unit 220 by the orientation unit 330 and the actual difference, the difference between the motion compensation of the outer tube 200 or the surgical instrument unit 240 by the compensation calculation unit 400 and the actual difference, and the difference between the motion compensation of each position in the organ of the target object 50 by the navigation unit 350 and the actual difference.

[0135] The compensation calculation part 400 may calculate the motion compensation using at least one of the first to fourth data D1 to D4 , or recalculate the motion compensation by correcting a difference between a value estimated by the motion estimation modeling part 500 and an actual value.

[0136] Even in the case where input data such as image data is insufficient or does not exist, the compensation calculation section 400 can receive information from the motion estimation modeling section 500 to calculate motion compensation.

[0137] The motion estimation modeling section 500 may update parameters for implementing the motion estimation model using the fourth data D4.

[0138] The fourth data D4 may include mechanical feedback information such as motor or encoder information connected to the driving unit 150, information indicating the current state of the flexible endoscope unit 170, hysteresis or backlash information of the line tension of the imaging unit 220 or the surgical instrument unit 240, and the like.

[0139] The data processing unit 310 may convert or unify the file formats so that the file formats of different formats can be used in an integrated manner in the compensation calculation unit 400 .

[0140] The compensation calculation unit 400 may calculate motion compensation including breathing compensation using the data collected or stored by the data processing unit 310 (step S310 ).

[0141] In order to perform motion compensation for the object 50 , the compensation calculation unit 400 may use at least one of the first to fourth data D1 to D4 .

[0142] In order to calculate motion compensation such as breathing compensation (step S310 ), the compensation calculation unit 400 may calculate motion information including a breathing cycle and the like (step S311 ).

[0143] Reference Fig.13 When the front end of the outer tube 200 reaches the target object position (step S130), its position can be fixed and data collection by the imaging unit 220 can be started (step S200).

[0144] The imaging unit 220 may capture a first image (initial image) while being stationary near the target 50, and may obtain image information (eg, differential image) having a high similarity to the first image from subsequently captured images based on the first image.

[0145] Acquiring highly similar image information (difference image) is equivalent to finding a sampling period that produces the most similar image in the repeated motion of the target object 50 observed by the imaging unit 220 , or is equivalent to finding an index that minimizes the histogram of the difference image.

[0146] The compensation calculation unit 400 may set the difference between the time of the image with the smallest total value of the acquired differential image and the first captured image as a reference for breathing, and then maintain the measured value within a predetermined range by repeated sampling to achieve normalization.

[0147] The navigation unit 350 can generate a map of a specific area or an organ obtained from the imaging unit 220 or the third data D3. Even in the same organ, depending on the position of the organ, the direction of the approach of the overtube 200, and the movement caused by breathing in the same human body, the three-dimensional motion obtained by the imaging unit 220 will be different, and the motion compensation information required for the compensation calculated will also be different.

[0148] The navigation unit 350 can be linked with the motion estimation modeling unit 500 to calculate or estimate the motion compensation value in any position and any situation as long as the map is completed.

[0149] The motion estimation modeling section 500 may perform modeling to estimate the motion of a living being including the kidney 21 , the stone 50 , and the like.

[0150] Motion compensation such as breathing compensation is not only used for a specific position, but needs to be generally applicable to all situations affected by breathing, such as during movement, close observation of the affected part, etc. The motion estimation modeling section 500 can provide the inherent motion of the target object 50 using the motion estimation model. To this end, the motion estimation modeling section 500 can learn from the input data, or can use a pre-input table or setting value.

[0151] The motion estimation modeling unit 500 can use at least one of the first data D1 and the second data D2 obtained from the shooting unit 220, the third data D3 (patient information that can be obtained from an external device such as a ventilator / anesthesia workstation or a C-arm), and the fourth data D4 (feedback data) as input data.

[0152] The motion estimation modeling unit 500 can calculate, based on the above-mentioned input data, information required for motion compensation including three-dimensional position or velocity, motion reference breathing action value within the respiratory cycle, current motion state information of the target object 50, performed motion compensation information, control input comparison motion information including hysteresis or recoil information based on the image data of the shooting unit 220, and at least one of the shape of the outer sleeve 200 or the front end position information within the target object 50.

[0153] In addition, the motion estimation model may be used when it is difficult to accurately grasp the motion of the target object 50 in the field of view of the imaging unit 220 .

[0154] When the first data D1 or the second data D2 can be received by the photographing unit 220, the values ​​predicted by the motion estimation model can be compared with each other based on the existing first data D1 or the second data D2. When there is a large difference between the actual value observed by the photographing unit 220 and the estimated value of the motion estimation model, or the difference deviates from the predetermined range for more than a certain period of time, the parameters for implementing the motion estimation model can be updated.

[0155] When there is no input of image information from the photographing unit 220 or the information is unavailable, the motion estimation modeling unit 500 may use the value of the motion estimation model as a compensation value for motion compensation.

[0156] Among them, unavailable information may be generated when the motion information of the target object 50 cannot be confirmed in the shooting unit 220, when the feedback on the motion compensation result cannot be fully received, or when it is affected by dangerous conditions such as mucosal collision or abnormal conditions such as input of erroneous data, confusion, etc.

[0157] For example, when the field of view of the shooting unit 220 is blurred due to the breakage of the stone 50, when it is difficult to judge the movement of the target object 50 or the surrounding conditions, the compensation calculation unit 400 receives data from the motion estimation modeling unit 500 and compensates for the motion as before or according to the prediction of the motion estimation modeling unit 500 until the field of view becomes clear again.

[0158] When the motion compensation calculation by the compensation calculation unit 400 is completed (step S310), the control unit 370 may issue an instruction to the drive unit 150 to perform synchronization (step S330). When performing synchronization (step S330), the first synchronization may be achieved by performing parallel motion compensation (step S331) by the translation drive unit 151. When the second motion compensation is required, compensation for the remaining degrees of freedom of motion other than the translation motion or additional motion compensation caused by mechanical factors may be performed (step S333).

[0159] The features, structures, effects, etc. described in the above embodiments are included in at least one embodiment of the present invention and are not necessarily limited to one embodiment. In addition, the features, structures, effects, etc. of each embodiment can be combined or modified with other embodiments by a technician with general knowledge of the field to which the embodiment belongs. Therefore, the contents related to the above combinations and modifications should be interpreted as being included within the scope of the present invention.

[0160] In addition, the above description is centered on the embodiment, but this is only an example and is not intended to limit the present invention. Anyone skilled in the art will know that various modifications and applications not shown in the above examples can be performed without departing from the essential features of the present embodiment. For example, the various components shown in detail in the embodiment can be deformed and implemented. And the differences related to such deformation and application should be understood to be included in the scope of the present invention as defined by the attached claims.

[0161] Description of Reference Numerals

[0162] 10: Patient 21: Kidney

[0163] 23: Ureter 25: Bladder

[0164] 27: Urethra 50: Target

[0165] 100: Motion compensation device 110: Monitoring unit

[0166] 130: Operation unit 140: Robotic arm

[0167] 150: driving unit 151: translation driving unit

[0168] 153: bending drive unit 155: rotation drive unit

[0169] 170: Flexible endoscope unit 172: Access sheath

[0170] 180: handle 200: outer sleeve

[0171] 220: Photographing unit 240: Surgical instrument unit

[0172] 310: Data processing unit 330: Orientation unit

[0173] 350: Navigation unit 370: Control unit

[0174] 400: Compensation calculation unit 500: Motion estimation modeling unit

[0175] 600: Measurement unit 610: C-arm

[0176] S100: Preparation step S200: Data collection step

[0177] S300: Compensation step S400: Surgery step

[0178] D1: first data D2: second data

[0179] D3: third data D4: fourth data

[0180] MS: Master station SS: Slave station

[0181] <Additional Embodiments>

[0182] Embodiment 1: A motion compensation device, comprising: an outer sleeve including a surgical instrument and configured to be inserted into a subject; and a compensation calculation unit for calculating motion compensation corresponding to the movement of an object in the subject, wherein the motion compensation device controls the distance between the object and the surgical instrument.

[0183] Embodiment 2: A motion compensation device according to the above embodiment, wherein the subject is a human being.

[0184] Embodiment 3: A motion compensation device according to any of the above embodiments, wherein the subject is an animal.

[0185] Embodiment 4: A motion compensation device according to any one of the above embodiments, wherein the motion compensation controls and / or adjusts the distance between the object and the surgical instrument in response to the movement of the object.

[0186] Embodiment 5: A motion compensation device according to any one of the above embodiments, wherein the motion compensation device maintains the distance between the object and the surgical instrument within a predetermined distance.

[0187] Embodiment 6: The motion compensation device according to any one of the above embodiments, wherein the outer sleeve is synchronized in response to the movement of the object.

[0188] Embodiment 7: A motion compensation device according to any one of the above embodiments, wherein the object of motion compensation is an outer sleeve.

[0189] Embodiment 8: A motion compensation device according to any one of the above embodiments, wherein the motion compensation device is a surgical instrument part, and the surgical instrument part is synchronized in response to the movement of the surgical instrument.

[0190] Embodiment 9: The motion compensation device according to any one of the above embodiments further comprises a shooting unit for acquiring image data of an object.

[0191] Embodiment 10: The motion compensation device according to any one of the above embodiments is characterized in that the distance between the shooting unit and the object is maintained within another predetermined distance in response to the movement of the object.

[0192] Embodiment 11: A motion compensation device according to any of the above embodiments, wherein the outer sleeve or the surgical instrument is configured to perform multiple movements.

[0193] Embodiment 12: A motion compensation device according to any one of the above embodiments, wherein the multiple motions include translation, bending or rotational motion.

[0194] Embodiment 13: A motion compensation device according to any one of the above embodiments, wherein the motion compensation is performed with respect to the translational motion direction of the outer sleeve relative to the object.

[0195] Embodiment 14: A motion compensation device according to any one of the above embodiments, wherein the motion compensation for the translational motion direction corresponds to breathing compensation.

[0196] Embodiment 15: The motion compensation device according to any of the above embodiments is characterized in that, in response to the movement of the object, the motion compensation of the surgical instrument is first performed along the direction of translational motion, and in response to the remaining movement of the object, the surgical instrument provides additional motion compensation through bending or rotational motion.

[0197] Embodiment 16: The motion compensation device according to any one of the above embodiments is characterized in that the compensation calculation unit uses continuous 2D image data from the shooting unit to calculate the motion of the object.

[0198] Embodiment 17: According to the motion compensation device described in any of the above embodiments, the outer sleeve includes a shooting part for shooting the object.

[0199] Embodiment 18: The motion compensation device according to any one of the above embodiments is characterized in that the compensation calculation unit calculates the distance or movement interval between the surgical instrument and the object.

[0200] Embodiment 19: The motion compensation device according to any one of the above embodiments, wherein the compensation calculation unit uses 2D image data from the motion interval of the object.

[0201] Embodiment 20: The motion compensation device according to any one of the above embodiments is characterized in that the compensation calculation unit uses at least one of the first data, the second data and the third data to calculate the motion compensation.

[0202] Embodiment 21: The motion compensation device according to any one of the above embodiments, wherein the second data is captured in real time by the shooting unit, and the third data includes external data received from an external device.

[0203] Embodiment 22: The motion compensation device according to any one of the above embodiments, wherein the external device includes a device other than the shooting unit.

[0204] Embodiment 23: The motion compensation device according to any one of the above embodiments is characterized in that the second data forms an optical flow according to time, and the outer sleeve or surgical instrument part is configured to compensate for the movement of the object in real time through the optical flow of the second data.

[0205] Embodiment 24: The motion compensation device according to any one of the above embodiments is characterized in that the third data is provided as an initial value for calculating the movement period of the object, and the compensation calculation unit calculates the motion compensation by associating the image data of the shooting unit with the third data.

[0206] Embodiment 25: The motion compensation device according to any one of the above embodiments is characterized in that it also includes a monitoring unit, and the monitoring unit is configured to display the image data obtained by the shooting unit to the outside.

[0207] Embodiment 26: A motion compensation device according to any one of the above embodiments, wherein when the observation direction of the photographing unit deviates from the moving direction of the object, the position of the photographing unit moves from an initial position to a correction position.

[0208] Embodiment 27: A motion compensation device according to any one of the above embodiments, wherein the observation position at the correction position is parallel to the moving direction of the object.

[0209] Embodiment 28: The motion compensation device according to any one of the above embodiments is characterized in that when the outer sleeve approaches the object, the position of the shooting unit is fixed at a fixed position, and the shooting unit collects image data of the object from the fixed position.

[0210] Embodiment 29: The motion compensation device according to any one of the above embodiments is characterized in that the compensation calculation unit sets a target part or a feature part in the captured image of the shooting unit, the target part is a reference part on the screen displaying the object, and the feature part represents the position change of the target part on the captured image, and the compensation calculation unit tracks the target part or the feature part from the captured image and calculates the motion of the object at predetermined time intervals.

[0211] Embodiment 30: The motion compensation device according to any one of the above embodiments is characterized in that the compensation calculation unit captures an initial image of the object and obtains a differential image, and the differential image is obtained from an image captured after the initial image, and the differential image corresponds to a sampling period in which the most similar image appears in the repeated motion of the object observed by the shooting unit.

[0212] Embodiment 31: A motion compensation device according to any one of the above embodiments, wherein the compensation calculation unit calculates the difference between the time of the image with the smallest sum of differences and the time of the initial image, wherein the difference is a benchmark for breathing.

[0213] Embodiment 32: The motion compensation device according to any one of the above embodiments further comprises a motion estimation modeling unit, wherein the motion estimation modeling unit performs modeling for estimating the motion of the object, and the motion estimation modeling unit learns the intrinsic motion of the object.

[0214] Embodiment 33: A motion compensation device according to any one of the above embodiments, wherein parameters are updated according to the difference between the value estimated by the motion estimation modeling unit and the actual value observed by the shooting unit, and the motion compensation device re-estimates the motion compensation based on the updated parameters.

[0215] Embodiment 34: Use of the motion compensation device according to any of the above embodiments for performing surgical operations.

[0216] Embodiment 35: A motion compensation device system, characterized in that it comprises: a main device having a screen and a controller; a motion compensation device comprising a compensation calculation unit and a surgical instrument configured to be inserted into a subject; and a driving unit accommodating the motion compensation device, wherein the motion compensation is configured to control the distance between the object and the surgical instrument in response to a signal from the main device.

[0217] Embodiment 36: A motion compensation device system according to the above embodiments, wherein the motion compensation device comprises an outer sleeve configured to be inserted into a subject.

[0218] Embodiment 37: A motion compensation device system according to any of the above embodiments, wherein the subject is a human.

[0219] Embodiment 38: A motion compensation device system according to any of the above embodiments, wherein the subject is an animal.

[0220] Embodiment 39: A motion compensation device system according to any of the above embodiments, wherein the motion compensation controls and / or adjusts the distance between the object and the surgical instrument in response to the movement of the object.

[0221] Embodiment 40: A motion compensation device system according to any one of the above embodiments, wherein the motion compensation device maintains the distance between the object and the surgical instrument within a predetermined distance.

[0222] Embodiment 41: A motion compensation device system according to any of the above embodiments, wherein the outer sleeve is synchronized in response to the movement of the object.

[0223] Embodiment 42: A motion compensation device system according to any one of the above embodiments, wherein the object of motion compensation is an outer sleeve.

[0224] Embodiment 43: The motion compensation device system according to any one of the above embodiments is characterized in that the motion compensation device system is a surgical instrument part, and the surgical instrument part is synchronized in response to the movement of the surgical instrument.

[0225] Embodiment 44: A motion compensation device system according to any one of the above embodiments, further comprising a shooting unit for acquiring image data of an object.

[0226] Embodiment 45: The motion compensation device system according to any one of the above embodiments is characterized in that the distance between the shooting unit and the object is maintained within another predetermined distance in response to the movement of the object.

[0227] Embodiment 46: A motion compensation device system according to any of the above embodiments, wherein the outer sleeve or the surgical instrument is configured to perform multiple movements.

[0228] Embodiment 47: A motion compensation device system according to any one of the above embodiments, wherein the multiple motions include translation, bending or rotational motion.

[0229] Embodiment 48: A motion compensation device system according to any one of the above embodiments, wherein the motion compensation is performed with respect to the translational motion direction of the outer sleeve relative to the object.

[0230] Embodiment 49: A motion compensation device system according to any one of the above embodiments, wherein the motion compensation for the translational motion direction corresponds to breathing compensation.

[0231] Embodiment 50: A motion compensation device system according to any of the above embodiments, characterized in that, in response to the movement of the object, the motion compensation of the surgical instrument is first performed along the direction of translational motion, and in response to the remaining movement of the object, the surgical instrument provides additional motion compensation through bending or rotational motion.

[0232] Embodiment 51: The motion compensation device system according to any one of the above embodiments is characterized in that it also includes a compensation calculation unit, which uses the continuous 2D image data from the shooting unit to calculate the motion of the object.

[0233] Embodiment 52: A motion compensation device system according to any one of the above embodiments, wherein the outer sleeve includes a photographing portion for photographing the object.

[0234] Embodiment 53: The motion compensation device system according to any one of the above embodiments is characterized in that the compensation calculation unit calculates the distance or movement interval between the surgical instrument and the object.

[0235] Embodiment 54: A motion compensation device system according to any one of the above embodiments, wherein the compensation calculation unit utilizes 2D image data from the motion interval of the object.

[0236] Embodiment 55: The motion compensation device system according to any one of the above embodiments is characterized in that the compensation calculation unit uses at least one of the first data, the second data and the third data to calculate the motion compensation.

[0237] Embodiment 56: A motion compensation device system according to any one of the above embodiments, wherein the second data is captured in real time by the shooting unit, and the third data includes external data received from an external device.

[0238] Embodiment 57: A motion compensation device system according to any one of the above embodiments, wherein the external device includes a device other than the shooting unit.

[0239] Embodiment 58: The motion compensation device system according to any one of the above embodiments is characterized in that the second data forms an optical flow according to time, and the outer sleeve or surgical instrument part is configured to compensate for the movement of the object in real time through the optical flow of the second data.

[0240] Embodiment 59: A motion compensation device system according to any one of the above embodiments, wherein the third data is provided as an initial value for calculating the movement period of the object, and the compensation calculation unit calculates the motion compensation by associating the image data of the shooting unit with the third data.

[0241] Embodiment 60: The motion compensation device system according to any one of the above embodiments is characterized in that it also includes a monitoring unit, and the monitoring unit is configured to display the image data obtained by the shooting unit to the outside.

[0242] Embodiment 61: A motion compensation device system according to any one of the above embodiments, wherein when the observation direction of the photographing unit deviates from the moving direction of the object, the position of the photographing unit moves from an initial position to a correction position.

[0243] Embodiment 62: A motion compensation device system according to any one of the above embodiments, wherein the observation position at the correction position is parallel to the moving direction of the object.

[0244] Embodiment 63: The motion compensation device system according to any one of the above embodiments is characterized in that when the outer sleeve approaches the object, the position of the shooting unit is fixed at a fixed position, and the shooting unit collects image data of the object from the fixed position.

[0245] Embodiment 64: A motion compensation device system according to any one of the above embodiments, wherein the compensation calculation unit sets a target portion or a feature portion in the captured image of the shooting unit, the target portion is a reference portion on the screen displaying the object, and the feature portion represents a position change of the target portion on the captured image, and the compensation calculation unit tracks the target portion or the feature portion from the captured image and calculates the motion of the object at predetermined time intervals.

[0246] Embodiment 65: A motion compensation device system according to any one of the above embodiments, characterized in that the compensation calculation unit captures an initial image of the object and obtains a differential image, the differential image is obtained from an image captured after the initial image, and the differential image corresponds to a sampling period in which the most similar image appears in the repeated motion of the object observed by the shooting unit.

[0247] Embodiment 66: A motion compensation device system according to any one of the above embodiments, wherein the compensation calculation unit calculates the difference between the time of the image with the smallest sum of differences and the time of the initial image, wherein the difference is a benchmark for breathing.

[0248] Embodiment 67: A motion compensation device system according to any one of the above embodiments, further comprising a motion estimation modeling unit, wherein the motion estimation modeling unit performs modeling for estimating the motion of the object, and the motion estimation modeling unit learns the intrinsic motion of the object.

[0249] Embodiment 68: A motion compensation device system according to any one of the above embodiments, wherein parameters are updated according to the difference between the value estimated by the motion estimation modeling unit and the actual value observed by the shooting unit, and the motion compensation device re-estimates the motion compensation based on the updated parameters.

[0250] Embodiment 69: Use of the motion compensation device system according to any of the above embodiments for performing surgical operations.

[0251] Example 70: A method for operating a motion compensation device, wherein the method comprises the following steps: inserting an endoscope comprising an outer sleeve and a surgical instrument into a subject; using a compensation calculation unit to calculate motion compensation corresponding to the movement of an object in the subject; and adjusting the endoscope to adjust the distance between the object and the surgical instrument.

[0252] Example 71: A method according to Example 70, wherein the distance between the object and the surgical instrument is within a predetermined range.

[0253] Example 72: The method according to Example 70 or Example 71, further comprising the step of synchronizing the outer sleeve in response to the movement of the object.

[0254] Embodiment 73: A method according to any of the above embodiments, further comprising a photographing unit for acquiring image data of an object.

[0255] Embodiment 74: The method according to any one of embodiments 70 to 73 is characterized in that the outer sleeve or surgical instrument is configured to perform multiple movements, and the multiple movements include translational movement, bending movement or rotational movement.

[0256] Embodiment 75: The method according to any one of the above embodiments further comprises the following step: performing motion compensation for the translational movement direction of the outer sleeve relative to the object.

[0257] Embodiment 76: The method according to any one of embodiments 70 to 75 is characterized in that the motion compensation device includes a compensation calculation unit, which uses continuous 2D image data from the shooting unit to calculate the motion of the object.

[0258] Embodiment 77: A method according to any one of the above embodiments, wherein compensation calculation is used to calculate motion compensation using at least one of the first data, the second data and the third data.

[0259] Embodiment 78: A method according to any one of the above embodiments, wherein the second data is captured in real time by the photographing unit, and the third data includes external data received from an external device.

[0260] Embodiment 79: The method according to any one of embodiments 70 to 78, wherein the external device includes a device other than the photographing unit.

[0261] Embodiment 80: The method according to any one of embodiments 70 to 79 further includes the following step: using the compensation calculation unit to calculate motion compensation by associating the image data of the shooting unit with the third data.

[0262] Embodiment 81: A method according to any one of embodiments 70 to 80, wherein the motion compensation device further comprises a monitoring unit, and the monitoring unit is configured to externally display the image data acquired by the shooting unit.

[0263] Embodiment 82: A method according to any one of the above embodiments, further comprising the following step: when the observation direction of the photographing unit deviates from the moving direction of the object, the position of the photographing unit is moved from an initial position to a correction position.

[0264] Embodiment 83: A method according to any one of the above embodiments, wherein the observation position at the correction position is parallel to the moving direction of the object.

[0265] Embodiment 84: The method according to any one of embodiments 70 to 83 further includes the following steps: when the outer sleeve approaches the object, the position of the shooting unit is fixed, and the shooting unit collects image data of the object from the fixed position.

[0266] Embodiment 85: A method according to any one of the above embodiments, wherein a target portion or a feature portion is set in the captured image of the shooting portion by the compensation calculation unit, the target portion is a reference portion on the screen displaying the object, and the feature portion represents a position change of the target portion on the captured image, and the compensation calculation unit tracks the target portion or the feature portion from the captured image and calculates the movement of the object at predetermined time intervals.

[0267] Embodiment 86: A method according to any one of the above embodiments, wherein the compensation calculation unit captures an initial image of the object and obtains a differential image, the differential image is obtained from an image captured after the initial image, and the differential image corresponds to a sampling period in which the most similar image appears in the repeated motion of the object observed by the shooting unit.

[0268] Embodiment 87: The method according to any one of embodiments 70 to 86 further includes the following step: using the compensation calculation unit to calculate the difference between the time of the image with the smallest sum of differences and the time of the initial image, wherein the difference is a benchmark for breathing.

[0269] Embodiment 88: A method according to any one of embodiments 70 to 87, wherein the motion compensation device further comprises a motion estimation modeling unit, which performs modeling for estimating the motion of the object, and the motion estimation modeling unit learns the intrinsic motion of the object.

[0270] Embodiment 89: A method according to any one of embodiments 70 to 88, wherein parameters are updated according to the difference between the value estimated by the motion estimation modeling unit and the actual value observed by the capturing unit, and motion compensation is re-estimated based on the updated parameters.

[0271] Embodiment 90: A method according to any of the above embodiments, wherein the object is an object of the surgical instrument.

[0272] Embodiment 91: The method according to any one of embodiments 70 to 90, wherein the object comprises stones.

[0273] Embodiment 92: The method according to any one of embodiments 70 to 90, wherein the object includes at least a portion of a tissue.

[0274] <Additional Embodiments>

[0275] Embodiment 1: A motion compensation device, comprising: an outer sleeve including a surgical instrument and configured to be inserted into a subject; and a compensation calculation unit for calculating motion compensation corresponding to the movement of an object in the subject, wherein the motion compensation device controls the distance between the object and the surgical instrument.

[0276] Embodiment 2: According to the motion compensation device described in the above embodiment, the subject is a human being.

[0277] Embodiment 3: A motion compensation device according to any one of the above embodiments, wherein the subject is an animal.

[0278] Embodiment 4: A motion compensation device according to any one of the above embodiments, wherein the motion compensation controls and / or adjusts the distance between the object and the surgical instrument in response to the movement of the object.

[0279] Embodiment 5: A motion compensation device according to any one of the above embodiments, wherein the motion compensation device maintains the distance between the object and the surgical instrument within a predetermined distance.

[0280] Embodiment 6: The motion compensation device according to any one of the above embodiments, wherein the outer sleeve is synchronized in response to the movement of the object.

[0281] Embodiment 7: According to the motion compensation device described in any one of the above embodiments, wherein the object of the motion compensation is the outer sleeve.

[0282] Embodiment 8: The motion compensation device according to any one of the above embodiments, wherein the motion compensation device is a surgical instrument part, and the surgical instrument part is synchronized in response to the movement of the object.

[0283] Embodiment 9: The motion compensation device according to any one of the above embodiments further comprises a shooting unit for acquiring image data of the object.

[0284] Embodiment 10: The motion compensation device according to any one of the above embodiments, wherein the distance between the photographing unit and the object is maintained within another predetermined distance in response to the motion of the object.

[0285] Embodiment 11: A motion compensation device according to any of the above embodiments, wherein the outer sleeve or surgical instrument is configured to perform multiple movements.

[0286] Embodiment 12: A motion compensation device according to any one of the above embodiments, wherein the multiple motions include translation, bending or rotational motion.

[0287] Embodiment 13: A motion compensation device according to any one of the above embodiments, wherein the motion compensation is performed with respect to a translational motion direction of the outer sleeve relative to the object.

[0288] Embodiment 14: A motion compensation device according to any one of the above embodiments, wherein the motion compensation for the translational motion direction corresponds to breathing compensation.

[0289] Embodiment 15: A motion compensation device according to any of the above embodiments, wherein, in response to the movement of the object, the motion compensation of the surgical instrument is first performed along the translational motion direction, and in response to the remaining movement of the object, the surgical instrument provides additional motion compensation through bending or rotational movement.

[0290] Embodiment 16: The motion compensation device according to any one of the above embodiments further comprises a compensation calculation unit, which calculates the motion of the object using the continuous 2D image data from the shooting unit.

[0291] Embodiment 17: According to the motion compensation device described in any of the above embodiments, the outer sleeve includes a shooting part for shooting the object.

[0292] Embodiment 18: A motion compensation device according to any one of the above embodiments, wherein the compensation calculation unit calculates the distance or movement interval between the surgical instrument and the object.

[0293] Embodiment 19: The motion compensation device according to any one of the above embodiments, wherein the compensation calculation unit uses 2D image data from the motion interval of the object.

[0294] Embodiment 20: The motion compensation device according to any one of the above embodiments, wherein the compensation calculation unit calculates the motion compensation using at least one of the first data, the second data and the third data.

[0295] Embodiment 21: A motion compensation device according to any one of the above embodiments, wherein the first data is used to calculate motion compensation corresponding to the motion of the object, the second data is captured in real time by the shooting unit, and the third data includes external data received from an external device.

[0296] Embodiment 22: The motion compensation device according to any one of the above embodiments, wherein the external device includes a device other than the shooting unit.

[0297] Embodiment 23: A motion compensation device according to any one of the above embodiments, wherein the second data forms an optical flow according to time, and the outer sleeve or surgical instrument part is configured to compensate for the motion of the object in real time through the optical flow of the second data.

[0298] Embodiment 24: A motion compensation device according to any one of the above embodiments, wherein the third data is provided as an initial value for calculating a movement period of the object, and the compensation calculation unit calculates the motion compensation by associating the image data of the shooting unit with the third data.

[0299] Embodiment 25: The motion compensation device according to any one of the above embodiments further comprises a monitoring unit, wherein the monitoring unit is configured to display the image data acquired by the shooting unit to the outside.

[0300] Embodiment 26: A motion compensation device according to any one of the above embodiments, wherein when the observation direction of the photographing unit deviates from the moving direction of the object, the position of the photographing unit moves from an initial position to a correction position.

[0301] Embodiment 27: A motion compensation device according to any one of the above embodiments, wherein the observation position at the correction position is parallel to the moving direction of the object.

[0302] Embodiment 28: A motion compensation device according to any one of the above embodiments, wherein when the outer sleeve approaches the object, the position of the shooting unit is fixed at a fixed position, and the shooting unit collects image data of the object from the fixed position.

[0303] Embodiment 29: A motion compensation device according to any one of the above embodiments, wherein the compensation calculation unit sets a target portion or a feature portion in the captured image of the shooting unit, the target portion is a reference portion on the screen displaying the object, and the feature portion represents a position change of the target portion on the captured image, and the compensation calculation unit tracks the target portion or the feature portion from the captured image and calculates the motion of the object at predetermined time intervals.

[0304] Embodiment 30: A motion compensation device according to any one of the above embodiments, wherein the compensation calculation unit captures an initial image of the object and obtains a differential image, the differential image is obtained from an image captured after the initial image, and the differential image corresponds to a sampling period in which the most similar image appears in the repeated motion of the object observed by the shooting unit.

[0305] Embodiment 31: A motion compensation device according to any one of the above embodiments, wherein the compensation calculation unit calculates the difference between the time of the image with the smallest sum of differences and the time of the initial image, wherein the difference is a benchmark for breathing.

[0306] Embodiment 32: The motion compensation device according to any one of the above embodiments further comprises a motion estimation modeling unit, wherein the motion estimation modeling unit performs modeling for estimating the motion of the object, and the motion estimation modeling unit learns the intrinsic motion of the object.

[0307] Embodiment 33: A motion compensation device according to any one of the above embodiments, wherein parameters are updated according to the difference between the value estimated by the motion estimation modeling unit and the actual value observed by the shooting unit, and the motion compensation device re-estimates the motion compensation based on the updated parameters.

[0308] Embodiment 34: Use of the motion compensation device according to any of the above embodiments for performing surgical operations.

[0309] Embodiment 35: A motion compensation device system, comprising: a main device having a screen and a controller; a motion compensation device comprising a compensation calculation unit and a surgical instrument configured to be inserted into a subject; and a drive unit accommodating the motion compensation device, wherein the motion compensation is configured to control the distance between the object and the surgical instrument in response to a signal from the main device.

[0310] Embodiment 36: A motion compensation device system according to the above embodiments, wherein the motion compensation device comprises an outer sleeve configured to be inserted into a subject.

[0311] Embodiment 37: A motion compensation device system according to any of the above embodiments, wherein the subject is a human being.

[0312] Embodiment 38: A motion compensation device system according to any of the above embodiments, wherein the subject is an animal.

[0313] Embodiment 39: A motion compensation device system according to any of the above embodiments, wherein the motion compensation controls and / or adjusts the distance between the object and the surgical instrument in response to the movement of the object.

[0314] Embodiment 40: A motion compensation device system according to any one of the above embodiments, wherein the motion compensation device maintains the distance between the object and the surgical instrument within a predetermined distance.

[0315] Embodiment 41: A motion compensation device system according to any of the above embodiments, wherein the outer sleeve is synchronized in response to the movement of the object.

[0316] Embodiment 42: A motion compensation device system according to any one of the above embodiments, wherein the object of the motion compensation is the outer sleeve.

[0317] Embodiment 43: A motion compensation device system according to any one of the above embodiments, wherein the motion compensation device system is a surgical instrument part, and the surgical instrument part is synchronized in response to the movement of the object.

[0318] Embodiment 44: A motion compensation device system according to any one of the above embodiments, further comprising a shooting unit for acquiring image data of the object.

[0319] Embodiment 45: A motion compensation device system according to any one of the above embodiments, wherein the distance between the shooting unit and the object is maintained within another predetermined distance in response to the movement of the object.

[0320] Embodiment 46: A motion compensation device system according to any of the above embodiments, wherein the outer sleeve or surgical instrument is configured to perform multiple movements.

[0321] Embodiment 47: A motion compensation device system according to any one of the above embodiments, wherein the multiple motions include translation, bending or rotational motion.

[0322] Embodiment 48: A motion compensation device system according to any one of the above embodiments, wherein the motion compensation is performed with respect to the translational motion direction of the outer sleeve relative to the object.

[0323] Embodiment 49: A motion compensation device system according to any one of the above embodiments, wherein the motion compensation for the translational motion direction corresponds to breathing compensation.

[0324] Embodiment 50: A motion compensation device system according to any of the above embodiments, wherein, in response to the movement of the object, the motion compensation of the surgical instrument is first performed along the translational motion direction, and in response to the remaining movement of the object, the surgical instrument provides additional motion compensation through bending or rotational movement.

[0325] Embodiment 51: The motion compensation device system according to any one of the above embodiments further comprises a compensation calculation unit, which calculates the motion of the object using continuous 2D image data from the shooting unit.

[0326] Embodiment 52: A motion compensation device system according to any one of the above embodiments, wherein the outer sleeve includes a photographing portion for photographing the object.

[0327] Embodiment 53: A motion compensation device system according to any one of the above embodiments, wherein the compensation calculation unit calculates the distance or movement interval between the surgical instrument and the object.

[0328] Embodiment 54: A motion compensation device system according to any one of the above embodiments, wherein the compensation calculation unit utilizes 2D image data from the motion interval of the object.

[0329] Embodiment 55: A motion compensation device system according to any one of the above embodiments, wherein the compensation calculation unit calculates motion compensation using at least one of the first data, the second data and the third data.

[0330] Embodiment 56: A motion compensation device system according to any one of the above embodiments, wherein the first data is used to calculate motion compensation corresponding to the motion of the object, the second data is captured in real time by the shooting unit, and the third data includes external data received from an external device.

[0331] Embodiment 57: A motion compensation device system according to any one of the above embodiments, wherein the external device includes a device other than the shooting unit.

[0332] Embodiment 58: A motion compensation device system according to any one of the above embodiments, wherein the second data forms an optical flow according to time, and the outer sleeve or surgical instrument part is configured to compensate for the movement of the object in real time through the optical flow of the second data.

[0333] Embodiment 59: A motion compensation device system according to any one of the above embodiments, wherein the third data is provided as an initial value for calculating the movement period of the object, and the compensation calculation unit calculates the motion compensation by associating the image data of the shooting unit with the third data.

[0334] Embodiment 60: The motion compensation device system according to any one of the above embodiments further comprises a monitoring unit, wherein the monitoring unit is configured to display the image data acquired by the shooting unit to the outside.

[0335] Embodiment 61: A motion compensation device system according to any one of the above embodiments, wherein when the observation direction of the photographing unit deviates from the moving direction of the object, the position of the photographing unit moves from an initial position to a correction position.

[0336] Embodiment 62: A motion compensation device system according to any one of the above embodiments, wherein the observation position at the correction position is parallel to the moving direction of the object.

[0337] Embodiment 63: A motion compensation device system according to any one of the above embodiments, wherein, when the outer sleeve approaches the object, the position of the shooting unit is fixed at a fixed position, and the shooting unit collects image data of the object from the fixed position.

[0338] Example 64: A motion compensation device system according to any one of the above embodiments, wherein the compensation calculation unit sets a target portion or a feature portion in the captured image of the shooting unit, the target portion is a reference portion on the screen displaying the object, and the feature portion represents a position change of the target portion on the captured image, and the compensation calculation unit tracks the target portion or the feature portion from the captured image and calculates the motion of the object at predetermined time intervals.

[0339] Embodiment 65: A motion compensation device system according to any one of the above embodiments, wherein the compensation calculation unit captures an initial image of the object and obtains a differential image, the differential image is obtained from an image captured after the initial image, and the differential image corresponds to a sampling period in which the most similar image appears in the repeated motion of the object observed by the shooting unit.

[0340] Embodiment 66: A motion compensation device system according to any one of the above embodiments, wherein the compensation calculation unit calculates the difference between the time of the image with the smallest sum of differences and the time of the initial image, wherein the difference is a benchmark for breathing.

[0341] Embodiment 67: A motion compensation device system according to any one of the above embodiments, further comprising a motion estimation modeling unit, wherein the motion estimation modeling unit performs modeling for estimating the motion of the object, and the motion estimation modeling unit learns the intrinsic motion of the object.

[0342] Embodiment 68: A motion compensation device system according to any one of the above embodiments, wherein parameters are updated according to the difference between the value estimated by the motion estimation modeling unit and the actual value observed by the shooting unit, and the motion compensation device re-estimates the motion compensation based on the updated parameters.

[0343] Embodiment 69: Use of the motion compensation device system according to any of the above embodiments for performing surgical operations.

[0344] Example 70: A method for operating a motion compensation device, wherein the method comprises the following steps: inserting an endoscope comprising an outer sleeve and a surgical instrument into a subject; using a compensation calculation unit to calculate motion compensation corresponding to the movement of an object in the subject; and adjusting the endoscope to adjust the distance between the object and the surgical instrument.

[0345] Example 71: A method according to Example 70, wherein the distance between the object and the surgical instrument is within a predetermined range.

[0346] Example 72: The method according to Example 70 or Example 71, further comprising the step of synchronizing the outer sleeve in response to the movement of the object.

[0347] Embodiment 73: The method according to any one of embodiments 70 to 72, further comprising a photographing unit for acquiring image data of the object.

[0348] Example 74: A method according to any one of Examples 70 to 73, wherein the outer sleeve or surgical instrument is configured to perform multiple movements, including translation, bending or rotational movements.

[0349] Embodiment 75: The method according to any one of embodiments 70 to 74 further comprises the step of performing motion compensation for the translational motion direction of the outer sleeve relative to the object.

[0350] Embodiment 76: A method according to any one of embodiments 70 to 75, wherein the motion compensation device includes a compensation calculation unit, which uses continuous 2D image data from the shooting unit to calculate the motion of the object.

[0351] Embodiment 77: The method according to any one of embodiments 70 to 76 further includes the following step: using compensation calculation to calculate motion compensation using at least one of the first data, the second data and the third data.

[0352] Embodiment 78: A method according to any one of embodiments 70 to 77, wherein the first data is used to calculate motion compensation corresponding to the movement of the object, the second data is captured in real time by the shooting unit, and the third data includes external data received from an external device.

[0353] Embodiment 79: The method according to any one of embodiments 70 to 78, wherein the external device includes a device other than the photographing unit.

[0354] Embodiment 80: The method according to any one of embodiments 70 to 79 further includes the following step: using the compensation calculation unit to calculate motion compensation by associating the image data of the shooting unit with the third data.

[0355] Embodiment 81: A method according to any one of embodiments 70 to 80, wherein the motion compensation device further comprises a monitoring unit, and the monitoring unit is configured to display the image data acquired by the shooting unit to the outside.

[0356] Embodiment 82: The method according to any one of embodiments 70 to 81 further comprises the following step: when the observation direction of the photographing unit deviates from the moving direction of the object, the position of the photographing unit is moved from an initial position to a correction position.

[0357] Embodiment 83: A method according to any one of embodiments 70 to 82, wherein the observation position at the correction position is parallel to the moving direction of the object.

[0358] Embodiment 84: The method according to any one of embodiments 70 to 83 further comprises the following step: when the outer sleeve approaches the object, the position of the photographing unit is fixed, and the photographing unit collects image data of the object from the fixed position.

[0359] Embodiment 85: A method according to any one of embodiments 70 to 84, wherein a target portion or a feature portion is set in the captured image of the shooting portion by the compensation calculation unit, the target portion is a reference portion on the screen displaying the object, and the feature portion represents a position change of the target portion on the captured image, and the compensation calculation unit tracks the target portion or the feature portion from the captured image and calculates the movement of the object at predetermined time intervals.

[0360] Embodiment 86: A method according to any one of embodiments 70 to 85, wherein the compensation calculation unit captures an initial image of the object and obtains a differential image, wherein the differential image is obtained from an image captured after the initial image, and the differential image corresponds to a sampling period in which a most similar image appears in the repeated motion of the object observed by the photographing unit.

[0361] Embodiment 87: The method according to any one of embodiments 70 to 86 further includes the following step: using the compensation calculation unit to calculate the difference between the time of the image with the smallest sum of differences and the time of the initial image, wherein the difference is a benchmark for breathing.

[0362] Embodiment 88: A method according to any one of embodiments 70 to 87, wherein the motion compensation device further comprises a motion estimation modeling unit, which performs modeling for estimating the motion of the object, and the motion estimation modeling unit learns the intrinsic motion of the object.

[0363] Embodiment 89: A method according to any one of embodiments 70 to 88, further comprising the steps of: updating parameters according to the difference between the value estimated by the motion estimation modeling unit and the actual value observed by the shooting unit; and re-estimating motion compensation based on the updated parameters.

[0364] Embodiment 90: The method according to any one of embodiments 70 to 89, wherein the object is an object of the surgical instrument.

[0365] Embodiment 91: The method according to any one of embodiments 70 to 90, wherein the object comprises stones.

[0366] Embodiment 92: The method according to any one of embodiments 70 to 90, wherein the object includes at least a portion of a tissue.

Claims

1. A motion compensation device, wherein: include: an outer cannula, which is inserted into the human body and reaches a target object to be operated on, and a compensation calculation unit, which calculates motion compensation corresponding to the motion of the target object in the human body; The outer sleeve includes a surgical instrument portion, and the surgical instrument portion processes the target object; In response to the movement of the target, a distance between the target and the surgical instrument portion or a distance between the target and the outer sleeve is maintained within a predetermined range.

2. The motion compensation device according to claim 1, wherein: The object of the motion compensation is the outer sleeve. The outer sleeve is synchronized in a manner corresponding to the movement of the target object.

3. The motion compensation apparatus according to claim 1, wherein: The object of the motion compensation is the surgical instrument part, The surgical instrument components are synchronized in a manner corresponding to the movement of the target object.

4. The motion compensation apparatus according to claim 1, wherein: The objects of the motion compensation are the outer tube and the surgical instrument part. The outer sleeve and the surgical instrument portion move in a manner related to each other in response to the movement of the target object, By the mutually linked actions of the overtube and the surgical instrument section, the distance between the target object and the surgical instrument section or the distance between the target object and the overtube is maintained within a predetermined range.

5. The motion compensation apparatus according to claim 1, wherein: The outer sleeve includes a photographing unit, and the photographing unit acquires image data of the target object; In response to the movement of the target object, the distance between the imaging unit and the target object is maintained within a predetermined range.

6. The motion compensation apparatus according to claim 1, wherein: The outer sleeve or surgical instrument part performs translational movement, bending movement or rotational movement, The motion compensation is performed with respect to the translational motion direction of the outer tube or the surgical instrument relative to the target object. The motion compensation for the translational motion direction is responsive to the motion of the target object caused by the breathing of the human body.

7. The motion compensation apparatus according to claim 1, wherein: In response to the movement of the target object, the surgical instrument portion or the outer sleeve first performs motion compensation along a translational motion direction relative to the target object, Remaining movements of the target object are additionally compensated by bending or rotating the surgical instrument part or the outer sleeve.

8. The motion compensation apparatus according to claim 1, wherein: The outer sleeve includes a photographing portion for photographing a target object; The compensation calculation unit calculates the motion of the target object using the continuous image data of the imaging unit.

9. The motion compensation apparatus according to claim 1, wherein: The outer sleeve includes a photographing portion for photographing the target object; The target repeatedly approaches or moves away from the imaging unit according to a motion cycle including a breathing cycle, The compensation calculation unit calculates the distance, the movement cycle, or the movement direction between the surgical instrument unit and the target object using image data according to the movement cycle of the target object.

10. The motion compensation apparatus according to claim 1, wherein: The outer sleeve includes a photographing portion for photographing the target object; The motion compensation device includes a motion estimation modeling unit that performs modeling to estimate the motion of the target object including biological motion; When there is no input of image data through the photographing section or data transmission from a measuring section of an external device, the compensation calculation section receives data from the motion estimation modeling section to calculate the motion compensation.

11. The motion compensation apparatus according to claim 1, wherein: The outer sleeve includes a photographing portion for photographing a target object; The first data is data for calculating motion compensation corresponding to the motion of the target object. The second data is data captured in real time by the capturing unit. The third data is external data received from a measuring unit which is an external device other than the imaging unit. The compensation calculation unit calculates motion compensation using one or more of the first data, the second data, and the third data.

12. The motion compensation apparatus according to claim 1, wherein: The outer sleeve includes a photographing portion for photographing a target object; The compensation calculation unit receives the second data captured by the capturing unit in real time, The overtube or the surgical instrument unit operates in a manner that compensates for the movement of the target object in real time according to the motion compensation calculated by the compensation calculation unit using the second data.

13. The motion compensation apparatus according to claim 1, wherein: The outer sleeve includes a photographing portion for photographing a target object; The compensation calculation unit receives third data related to the patient information from a measurement unit of an external device other than the imaging unit, The third data is used to calculate the movement period of the target object including the breathing period or to calculate the movement direction, The compensation calculation section associates the image data of the imaging section with the third data to calculate motion compensation.

14. The motion compensation apparatus according to claim 1, wherein: The outer sleeve includes a photographing unit, and the photographing unit acquires image data of the target object; The motion compensation device includes a monitoring unit that displays the image data acquired from the shooting unit to the outside; Whether the target object on the monitoring unit is stationary or moving is determined by whether the object of motion compensation for the movement of the target object is the outer sleeve, the surgical instrument unit and the imaging unit, or by the configuration structure among the outer sleeve, the imaging unit and the surgical instrument unit.

15. The motion compensation apparatus according to claim 1, wherein: The outer sleeve includes a photographing portion for photographing a target object; When the field of view of the photographing unit at the initial position deviates from the moving direction of the target object, the outer sleeve or the photographing unit moves to a correction position. The field of view direction of the photographing unit at the calibration position is set to be consistent with or parallel to the moving direction of the target object.

16. The motion compensation apparatus according to claim 1, wherein: When the outer tube reaches the target object, the position of the imaging unit is fixed. The photographing unit collects image data of the target object at a fixed position. Based on the image data from the imaging unit, the overtube or the surgical instrument unit moves in synchronization with the movement of the target object.

17. The motion compensation apparatus according to claim 1, wherein: The outer sleeve includes a photographing portion for photographing a target object; The compensation calculation unit sets a target portion or a feature portion on the image captured by the imaging unit, The target portion is a portion that serves as a reference on the screen including the target object. The characteristic portion indicates a position change of the target portion on the captured image. The compensation calculation unit tracks the target portion or the feature portion from the images captured by the capturing unit at predetermined time intervals to calculate the movement of the target object.

18. The motion compensation apparatus according to claim 1, wherein: The outer sleeve includes a photographing portion for photographing a target object; The compensation calculation unit captures an initial image of the target object and obtains image information having a high similarity to the initial image from images captured after the initial image. The compensation calculation unit sets the difference between the time of the acquired image having the smallest total value of the image information with high similarity and the initial image as a reference of the respiratory cycle.

19. The motion compensation apparatus according to claim 1, wherein: The outer sleeve includes a photographing portion for photographing the target object; The motion compensation device includes a motion estimation modeling unit that performs modeling to estimate the motion of the target object including biological motion; The motion estimation modeling unit learns the intrinsic motion of the target object, The motion estimation modeling unit, updating the parameters based on a difference between a value estimated by modeling and an actual value observed by the imaging unit, Motion compensation is re-estimated based on the updated parameters.

Images