Compensating movement device

Abstract

The motion compensation device of the present invention may include an overtube inserted into the human body and reaching a subject to surgery, and a compensation calculation unit that calculates a motion compensation corresponding to the movement of said subject within the human body, and the overtube may include a surgical tool unit that applies treatment to said subject, and in response to the movement of said subject, the distance between said subject and said surgical tool unit, or the distance between said subject and said overtube, may be maintained within a predetermined range.

Description

Technology Field

[0001] The present invention relates to a motion compensation device that calculates motion compensation for a surgical tool corresponding to the movement of a surgical target within the human body. Background Technology

[0002] In the field of medicine, surgery using robots that include master and slave stations, implemented independently or as a single unit, has recently been gaining attention.

[0003] Even in the field of surgery involving the insertion of a flexible endoscope without open surgery, controlling the movement of the flexible endoscope with a robot can improve the problems that arise from operating in a constrained environment while passing through internal tissues or organs.

[0004] In particular, flexible endoscopes connected to peripheral devices such as external slave stations or robotic arms can have their coordinate systems or positions fixed relative to the target object, such as a lesion or organ within the human body. On the other hand, the surgical target object may exhibit regular or irregular movements over time due to anatomical factors such as respiration and heartbeats, or mechanical factors such as wire tension. In such cases, the surgeon faces the difficulty of having to perform surgery while the target object is moving, while viewing images transmitted via imaging means, such as a camera installed at the tip of the endoscope. The problem to be solved

[0005] The present invention can provide a technology that recognizes the movement of a surgical subject within the human body and synchronizes a surgical tool part provided at the tip of an over tube with the movement of the subject. means of solving the problem

[0006] The motion compensation device of the present invention may include an overtube inserted into the human body and reaching a subject to surgery, and a compensation calculation unit that calculates a motion compensation corresponding to the movement of said subject within the human body, and the overtube may include a surgical tool unit that applies treatment to said subject, and in response to the movement of said subject, the distance between said subject and said surgical tool unit, or the distance between said subject and said overtube, may be maintained within a predetermined range. Effects of the invention

[0007] The present invention can calculate the movement of a subject including a breathing cycle by using image data obtained at a predetermined time interval from a shooting unit provided at the end of an over tube.

[0008] In addition, the present invention can calculate the movement of a target object by a motion estimation modeling unit even when there is no input data or only minimal information including a pre-entered table or preset values.

[0009] The subject of surgery may have three degrees of freedom of movement, and the three-dimensional movement of the subject may be compensated through translational, bending, or rotational movement of the surgical tool part or over tube of the present invention.

[0010] Compared to the method in which a doctor performs treatment operations such as crushing at a specific time in accordance with the movement of the object to be treated, according to the present invention, the movement of the surgical tool part or the over tube is compensated so that the distance between the surgical tool part or the surgical means and the object to be treated, which is a stone, is maintained at a constant level, thereby increasing efficiency such as the probability or strength of hitting the stone with a crushing means such as a laser, and can lead to a reduction in surgery time or a reduction in the frequency of mucosal contact.

[0011] The main movement of the subject may correspond to the extension direction of the surgical tool part of the present invention. After the movement in the translational direction of the surgical tool part is compensated, the physician may decide through the monitoring unit whether to compensate for the remaining degrees of freedom of movement or perform surgical operations such as crushing, or perform surgical operations after the movement compensation according to the present invention is completed.

[0012] When an image with motion compensation applied to a subject according to the present invention is displayed on a monitoring unit, the surgeon performing the operation can ensure intuitiveness of the motion. If it is determined that motion compensation is not functioning smoothly, immediate visual confirmation and additional correction are possible, thereby enabling a more efficient response to unexpected situations during surgery.

[0013] Accordingly, if the movement of the subject due to breathing, etc., is compensated for according to the present invention and the distance between the over tube or the surgical instrument part and the subject is maintained, the physician does not need to manually operate to maintain a constant distance from the subject or perform surgery while forcibly stopping breathing, so the dependence on the individual physician's skill level can be reduced, the surgical risk due to the patient's condition can be reduced, and the burden on the operating physician can be reduced. Brief explanation of the drawing

[0014] FIG. 1 is an overall explanatory diagram of the driving part and flexible endoscope part of the motion compensation device of the present invention from a surgical perspective. FIG. 2 is an explanatory diagram of the flexible endoscope part of the present invention. Figure 3 is a configuration diagram of the motion compensation method of the present invention. Figure 4 is a configuration diagram of the motion compensation device of the present invention. FIG. 5 is an explanatory diagram of the movement of the over tube or surgical tool part of the present invention. FIG. 6 is an explanatory diagram of the motion compensation calculation of the compensation calculation unit of the present invention. Figure 7 shows the relative movement of the object to the over tube of the present invention. FIG. 8 is an explanatory diagram of the orientation of the present invention. FIG. 9 is an explanatory diagram of the case where the object of movement compensation according to the movement of the subject of the present invention, or the driving object of movement compensation, is a surgical tool part. FIG. 10 is an explanatory diagram of the case where the object of motion compensation according to the movement of the object of the present invention, or the driving object of motion compensation, is an over tube. FIG. 11 is an explanatory diagram of the first to fourth data flows obtained according to the present invention. FIG. 12 is an explanatory diagram of data collection or calculation of breathing movements by the imaging unit of the present invention. FIG. 13 is an explanatory diagram of an embodiment of the calculation of breathing movements, such as breathing cycles, by the compensation calculation unit of the present invention. FIG. 14 is an embodiment of obtaining third data of the present invention from an artificial respirator / anesthesia workstation, which is an embodiment of a measurement unit. FIG. 15 is another embodiment of obtaining third data of the present invention from a C-arm, which is another embodiment of the measurement unit. Specific details for implementing the invention

[0015] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings. However, the technical concept of the present invention is not limited to some of the described embodiments and can be implemented in various forms, and at least one of the components among the embodiments may be optionally combined and / or substituted within the scope of the technical concept of the present invention.

[0016] In addition, unless specifically defined otherwise, the terms of the embodiments of the present invention may be interpreted in a sense that is generally understood by those skilled in the art, and commonly used terms may be interpreted by considering the contextual meaning of the relevant technology.

[0017] Furthermore, the terms of the embodiments of the present invention are for the purpose of describing the embodiments and are not intended to limit the present invention, and the singular can be interpreted to include the plural unless otherwise stated in the text.

[0018] In addition, in the components of the embodiments of the present invention, terms such as first, second, third, or A, B, C, etc. may be used, and such terms are used merely to distinguish one component from another and do not limit the order or sequence.

[0019] In addition, when a component is described as being “connected,” “joined,” or “combined” with another component in an embodiment of the present invention, it may mean not only that one component is directly connected, joined, or combined with another component, but also that it is indirectly connected, joined, or combined by another component between the two components.

[0020] In addition, in an embodiment of the present invention, one component being placed, formed, or positioned above or below another component may include one component being placed, formed, or positioned directly or indirectly on the other component. The expressions "above or below" or "above or below" may mean not only an upward direction but also a downward direction relative to one component.

[0021] With reference to FIGS. 1 to 15, the motion compensation device and motion compensation method of the present invention will be described.

[0022] Referring to FIGS. 1 and 2, the motion compensation device (100) of the present invention can be applied to surgery for inserting a flexible endoscope (170) into the human body.

[0023] In one embodiment, the present invention may be applied to retrograde intrarenal surgery (RIRS) for crushing stones around the kidney (21) by inserting an over tube (200) through the urethra (27). The present invention may be applied to crushing or extracting surgery of at least one of kidney stones, renal pelvis stones, and ureteral stones.

[0024] Here, the subject (50) is the part that is the target of the surgery, and may be a part of an organ that is subject to crushing or external extraction, such as a lesion or a stone. The movement of the subject (50) may include human factors such as internal respiration and heartbeat, or mechanical factors. In one embodiment, the main movement compensation of the present invention may mean compensation for the movement of the subject (50) caused by respiration, and compensation for the movement of the subject (50) caused by other human factors or mechanical factors may be treated as incidental.

[0025] The motion compensation device (100) may include a monitoring unit (110) in which an image captured through a shooting unit (220) is displayed, or an operating unit (130) in which a doctor operates a driving unit (150) through means including a joystick, a handle, a gripper, etc.

[0026] The driving unit (150) may include at least one of a translational driving unit (151), a bending driving unit (153), and a rotational driving unit (155), and the target of the driving may be at least one of an overtube (200), a shooting unit (220), and a surgical tool unit (240).

[0027] The imaging section (220) or surgical tool section (240) may be extended along the length inside the over tube (200) in a tubular shape.

[0028] In one embodiment, the motion compensation device (100) may include a master station (MS) or a slave station (SS).

[0029] The master station (MS) allows the surgeon performing the operation to monitor the surgical situation and operate at least one of the flexible endoscope unit (170), over tube (200), imaging unit (220), and surgical tool unit (240). The slave station (SS) may be operated by receiving instructions from the surgeon from the master station (MS) or by operating through a separate control unit that can be linked with other display means.

[0030] The master station (MS) or slave station (SS) may be implemented independently or as a single unit, depending on the surgical site or component layout. The monitoring unit (110) or the operation unit (130) may be included in the master station (MS), and the drive unit (150), the flexible endoscope unit (170), or the robot arm (140) may be included in the slave station (SS).

[0031] The flexible endoscope (170) can be mounted on a slave station (SS) including a robot arm (140), and can be adjusted to a height or angle suitable for surgery by the robot arm (140).

[0032] The flexible endoscope (170) may include a flexible tube-type over tube (200) that is inserted into the human body (S110). The over tube (200) may be composed of a plurality of joints having degrees of freedom under certain constraint conditions, and as a result, the end (end tip) of the over tube (200) can freely approach the object (50) with three degrees of freedom.

[0033] In one embodiment, the over tube (200) is inserted into the urethra (27) and can reach the kidney (21) or its vicinity through the bladder (25) and ureter (23). As such, the over tube (200) is flexible so as to extend along irregular, winding, or smooth organs and can be moved in three dimensions by the driving unit (150).

[0034] The imaging unit (220) or surgical tool unit (240) may include a wire extending along a path inside the over tube (200) and an end effector protruding from the tip of the over tube (200). In the case of the imaging unit (220), the end effector may be a imaging means such as a camera capable of collecting image data inside the body. The image data collected by the imaging unit (220) may comprehensively include not only two-dimensional data or three-dimensional data including RGB-D or stereo, but also data mounted on the over tube (200) capable of obtaining body information, such as Lidar, IR, sonar, etc.

[0035] In the case of the surgical tool part (240), the end effector may be a surgical means. The surgical means may include at least one of a crushing means capable of crushing the target object (50), such as a laser; a collecting means, such as a basket, for collecting the target object (50) that needs to be extracted from the body; and a suction means for spraying water onto the target object (50) or suctioning the target object (50).

[0036] The surgical tool unit (240) can apply treatment to the subject (50). Here, 'treatment' may include all actions that can be applied to the subject (50) during surgery, excluding the acquisition of image data by the imaging unit (220), etc. Accordingly, the treatment may include crushing means, collection means, or suction means by the surgical means described above.

[0037] The portion of the flexible endoscope (170) connected to the slave station (SS) or the driving unit (150) may be called the proximal portion, and the tip portion of the flexible endoscope (170) connected to the imaging means or surgical means may be called the distal portion. The proximal portion or the distal portion may be used to refer to the upstream or downstream direction, respectively, along the path extending along the over tube (200).

[0038] The flexible endoscope (170) may be provided with a handle (180) or an access sheath (172) or an operating part in the proximal part.

[0039] The handle portion (180) may include an insertion means into which a shooting portion (220) or a surgical tool portion (240) can be inserted, or an operating means capable of operating at least one of the overtube (200), the shooting portion (220), and the surgical tool portion (240). For example, the shooting portion (220) may be provided integrated with the overtube (200), and the surgical tool portion (240) may be inserted or replaced through the handle portion (180).

[0040] The present invention can provide movement compensation, including compensation for movement of the subject (50) caused by respiration, using image data obtained from the imaging unit (220). The imaging unit (220) or the surgical tool unit (240) may be included in the over tube (200) or provided separately. Although the drawings of the present invention describe an embodiment in which the imaging unit (220) or the surgical tool unit (240) is included in the over tube (200), the invention is not limited thereto and may also be applied in cases where the imaging unit (220) or the surgical tool unit (240) is provided separately from the over tube (200).

[0041] The surgical tool part (240) equipped with a laser, basket, etc. may be replaced through the insertion means of the handle part (180) as the surgery progresses, or multiple surgical tool parts (240) may be provided at a thickness allowed in the insertion environment.

[0042] The operating means of the handle part (180) is provided separately from the drive part (150) and can serve to assist the drive part (150).

[0043] The access sheath (172) may be provided at the entrance of the urethra and may guide the insertion of the over tube (200) into the human body. Without the access sheath (172), damage due to internal friction is likely to occur during the insertion or withdrawal of the over tube (200). In particular, according to the present invention, the surgical means or surgical tool part (240) may be operated in response to the movement of the subject (50) to compensate for the movement.

[0044] With reference to FIGS. 5 to 10, the mechanism and structure of motion compensation, such as respiration compensation and heart rate compensation, of the present invention will be described.

[0045] According to FIG. 5, the surgical tool portion (240) of the over tube (200) can perform translational, bending, or rotational movements, and through at least some of these or a combination thereof, the surgical tool portion (240) can perform all three degrees of freedom of movement. From this, the tip of the over tube (200) can pass through a winding organ and approach the target (50) in a desired direction.

[0046] Referring to FIG. 5, the surgical tool part (240) is illustrated as translating, bending, or rotating (rolling), but depending on the structure, the translation, bending, or rotation of the over tube (200) is transmitted to the surgical tool part (240), and consequently, the translation, bending, or rotation of the surgical tool part (240) may be implemented.

[0047] The image or video captured by the shooting unit (220) or the shooting means may be determined according to the direction in which the tip of the over tube (200) is facing. Image data including a two-dimensional image of the shooting unit (220) may be implemented as a coordinate system consisting of an axis (z-axis) parallel to the direction in which the shooting unit (220) is facing and a plane (xy-plane) perpendicular to that axis. This coordinate system of the two-dimensional image of the shooting unit (220) may be a relative coordinate system that changes according to the position of the tip of the over tube (200).

[0048] The main objective of the present invention may be to maintain the change in the gap between the surgical tool part (240), such as a laser or basket, which may be provided at the tip of the over tube (200), i.e., the distal part, and the object (50), or between the over tube (200) and the object (50), at a level below a certain limit, or to maintain the gap within a predetermined range.

[0049] In most cases, the purpose is to maintain the distance between the surgical tool part (240) and the target (50) within a predetermined range. However, in cases where the surgical tool part (240) is detached and at least some of the target (50) is sucked using an empty tube (or empty channel) rather than a suction means of the surgical tool part (240), it may be necessary to maintain the distance between the over tube (200), the tip of the over tube (200), and the target (50) within a predetermined range.

[0050] The compensation calculation unit (400) or the control unit (370) can ensure that the distance between the surgical tool unit (240) and the object (50), or the distance between the over tube (200) and the object (50), is maintained within a predetermined range. The compensation calculation unit (400) or the control unit (370) can determine that there is normal movement compensation if the distance between the surgical tool unit (240) and the object (50), or the distance between the over tube (200) and the object (50), is maintained within a predetermined set range, and can recalculate the movement compensation if it is determined that it deviates from the predetermined set range.

[0051] Unless otherwise specifically distinguished or mentioned in this specification, maintaining a predetermined range of spacing between the surgical tool part (240) and the object to be treated (50), and maintaining the spacing between the over tube (200) and the object to be treated (50) within a predetermined range are common objectives of the present invention.

[0052] The distance between the surgical tool part (240) and the object (50) can have all three-dimensional components as a vector concept, and the movement of the object (50) has regularity due to its characteristics and can have repetitive movement along a specific direction. At this time, the position of the object (50) attached to the body, such as a stone, can be assumed to follow body movements such as breathing. For example, if the position of the object (50) changes or shows irregular movement with each breath, it may be necessary to identify additional movements to compensate for it.

[0053] The main direction of movement or the direction of movement compensation of the object (50) may vary depending on the position of the object (50) or the position of the over tube (200).

[0054] Therefore, if movement compensation is provided so that the direction of the tip of the over tube (200) matches the specific direction movement of the object (50) as much as possible, it can have the effect of offsetting most of the movement of the object (50).

[0055] The present invention may be intended to provide convenience when a doctor performs surgery using a surgical robot, and even if the movement of the subject (50) is not completely offset, if the most dominant direction of movement is compensated and offset, it may have the effect of offsetting most of the movement of the subject (50) so that the distance between the subject (50) and the surgical tool part (240), the distance between the subject (50) and the over tube (200), or the distance between the subject (50) and the imaging part (220) is well maintained within a predetermined range. An additional step may be taken to make the direction of movement compensation of the surgical tool part (240), the over tube (200), or the imaging part (220) align with or parallel to the most dominant direction of movement of the subject (50).

[0056] The direction of translational movement is one example of the main direction of movement of the subject (50), and the process of movement compensation (or main direction of movement) may vary depending on the relative visual condition or position of the over tube (200) located within an organ such as a kidney.

[0057] When described in terms of the relative coordinate system of the imaging unit (220), if the surgical tool unit (240) compensates for movement along the translational motion direction (z-axis direction) in response to the movement of the subject (50) (S331), the largest movement of the subject (50) can be offset. The remaining degrees of freedom along the plane perpendicular to the translational motion direction (xy plane) can be further compensated for by bending motion or rotational motion (S333). That is, if the movement along the z-axis, which is the translational motion direction, is compensated for and offset, only movement along the plane composed of the x-axis and y-axis will be visible.

[0058] Additional movement compensation (S333) may include compensation for movement caused by mechanical factors, such as wire tension, in addition to human factors, such as friction, restraint, and elastic deformation of tissues caused by soft internal tissues or organs, excluding the main direction of movement. The direction of additional movement compensation may vary depending on the position of the object (50) or the position of the over tube (200).

[0059] With reference to FIGS. 7 to 10, the motion compensation mechanism of the present invention will be described.

[0060] FIG. 7 shows that the tip of the over tube (200) has reached the target object (50), such as a calculus (S130), and FIG. 9 and FIG. 10 show that movement compensation is performed so that the movement between the surgical tool part (240) or the over tube (200) and the target object (50) is mutually synchronized.

[0061] The over tube (200) can be inserted into the body (S110) and reach the target object (50) (S130) through translation, bending, or rotational movement. The image data may include image data such as photographs or videos that can be monitored by a doctor while performing surgery.

[0062] At this time, the imaging unit (220) can collect image data at a fixed position. Here, fixation may mean stopping with respect to an absolute coordinate system associated with external surgical equipment, such as a master station (MS) and a slave station (SS), independent of the movement of the human body.

[0063] In order to maintain the distance between the surgical tool part (240) and the subject (50) within a predetermined range by synchronizing the surgical tool part (240) with the dynamic movement of the subject (50) over time and compensating for the movement, an absolute coordinate system that can serve as a reference for the movement of the subject (50) is required. Unlike conventional endoscopic surgery, robot-assisted endoscopic surgery has the advantage of ensuring reliable position fixation of the end effector at the end of the over tube (200), which consequently leads to accurate movement compensation.

[0064] When the tip of the surgical tool part (240) or the over tube (200) reaches the target point and is fixed or stopped, as shown in FIG. 7, the distance (L+α) between the surgical tool part (240) and the target (50) also changes as much as the target moves (α).

[0065] In FIG. 9 or FIG. 10, the movement compensation of the present invention is applied so that the change in distance between the surgical tool part (240) and the subject (50) is maintained below a certain value or within a predetermined range (L-β to L+β, where β is the error).

[0066] Since the direction of movement may vary depending on the position of the subject (50) even within the same organ, such as the kidney, the viewing angle of the imaging unit (220) needs to be changed when the subject (50) changes, and the size of the movement that needs to be compensated for, the degree of movement observed, and the direction of movement may change accordingly.

[0067] If the viewing direction of the imaging unit (220) at the initial position (P1) where the over tube (200) approaches the subject (50) to collect data is significantly different from the direction of movement (α) of the subject (50), then an orientation change in the imaging direction of the imaging unit (220) may be required.

[0068] Referring to FIG. 8, if the viewing direction of the imaging unit (220) at the initial position (P1) is slightly misaligned with the movement direction (α) of the subject (50), it can be moved to a modified position (P1') to align the viewing direction with the movement direction (α). In this case, a conversion of the relative coordinate system can occur from the coordinate system (A) (x1 axis, y1 axis, z1 axis) of the initial position (P1) to the coordinate system (A') (x2 axis, y2 axis, z2 axis) of the modified position (P1') by orientation adjustment. The z-axis direction of the modified coordinate system (A') can be parallel to the movement direction of the subject (50). The absolute coordinate system (x-axis, y-axis, z-axis) may be contained or integrated into external equipment of the surgical robot or motion compensation device (100) that does not change according to movements such as breathing, or it may be a coordinate system that is converted to move together with the subject (50).

[0069] The relative coordinate values ​​for the shooting unit (220) or the subject (50) that moves according to breathing, etc., can be accurately calculated by comparing the absolute coordinate system and the relative coordinate system (A or A').

[0070] FIGS. 9 and FIGS. 10 illustrate embodiments of the motion compensation of the invention. FIG. 9 may show that the surgical tool part (240) is motion compensated by synchronization corresponding to the movement (α) of the object (50). FIG. 10 may show that the over tube (200) is motion compensated by synchronization corresponding to the movement (α) of the object (50).

[0071] When the surgical tool unit (240) is synchronized, the risk of damage to the organ surface, etc. caused by the over tube (200) can be reduced. Additionally, minimizing movement on the organ surface due to the irregular internal structure of the organ can help reduce the accumulation of over tube (200) control errors.

[0072] When the over tube (200) is synchronized, a control step for issuing subsequent control commands to end effectors, such as the imaging unit (220) or surgical tool unit (240), can be made clear during the process of performing surgery after movement compensation.

[0073] When the surgical tool unit (240) is synchronized, the surgical tool unit (240) continues to perform translational movements, and subsequent commands such as additional compensation, crushing, or capture may be issued in a superposition manner. On the other hand, when the over tube (200) is synchronized, the imaging unit (220) or the surgical tool unit (240) can be fixed in position on the over tube (200), and if only the over tube (200) is movement compensated, the remaining imaging unit (220) or the surgical tool unit (240) can automatically obtain a movement compensation effect, which is an advantage.

[0074] The imaging unit (220) may have various arrangement structures, such as being fixed in position on the over tube (200), protruding from the tip of the over tube (200) like the surgical tool unit (240) to enable operation, or being implemented to be integrated with the surgical tool unit (240).

[0075] The shooting unit (220) can be the target of motion compensation or the driving target of motion compensation according to the movement of the target object (50).

[0076] Depending on the object of motion compensation or the driving object of motion compensation according to the movement of the subject (50), the installation location of the imaging unit (220) or the arrangement structure between the over tube (200), the imaging unit (220), and the surgical tool unit (240), the image displayed to the doctor through the monitoring unit (110) may appear almost stationary (first image) synchronized with the subject (50), or the subject (50) may appear to be moving (second image).

[0077] In either case of the two types (first image and second image) that are mutually stopped or mutually operated, the distance between the surgical tool unit (240) and the subject (50) remains constant, so one of the two types may be displayed manually or automatically depending on the surgeon's preference or the situation during surgery. Depending on the surgeon's choice, an option may be provided for the two types of images to be switched between each other. Depending on the mutual arrangement structure between the over tube (200), the imaging unit (220), or the surgical tool unit (240), the first image or the second image may be switched between through image data compensation.

[0078] When the movement of the surgical tool unit (240) is compensated (Fig. 9) and the imaging unit (220) is fixed in position on the over tube (200), the image displayed to the doctor through the monitoring unit (110) may appear as if the subject (50) is moving in accordance with the breathing cycle, breathing direction, etc. For the monitoring unit (100) to display the subject (50) as being fixed, additional image-based calculations may be required.

[0079] When the movement of the surgical tool unit (240) is compensated (Fig. 9) and the imaging unit (220) is not fixed to the over tube (200) and can operate separately, the imaging unit (220) and the subject (50) can be synchronized through additional synchronization between the imaging unit (220) and the surgical tool unit (240) so that the image displayed to the doctor through the monitoring unit (110) appears as if the subject (50) has stopped.

[0080] When the movement of the surgical tool unit (240) is compensated (Fig. 9) and the imaging unit (220) is implemented to be integrated with the surgical tool unit (240), the image displayed to the doctor through the monitoring unit (110) may appear as if the subject (50) has stopped. When crushing or capturing is performed while the movement of the surgical tool unit (240) is compensated, the subject (50) will remain in a stopped state, but if there is additional compensation or additional movement by the doctor's operation of the surgical tool unit (240), the monitored screen may change.

[0081] When the movement of the over tube (200) is compensated (Fig. 10) and the shooting unit (220) is fixed in position on the over tube (200), the image displayed to the doctor through the monitoring unit (110) may appear as if the subject (50) has stopped.

[0082] When the movement of the over tube (200) is compensated (Fig. 10) and the imaging unit (220) is not fixed to the over tube (200) and can operate separately, additional synchronization between the imaging unit (220) and the over tube (200) may be required so that the image displayed to the doctor through the monitoring unit (110) appears as if the subject (50) has stopped.

[0083] When the movement of the over tube (200) is compensated (Fig. 10) and the imaging unit (220) is implemented to be integrated with the surgical tool unit (240), the image displayed to the doctor through the monitoring unit (110) may appear as if the subject (50) has stopped. When crushing or capturing is performed while the movement of the surgical tool unit (240) is compensated, the subject (50) will remain in a stopped state; however, if there is additional compensation for the surgical tool unit (240) or additional movement caused by the doctor's operation, the monitored screen may change.

[0084] In addition, the over tube (200) and the surgical tool part (240) may not only be individually compensated for movement, but both may also be compensated for movement and operate in conjunction with each other. That is, through the coordinated operation of the over tube (200) and the surgical tool part (240), the distance between the object (50) and the surgical tool part (240), or the distance between the object (50) and the over tube (200), can be maintained within a predetermined range.

[0085] The motion compensation device (100) of the present invention may include at least one of a data processing unit (310), a navigation unit (350), a compensation calculation unit (400), and a motion estimation modeling unit (500).

[0086] Referring to FIGS. 6 and FIGS. 12, an image captured by the imaging unit (220) is represented in the relative coordinate system of the imaging unit (220). For example, the subject (50) may repeatedly approach or move away from the imaging unit (220) according to a breathing cycle consisting of exhalation and inhalation.

[0087] Figure 12 is an example of breathing compensation as an embodiment of movement compensation.

[0088] Referring to FIG. 12, during inhalation (B1), the subject (50) may approach the imaging unit (220), and during exhalation (B2), the subject (50) may move away from the imaging unit (220). The images or video of the imaging unit (220) collected over time may exhibit a periodic pattern according to the breathing cycle. The waveform pattern for the pressure or volume of breathing may include at least one of an exhalation section or exhalation peak (B31, B32), an inhalation section or inhalation peak (B33, B34), and a functional residual volume section or peak (B35, B36), which is the amount of exhaust that is not completely expelled. The compensation calculation unit (400) may correspond the characteristics of such breathing waveform patterns from the images or video of the imaging unit (220) and calculate breathing compensation for the subject (50) due to breathing, including the breathing cycle, breathing direction (movement direction), etc.

[0089] The compensation calculation unit (400) of the present invention can calculate movement information including the distance between the surgical tool unit (240) and the subject (50), or the movement time of the subject (50), including the breathing cycle, by using image data such as a two-dimensional type obtained from a shooting unit (220) provided at the end of the over tube (200) according to a predetermined time interval.

[0090] Referring to FIG. 6, the compensation calculation unit (400) can set a target part (TP) or a feature part (FP) on the captured image of the shooting unit (220). The target part (TP) may be a part that serves as a reference on the screen, such as a target object (50), and the feature part (FP) may measure or express the change in position of the target part (TP) on the captured image over time.

[0091] The compensation calculation unit (400) can calculate the movement of the target object (50) by tracking the target part (TP) or feature part (FP) from the image captured according to a predetermined time interval (e.g., 100ms).

[0092] According to FIG. 11, image data captured by the capturing unit (220) can be transmitted to the data processing unit (310) or the motion estimation modeling unit (500).

[0093] The data processing unit (310) may be provided separately from the motion estimation modeling unit (500) or may be included in the motion estimation modeling unit (500), and may perform the same function as when provided separately.

[0094] The data processing unit (310) can transmit data received from the motion estimation modeling unit (500) or data with a converted data format.

[0095] The motion estimation modeling unit (500) can perform motion compensation in specific scenarios or situations using data from a motion estimation model even when there is no input of image data from the shooting unit (220) or external data from the measurement unit (600) of an external medical device. Additionally, the motion estimation modeling unit (500) can perform motion compensation using data that has been pre-entered.

[0096] The compensation calculation unit (400) may selectively receive data from the data processing unit (310) or the motion estimation modeling unit (500), or receive data from both.

[0097] Image data captured by the capturing unit (220) may include first data (D1) used to compensate for movement of the subject (50), such as breathing compensation, or second data (D2) that is continuously captured in real time by the capturing unit (220).

[0098] The first data (D1) may include movement information of the object (50) required for the movement compensation calculation of the compensation calculation unit (400).

[0099] The first data (D1) may include a movement cycle including a breathing cycle used by the compensation calculation unit (400) to calculate (S310) the movement compensation of the over tube (200) or surgical tool unit (240), or information such as the target unit (TP) and feature unit (FP).

[0100] The second data (D2) may be image data that is continuously captured in real time by the imaging unit (220). The second data (D2) may conceptually overlap with at least some of the first data (D1) and may be used as real-time image data collected by the imaging unit (220) to monitor the subsequent surgical process after the calculation of motion information including a motion cycle.

[0101] The second data (D2) can be used alone to calculate the motion compensation.

[0102] The compensation calculation unit (400) can obtain the second data (D2) captured in real time by the shooting unit (220).

[0103] The compensation calculation unit (400) or control unit (370) can operate the over tube (200) or surgical tool unit (240) through the driving unit (150) to compensate for the movement of the subject (50) in real time by calculating the image captured at a predetermined time interval by the shooting unit (220) using the second data (D2) through an image processing technique or machine learning model including depth information or optical flow.

[0104] The compensation calculation unit (400) or the control unit (370) can calculate the direction in which the amount of optical flow is minimized using the second data (D2) observed in real time from the shooting unit (220), and can adjust each joint of the over tube (200), etc. in real time in the calculated direction.

[0105] When the breathing cycle is calculated using the first data (D1), the distance between the surgical tool part (240) and the subject (50) according to the breathing cycle is calculated, and the translational movement of the over tube (200) or the surgical tool part (240) is compensated (S331), and then the movement compensation is performed in real time using the second data (D2) alone without the need for a series of additional measures to compensate for the remaining movement (S333), the over tube (200) or the surgical tool part (240) can be controlled to compensate for the movement of the subject (50) in real time.

[0106] The data processing unit (310) can receive third data (D3), which is external data received from an external measurement unit (600) rather than a motion compensation device (100) including a shooting unit (220).

[0107] Figure 14 (a) may be a ventilation / anesthesia workstation, and Figure 14 (b) may include patient information such as respiratory pressure or respiratory volume that can be obtained from the ventilation / anesthesia workstation.

[0108] Referring to FIGS. 14 and 15, the third data (D3) may include data (D3) transmitted from the measurement unit (600) of an external medical device, such as a respiration / anesthesia workstation or a C-arm (610).

[0109] The third data (D3) may include data received from a measurement unit (600) of an external medical device, rather than a shooting unit (220).

[0110] The third data (D3) transmitted from the measurement unit (600) of the artificial respiration / anesthesia workstation may include information on at least one of the following: respiratory volume per minute, inspiratory / expiratory ratio, number of breaths per minute, exhalation / inspiratory start time, inspiratory / expiratory transition time, and functional residual volume interval time.

[0111] Referring to FIG. 14(b), one embodiment of the third data (D3) obtained from the measuring unit (600) may be pressure control or volume control per minute. The third data (D3) may include information on the breathing cycle calculated from information on the peaks (1230, 1240) for pressure or volume measured from the measuring unit (600). The compensation calculation unit (400) may use the third data (D3), which includes the breathing cycle transmitted from the measuring unit (600) of an external medical device, to calculate the movement compensation of the subject (50) caused by breathing. The compensation calculation unit (400) may use the third data (D3) as an initial value or reference value for movement compensation, and may link the image data (D1 or D2) collected from the imaging unit (220) with the third data (D3) for the movement compensation calculation.

[0112] The third data (D3) transmitted from the measurement unit (600) of the C-arm (610) may include the forward / backward time, respiration rate per minute, cycle information, or amplitude (forward / backward movement amount) of the subject (50). The third data (D3) of the C-arm may include the shape or location of the kidney (21) or the kidney stone (50), as shown in FIG. 15. The third data (D3) may be used in conjunction with the map formation information of the navigation unit (350).

[0113] The third data (D3) transmitted from the measurement unit (600) of the C-arm may include the degree of two-dimensional movement in a plane of the organ or subject (50). By using the third data (D3), relative movement can be estimated based on the thickness of the over tube (200), the amount of movement loss of the over tube (200) due to the mucous membrane in contact can be calculated, and the remaining time in the surgical process (e.g., the remaining time in the surgical process based on the amount of remaining stones (50)) can be estimated.

[0114] The third data (D3) may partially overlap with the first data (D1) or second data (D2) of the imaging unit (220), or may be estimated from the first data (D1) or second data (D2).

[0115] The third data (D3) can be provided as an initial value for calculating the breathing cycle or modeling the motion estimation. In this way, if the third data (D3) is used together with the first data (D1) or the second data (D2), a more accurate motion compensation value can be calculated.

[0116] The data processing unit (310) can receive the fourth data (D4), which is feedback data received from at least one of the compensation calculation unit (400), the navigation unit (350), and the orientation unit (330).

[0117] The fourth data (D4) may include feedback data for at least one of the difference between the actual and the adjustment of the viewing direction of the tip of the over tube (200) or the imaging unit (220) by the orientation unit (330), the difference between the actual and the movement compensation of the over tube (200) or the surgical tool unit (240) by the compensation calculation unit (400), and the difference between the actual and the movement compensation for each position within the organ of the subject (50) by the navigation unit (350).

[0118] The compensation calculation unit (400) can calculate the compensation using at least one of the first data (D1) to the fourth data (D4), or calculate the motion compensation again by correcting the difference between the value estimated from the motion estimation modeling unit (500) and the actual value.

[0119] Even if there is no input data such as image data or there is insufficient data, the compensation calculation unit (400) can receive information from the motion estimation modeling unit (500) and calculate the motion compensation.

[0120] The motion estimation modeling unit (500) can update the parameters used in the implementation of the motion estimation model using the fourth data (D4).

[0121] The fourth data (D4) may include mechanical feedback information such as motor or encoder information connected to the drive unit (150), information indicating the current state of the flexible endoscope unit (170), and hysteresis or backlash information of the wire tension of the imaging unit (220) or surgical tool unit (240).

[0122] The data processing unit (310) can convert or unify file formats so that file formats of different formats can be utilized integrally in the compensation calculation unit (400).

[0123] The compensation calculation unit (400) can calculate movement compensation including breathing compensation using data collected or stored in the data processing unit (310) (S310).

[0124] The compensation calculation unit (400) may use at least one of the first data (D1) to the fourth data (D4) to compensate for the movement of the object to be compensated (50).

[0125] The compensation calculation unit (400) can calculate movement information including breathing cycles, etc. (S311) to calculate movement compensation such as breathing compensation (S310).

[0126] Referring to FIG. 13, when the tip of the over tube (200) arrives at the target location (S130), the location is fixed and data collection by the imaging unit (220) can begin (S200).

[0127] The camera unit (220) can take a primary image (initial image) while stationary near the object (50), and can obtain image information (e.g., a differential image) that is highly similar to the primary image from images taken thereafter based on the primary image.

[0128] The acquisition of image information (difference image) with high similarity corresponds to finding the sampling period at which the most similar image appears among the repeated movements of the object (50) observed by the shooting unit (220), or to finding the index at which the histogram of the difference image is minimized.

[0129] The compensation calculation unit (400) can set the difference between the time of the image with the smallest sum of the acquired difference images and the first captured image as the standard for breathing, and then normalize the measured value within a predetermined range through repeated sampling.

[0130] The navigation unit (350) can generate a map of a specific area or within an organ obtained from the imaging unit (220) or the third data (D3). Even within the same organ, depending on the location and the direction in which the over tube (200) approaches, the three-dimensional movement obtained through the imaging unit (220) will differ despite the same movement caused by breathing, etc. of the human body, and accordingly, the movement compensation information requiring compensation may also differ.

[0131] The navigation unit (350) can be interconnected with the motion estimation modeling unit (500) and can calculate or estimate motion compensation values ​​at any location or in any situation within the area where the map is completed.

[0132] The motion estimation modeling unit (500) can model the biological motion including kidneys (21), stones (50), etc.

[0133] Movement compensation, such as breathing compensation, is not required to be utilized only in specific locations but must be universally applicable across all situations affected by breathing, and such situations may include movement and precise observation of the affected area. The movement estimation modeling unit (500) can provide the unique movement of the subject (50) using a movement estimation model. To this end, the movement estimation modeling unit (500) may learn from input data or use a pre-entered table or setting value.

[0134] The motion estimation model unit (500) may use at least one of the first data (D1) and second data (D2) obtained from the imaging unit (220), the third data (D3) which is patient information obtainable from an external device such as a ventilator / anesthesia workstation or C-arm, and the fourth data (D4) which is feedback data as input data.

[0135] The motion estimation model unit (500) can calculate at least one of the following based on the input data: information required for motion compensation including three-dimensional position or velocity, a motion reference breathing motion value within a breathing cycle, information on the current movement state of the subject (50), the degree of motion compensation performed, motion information relative to the control input including hysteresis or backlash information based on image data of the shooting unit (220), and the shape of the over tube (200) or the leading edge position information within the subject (50).

[0136] Additionally, the motion estimation model can be utilized in cases where it is difficult to accurately determine the movement of the object (50) within the field of view of the camera unit (220).

[0137] If the first data (D1) or the second data (D2) can be obtained through the imaging unit (220), the value predicted by the motion estimation model can be compared with the existing first data (D1) or second data (D2). If the actual value observed through the imaging unit (220) and the value estimated by the motion estimation model show too large a difference from each other, or if a difference outside a predetermined range is maintained for more than a certain period of time, the parameters implementing the motion estimation model can be updated.

[0138] If there is no video information input through the shooting unit (220) or if the information is unusable, the motion estimation model unit (500) can use the value of the motion estimation model as a compensation value for motion compensation.

[0139] Here, unusable information may occur when the motion information of the subject (50) cannot be confirmed in the shooting unit (220), when feedback on the motion compensation result cannot be fully received, or when it is affected by dangerous situations such as mucosal collision, input of error data, or abnormal situations such as disturbance.

[0140] For example, if the field of view of the imaging unit (220) becomes blurred due to the crushing of the stone (50), or if it is difficult to judge the movement of the subject (50) or the surrounding situation, the compensation calculation unit (400) can receive data from the motion estimation model unit (500) and compensate for the movement as before or according to the prediction of the motion estimation model unit (500) until the field of view becomes clear again.

[0141] When the motion compensation calculation by the compensation calculation unit (400) is completed (S310), the control unit (370) can issue a command to the driving unit (150) to perform synchronization (S330). Synchronization (S330) can first be performed by translational motion compensation by the translational driving unit (151) (S331), and secondarily, if motion compensation is required, compensation for the remaining degrees of freedom of motion excluding translational motion or additional motion compensation due to mechanical factors can be performed (S333).

[0142] The features, structures, effects, etc. described in the embodiments above are included in at least one embodiment of the present invention and are not necessarily limited to only one embodiment. Furthermore, the features, structures, effects, etc. of each embodiment may be combined or modified and implemented in other embodiments by a person skilled in the art to which the embodiments belong. Therefore, details regarding such combinations and modifications should be interpreted as being included within the scope of the present invention.

[0143] Furthermore, although the above description has focused on exemplary embodiments, this is merely illustrative and does not limit the invention. Those skilled in the art will understand that various modifications and applications not exemplified above are possible within the scope of the essential characteristics of the embodiments. For example, each component specifically shown in the embodiments may be modified. Differences arising from such modifications and applications should be interpreted as being included within the scope of the invention as defined in the appended claims. Explanation of the symbols

[0144] 10... Patient 21... Kidney 23... ureter 25... bladder 27... urethra 50... subject to 100... Motion compensation device 110... Monitoring unit 130... Control unit 140... Robot arm 150... Drive unit 151... Translational drive unit 153... Bending drive unit 155... Rotation drive unit 170... Flexible endoscope section 172... Access system 180... handle section 200... over tube 220... Camera Department 240... Surgical Instruments Department 310... Data Processing Department 330... Orientation Department 350... Navigation unit 370... Control unit 400... Compensation Calculation Unit 500... Motion Estimation Modeling Unit 600... Measurement unit 610... C-arm S100... Preparation phase S200... Data collection phase S300... Compensation stage S400... Surgery stage D1... 1st data D2... 2nd data D3... 3rd data D4... 4th data MS... Master Station SS... Slave Station

Claims

Claim 1 A movement compensation device comprising: an overtube inserted into a human body and reaching a subject to surgery; and a compensation calculation unit that calculates movement compensation corresponding to movement of the subject within the human body caused by respiration according to human respiration, wherein the overtube includes a surgical tool unit that applies treatment to the subject, and the overtube and the surgical tool unit each perform translational, bending, or rotational movements, and the movement compensation is performed for at least one translational movement direction of the overtube and the surgical tool unit with respect to the subject, and maintains the distance between the subject and the surgical tool unit, or the distance between the subject and the overtube, within a predetermined range. Claim 2 In claim 1, the object of the motion compensation is the over tube, and the over tube is a motion compensation device that is synchronized in response to the movement of the object to be compensated. Claim 3 In claim 1, the object of the motion compensation is the surgical tool part, and the surgical tool part is a motion compensation device that is synchronized in response to the movement of the subject. Claim 4 A motion compensation device according to claim 1, wherein the objects of the motion compensation are the over tube and the surgical tool part, and the over tube and the surgical tool part operate in conjunction with each other in response to the movement of the object to be treated, and the distance between the object to be treated and the surgical tool part, or the distance between the object to be treated and the over tube, is maintained within a predetermined range by the coordinated operation of the over tube and the surgical tool part. Claim 5 In claim 1, the over tube includes a shooting unit that acquires image data of a subject, and a motion compensation device in which the distance between the shooting unit and the subject is maintained within a predetermined range in response to the movement of the subject. Claim 6 delete Claim 7 In claim 1, the motion compensation device further performs the motion compensation through bending or rotational movement of at least one of the over tube and the surgical tool part. Claim 8 In claim 1, the over tube includes a shooting unit that photographs a subject, and the compensation calculation unit calculates the movement of the subject using continuous image data from the shooting unit. Claim 9 In claim 1, the over tube includes a shooting unit for photographing the subject, and the subject repeatedly approaches or moves away from the shooting unit according to a movement cycle including a breathing cycle, and the compensation calculation unit calculates the distance, movement cycle, or movement direction between the surgical tool unit and the subject using image data according to the movement cycle of the subject. Claim 10 In claim 1, the over tube includes a shooting unit for photographing the subject and a motion estimation modeling unit for modeling the movement of the subject including biological movement, and even when there is no input of image data through the shooting unit or data transmission from a measurement unit of an external device, the compensation calculation unit receives data from the motion estimation modeling unit and calculates the motion compensation. Claim 11 In claim 1, the over tube includes a shooting unit for photographing a subject, the first data is data used to calculate motion compensation corresponding to the movement of the subject, the second data is data captured in real time by the shooting unit, the third data is external data transmitted from a measurement unit of an external device other than the shooting unit, and the compensation calculation unit calculates motion compensation using at least one of the first data, the second data, and the third data. Claim 12 In claim 1, the over tube includes a shooting unit for photographing a subject, the compensation calculation unit obtains second data photographed in real time by the shooting unit, and at least one of the over tube and the surgical tool unit is operated to compensate for the movement of the subject in real time by the movement compensation of the compensation calculation unit using the second data. Claim 13 In claim 1, the over tube includes a shooting unit for photographing a subject, and the compensation calculation unit obtains third data related to patient information transmitted from a measurement unit of an external device other than the shooting unit, and the third data is provided for calculating the movement cycle or movement direction of the subject including a breathing cycle, and the compensation calculation unit calculates a movement compensation by linking the image data of the shooting unit with the third data. Claim 14 A motion compensation device according to claim 1, wherein the over tube includes a shooting unit for acquiring image data of a subject and a monitoring unit for displaying the image data acquired from the shooting unit externally, and whether the subject is stopped or moving on the monitoring unit is determined by whether the object of motion compensation for the movement of the subject is at least one of the over tube, the surgical tool unit, and the shooting unit, or by the arrangement structure between the over tube, the surgical tool unit, and the shooting unit. Claim 15 A motion compensation device according to claim 1, wherein the over tube includes a shooting unit for photographing a subject, and when the viewing direction of the initial position of the shooting unit deviates from the direction of movement of the subject, at least one of the over tube and the shooting unit is moved to a modified position, and the viewing direction of the shooting unit at the modified position is set to coincide with or parallel to the direction of movement of the subject. Claim 16 In claim 1, the over tube includes a shooting unit for photographing a subject, and when the over tube arrives at the subject, the position of the shooting unit is fixed, and the shooting unit collects image data of the subject at the fixed position, and based on the image data of the shooting unit, at least one of the over tube and the surgical tool unit operates in synchronization with the movement of the subject. Claim 17 A motion compensation device according to claim 1, wherein the over tube includes a shooting unit that photographs a subject, and the compensation calculation unit sets a target part or a feature part on the image captured by the shooting unit, the target part is a reference part on the screen that includes the subject, the feature part indicates a change in the position of the target part on the captured image, and the compensation calculation unit calculates the movement of the subject by tracking the target part or the feature part from the image of the shooting unit captured according to a predetermined time interval. Claim 18 A motion compensation device according to claim 1, wherein the over tube includes a shooting unit for photographing a subject, the compensation calculation unit photographs an initial image of the subject and obtains image information with high similarity to the initial image from an image taken after the initial image, and the compensation calculation unit sets the difference between the time of the image with the smallest sum of the obtained image information with high similarity and the initial image as the basis for the breathing cycle. Claim 19 In claim 1, the over tube includes a shooting unit for photographing the subject and a motion estimation modeling unit for modeling the movement of the subject including biological movement, the motion estimation modeling unit learns the inherent movement of the subject, and the motion estimation modeling unit updates parameters from the difference obtained by comparing the value estimated by modeling with the actual value observed by the shooting unit, and re-estimates the motion compensation device based on the updated parameters.

Images