Surgical robot system for compensating for hysteresis of medical instrument, and control method

Abstract

The present invention relates to a surgical robot system comprising: a surgical instrument device including a manipulator on which one or more motors and a medical instrument are mounted to deflect a tip of the medical instrument; and a processor for controlling the manipulator by driving the motors, wherein the processor outputs a motor driving signal for controlling that the tip of the medical instrument is deflected in a predetermined section of a bending range in order to estimate intrinsic hysteresis of the medical instrument modified by a use state of the medical instrument, and thus calibration for performing an operation in which the tip of the medical instrument is deflected at least once within the bending range before the control compensating for the hysteresis of the medical instrument is executed.

Description

Surgical robot system and control method for compensating for hysteresis of medical devices

[0001] The present invention relates to a surgical robot system for remotely controlling a steerable medical device, and more specifically, to a surgical robot system and control method capable of compensating for hysteresis of the medical device.

[0002] Minimally invasive surgery is a surgical method in which surgical instruments are inserted and procedures are performed by creating a small incision inside the patient's body. This surgical method offers numerous advantages over open surgery, including faster patient recovery, reduced risk of infection, and superior cosmetic results. With the continuous advancement of medical technology, minimally invasive surgery is evolving into robot-assisted surgical systems capable of remote operation. Reference may be made to the applicant's prior application No. 10-2024-0170032 as an example of a surgical robot system for minimally invasive surgery.

[0003] A surgical robot system for remotely controlling medical instruments can be utilized not only for minimally invasive surgery but also for invading through natural openings of the human body, steering to target tissue, and performing intended surgery. An example of such a surgical robot system may be referenced in the applicant's prior art, Korean Published Patent No. 10-2024-0009905.

[0004] Medical instrument control technology applied to remote surgical robot systems requires precise control in environments where medical instruments are invasive within the human body. In such systems, medical instruments in the form of long, flexible tubes are primarily used, with endoscopes or various surgical instruments attached to the tips to perform invasive operations. To move the tip of the medical instrument to a target position, control units (e.g., pull wires, end effectors, etc.) must be controlled remotely. However, in this case, the actual position of the tip often differs from the target position due to complex factors such as the material properties, internal friction, and structural flexibility of the medical instrument. Furthermore, hysteresis—a physical discrepancy occurring between the starting and ending points during repetitive movements—acts as a major technical obstacle. Hysteresis refers to the discrepancy that occurs between the angle value input by the control unit and the actual position of the medical instrument tip that must move in response. In other words, the tip does not respond accurately according to the angle commanded by the user, and this positional difference gradually accumulates and expands during the repetitive operation process.

[0005] Against this backdrop, various automatic calibration systems have been developed for hysteresis compensation and precise control of medical devices. As related prior art, U.S. Patent 2016 / 0374541 discloses an automatic calibration system that quantitatively models the nonlinear motion of a device, such as hysteresis, based on data from an internal imaging sensor and an external sensor.

[0006] However, conventional medical device hysteresis compensation systems have limitations in accurately reflecting hysteresis behavior that varies depending on the actual usage environment and conditions of the device. Even for identical medical devices, hysteresis characteristics can differ due to subtle differences in the manufacturing process or usage history, and the behavior can vary depending on the frequency of use and changes in the usage environment of individual devices. Furthermore, since the addition of sensors is restricted due to the design of fine and lightweight medical devices, new compensation technologies capable of estimating the device's inherent hysteresis without the need for sensor attachment are required. Accordingly, technologies that rapidly and accurately track and compensate for hysteresis behavior caused by the flexibility, structural complexity, and characteristic changes resulting from repeated use of medical devices are being continuously researched to ensure the precision control and safety of remote surgical robot systems.

[0007] [Patent Literature]

[0008] (Patent Document 1) U.S. Published Patent US 2016 / 0374541

[0009] The objective of the present invention is to systematically compensate for positional errors of medical devices caused by hysteresis by utilizing artificial intelligence software.

[0010] In addition, the present invention aims to provide a method for modeling medical instruments that change during surgery or the unique hysteresis behavior of each medical instrument in real time. Accordingly, the present invention aims to support the immediate estimation and application of hysteresis characteristics, which may vary depending on usage conditions even for the same medical instrument, without the need for separate sensors.

[0011] The present invention relates to a surgical robot system comprising: a surgical instrument device including a manipulator configured to deflect the tip of the medical instrument, the manipulator being equipped with one or more motors and a medical instrument; and a processor that drives the motor to control the manipulator, wherein the processor outputs a motor driving signal to control the tip of the medical instrument to deflect within a predetermined section of a bending range in order to estimate the inherent hysteresis of the medical instrument deformed by the usage state of the medical instrument, and thereby performs a calibration in which the tip of the medical instrument performs a deflection operation at least once within the bending range before control to compensate for the hysteresis of the medical instrument is executed.

[0012] In one embodiment, the processor controls the tip of the medical device to bend at least once additionally in a range of deflection angles different from the deflection angle bent in the predetermined section when the calibration is executed, and the motor driving signal may be a signal that controls the tip of the medical device to bend at least twice within the bending range before the control that compensates for hysteresis of the medical device is executed.

[0013] In one embodiment, the motor driving signal is a waveform extending from a first period to the nth period (n is a natural number) in which a motor command value for bending the tip of the medical device at a specific angle is specified, and the tip of the medical device can be bent within the range of a first section in one period of the waveform.

[0014] In one embodiment, the motor driving signal may be specified such that the waveform of each cycle is a standing wave, and the motor command value may be specified such that the range of bending the tip of the medical device in each cycle is different.

[0015] In one embodiment, the motor driving signal may be a damping signal in which, when bending the tip of the medical device into the range of the first section in the first cycle, the first section is the predetermined section, and when bending the tip of the medical device into the range of the second section in the second cycle, the second section is set to a smaller range than the first section.

[0016] In one embodiment, the motor driving signal may be a signal that, when bending the tip of the medical device into the range of the first section in the first cycle, the first section is the minimum bending range in which the tip can bend in the up and down direction from the correct position, and when bending the tip of the medical device into the range of the second section in the second cycle, the second section is a range larger than the first section and is set to a range less than or equal to the bending range.

[0017] In one embodiment, the processor can estimate the unique hysteresis of the medical device in a non-measurement manner by executing an angle estimation algorithm that estimates the angle of bending of the tip of the medical device through the execution of the calibration.

[0018] In one embodiment, the angle estimation algorithm is a deep learning-based time series analysis model that forms the torque data of the motor or the motor driving signal into residual blocks, has a network structure in which the residual blocks are sequentially stacked, and processes the residual blocks in parallel to learn the angle at which the tip of the medical device is bent according to the motor driving signal.

[0019] In one embodiment, the number of residual blocks in the angle estimation algorithm may be set based on the number of angle data collected over the total cycle of the motor driving signal.

[0020] In one embodiment, the processor may execute a hysteresis modeling algorithm that mathematically simulates non-linear hysteresis characteristics after the execution of the calibration.

[0021] In one embodiment, the hysteresis modeling algorithm models the hysteresis behavior by summing one or more Generalized Play Operator (GPO) operators with weights assigned to each, and calculates a plurality of parameters (P) constituting the GPO operator based on the bending angle of the medical device tip output as the result of the calibration execution, thereby modeling the unique hysteresis of the medical device.

[0022] In one embodiment, the surgical robot system according to the present invention is characterized in that the first hysteresis modeled as the calibration and the hysteresis modeling algorithm are executed in the first medical device mounted on the manipulator, and the second hysteresis modeled as the calibration and the hysteresis modeling algorithm are executed for the second medical device when the first medical device is replaced with a second medical device on the manipulator, are different from each other.

[0023] In one embodiment, the surgical robot system according to the present invention has a first hysteresis modeled as the calibration and hysteresis modeling algorithm are executed in a first state of the medical device mounted on the manipulator, and a second hysteresis modeled as the calibration and hysteresis modeling algorithm are executed in a second state of the medical device, and the first hysteresis is different from each other, and the state at this time can be defined as the time during which the medical device is operated.

[0024] In one embodiment, the processor can execute a feedforward control algorithm that compensates for a control signal of the medical device based on hysteresis behavior modeled by the hysteresis modeling algorithm in a medical environment where the manipulator inserts the medical device into the body for a procedure.

[0025] In one embodiment, the feedforward control algorithm can compensate the control signal of the medical device based on an inverse model of the hysteresis behavior modeled by the hysteresis modeling algorithm.

[0026] In one embodiment, the manipulator includes a mounting portion to which the medical device is attached and detached, and the mounting portion includes a lever holder that grips an operating lever of the medical device, and the motor can control the lever holder by means of the motor driving signal to bend the tip of the medical device.

[0027] In one embodiment, the motor may be a frameless motor.

[0028] In addition, the present invention is executed in a processor that controls a surgical robot system and, in a control method for compensating for hysteresis of a medical device mounted on a manipulator of a surgical robot system, the method comprises a step of outputting a motor driving signal to control a motor so that the tip of the medical device deflects within a predetermined section of a bending range, and performing a movement in which the tip of the medical device deflects at least once within the bending range before the control for compensating for hysteresis of the medical device is executed, and a calibration step for obtaining the angle at which the motor driven by the motor driving signal deflects the tip of the medical device, and the method can compensate for the hysteresis of the medical device by modeling the unique hysteresis behavior of the medical device based on the information of the medical device obtained in the calibration step.

[0029] In addition, the present invention relates to a processor encoded with a control command for compensating for hysteresis of a medical device mounted on a manipulator in a surgical robot system, wherein the processor outputs a command for a motor driving signal that controls a motor so that the tip of the medical device bends within a predetermined section of a bending range, and the motor driving signal is a waveform extending from a first period to the nth period (where n is a natural number) in which a command value of the motor is specified to bend the tip of the medical device at a specific angle, and the processor can generate a feedforward control command for compensating for hysteresis of the medical device by estimating the bending angle of the tip of the medical device according to the motor driving signal.

[0030] According to the present invention, hysteresis of a medical device can be systematically implemented using software with an artificial intelligence algorithm. The control method according to the present invention can be installed in a surgical robot system in the form of a solution and does not require design changes, such as separate sensor equipment, to existing medical devices, thereby allowing for universal application to medical devices and offering the advantage of high system compatibility. Furthermore, by utilizing calibration drive that can be applied immediately on-site, the present invention enables real-time reflection of changes in the usage environment or the inherent hysteresis behavior of the medical device, even during surgery or when replacing the medical device.

[0031] Figure 1 is an example of a surgical robot system.

[0032] Figure 2 is an example of a surgical instrument device of a surgical robot system.

[0033] FIG. 3 illustrates a manipulator according to one embodiment of a surgical robot system.

[0034] FIG. 4 is a flowchart of a control method for compensating hysteresis performed in a processor according to an embodiment of the present invention.

[0035] Figure 5 shows the operation of a medical device according to an embodiment of the present invention.

[0036] Figure 6 shows the appearance of a tip of a medical device being bent as a motor driving signal is input while a calibration step according to an embodiment of the present invention is being performed.

[0037] Figure 7 (a) is a graph showing the inherent hysteresis of a medical device, and Figure 7 (b) is a graph showing the motor torque that varies according to the hysteresis of the medical device.

[0038] FIG. 8 shows the architecture of an angle estimation algorithm executed in the angle estimation step according to an embodiment of the present invention.

[0039] FIG. 9 is a schematic diagram of the residual block of an angle estimation algorithm according to an embodiment of the present invention.

[0040] FIG. 10 is an example evaluating the accuracy of the tip angle estimated by executing the angle estimation algorithm according to FIG. 8.

[0041] FIG. 11 is another embodiment evaluating the accuracy of the tip angle estimated by executing the angle estimation algorithm according to FIG. 8.

[0042] FIG. 12 is another example evaluating the accuracy of the tip angle estimated by executing the angle estimation algorithm according to FIG. 8.

[0043] FIG. 13 is an overview of the operation of a hysteresis modeling algorithm executed in the hysteresis modeling step according to an embodiment of the present invention.

[0044] FIG. 14 is a schematic diagram of the results of hysteresis modeling according to an embodiment of the present invention and the operation of a feedforward control algorithm executed in the hysteresis compensation step.

[0045] FIG. 15 shows an experimental environment for testing a control method that compensates for hysteresis according to an embodiment of the present invention.

[0046] Figure 16 is a control process flowchart performed in the experimental environment according to Figure 15.

[0047] Figure 17 is an experimental example of angle estimation performed in the experimental environment according to Figure 15.

[0048] Figure 18 is an example of a hysteresis modeling experiment performed in the experimental environment according to Figure 15.

[0049] Figure 19 is an example of a hysteresis compensation experiment performed in the experimental environment according to Figure 15.

[0050] The various embodiments described in this document are illustrative for the purpose of clearly explaining the technical concept of the invention and disclosure, and are not intended to limit them to specific embodiments. The technical concept of the invention and disclosure includes various modifications, equivalents, alternatives, and embodiments selectively combined from all or part of each embodiment described in this document. Furthermore, the scope of rights of the technical concept of the invention and disclosure is not limited to the embodiments presented below or the specific descriptions thereof.

[0051] Terms used in this document, including technical or scientific terms, may have the meaning generally understood by those skilled in the art to which the invention and disclosure pertain.

[0052] Expressions used in this document, such as "includes," "may include," "is equipped," "may be equipped," "has," and "may have," imply that functions, operations, or components exist as the subject feature and do not exclude the existence of other additional features. In other words, such expressions should be understood as open-ended terms implying the possibility of including other embodiments.

[0053] Singular expressions used in this document may include the meaning of the plural form unless the context otherwise indicates, and this applies likewise to singular expressions described in the claims.

[0054] Expressions used in this document such as “A, B, and C,” “A, B, or C,” “A, B, and / or C,” or “at least one of A, B, and C,” “at least one of A, B, or C,” “at least one of A, B, and / or C,” “at least one selected from A, B, and C,” “at least one selected from A, B, or C,” “at least one selected from A, B, and / or C,” etc., may mean each of the listed items or all possible combinations of the listed items. For example, “at least one selected from A and B” may refer to (1) A, (2) at least one of A, (3) B, (4) at least one of B, (5) at least one of A and at least one of B, (6) at least one of A and B, (7) at least one of B and A, and (8) all of A and B.

[0055] The expression "based on" as used in this document is used to describe one or more factors affecting an act or action of a decision or judgment described in the phrase or sentence containing such expression, and this expression does not exclude additional factors affecting said act or action of a decision or judgment.

[0056] As used in this document, the expression that a certain component (e.g., a first component) is "connected" or "connected" to another component (e.g., a second component) may mean not only that the said certain component is directly connected or connected to the said other component, but also that it is connected or connected through a new other component (e.g., a third component).

[0057] The expression "configured to" as used in this document may have meanings such as "set to," "capable of," "modified to," "made to," or "able to," depending on the context, and is distinguished from the meaning of "consist."

[0058] Various embodiments of the present disclosure will be described below with reference to the accompanying drawings. In the accompanying drawings and the description thereof, identical or substantially equivalent components may be given the same reference numerals. Furthermore, in the description of the various embodiments below, the description of identical or corresponding components may be omitted, but this does not mean that such components are not included in the embodiments.

[0059] Surgical systems, such as those used in minimally invasive medical procedures, may include large and complex equipment to precisely control and operate relatively small tools or instruments.

[0060] A remotely controlled surgical robot system having a single entry port can be used for various surgeries to use various surgical instruments in a single system. Depending on the embodiment, the surgical robot system may be disclosed as a remote surgical system capable of operating various medical instruments through a single entry port. The surgical robot system may also be disclosed as a surgical robot system capable of operating a single medical instrument.

[0061] Depending on the characteristics of the shaft, the types of driven medical instruments may include overtubes, general surgical instruments, advanced surgical instruments, and camera instruments such as endoscopes. Medical instruments can be controlled manually, via computer-assisted control, or remotely via an operator console.

[0062] A surgical robot system according to an embodiment of the present invention enables surgery using a single entry port in various parts of the patient's body. Specifically, surgery can be performed by inserting a medical instrument through the patient's oral cavity, intercostal space, thigh, or other natural openings or incisions of the body. An overtube, which is an embodiment of a medical instrument, has a plurality of lumens formed therein, and an endoscope or a surgical instrument can be inserted through the lumens and inserted into the body together with the overtube. Control from the operator console can independently perform remote control of not only the overtube but also the surgical instrument inserted into the overtube.

[0063] The medical device referred to in this specification may collectively refer to an overtube and other surgical instruments inserted into the overtube, and regarding the steering and control of the overtube, the applicant’s prior Korean patents No. 10-2740479 and No. 10-2684546 may be referenced.

[0064] FIG. 1 is an embodiment of a surgical robot system (1). Referring to FIG. 1, the surgical robot system (1) may include a positioning cart (3), a surgical instrument device (10), and an operator console (5). In an embodiment, the positioning cart (3) may be a mobile platform on which the surgical instrument device (10) is mounted. The positioning cart (3) may perform the function of precisely positioning the surgical instrument device (10) at the surgical site of a patient. The positioning cart (3) may be equipped with wheels at the bottom for access to the operating table. The positioning cart (3) may include a frame composed of a plurality of arms and links. The frame may be configured to enable vertical movement of the surgical instrument device (10), movement in the direction of approach to the patient, and movement away from the patient. The frame of the positioning cart (3) may be configured in various forms considering a kinematic structure that enables access to the operating table and access to the surgical instrument device (10) at the surgical site of the patient. A medical instrument may be mounted on the surgical instrument device (10). The surgical instrument device (10) may be provided in a form mounted on a frame with controlled degrees of freedom, as in the embodiment of FIG. 1, or in a form mounted on a fixed frame, as in the embodiment of FIG. 2. The medical instrument mounted on the surgical instrument device (10) may include an overtube and a surgical instrument.

[0065] The operator console (5) may refer to medical control equipment capable of verifying surgery-related images and controlling the positioning cart (3) and the surgical instrument device (10). The operator console (5) is a remote control device that controls the entire system and can be installed inside the surgical space or in a separate space. The operator console (5) may include one or more actuators. One embodiment of the actuator may be an input device such as a handgrip, joystick, trackball, data glove, trigger gun, or manual controller. Another embodiment of the actuator may be a voice recognition device or a touchscreen. Yet another embodiment of the actuator may be a control device such as a clutch or a foot motion controller.

[0066] Although not shown in the drawings, depending on the embodiment, the surgical robot system (1) may further include a vision cart, which is an auxiliary imaging device. The vision cart can provide visual prompts and step-by-step surgical guidance through a touchpad or touchscreen monitor. Additionally, if necessary, medical imaging equipment for surgical assistance, such as CT and X-ray, may be linked to the system.

[0067] In an embodiment, the system configuration equipment including the positioning cart (3) and the operator console (5) may be equipped with one or more processors. The processors may process information input from various sensors. For example, the sensor information may include the drape state, the state of the positioning cart (3), the state of the surgical instrument device (10), and the state of the medical instrument.

[0068] In an embodiment, the processor may be mounted on the positioning cart (3), on the operator console (5), on the surgical instrument device (10), or on every piece of equipment. The processor (500) may be provided as a separate device, in which case it can process data via wired or wireless communication with each piece of equipment constituting the surgical robot system (1).

[0069] FIG. 2 is an example of a surgical instrument device (10) of a surgical robot system (1).

[0070] Referring to FIG. 2, the surgical instrument device (10) may include a medical instrument (100) and an actuator (200). The operator console (5) may include a display device (50), an input device (51), a control device (52), a memory (400), and a processor (500).

[0071] The medical device (100) may be implemented in an elongated shaft shape having a steering-capable joint structure, a flexible structure, or a bendable structure. The medical device (100) may be controlled by three or more degrees of freedom, including pitch, yaw, and rolling, through a wire-based drive mechanism, or by bending control that deflects in one axis. As an embodiment, the medical device (100) may be realized in a configuration of 3 degrees of freedom, 4 degrees of freedom, or 6 or more degrees of freedom. In a 3-degree-of-freedom configuration, 1 bending (pitch or deflection, up / down direction), rolling, and forward / backward movement are implemented. In a 4-degree-of-freedom configuration, 2 bending (pitch, up / down direction), yaw (left / right direction), rolling, and forward / backward movement are implemented. Among the degrees of freedom, the bending movement may be controlled through a wire mechanism.

[0072] In one embodiment, the medical device (100) is implemented in the form of an overtube to guide a surgical instrument into the body. An endoscope probe may be inserted into the overtube via a separate shaft, or an endoscope camera may be installed on the overtube itself to capture images of the steering path in real time. In this case, the control of the inserted surgical instrument may be performed independently of the overtube.

[0073] In another embodiment, the medical device (100) may be implemented in the form of a single shaft in which an endoscope and a surgical instrument are integrated. For example, as a surgical instrument for removing kidney stones, an integrated medical device may be utilized in which a basket or a shaft from which a laser is emitted and a camera is provided at the end thereof. FIG. 2 illustrates a surgical instrument for removing kidney stones as an example of the medical device (100) as one of these embodiments.

[0074] As another embodiment, the medical device (100) may be implemented in a combined form of a tissue collection function for a biopsy procedure and an image acquisition function. In this case, the medical device (100) may be equipped with a basket capable of collecting a fine tissue sample and may simultaneously include a mechanical mechanism and a small camera that provides real-time images of the surrounding tissue.

[0075] The image acquired through the camera provided in the medical device (100) can be configured as visual data in the form of an image or a recording image and can be transmitted to and stored in memory (400).

[0076] The actuator (200) is a core component for driving the medical device (100) and may be equipped with one or more motors and encoders. The motor mounted on the actuator (200) can provide precise driving force to steer the medical device (100) in various directions. Individual actuators (200) may be designed to implement a single movement in a specific axial direction, and multiple actuators (200) may be organically combined to form an integrated motor mechanism. Through the cooperative operation of such a motor mechanism, the medical device (100) can be precisely steered along complex anatomical paths. The actuator (200) may be provided on the manipulator (13) of the surgical device (10). In the following embodiments, a motor (1300, FIG. 3) is described as an example of the actuator (200).

[0077] In one embodiment, the actuator (200) may include a frameless motor, a high-precision servo motor, or an optical or magnetic encoder to provide real-time feedback on the position and orientation of the medical device (100). Such a feedback system may be an essential element for controlling the movement of the medical device (100) with micrometer-level accuracy. In another embodiment, the actuator (200) may be configured based on a stepper motor to enable fine steering of the medical device (100) through precise stepper movements. This configuration may be particularly suitable for manipulation around narrow lumens or sensitive tissues.

[0078] The operator console (5) may include a display device (50), an input device (51), an operating device (52), a processor (500), and a memory (400). The devices exemplified in the operator console (5) may refer to the structure of the device presented in the aforementioned Korean Patent Publication No. 10-2024-0009905 or Application No. 10-2024-0170032.

[0079] The display device (50) can display real-time visual data captured by an endoscope camera mounted on a medical device (100) in high resolution. In addition to visual data, the display device (50) can also display the user's operating environment and various control interfaces in an integrated manner. This display device (50) can support medical staff in intuitively monitoring and controlling the surgical process. In one embodiment, the display device (50) is implemented as a medical display supporting 4K or 8K resolution, so that even fine features of anatomical structures can be clearly displayed. In another embodiment, the display device (50) may be configured to include a touchscreen function so that medical staff can control the system by directly manipulating the screen. Additionally, the display device (50) may be configured as a single display or multiple monitors to simultaneously display various information such as biosignals, patient information, and system status along with the endoscopic image.

[0080] The input device (51) can be defined as a precision operating tool operated by a medical professional using their hands. The input device (51) may be provided in an ergonomic form such as a trackball, a multi-axis joystick, or a precision gimbal, thereby allowing the medical professional to intuitively and accurately control the medical device (100). In one embodiment, the input device (51) may be composed of multiple operating devices separated for the left hand and the right hand, enabling precise operation using both hands.

[0081] In another embodiment, the input device (51) may be implemented to include a haptic feedback function so that the user can feel the resistance of the tissue that the medical device (100) contacts with their hand. This haptic feedback function can help determine the physical characteristics of the tissue that are difficult to perceive with visual information alone. The input device (51) can serve as a key interface for accurately controlling the movement of the medical device (100) by precisely detecting the user's manual control command, converting it into a digital signal, and transmitting it to the processor (500).

[0082] The control device (52) can function as a specialized interface device that outputs a path playback request command to the processor (500). The control device (52) can be simply implemented as a touchscreen of the display device (50). In one embodiment, the control device (52) can be provided in the form of a foot pedal, clutch, etc., which is a means of control by the user through the foot. The control device (52) can be configured separately from the input device (51) to provide specialized operation functions.

[0083] The control device (52) may include an input unit that remains active in response to continuous physical operation by a user, and an output unit configured to continuously output commands to control an actuator (200) while the input unit is active.

[0084] In the embodiment of FIG. 2, the operating device (52) is implemented in the form of a foot pedal. Although not explicitly shown in the drawing, the input unit may be realized as an internal configuration of the foot pedal including a sensor and a signal processing circuit that detect pressure applied to the foot pedal and convert it into an electrical signal. That is, the input unit may be implemented as a signal processing circuit of the foot pedal. When the user steps on the foot pedal with a pressure greater than a certain amount, the input unit may generate an active signal.

[0085] In this embodiment, the configuration may be such that an active signal is generated when the user steps on the foot pedal, and a stop signal (or cessation of the active signal) is generated when the user takes their foot off the foot pedal. As another embodiment of the present invention, the operating device (52) may be implemented in the opposite manner, that is, a configuration in which a stop signal is generated when the user steps on the foot pedal and an active signal is maintained when the user takes their foot off the foot pedal, and such a modified embodiment may also be included within the technical scope of the present invention.

[0086] Likewise, an output unit not specified in the drawing may include a communication interface circuit that converts an active signal generated from an input unit into a standardized command signal that can be recognized by the processor (500) and transmits it. The output unit may be realized as an internal component of the foot pedal, just like the input unit. That is, the output unit may be implemented as a signal processing circuit of the foot pedal. In this embodiment, the signal activated through the foot pedal may be interpreted and processed by the processor (500) as a path playback request command. As an example, the operating device (52) may include a multi-stage pressure sensing function to provide a function that can adjust the speed or intensity of path driving according to the degree of pressure applied to the foot pedal.

[0087] The memory (400) is configured to store record information of the path in which the medical device (100) is steered within the body, and the record information may include visual data of the anatomical structure (20) in which the medical device (100) is steered and control data of the medical device (100).

[0088] The memory (400) stores recording information in the form of time-series data, each assigned a time index to the visual data and the control data, and the stored recording information may have a data structure in which visual data and control data having the same time index are matched with each other. This time-synchronized data structure can ensure an accurate correspondence between the anatomical image at a specific point in time and the control command at that moment.

[0089] The processor (500) may be implemented in the form of physical hardware such as a controller, a microprocessor, a CPU (Central Processing Unit), or an ASIC (Application Specific Integrated Circuit). The processor (500) may include memory, which is a non-transient computer-readable storage medium for storing logic circuits and software instructions, and these instructions are configured to perform specific functions of the processor (500) as described in this specification when executed by the processor.

[0090] In terms of implementation form, the memory (400) and the processor (500) may be implemented as separate semiconductor circuits that are physically separated. In another implementation form, the memory (400) and the processor (500) may be implemented together within a single integrated semiconductor circuit (SoC: System on Chip). The processor (500) may be implemented as a single processor having a single core or multi-core structure, or as a distributed processing architecture including a plurality of individual processors.

[0091] In this embodiment, the recorded information may collectively refer to all types of data acquired while the medical device (100) travels along the internal path (21) of the anatomical structure (20). The recorded information can be broadly divided into two main categories: visual data and control data.

[0092] As one example of visual data, high-resolution image data of anatomical structures (20) and internal pathways (21) captured by an endoscope camera mounted on a medical device (100) may be included. As another example, continuous recording image data of anatomical structures (20) and internal pathways (21) captured by an endoscope camera may be included, which may be captured at a frame rate of 30fps or higher to enable smooth visual reproduction. As yet another example, additional image data of anatomical structures (20) obtained from external medical imaging equipment such as ultrasound, X-ray, CT, MRI, etc. may be included.

[0093] As one example of control data, precise control signal information for each motor driving the actuator (200) may be included. This may include parameters such as the motor's rotational speed, direction, and acceleration. As another example, it may be detailed information such as the motor's actual position, rotation angle, and torque recorded in the motor's encoder. Additionally, the control data may include low-level command data such as current values, voltage values, and PWM (Pulse Width Modulation) signals that drive the motor, which may be essential elements for accurately reproducing the movement of the medical device (100). As yet another example, the angle at which the tip of the medical device (100) is bent may be used as the indicated value. The motor may output a motor power signal corresponding to the indicated angle value and may bend the tip of the medical device (100) by interacting with the motor's encoder (angle) information. The motor may receive current from the current controller and receive current corresponding to the indicated angle by receiving feedback on the angle information from the encoder. In one embodiment, the motor may receive current through a current sensing function in the motor driver.

[0094] In the embodiment of the surgical robot system (1) described above, the internal path (21) of the anatomical structure (20) requires precise drive control. However, depending on the usage state of the medical device (100), hysteresis occurs, and a driving error occurs in which the tip of the actual medical device (100) does not move that much even when the same amount of movement is applied as a control signal. Since such errors in fine motion control can cause serious tissue damage within the internal path (21), hysteresis compensation is required.

[0095] FIG. 3 illustrates a manipulator (13) according to one embodiment of a surgical robot system (1). In the embodiment of FIG. 3, the manipulator (13) includes a mounting part (130) to which a medical device (100) is attached and detached, and the mounting part (130) includes a lever holder (131) that grips an operating lever (110) of the medical device (100), and a motor (1300) can control the lever holder (131) by a motor driving signal to bend the tip of the medical device (100). In this embodiment, an embodiment of the medical device (100) to be attached and detached is described later through FIG. 4. The lever holder (131) may have a receiving groove or an opening into which the operating lever (110) of the medical device (100) is inserted. The lever holder (131) may have a fixing projection or a friction surface on its inner side so as to stably fix the operating lever (110) of the medical device (100), and the fixing force may be adjustable to correspond to operating levers (110) of various sizes.

[0096] The motor (1300) can control the lever holder (131) according to a driving signal provided by the control unit to slide the operating lever (110) of the medical device (100), and the tip of the medical device (100) can be bent in response. At this time, the motor (1300) may be a stepper motor, a servo motor, or a DC motor, and preferably a frameless motor. A frameless motor is a motor that is assembled integrally with a mounting structure by providing only a stator and a rotor without a separate housing or shaft. Such a frameless motor is advantageous for lightweighting and miniaturization, and can increase the design freedom of the drive unit. In addition, since the frameless motor is formed with a low gear ratio and enables direct drive, it can secure precise control characteristics and is advantageous for compensating for hysteresis that occurs during the torque transmission process.

[0097] In this embodiment, the manipulator (13) may include a driving mechanism that advances or retracts the mounting portion (130) to perform insertion or withdrawal of the medical device (100). The surgical device device (10) may include a guide rail on which the mounting portion (130) moves linearly and an electric transfer mechanism that drives it. The manipulator (13) may be designed to operate the medical device (100) in a remote control manner, and a control signal from the operator console (5) is transmitted to the motor (1300) of the manipulator (13), so that the operating lever (110) can be operated without the medical practitioner directly operating the medical device (100). Through this, the fatigue of the practitioner who would otherwise have to directly operate the operating lever (110) during a long procedure can be reduced, and the tip of the medical device (100) can be precisely controlled.

[0098] At this time, the medical device (100) mounted on the manipulator (13) can be compatible with a ready-made flexible ureteroscope (fURS) and can be attached and detached without separate modification. Therefore, the medical device (100) having various shapes of operating levers (110) can be easily compatible with the mounting part (130) and can be used universally in the same structure.

[0099] FIG. 4 is a flowchart of a control method for compensating hysteresis performed in a processor (500) according to an embodiment of the present invention. The processor (500) can control a manipulator (13) by driving one or more motors.

[0100] Referring to FIG. 4, a control method for compensating for hysteresis performed in a processor (500) may include (a) a calibration step (S10), (b) an angle estimation step (S20), (c) a hysteresis modeling step (S30), and (d) a hysteresis compensation step (S40).

[0101] The calibration step (S10) may be a step in which a motor driving signal (1310, FIG. 6) is output to control the motor so that the tip (101, FIG. 6) of the medical device (100) deflects over a predetermined section of the bending range, and the tip (101) of the medical device (100) performs a bending operation at least once within the bending range before the control to compensate for hysteresis of the medical device (100) is executed.

[0102] In this specification, a predetermined section may refer to a range of operating angles in which the operating lever (110) can be operated in one embodiment, and this may form a bending range of approximately ±40°. In another embodiment, the predetermined section may be the first section or the second section to be described below. In yet another embodiment, the predetermined section may be a section set according to the purpose of hysteresis modeling. If hysteresis modeling is performed for the purpose of deadzone compensation, the tip (101) can be bent by operating a narrow area based on the neutral position of the operating lever (110) to model the central local part of the hysteresis, and in this case, the predetermined section may be set to have a bending range of approximately ±20°. On the other hand, if hysteresis modeling of the entire section is performed, the tip (101) can be bent over the entire operating range of the operating lever (110), and the operating angle corresponding to the entire operating range of the operating lever (110) may be the predetermined section. An embodiment of the operating lever (110) will be described later.

[0103] The calibration step (S10) may be a step for obtaining the angle at which the motor (1300, FIG. 6), driven by the motor driving signal (1310), bends the tip (101) of the medical device (100).

[0104] In the control method according to the present embodiment, information regarding the usage state of the medical device (100) is obtained through a calibration step (S10), and the inherent hysteresis behavior of the medical device (100) can be modeled and compensated based on the information. Here, 'calibration' may refer to a series of processes for obtaining actual physical characteristics and driving response data of the medical device (100) for hysteresis compensation.

[0105] The calibration process can be performed by reflecting the usage status of the medical device (100) and can be performed immediately before surgery, prior to the insertion of the medical device (100) into the internal pathway (21). Alternatively, even during surgery, if the medical staff determines that control precision has decreased after a predetermined surgical time has elapsed, the medical device (100) can be withdrawn from the body, calibration can be performed, and then the device can be reinserted into the internal pathway (21). In this process, if a complex process or a significant amount of time is required for calibration, usability may be greatly reduced; therefore, the system should be constructed to compensate for hysteresis with a minimized calibration process.

[0106] It should be considered that hardware factors occurring during the manufacturing process, such as the elasticity of the tube constituting the medical device (100), friction characteristics between the wire and the tube, and the degree of tensile strength, may not be formed completely uniformly even for the same medical device (100), and accordingly, different hysteresis may occur for each medical device (100). In another case where different hysteresis occurs, even if the medical device (100) initially has the same hysteresis, the hysteresis characteristics may change over time of use. For example, as the contact friction between the wire and the tube increases or strain accumulates due to repeated use of the medical device (100), behavior different from the hysteresis pattern observed in the initial state may appear. Hysteresis that fluctuates according to the aforementioned hardware characteristics or usage conditions may be defined in this specification as 'inherent hysteresis,' which takes into account both mechanical deviations originating from the manufacturing stage of the medical device (100) and physical changes accumulated during the use process.

[0107] Therefore, in this embodiment, the inherent hysteresis of the medical device (100) is determined through calibration (S10), and by modeling and compensating for it during the control process, positional errors that may occur during the operation of the medical device (100) can be minimized. Since this compensation can precisely reflect hysteresis that differ from one another or change over time even for the same medical device (100), the operational accuracy of the medical device (100) can be improved.

[0108] Hereinafter, the characteristics of the medical device (100) applied in this embodiment and the process of performing calibration (S10) with the medical device (100) are explained. FIG. 5 shows the operation of the medical device (100) according to an embodiment of the present invention.

[0109] Referring to FIG. 5, the medical device (100) includes an operating lever (110) to deflect the tip (101). In this embodiment, the medical device (100) illustrated in FIG. 5 exemplifies a Flexible Ureteroscopy (fURS), that is, a flexible ureteroscope. The medical device (100) may be a commercially available endoscope product and can control the deflection of the tip (101) through an operating lever (110) provided on the body of the handpiece. When a user mechanically operates the operating lever (110), the corresponding operation input is transmitted to the tip (101) to cause the tip to deflect. The exemplified ureteroscope (fURS) is configured with a tendon-sheath mechanism and can be driven by a bending motion in which the tip at the end deflects while having flexible characteristics. However, a dead zone may occur in the ureteroscope (fURS) due to tendon slack in the non-bending position (zero). Alternatively, backlash may occur in the ureteroscope (fURS) due to non-linear friction and elongation between the tendon and the sheath. These elements of dead zone and backlash combine to cause hysteresis inherent to the medical device (100), which may reduce the precision and repeatability of the bending motion. In particular, the hysteresis characteristics of the ureteroscope (fURS) change due to tendon elongation and other factors as it is used more.

[0110] In this embodiment, the medical device (100) has an operating angle (θ) of the operating lever (110). in The tip (101) may be bent in response to ). Depending on the structural form of the medical device (100), the operating lever (110) may be driven by sliding in a linear motion along a vertical or horizontal direction, but it may also be implemented by pivoting in a circumferential direction for operability and ease of use. In this example, a mechanism is disclosed in which the operating lever (110) is driven by pivoting in a circumferential direction, and in this case, the pivot angle formed by the rotation of the operating lever (110) is the operating angle (θ in It becomes.

[0111] In this embodiment, the change in inclination of the operating lever (110) is directly reflected in the displacement path of the tip (101), thereby forming a correlation between the input operation and the output bending. The degree of bending of the tip (101) can be deformed within a certain range, and the actual amount of bending of the tip is the deflection angle (θ ff It can be defined as ). The range in which the tip (101) can be bent may be referred to as the bending range. In this embodiment, the bending range of the medical device tip (101) may be up to (-)270° to (+)270°. In this case, the deflection angle (θ ff ) can be formed up to a maximum of 540°. The operating angle (θ) of the operating lever (110) in ) and the deflection angle (θ) of the tip (101) ff The correspondence between ) may change depending on the product specifications. As an example, the operation angle (θ in The range of ) can generally have a maximum range of ±35° to 40°, and the deflection angle (θ ff ) can form up to 80°.

[0112] FIG. 6 shows the appearance in which a calibration step (S10) according to an embodiment of the present invention is performed when a medical device (100) according to the embodiment of FIG. 5 is mounted on a manipulator (13), and a motor driving signal (1310) is input so that the tip (101) of the medical device (100) bends.

[0113] The calibration step (S10) is such that the tip (101) of the medical device (100) is bent at a deflection angle (θ) in a predetermined section. max ) and a different deflection angle (θ ff It can be controlled to bend at least once additionally within the range of ). Different deflection angles (θ ff ) refers to the deflection angle (θ) in this embodiment. maxIt may mean a reduced or increased deflection angle compared to ). In this case, the motor driving signal (1310) may be a signal that controls the tip (101) of the medical device (100) to bend at least twice within the bending range before the control that compensates for the hysteresis of the medical device (100) is executed.

[0114] In one embodiment, the motor driving signal (1310) is a waveform extending from the first cycle to the nth cycle (n is a natural number) in which a motor command value is specified to bend the tip (101) of the medical device (100) at a specific angle, and the tip (101) of the medical device (100) can be bent within the range of the first section in one cycle of the waveform. The motor driving signal (1310) may be formed with the bending angle of the tip (101) as an indicated value. In the example of FIG. 6, the motor driving signal (1310) illustrates a signal in which the x-axis is time and the y-axis is the bending angle of the tip (101).

[0115] The motor driving signal (1310) may have a waveform of a standing wave for each cycle, and the motor command value may be specified such that the range of bending the tip of the medical device in each cycle is different. When the motor driving signal (1310) bends the tip (101) of the medical device (100) to the range of a first section in the first cycle (T), the first section is the bending range of the tip (101) according to the entire operating range of the operating lever (110), and when the motor driving signal (1310) bends the tip (101) of the medical device (100) to the range of a second section in the second cycle (2T), the second section may be a damping signal set to a range smaller than the first section. In this case, the first section is a deflection angle (θ max ) can be formed at a maximum value, and the second section is the deflection angle (θ out ) is formed as a value smaller than the maximum value. The motor driving signal (1310) illustrated in FIG. 6 exemplifies a damped signal. In the example of FIG. 6, the motor driving signal (1310) has a standing wave signal of 4 periods (T to 4T), and in the 4th section, which is the last period, the deflection angle (θ min ) can be formed within the minimum range.

[0116] The motor driving signal (1310) has a deflection angle (θ) based on one cycle. out ) is extended to a positive value, and then extended to a negative value. When the motor driving signal (1310) is a damping signal, the deflection angle may be at its maximum range in the first period (T). In one embodiment, the first section is the maximum deflection angle (θ max It may refer to a section that bends within the range of ). In another embodiment, the first section is specified as a region of the range where the tip (101) is bent, and the maximum deflection angle (θ max Some intervals may be set in a specific form, such as 10%, 30%, and 50% of ).

[0117] In this embodiment, the calibration (S10) is performed to determine the usage state of the medical device (100) by indicating the angle (θ) in the entire range of hysteresis behavior. in ) and actual driving angle (θ out It can be implemented as a process of measuring the relationship between ). In one embodiment, since there is a limitation in that the entire shape of the hysteresis curve cannot be determined when the tip (101) is repeatedly bent only in a limited narrow section, in calibration (S10), the tip (101) can be driven at least once to the maximum bending range to obtain a relationship equation for the entire loop section.

[0118] In addition, since the dead-zone and backlash within the hysteresis behavior may appear differently depending on the driving range, information that the tip (101) is driven back and forth at least once in the intermediate bending range and the small bending range, as well as in the maximum range, is required to accurately determine this. Therefore, in this embodiment, the motor driving signal (1310) is provided as a waveform of four cycles that gradually attenuates rather than a single amplitude, thereby allowing the characteristics of each hysteresis region that occur when the tip (101) bends sequentially from a large range to a small range to be measured, and thereby the unique hysteresis of the medical device (100) can be modeled.

[0119] That is, the motor driving signal (1310) has a deflection angle (θ) that gradually decreases during four repeated cycles. out It is provided to form ), thereby allowing the determination of how the response of the tip (101) varies not only in the entire area but also in partial sections. Calibration (S10) can collect a dataset including the entire area of ​​the hysteresis loop and further model the changes in dead zone and backlash for each section. As will be described later, the indicated value according to the motor driving signal (1310) and the actual driving response of the tip (101) are the torque (τ) of the motor (1300). m It can be extracted from the signal.

[0120] In an embodiment different from the damping signal, the motor driving signal (1310) may be a damping signal in which, when the tip (101) of the medical device (100) is bent to the range of the first section in the first cycle, the first section is the minimum bending range in which the tip can bend up and down from the correct position, and when the tip (101) of the medical device (100) is bent to the range of the second section in the second cycle, the second section is a range larger than the first section and is set to a range less than or equal to the bending range. That is, the motor driving signal may be configured in the form of a damping signal that is a left-right flipped version of the motor driving signal (1310) of FIG. 6.

[0121] FIG. 7(a) is a graph showing the inherent hysteresis behavior of a medical device (100), and FIG. 7(b) is a graph showing the torque (τ) formed by the motor (1300) in the corresponding hysteresis behavior. m This is a graph showing the change of ). Referring to FIG. 7(a), depending on the usage state of the medical device (100), hysteresis can be formed differently from the first plot (H1, yellow) to the second plot (H2, purple). In FIG. 7(a), the x-axis indicates the angle indicating the bending of the tip (101), and the y-axis indicates the actual bending angle of the tip (101). At this time, the hysteresis curve can appear in various forms between H1 and H2.

[0122] FIG. 7(b) shows the torque (τ) measured at the motor (1300) in response to the hysteresis behavior. m ) information is plotted, and the torque curve corresponding to the first plot (H1) may be MT1 (yellow), and the torque curve corresponding to the second plot (H2) may be MT2 (purple). In Fig. 7 (b), the x-axis represents time and the y-axis represents torque value. Referring to Fig. 7, the first plot (H1) corresponds to a shape with significant hysteresis deformation when the usage state has accumulated for a long time, and the second plot (H2) corresponds to a case where the usage state is relatively short and the hysteresis deformation is small.

[0123] Ideally, it is desirable for the angle value indicated in FIG. 7 (a) to match the actual bending angle of the tip (101), thereby forming a straight line graph corresponding to x=y. However, in the actual hysteresis behavior, a dead zone region exists in the center where the tip (101) does not respond even when the indicated value increases, and a non-linear closed loop may be formed in the lower left and upper right sections where the indicated value and the actual value do not match. Therefore, separate control compensation is required for these hysteresis characteristics. Furthermore, the aforementioned hysteresis behavior may change moment by moment depending on the usage history of the medical device (100), and the difference is the motor torque (τ).m It can appear as a relatively subtle change at the ) level.

[0124] FIG. 7(a) illustrates an example of hysteresis behavior experimentally measured for the medical device (100). To determine which hysteresis pattern the current medical device (100) exhibits among these various behaviors, the bending angle of the tip (101) must be measured. Although actual measurement can be performed by attaching a separate angle sensor to the tip (101), a system processing capable of estimating the angle without a sensor is required to utilize medical devices with space constraints, such as commercially available ureteroscopes, as they are. Accordingly, in this embodiment, an angle estimation step (S20) may be performed after the calibration step (S10), and as illustrated in FIG. 7(b), the torque (τ) of the motor (1300) m It can be implemented to estimate the bending angle of the tip (101) based on the value.

[0125] In the angle estimation step (S20), an angle estimation algorithm can be executed to estimate the angle at which the tip (101) of the medical device (100) is bent by executing calibration.

[0126] FIG. 8 shows the architecture of an angle estimation algorithm (520) executed in the angle estimation step (S20) according to an embodiment of the present invention. Referring to FIG. 8, the angle estimation algorithm (520) is a deep learning-based time series analysis model and the motor torque data (τ m The motor driving signal (1310) or the residual block (522) is configured to have a network structure in which the residual blocks (522) are stacked sequentially, and the residual blocks (522) are processed in parallel to learn the angle at which the tip (101) of the medical device (100) is bent according to the motor driving signal (1310).

[0127] The learning process of the angle estimation algorithm (520) is as follows. When constructing the angle estimation algorithm (520), a fixed input signal (y) is applied to each medical device (fURS) with a different usage state to the entire dataset (dataset ) is obtained. In this embodiment, the input signal (y) may be a signal simulating the motor driving signal (1310), and an example thereof is [Relationship 1].

[0128]

[0129] [Relationship 1]

[0130]

[0131] Here, t represents time, f represents frequency, ζ represents the damping constant, and T represents the period. During training, an electromagnetic field tracking sensor is attached to the end of the medical device (fURS) to determine the actual deflection angle (θ out ) can be used as a measure. dataset ) can collect 1,000 samples (N=1,000) based on a single medical device (fURS). dataset The composition of ) can be formed as shown in [Relation 2] below.

[0132] [Relationship 2]

[0133]

[0134] Here, N is the number of samples, D is the sub-dataset, and θ in is the input angle, θ out is the angle at which the tip is bent, τ m represents the motor torque value. In the dataset, constraints can be set by considering the maximum and minimum ranges of the deflection angle. dataset ) can subsequently be constructed into a training dataset (521) by a batch generator. The batch generator uses a dataset ) is divided into N batch units, each of which is a sample unit. Each batch constitutes a training dataset (521).

[0135] In this embodiment, the angle estimation algorithm (520) may be configured with an architecture of a TCN model, which is one of the deep learning-based time series analysis models. The TCN model has a structure in which a plurality of residual blocks (522) are stacked sequentially. The residual blocks (522) contain torque data (τ m Two pieces of information, an input signal (y) corresponding to the motor driving signal (1310) and the input signal (y), are configured and act as inputs to a deep learning model.

[0136] FIG. 9 is a schematic diagram of a residual block (522) of an angle estimation algorithm (520) according to an embodiment of the present invention. Each residual block (522) includes a structure for effectively extracting long-range dependency and non-linear characteristics of time series data. Each residual block (522) can acquire two pieces of information together as input signals: torque data (τm) and an input signal (y) corresponding to a motor driving signal (1310). These input signals operate to characteristically learn time series patterns by passing through multiple layers including Dilated Causal Convolution, Weight Normalization, ReLU, Dropout, etc. Within the residual block (522), the input signals undergo a series of convolution operations and can be transmitted to an upper layer through a 1x1 convolution or an Identity Map. The purpose of the residual block (522) is to effectively identify the temporal correlation between the torque change and the motor driving characteristics included in the input time series signal, so that the angle estimation algorithm (520) can accurately predict the actual bending angle of the tip.

[0137] Here, the number of residual blocks (522) varies according to the sequence length. In this embodiment, the number of residual blocks (522) is calculated according to the sequence length using [Relationship 3] to set the desired number of residual blocks (n).

[0138] [Relationship 3]

[0139]

[0140] In this embodiment, the angle estimation algorithm (520) may set the number of residual blocks (522) based on the number of angle data collected during the total cycle of the motor driving signal (1310). If the motor driving signal (1310) samples 1000 data points as in the previously described embodiment, the number of residual blocks (522) may be set to 1000. In this case, n may be composed of 8 layers, and variables may be set to k = 3, b = 2, and l = 1000.

[0141] For the validation of learning, well-known machine learning evaluation techniques such as RMSE or MAE can be used.

[0142] FIG. 10 is an example of evaluating the accuracy of the tip angle estimated by executing the angle estimation algorithm (520) according to FIG. 8. As a result using validation data, FIG. 10 (a) shows the best evaluation data for angle estimation, and FIG. 10 (b) shows the worst evaluation result for angle estimation. In each graph, the red graph represents the estimated tip angle of the medical device (fURS), the dashed graph represents the indicated tip angle of the medical device (fURS), and the blue graph represents the actual measured tip angle of the medical device (fURS). In both the red and blue graphs, the actual measured angle and the estimated angle of the medical device (fURS) are not aligned due to hysteresis, but the tip angle of the medical device (fURS) shows that the actual measured angle and the estimated angle are almost aligned in both the Best Case and the Worst Case. At this time, the mean of RMSE[deg] was 7.8310±1.3604 and the mean of MAE[deg] was 6.2129±1.1047, both of which were evaluated as being less than 10.

[0143] FIG. 11 is another embodiment evaluating the accuracy of the tip angle estimated by executing the angle estimation algorithm (520) according to FIG. 8. FIG. 11 (a) shows the Root Mean Square Error (RMSE) evaluation results according to the sequence length. FIG. 11 (b) shows the Maximum Absolute Error (MAE) evaluation results according to the sequence length. Combining FIG. 10 and 11, the best learning performance was observed when the sequence length was 1000 in both the Best Case and Worst Case.

[0144] FIG. 12 is another example evaluating the accuracy of the tip angle estimated by executing the angle estimation algorithm according to FIG. 8. In this example, the average RMSE was measured to be 15, and the average MAE was measured to be 12. When the RMSE was 10 or higher, a relatively large difference in angle estimation occurred at the maximum angle.

[0145] Therefore, the case where the RMSE is 10 or less is assumed to be a desirable angle estimation performance, and in this case, it can be expected that matching the number of residual blocks (522) with the number of angle data (N=1000) collected in the total cycle of the motor drive signal (1310) will result in the most desirable angle estimation performance.

[0146] In summary, by performing the calibration step (S10) and the aforementioned angle estimation step (S20), the angle at which the tip (101) of the medical device (100) moved as a result of the motor driving signal (1310) is systematically estimated. Subsequently, the hysteresis modeling step (30) is performed by loading the estimated angle information of the tip (101). In the hysteresis modeling step (S30), after the calibration is executed, a hysteresis modeling algorithm that mathematically simulates non-linear hysteresis characteristics may be executed.

[0147] FIG. 13 illustrates an overview of the operation of a hysteresis modeling algorithm (530) executed in the hysteresis modeling step (S30) according to an embodiment of the present invention. In one embodiment, the hysteresis modeling algorithm (530) may be configured with the architecture of a Symmetric Generalized Prandtl-Ishlinskii (SGPI) model. The SGPI model is a phenomenon-based model that constructs a synthetic response of a model by arranging a plurality of GPO (523) operators in parallel to describe a hysteresis response and applying individual weights (w) to each operator. Since the SGPI model has symmetric characteristics, it can guarantee left-right equivalent behavior when the directionality of the input signal changes, and it has the characteristic of being able to describe various hysteresis patterns (asymmetry, dead zone, region-specific nonlinearity) in detail.

[0148] FIG. 13(a) shows an overview of the architecture of a hysteresis modeling algorithm (530), wherein one or more Generalized Play Operators (GPO, 523) each operate independently with respect to an input signal x(t), and weights (w₁~w) corresponding to the output of each GPO (523) n After ) is applied, it is finally totaled to y SGPI It can be calculated as (t). Each GPO operator (523) non-linearly transforms the hysteresis characteristics of the input signal, and in this process, a plurality of parameters (P) constituting the GPO operator (523) can be calculated based on the estimated bending angle of the medical device tip (101) derived in the calibration step (S10).

[0149] FIG. 13(b) illustrates, exemplarily, the general shape of a hysteresis curve implemented by a GPO operator (523), where the solid line represents the actual non-linear response curve of the GPO operator (523) and the dashed line represents the linear characteristics of the theoretical ideal model. Each curve can graphically explain the residual phenomenon of the output and the non-linear closed-loop characteristics (γ1, γ2, ζ1, ζ2, etc.) according to the change in input. The hysteresis modeling algorithm (530) of this embodiment provides an SGPI model structure capable of modeling the overall hysteresis behavior by appropriately combining and weighting a plurality of GPOs (523) to suit the actual usage environment of the medical device (100).

[0150] In this embodiment, the output value of the GPO operator (523) for the input signal x(t) can be derived as [Relation 4]. The operator of the GPO operator (523) can be composed of two piecewise linear envelope functions.

[0151] [Relationship 4]

[0152]

[0153] Here, the parameter (P) that simulates the characteristics of γ(x) and hysteresis in [Equation 4] can be calculated through the following [Equation 5].

[0154] [Relationship 5]

[0155]

[0156] The parameters (P) that simulate the characteristics of hysteresis include P1 and P2, which can reflect the left and right boundary points. P3 can reflect the linear slope of the central section. P4 reflects the offset of the central section. P5 and P6 reflect the slopes of the left and right outer sections. P7 reflects the magnitude of the model threshold. P8 and P9 relate to the magnitude and exponent of the model weights. If the parameters (P) of P1 through P9 are defined, the output value y of the hysteresis modeling algorithm (530) is obtained through [Equation 6]. SGPI (t) can be calculated.

[0157] [Relationship 6]

[0158]

[0159] According to the present embodiment, the first hysteresis modeled as the calibration and hysteresis modeling algorithm (530) is executed on the first medical device mounted on the manipulator (13) and the second hysteresis modeled as the calibration and hysteresis modeling algorithm (530) is executed on the second medical device when the first medical device is replaced with a second medical device on the manipulator (13) may be different from each other.

[0160] In addition, according to the present embodiment, the first hysteresis modeled as the calibration and hysteresis modeling algorithm (530) is executed in the first state of the medical device (100) mounted on the manipulator (13) and the first hysteresis modeled as the calibration and hysteresis modeling algorithm is executed in the second state of the medical device (100) may be different from each other. Since the state at this time is defined by the time the medical device (100) is operated, the medical device (100) used for different times may be defined as different states.

[0161] FIG. 14 shows the results of hysteresis modeling according to an embodiment of the present invention and an overview of the operation of a feedforward control algorithm (540) executed in the hysteresis compensation step. FIG. 14(a) exemplarily illustrates the results of modeling four different hysteresis behaviors with different usage states after the hysteresis modeling step (S30) according to FIG. 13 is performed. In FIG. 14(a), the red graph is the hysteresis curve predicted by the hysteresis modeling algorithm (530), and the black graph is the hysteresis behavior actually observed. It is shown that all four hysteresis closed loops have different nonlinear characteristics depending on the state change of the medical device, and it can be confirmed that the actual observed curve and the modeling result are followed with high similarity.

[0162] In the hysteresis compensation step (S40), a feedforward control algorithm that compensates the control signal of the medical device (100) based on the hysteresis behavior modeled by the hysteresis modeling algorithm (530) may be executed. At this time, the hysteresis compensation step (S40) may be performed in a medical environment where the manipulator (13) inserts the medical device (100) into the body and operates it for a procedure.

[0163] The feedforward control algorithm (540) can compensate the control signal of the medical device (100) based on an inverse model of the hysteresis behavior modeled by the hysteresis modeling algorithm (530).

[0164] In the feedforward control algorithm (540), the inversion solution for the inversion can be referenced by the variable of [Relation 6] and [Relation 7].

[0165] [Relationship 7]

[0166]

[0167] The envelope function of the inverse transformation model according to [Equation 7] is as shown in [Equation 8].

[0168] [Relationship 8]

[0169]

[0170] Here, b1 has the relationship P1P3+P4, and b2 has the relationship P2P3+P4.

[0171] In this embodiment, parameters (P1~P8) can be optimized more precisely by performing an additional algorithm. The parameter optimization algorithm can be executed in the processor (500) during the hysteresis modeling step (S30). As an example of the parameter optimization algorithm, the LM (Levenberg-Marquardt) algorithm may be applied. The LM algorithm is an algorithm that combines the Gradient Desecent method and the Gauss-Newton method to form a mutually complementary structure. Since the LM algorithm demonstrates faster and more stable convergence than the Gauss-Newton method, it can be primarily used in non-linear optimization problems. The cost function of the LM algorithm and the boundary conditions of the parameters (P) are as shown in [Equation 9]. After the parameters (P) are adjusted through optimization according to [Equation 9], [Equation 8] can be applied to perform hysteresis modeling.

[0172] [Relationship 9]

[0173]

[0174] FIG. 14(b) shows an overview of the operation of a feedforward control algorithm (540) executed in the hysteresis compensation step. The hysteresis shown in the feedforward compensator of the feedforward control algorithm (540) shown in FIG. 14(b) is an example of an output according to [Equation 8], and the indicated angle θ d When is input, θ based on the inverse transformed hysteresis model of the feedforward compensator cFeedforward compensation of is performed and θ a Angle control of the tip compensated by can be performed. In Fig. 14 (b), θ d is the angle command value of the tip (101) of the medical device (100), and θ c is the input angle value corrected by the inverse transformed model of hysteresis, and θ a is the actual angle at which the tip (101) of the medical device (100) is bent. As a result of the feedforward control algorithm (540), the indicated angle (θ) as shown in the linearized system d) The actual bending angle (θ) corresponding to a ) can be formed.

[0175] FIG. 15 shows an experimental environment for testing a control method for compensating for hysteresis according to an embodiment of the present invention. A frameless motor was used as the motor (1300), and a ureteroscope (fURS) was used as the medical device (100). An electromagnetic field sensor (trakSTAR) for measuring angles was used at the tip (101) of the medical device (100) for learning angle estimation and evaluating results. A processor (500) encoded with control commands for compensating for hysteresis of the medical device was implemented as a PC.

[0176] Figure 16 is a control process flowchart performed in the experimental environment according to Figure 15.

[0177] First, calibration is started by performing the calibration step (S10).

[0178] With the execution of calibration, a motor drive signal (1310) is applied and the tip trajectory of the ureteroscope (fURS) is executed. In this experimental example, the motor drive signal (1310) inputs a 4-cycle damping signal as exemplified in FIG. 6, and 1,000 angles of bending of the tip (101) are collected. Once 1,000 data points are collected, data preprocessing is performed, and an angle estimation step (S20) is executed. In this experimental example, angle inference was performed using a TCN model as the angle estimation algorithm. At this time, for the learning settings, the loss function was set to MSE, the learning rate to 0.001, the epoch to 500, and the kernel size (k) to 3. As a result of the inference of the TCN model, 1,000 angle data points of the ureteroscope (fURS) were acquired. FIG. 17 is an experimental example of angle estimation performed in the experimental environment according to FIG. 15. As a result of the experiment as shown in FIG. 17, the results of FIG. 10 to FIG. 12 described above were derived to set the optimal sequence length and residual block (522). Subsequently, a hysteresis modeling step (S30) was executed, and the SGPI model was used as the hysteresis modeling algorithm. In this experimental example, a parameter optimization algorithm was performed to optimize the parameter (P) that can define the hysteresis behavior. Subsequently, a hysteresis compensation step (S40) was executed, and a feedforward algorithm that inversely transforms the modeled hysteresis was applied.

[0179] Figure 18 is an example of a hysteresis modeling experiment performed in the experimental environment according to Figure 15. As a result of the evaluation, the RMSE was calculated to be approximately 16. In Figure 18, the top graph represents the SGPI performance, where the blue graph represents the input angle value, the red graph represents the angle of bending of the actual ureteroscope (fURS) tip, and the green graph represents the angle predicted by the SGPI model. In Figure 18, the bottom graph represents the modeled hysteresis, where the red graph represents the hysteresis of the actual ureteroscope (fURS) and the green graph represents the hysteresis predicted and modeled by SGPI.

[0180] Figure 19 is a graph showing the results of the hysteresis compensation performance evaluation performed in the experimental environment according to Figure 15. The top graph (a) of Figure 19 shows the state where feedforward compensation is not applied, and in this case, it can be seen that a discrepancy occurs between the ideal reference signal (dashed line) and the actual measured signal (blue line) due to the hysteresis phenomenon.

[0181] On the other hand, the bottom graph of FIG. 19 (a) shows the state in which feedforward-based hysteresis compensation is applied according to the present embodiment. At this time, it can be seen that the red measured signal graph matches the ideal reference signal plot with relatively high precision, which can be understood as an experimental result in which the error caused by hysteresis is effectively compensated and the tracking performance for the target angle is significantly improved.

[0182] Figure 19(b) is a hysteresis plot representing the result of Figure 19(a). Referring to Figure 19(b), it can be seen that the closed loop of the actual measured signal (blue) in which hysteresis occurred is compensated into a red graph that is close to linear as feedforward compensation is performed. Therefore, it has been experimentally verified that the feedforward-based compensation control of the present embodiment minimizes non-linear errors caused by hysteresis, thereby enabling precise tracking control for an ideal signal.

[0183] Although the technical concept of the present invention and disclosure has been explained by the embodiments described above, the technical concept of the present invention includes various substitutions, modifications, and changes that can be understood by those skilled in the art to which the present invention pertains. Furthermore, it should be understood that such substitutions, modifications, and changes may be included within the scope of the appended claims.

Claims

1. In surgical robot systems, A surgical instrument device comprising a manipulator configured to deflect the tip of said medical instrument, equipped with one or more motors and a medical instrument; and A processor that drives the motor to control the manipulator; is included, The above processor is, A surgical robot system characterized by outputting a motor driving signal that controls the tip of the medical device to deflect within a predetermined section of a bending range in order to estimate the inherent hysteresis of the medical device deformed by the usage state of the medical device, and thereby performing a calibration in which the tip of the medical device performs a bending motion at least once within the bending range before control to compensate for the hysteresis of the medical device is executed.

2. In Paragraph 1, The above processor is, When executing the above calibration, the tip of the medical device is controlled to bend at least once additionally in a range of deflection angles different from the deflection angle bent in the above predetermined section, and The above motor driving signal is, A surgical robot system characterized by a signal that controls the tip of the medical device to bend at least twice within the bending range before control compensating for hysteresis of the medical device is executed.

3. In Paragraph 2, The above motor driving signal is, The command value of the motor that bends the tip of the medical device at a specific angle is a waveform extending from the first period to the nth period (n is a natural number), and A surgical robot system characterized by bending the tip of the medical device into the range of a first section in a waveform of one cycle.

4. In Paragraph 3, The above motor driving signal is, A surgical robot system characterized by the waveform of each cycle being a standing wave and the motor command value being specified such that the range of bending the tip of the medical device in each cycle is different.

5. In Paragraph 3, The above motor driving signal is, When the tip of the medical device is bent to the range of the first section in the first cycle, the first section is the predetermined section, and A surgical robot system characterized in that, when the tip of the medical device is bent into the range of the second section in the second cycle, the second section is a damping signal set to a smaller range than the first section.

6. In Paragraph 3, The above motor driving signal is, When the tip of the medical device is bent to the range of the first section in the first cycle, the first section is the minimum bending range in which the tip can be bent in the up and down direction from the correct position, and A surgical robot system characterized by the fact that when the tip of the medical device is bent into the range of the second section in the second cycle, the second section is a range larger than the first section and is a bending signal set to a range less than or equal to the bending range.

7. In Paragraph 1, The above processor is, By executing the above calibration, an angle estimation algorithm is executed to estimate the angle of bending of the tip of the medical device, A surgical robot system characterized by estimating the inherent hysteresis of the medical device using a non-measurement method.

8. In Paragraph 7, The above angle estimation algorithm is, As a deep learning-based time series analysis model, the torque data of the motor or the motor driving signal is configured as residual blocks, and the residual blocks have a network structure in which they are sequentially stacked. A surgical robot system characterized by processing the above residual blocks in parallel to learn the angle at which the tip of the medical device is bent according to the motor driving signal.

9. In Paragraph 8, The above angle estimation algorithm is, A surgical robot system characterized by the number of residual blocks being set based on the number of angle data collected during the total cycle of the motor driving signal.

10. In Paragraph 1, The above processor After performing the above calibration, A surgical robot system characterized by executing a hysteresis modeling algorithm that mathematically simulates nonlinear hysteresis characteristics.

11. In Paragraph 10, The above hysteresis modeling algorithm is, Model hysteresis behavior by summing one or more Generalized Play Operators (GPO) with weights assigned to each, A plurality of parameters (P) constituting the above GPO operator are calculated based on the bending angle of the medical device tip output as the result of the calibration execution, A surgical robot system characterized by modeling the inherent hysteresis of the above-mentioned medical device.

12. In Paragraph 10, In the first medical device mounted on the above manipulator, the first hysteresis modeled as the calibration and the hysteresis modeling algorithm are executed, and A surgical robot system characterized in that the first medical device is replaced with a second medical device in the manipulator, and the second hysteresis modeled as the calibration and hysteresis modeling algorithm for the second medical device are executed is different from each other.

13. In Paragraph 10, In the first state of the medical device mounted on the manipulator, the first-1 hysteresis modeled as the calibration and the hysteresis modeling algorithm are executed, and A surgical robot system characterized in that, in a second state of the medical device, the first and second hysteresis modeled as the calibration and the hysteresis modeling algorithm are executed are different from each other, and the state at this time is defined by the time the medical device has been operated.

14. In Paragraph 10, The above processor is, In a medical environment where the above manipulator inserts and operates the above medical device into the body for a procedure, A surgical robot system characterized by executing a feedforward control algorithm that compensates for the control signal of the medical device based on the hysteresis behavior modeled by the hysteresis modeling algorithm.

15. In Paragraph 14, The above feedforward control algorithm is, A surgical robot system characterized by compensating the control signal of the medical device based on an inverse model of the hysteresis behavior modeled by the hysteresis modeling algorithm.

16. In Paragraph 1, The above manipulator is, The above medical device includes a detachable mounting portion, The above-mentioned mounting portion includes a lever holder that grips the operating lever of the medical device, and A surgical robot system characterized by the motor controlling the lever holder with the above motor driving signal to bend the tip of the medical device.

17. In Paragraph 1, The above motor is, A surgical robot system characterized by being a frameless motor.

18. A control method that is executed in a processor controlling a surgical robot system and compensates for hysteresis of a medical device mounted on a manipulator of a surgical robot system, wherein A step of outputting a motor driving signal to control the motor so that the tip of the medical device deflects within a predetermined section of the bending range, wherein the tip of the medical device performs a bending operation at least once within the bending range before control to compensate for hysteresis of the medical device is executed. It includes a calibration step for obtaining the angle at which the motor driven by the motor driving signal bends the tip of the medical device, and A control method characterized by compensating for hysteresis of a medical device by modeling the behavior of the unique hysteresis of the medical device based on information of the medical device obtained in the calibration step.

19. A processor encoded with control commands for compensating for hysteresis of a medical device mounted on a manipulator in a surgical robot system, Outputting a command for a motor driving signal that controls the motor so that the tip of the medical device bends within a predetermined section of the bending range, and the motor driving signal is a waveform extending from the first period to the nth period (n is a natural number) in which the command value of the motor that bends the tip of the medical device at a specific angle is specified. A processor characterized by estimating the bending angle of the tip of the medical device according to the motor driving signal and generating a feedforward control command to compensate for hysteresis of the medical device.

Images