Hysteretic compensation control device and method for flexible tube

Abstract

The present invention relates to a hysteretic compensation control device for a flexible tube and a method therefor, and more particularly, to a hysteretic compensation control device for a flexible tube and a method therefor for performing compensation control on hysteretic of a surgical tool disposed in a channel of an outer cannula. To this end, disclosed is a hysteresis compensation control method for a flexible tube, which is characterized by comprising the steps of: performing compensation control on a flexible surgical instrument having a hysteresis characteristic on the basis of image information of the flexible surgical instrument as the flexible tube; judging whether the flexible surgical instrument is visible or not based on the image information used for judging the current state of the flexible surgical instrument; if it is judged that the flexible surgical instrument is invisible, a control mode is switched from compensation control based on the image information to compensation control based on tension information; based on the tension information of the flexible surgical instrument, compensation control is conducted on the flexible surgical instrument with the hysteresis characteristic.

Description

[0001] This application is a divisional application of patent application No. 202211267312.X, filed on October 17, 2022, entitled "Hysteresis Compensation Control Device and Method for Flexible Tube". Technical Field

[0002] The present invention relates to a hysteresis compensation control device and method for a flexible tube, and more specifically, to a hysteresis compensation control device and method for a flexible tube for compensating and controlling the hysteresis of surgical instruments disposed within a channel of an outer sheath. Background Technology

[0003] like Figure 1 As shown, the flexible outer sheath 10 includes a plurality of flexible surgical instruments 21, 22. The plurality of flexible surgical instruments 21, 22 are inserted into various channels 11, 12 disposed within the main body of the outer sheath. As an example, the first surgical instrument 21 is tractioned by first and second traction lines (shown in the attached figure) to control its position and orientation.

[0004] like Figure 2 As shown, flexible surgical instruments have a hysteresis range. That is, even when the same traction force is applied, a hysteresis range will occur depending on the shape of the flexible sheath or surgical instrument. This results in a disadvantage where the sheath or surgical instrument cannot precisely control this hysteresis range.

[0005] Existing technical documents

[0006] Patent documents

[0007] US 2019 / 0290109 Summary of the Invention

[0008] Technical issues

[0009] Therefore, this invention is proposed to solve the above-mentioned problems, and its purpose is to provide an invention that can precisely control a flexible surgical instrument even in the hysteresis range.

[0010] However, the purpose of this invention is not limited to the purposes mentioned above, and those skilled in the art can clearly understand other purposes not mentioned from the following description.

[0011] Problem-solving methods

[0012] The objective of this invention is achieved by providing a hysteresis compensation control device for a flexible outer sheath, characterized by comprising: an input unit for receiving image information and tension information of the flexible surgical instrument required for mode switching; a mode switching judgment unit for judging the current state of the surgical instrument based on the image information and generating a control mode change signal based on the current state of the surgical instrument; a mode switching unit for switching the control mode from flexible outer sheath control based on image information to flexible outer sheath control based on tension information according to the control mode change signal from the mode switching judgment unit; and a compensation control unit for compensating the flexible outer sheath with hysteresis characteristics based on the tension error value calculated by a learning model.

[0013] Furthermore, the image information used to determine the current state of the surgical instrument is an image of the tip area of ​​the surgical instrument that is inserted into the channel of the body of the outer sheath and is pulled by the traction line.

[0014] Furthermore, the input unit includes: a surgical instrument image capturing unit, which is inserted into the channel of the main body of the outer sheath to capture images of the surgical instrument; and a tension measuring unit, which measures the traction force of the traction line of the surgical instrument and transmits the image information of the tip area of ​​the surgical instrument captured by the surgical instrument image capturing unit to the mode switching judgment unit, thereby determining the current state of the surgical instrument.

[0015] Furthermore, the mode switching judgment unit determines whether it can be controlled using the flexible outer tube control mode based on the image information of the tip area of ​​the surgical instrument.

[0016] Furthermore, it also includes: an image information learning unit, which learns based on the image information of the surgical instruments according to a preset learning model and calculates the image error value through learning; and a tension information learning unit, which learns based on the tension information of the surgical instruments according to a preset learning model and calculates the tension error value through learning.

[0017] Furthermore, the objective of this invention is achieved by providing a hysteresis compensation device for a flexible surgical instrument, characterized by comprising: a mode switching judgment unit, which determines the current state of the surgical instrument based on image information of the flexible surgical instrument, thereby generating a first control mode change signal, or, based on tension information of the flexible surgical instrument, determines whether it conforms to a preset tension control model, thereby generating a second control mode change signal; an image mode switching unit, which switches the control mode from flexible surgical instrument control based on tension information to flexible surgical instrument control based on image information; a tension mode switching unit, which switches the control mode from flexible surgical instrument control based on image information to flexible surgical instrument control based on tension information; and a compensation control unit, which uses a compensation error value generated based on tension information or image information to perform compensation control on the flexible surgical instrument with hysteresis characteristics.

[0018] Furthermore, when the tip region of the surgical instrument included in the image information used to determine the current state of the surgical instrument is not visible or is obscured by other surgical instruments, the first control mode change signal is generated.

[0019] Furthermore, the tension mode switching unit switches the control mode according to the first control mode change signal.

[0020] Furthermore, when the load on the surgical instrument changes, or when any of the multiple tension measurements of the surgical instrument is zero and does not conform to the tension control modeling, a second control mode change signal is generated.

[0021] Furthermore, the image mode switching unit switches the control mode according to the second control mode change signal.

[0022] Furthermore, the objective of this invention can be achieved by providing a hysteresis compensation control method for a flexible outer sleeve, characterized by the following steps: performing compensation control on a flexible surgical instrument with hysteresis characteristics based on image information of the flexible surgical instrument; determining whether the surgical instrument is visible based on image information used to determine the current state of the surgical instrument; if the surgical instrument is determined to be invisible, switching the control mode from image information-based compensation control to tension information-based compensation control; and performing compensation control on a flexible surgical instrument with hysteresis characteristics based on tension information of the flexible surgical instrument.

[0023] Furthermore, after changing the control mode from image-based compensation control to tension-based compensation control, the following steps are also included: Based on the tension information of the flexible surgical instrument, it is determined whether it conforms to the predefined tension control model. If it is determined that it does not conform to the predefined tension control model, the control mode is switched from tension information-based compensation control to image information-based compensation control. Based on the image information of the flexible surgical instrument, compensation control is performed on the flexible surgical instrument with hysteresis characteristics.

[0024] Invention Effects

[0025] According to the present invention described above, it has the effect of precisely controlling flexible surgical instruments even within the hysteresis range. Attached Figure Description

[0026] The following figures are examples of preferred embodiments of the present invention, which are used together with the detailed description of the invention to illustrate the technical concept of the invention, and should not be construed as limiting the invention to the matters described in these figures.

[0027] Figure 1 The accompanying drawings illustrate a flexible outer sleeve according to an embodiment of the present invention; Figure 2The accompanying drawings illustrate the hysteresis characteristics or curves of a flexible surgical instrument according to an embodiment of the present invention. Figure 3 The accompanying drawings illustrate a simplified structure of the hysteresis compensation control device for the flexible outer sleeve according to a first embodiment of the present invention. Figure 4 The accompanying drawings illustrate the switching of the control mode from image-based hysteresis compensation control to tension-based hysteresis compensation control in the first embodiment of the present invention. Figure 5 The accompanying drawings illustrate how, in the first embodiment of the present invention, the control mode is switched from hysteresis compensation control based on image information to hysteresis compensation control based on tension information, and then the control mode is switched again from hysteresis compensation control based on tension information to hysteresis compensation control based on image information. Figure 6 The accompanying drawings illustrate a simplified structure of the hysteresis compensation control device for the flexible outer jacket according to a second embodiment of the present invention. Figure 7 The accompanying drawings illustrate how, in the second embodiment of the present invention, the control mode is switched from hysteresis compensation control based on tension information to hysteresis compensation control based on image information, and then the control mode is switched again from hysteresis compensation control based on image information to hysteresis compensation control based on tension information. Figure 8 The accompanying drawings illustrate a simplified structure of the learning process of the hysteresis compensation model according to the third embodiment of the present invention. Figure 9 The accompanying drawings illustrate a simplified structure of the hysteresis compensation control according to a third embodiment of the present invention.

[0028] Explanation of reference numerals in the attached figures: 10: Flexible outer sleeve; 10a: Outer sleeve body 10b: First traction line; 10c: Second traction line 11: First Channel 12: Second Channel 13: Third channel; 21: First surgical instrument 21a: End effector or tip of surgical instrument 22: Second surgical instrument 22a: End effector or surgical instrument tip; 23: Camera unit 110: Input unit; 111: Surgical instrument image acquisition unit 112: Tension Measurement Unit; 113: Tension Change Calculation Unit 114: Tension input value comparison and judgment unit; 120: Mode switching judgment unit 121: Image Analysis and Judgment Unit; 122: Tension Modeling Analysis and Judgment Unit 130: Mode switching unit; 131: Image mode switching unit 132: Tension mode switching unit; 140: Learning unit 141: Image Information Learning Unit; 142: Tension Information Learning Unit 150: Compensation Control Unit 151: Image Information-Based Hysteresis Compensation Control Unit 152: Hysteresis Compensation Control Unit Based on Tension Information 210: Input unit; 211: Surgical instrument image acquisition unit 212: Tension Measurement Unit 213: Tension Change Calculation Unit; 214: Tension Input Value Comparison and Judgment Unit 220: Mode switching judgment unit; 221: Image analysis and judgment unit 222: Tension Modeling Analysis and Judgment Unit; 230: Mode Switching Unit 231: Image mode switching unit; 232: Tension mode switching unit 240: Learning Unit 241: Image Information Learning Unit 242: Tension Information Learning Unit; 250: Compensation Control Unit 251: Image Information-Based Hysteresis Compensation Control Unit 252: Hysteresis Compensation Control Unit Based on Tension Information 260: Control Unit; 261: Line Control Unit 262: Surgical instrument control unit; 310: Flexible surgical instrument pose calculation unit. 311: Flexible Surgical Instrument Position Detection Unit 312: Flexible Surgical Instrument Image Acquisition Unit 313: Flexible Surgical Instrument Posture Measurement Unit 320: Learning Unit; 321: Model Condition Setting Unit 322: Tension information and image information acquisition unit; 323: Flexible surgical instrument pose export unit. 324: Lag Compensation Model Generation Unit; 325: Compensation Model Learning Unit 330: Model search condition input unit; 340: Lag compensation model search unit 350: Determining unit for delayed compensation model; 360: Compensation control unit. 370: Hysteresis Change Detection Unit; 371: Outer Tube Movement Detection Unit 372: Patient Movement Detection Unit Detailed Implementation

[0029] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings. Furthermore, the embodiments described below do not unreasonably limit the scope of the invention as described in the claims, and it cannot be said that all structures described in this embodiment are necessary solutions for the present invention. In addition, descriptions of matters obvious to those skilled in the art may be omitted, and the omitted descriptions of components (methods) and functions can be fully referenced without departing from the technical spirit of the present invention.

[0030] The hysteresis compensation control device for a flexible tube according to an embodiment of the present invention is a device for compensating and controlling hysteresis occurring in a flexible tube. Here, the flexible tube is a wire-driven or traction tube, and as an example, it may be a flexible outer tube or a flexible surgical instrument or flexible robotic surgical instrument inserted into a channel within the flexible outer tube. In the following description, as an example of a flexible tube, the flexible outer tube 10 or the flexible surgical instruments 21 and 22 will be used as examples.

[0031] The flexible outer tube 10 has a flexible tube body and flexible surgical instruments inserted into the channel of the flexible tube. The flexible outer tube 10 is inserted into the human body, and the position and orientation of the flexible tube and surgical instruments are controlled by traction lines to perform surgery on the human body.

[0032] like Figure 1 As shown, the flexible outer sheath 10 (or tube) includes: an outer sheath body 10a, having a flexible body; first, second, and third channels 11, 12, and 13, formed inside the outer sheath 10a with preset diameters; surgical instruments inserted into each channel; and a camera unit. Figure 1As shown, a first surgical instrument 21 is inserted into the first channel 11, a second surgical instrument 22 is inserted into the second channel 12, and a camera unit 23 is inserted into the third channel 13. However, the number of channels and the number of surgical instruments can be varied as needed. The camera unit 23 and surgical instruments inserted into each channel protrude from the front end of the outer sheath body 10a. Therefore, the camera unit 23 can capture images of the surgery in front, where each surgical instrument grips or removes tissue according to the operation control of the traction line (or drive line). An operation unit (or control unit, not shown in the figure) for operating the traction line (not shown) is arranged at the rear end of the outer sheath body 10a. The traction line is connected from the operation unit to the front end of the outer sheath body or surgical instrument. Therefore, the user can control the position and direction of the traction line by operating the operation unit. For example, to control the position and direction of the flexible outer sheath body 10a, the first traction line 10b and the second traction line 10c are adjusted by the operation unit, thereby changing the position and direction according to the traction force. Furthermore, the position and direction control of each surgical instrument can also be described according to the same principle as the position and direction control of the flexible outer sheath body 10a.

[0033] In addition, such as Figure 2 As shown, the flexible sheath body 10a or the flexible surgical instruments 21 and 22 have a linear slope range due to the traction force of the traction line, thus enabling control of the position and orientation of the surgical instruments. However, when using traction lines to control surgical instruments, even with such... Figure 2 The same traction force, as shown, also exhibits a range where position and direction cannot be consistently controlled, corresponding only to the traction force. This range can be called the hysteresis region, where the position and direction control of surgical instruments may differ even with the same traction force. One possible cause of this hysteresis is the change in shape or twisting of the flexible surgical instrument or flexible sheath when it is inserted into the body, resulting in friction between the suture and the sheath. It could also be caused by suture elongation due to the use of traction sutures. Furthermore, it can also occur even if the input control command pulls the suture through slack, resulting in a so-called dead zone where the output remains unchanged. Additionally, it could be due to the un-uniform characteristics of the tubing caused by the sheath or suture within the flexible sheath.

[0034] Therefore, by compensating for the hysteresis characteristic of the flexible surgical instrument, the surgical instrument can be precisely controlled in this invention.

[0035] (Structure and function of the hysteresis compensation control device for flexible outer jacket: First embodiment)

[0036] like Figure 3As shown, the hysteresis compensation control device for a flexible tube according to the first embodiment of the present invention includes an input unit 110, a mode switching judgment unit 120, a mode switching unit 130, a learning unit 140, and a compensation control unit 150.

[0037] The input unit 110 receives image information and tension information of the flexible surgical instrument required for mode switching in hysteresis control. To this end, the input unit 110 includes a surgical instrument image acquisition unit 111, a tension measurement unit 112, a tension change calculation unit 113, and a tension input value comparison and judgment unit 114.

[0038] The surgical instrument image capturing unit 111 is inserted into the third channel 13 to generate an image of the end effector of the first surgical instrument 21 or the tip 21a region of the surgical instrument, which is the hysteresis-compensated controlled object inserted into the third channel 13. At this time, when the controlled object is the second surgical instrument 22, the second surgical instrument can also be captured, and the first and second surgical instruments 21 and 22 can be captured together as needed. Furthermore, the surgical instrument image capturing unit 111 can be fixed, or it can be fixed by inserting it into the channel with a movable controllable structure as needed. Preferably, the shooting conditions of the camera actually inserted into the body for surgery are completely consistent with the shooting position, shooting angle, or shooting area of ​​the surgical instrument used for learning in the learning unit 120.

[0039] The tension measuring unit 112 measures the traction force of the traction line of the surgical instrument. To measure the traction force, for example, a tension measuring sensor can be provided to measure the tension of the traction line, or the load on the motor mounted on the operating unit for driving the traction line can be measured.

[0040] like Figure 5 As shown, after switching the control mode from image-based hysteresis compensation control to tension-based hysteresis compensation control, and then switching the mode back to image-based surgical instrument hysteresis compensation control, the system also includes a tension change calculation unit 113 and a tension input value comparison and judgment unit 114.

[0041] The tension change calculation unit 113 calculates the change in tension value when a load is suddenly applied to the first surgical instrument 21, which is in a state of no load, or conversely, when a load is suddenly lost while the first surgical instrument 21 is under load. This is done to control a surgical instrument via a traction line, as described above. Figure 1 As shown, two traction lines need to be operated separately, thus requiring tension or traction force measuring sensors for the first and second traction lines.

[0042] However, for ease of explanation, we use one traction line for each line. In reality, two traction lines are needed to move one degree of freedom, and four traction lines are needed for two degrees of freedom. Furthermore, each traction line requires a drive motor, or each pair of traction lines is driven by one drive motor.

[0043] The tension change calculation unit 113 calculates the change in tension value measured by each measuring sensor. By comparing the changes in tension values, it is possible to determine whether the load changes from no-load to load or from load to no-load. In these two cases, it is difficult to perform hysteresis compensation control based on tension information, and a switch in control mode is required. That is, in this case, it is necessary to change from tension information-based control to image information-based control.

[0044] The tension input value comparison and judgment unit 114 searches for a case where either the tension or traction force measurement value (or tension value) of the first and second lines has an input value of "0". When either the tension value of the first or second line has an input value of "0", it is difficult to perform hysteresis compensation control based on tension information, and a control mode switch is required. That is, in this case, it is necessary to change from tension information-based control to image information-based control. In the following description, the first surgical instrument 21 will be assumed to be the hysteresis compensation control object. However, the same principle can be used to explain the hysteresis compensation control of the second surgical instrument 22.

[0045] In one embodiment of the present invention, the mode switching judgment unit 120 determines the current state of the first surgical instrument 21 based on the image information of the flexible first surgical instrument 21, thereby generating a first control mode change signal (a signal generated to switch the mode from image information-based hysteresis compensation control to tension information-based hysteresis compensation control).

[0046] The mode switching judgment unit 120 determines whether the tension control modeling is satisfied based on the tension information of the flexible first surgical instrument 21, thereby generating a second control mode change signal (a signal generated to switch the mode from tension information-based hysteresis compensation control to image information-based hysteresis compensation control).

[0047] The image information used to determine the current state of the first surgical instrument 21 is the image information of the tip 21a region of the first surgical instrument 21.

[0048] The mode switching judgment unit 120 includes an image analysis and judgment unit 121 and a tension modeling analysis and judgment unit 122.

[0049] The image analysis and judgment unit 121 receives an image of the tip region of the first surgical instrument 21 from the surgical instrument image capturing unit 111. The current state of the first surgical instrument 21 can be determined from the received image. Specifically, by analyzing the image, if the tip region of the first surgical instrument 21 is not visible within the image or if the first surgical instrument 21 is obscured by the second surgical instrument 22, image analysis and judgment unit 121 generates a first control mode because image-based compensation control is not possible. With the generation of the first control mode change signal, the control mode switches from image-based hysteresis compensation control to tension-based hysteresis compensation control.

[0050] The tension modeling analysis and judgment unit 122 determines whether the input value of the tension change calculation unit 113 or the tension input value comparison judgment unit 114 conforms to the tension control modeling. If the modeling is deemed not to conform, hysteresis compensation control based on tension information cannot continue, and a second control mode is generated for switching control modes. That is, the control mode is switched from hysteresis compensation control based on tension information to hysteresis compensation control based on image information. An example of a situation where tension control modeling is not conforming is when the load on the first surgical instrument 21 changes (a load is applied when there is no load, and the load disappears when a load is applied), or when either the tension measurement value of the first or second traction line pulling the first surgical instrument 21 is "0". In the above example, this is not the prediction model assumed during the deep learning process of the tension information learning unit 142 described later. In this case, hysteresis compensation control based on tension information cannot be performed. Therefore, a switch in control mode is necessary.

[0051] According to an embodiment of the present invention, the mode switching unit 130 includes an image mode switching unit 131 and a tension mode switching unit 132.

[0052] The image mode switching unit 131 switches the control mode from image information-based first surgical instrument 21 compensation control to tension information-based first surgical instrument 21 compensation control according to the first control mode change signal of the image analysis and judgment unit 121.

[0053] According to the second control mode change signal from the tension modeling analysis and judgment unit 122, the tension mode switching unit 132 switches the control mode from image information-based first surgical instrument 21 compensation control to tension information-based first surgical instrument 21 compensation control.

[0054] According to an embodiment of the present invention, the learning unit 140 performs deep learning or machine learning based on image information or tension information, and calculates image error values ​​or tension error values ​​respectively through learning. For this purpose, the learning unit 140 includes an image information learning unit 141 and a tension information learning unit 142.

[0055] The image information learning unit 141 performs machine learning or deep learning based on photographs taken of the tip 21a region of the first surgical instrument 21. Therefore, preferably, the shooting conditions of the camera 23 for obtaining the images required for learning are consistent with the shooting conditions of the camera 23 when actually inserted into the body. The image information learning unit 141 calculates an image compensation control value or an image compensation error value corresponding to the image of the first surgical instrument 21 inserted into the body and currently captured through a predefined learning model. The image compensation error value is transmitted to the compensation control unit 150. Of course, the image compensation error value generated at this time can be information related to the drive of the traction cable drive motor actually used to drive the first surgical instrument.

[0056] The tension information learning unit 142 performs machine learning or deep learning based on the tension information of the traction line of the first surgical instrument 21. Through learning a predefined learning model, the tension information learning unit 142 calculates a tension compensation control value or a tension compensation error value corresponding to the tension measurement value of the first surgical instrument 21 inserted into the body. The tension compensation error value is transmitted to the compensation control unit 150.

[0057] The compensation control unit 150 performs compensation control on the first surgical instrument 21 with a hysteresis curve based on the image compensation error value or the tension compensation error value. For this purpose, the compensation control unit 150 includes a hysteresis compensation control unit 152 based on image information and a hysteresis compensation control unit 152 based on tension information.

[0058] The image-based hysteresis compensation control unit 151 performs hysteresis compensation control on the first surgical instrument 21 using image compensation error values. That is, based on the currently captured image information of the first surgical instrument, hysteresis compensation control is performed using image compensation error values ​​calculated through learning.

[0059] The hysteresis compensation control unit 152 based on tension information performs hysteresis compensation control on the first surgical instrument 21 using the tension compensation error value. That is, based on the currently measured tension information of the first surgical instrument, hysteresis compensation control is performed using the tension compensation error value calculated through learning.

[0060] (Hysteresis compensation control method for flexible outer jacket)

[0061] A hysteresis compensation control method for a flexible outer jacket according to an embodiment of the present invention includes as follows: Figure 4The first method shown changes the control mode from image-based compensation control to tension-based compensation control, and as... Figure 5 The second method, which changes the control mode from image-based compensation control to tension-based compensation control and then changes the control mode back from tension-based compensation control to image-based compensation control, is explained below.

[0062] In addition, such as Figure 4 As shown, the transition from image-based compensation control to image-based compensation control is achieved by performing the following steps.

[0063] The compensation control unit 150 first performs compensation control on the first surgical instrument 21, which has a hysteresis curve, based on the image information of the first surgical instrument 21 (S11). More specifically, the hysteresis compensation control unit 151 based on the image information controls the tension of the traction line of the first surgical instrument 21 based on the image information of the first surgical instrument 21.

[0064] At this time, the image analysis and judgment unit 121 determines whether the first surgical instrument 21 is visible based on the image information used to determine the current state of the first surgical instrument 21. Since the first surgical instrument 21 cannot continue to perform compensation control based on image information when it is obscured or disappears from the field of view of the camera unit 23, the current state of the first surgical instrument 21 is determined (S12).

[0065] Based on the judgment result, when it is determined that the surgical instrument is not visible, the mode switching unit 130 switches the control mode from compensation control based on the image information to compensation control based on tension information (S13).

[0066] According to the switching of control mode, the hysteresis compensation control unit 152 based on tension information performs compensation control on the first surgical instrument 21 with hysteresis curve based on the tension information of the first surgical instrument 21.

[0067] In addition, such as Figure 5 As shown, compensation control based on image information is converted to compensation control based on tension information, and then to compensation control based on image information again, by performing the following steps.

[0068] First, such as Figure 4 and Figure 5 As shown, steps S11 to S14 are the same. The hysteresis compensation control unit 152 based on tension information performs compensation control on the first surgical instrument 21 based on the tension information of the first surgical instrument 21.

[0069] After changing the control mode from image-based compensation control to tension-based compensation control, the tension modeling analysis and judgment unit 122 determines whether the control model conforms to the predefined control modeling based on the tension information of the first surgical instrument 21 (S15). This is because if the tension control model does not conform, tension-based compensation control cannot continue.

[0070] Based on the judgment result, when the judgment does not conform to the predefined tension control modeling, the image mode switching unit 131 switches the control mode from tension information-based compensation control to image information-based compensation control (S16).

[0071] According to the switching of control mode, the image information-based hysteresis compensation control unit 151 performs compensation control on the first surgical instrument 21 with hysteresis curve based on the image information of the first surgical instrument 21.

[0072] (Structure and function of the hysteresis compensation control device for flexible outer sleeve: Second embodiment)

[0073] The following is combined Figure 6 and Figure 7 The hysteresis compensation control device for the flexible outer jacket according to the second embodiment of the present invention will be described in detail. However, the same descriptions can be omitted as needed, referring to the descriptions of the input unit 110, mode switching judgment unit 120, mode switching unit 130, learning unit 140 and compensation control unit 250 described above.

[0074] like Figure 6 As shown, input unit 210 receives image information of the flexible surgical instrument and line tension information required for mode switching. Input unit 210 includes a surgical instrument image capturing unit 211, a tension measurement unit 212, a tension change calculation unit 213, and a tension input value comparison and determination unit 214, which are replaced by the description of input unit 110 above. However, tension input value comparison and determination unit 214 receives the tension measurement values ​​of the first line and the second line, and searches for the case where any of the tension input values ​​is "0". For this purpose, it is preferable to adjust the traction force so that even under no-load conditions, the initial tension measurement value based on the traction force of the line is "0" or higher.

[0075] The mode switching judgment unit 220 includes an image analysis judgment unit 221 and a tension modeling analysis and judgment unit 222. The mode switching judgment unit 220 determines the current state of the surgical instrument based on image and tension information, and generates a control mode change signal for switching control modes according to the current state of the surgical instrument. A control mode change signal is generated for each switching condition. The mode switching judgment unit 220 generates a first switching condition control mode change signal, a second switching condition control mode change signal, a third forced switching condition control mode change signal, and a signal guiding the state change of the surgical instrument.

[0076] When the tension modeling analysis and judgment unit 222 performs tension-information-based control, if it determines that the first switching condition of tension-information-based flexible tube control cannot be performed, it generates a first switching condition control mode change signal that switches the control mode from tension-information-based flexible tube control to image-information-based flexible tube control. An example of the first switching condition is provided by the above explanation of the inability to perform tension-information-based control.

[0077] Furthermore, when performing flexible tube control based on image information according to the first switching condition control mode change signal, if the second switching condition is met, that is, when it is determined that flexible tube control based on image information cannot be performed based on the signal of the image analysis and judgment unit 221 and the first switching condition has been released, the tension modeling analysis and judgment unit 222 generates a second switching condition control mode change signal to switch the control mode from flexible tube control based on image information to flexible tube control based on tension information.

[0078] When performing flexible tube control based on image information according to the first switching condition control mode change signal, if the third switching mandatory condition is met, i.e., it is determined that flexible tube control based on image information cannot be performed and the first switching condition is not released, the image analysis and judgment unit 221 generates a third switching mandatory condition control mode change signal to switch the control mode from flexible tube control based on image information to flexible tube control based on tension information. Examples of the third switching mandatory condition include situations where at least one of a plurality of surgical instruments has a portion of its area not visible in the read image, or two or more surgical instruments overlap and an overlapping area appears in the read image, making it difficult to read the image, or two or more surgical instruments are in contact with each other, making it difficult to read the image. Furthermore, as another example, situations include when the camera lens or surgical instrument is stained with blood, the camera's field of view or the surgical instrument is obscured by organs or tissues inside the body or smoke generated during removal, the overall image outline of the surgical instrument is altered due to organs or the surrounding environment, making it difficult to identify the surgical instrument, or any surgical instrument obscures the camera's field of view, or the camera's LED light reflects off the surgical instrument, making it difficult to identify the image of the surgical instrument.

[0079] When performing flexible tube control based on image information according to the first switching condition control mode change signal, if the fourth switching condition is met, i.e., it is determined that flexible tube control based on image information cannot be performed and the first switching condition is not lifted, the mode switching determination unit 220 generates a signal to guide the state change of the surgical instrument to control the line or surgical instrument, thereby realizing flexible tube control based on tension information or image information. The generated signal is sent to the control unit 260 described later.

[0080] Control unit 260 includes wire control unit 261 and surgical instrument control unit 262. Wire control unit 261 receives signals from mode switching determination unit 220 to guide changes in the state of the surgical instrument, thereby controlling the wire to achieve flexible tube control based on tension information. For example, if the wire is not broken, control can be performed to measure tension values ​​by repeatedly pulling or releasing the traction wire.

[0081] In addition, the surgical instrument control unit 262 receives signals from the mode switching judgment unit 220 to guide the state changes of the surgical instrument and controls the surgical instrument imaging unit 211 or the surgical instrument, thereby realizing control based on the flexible tube capable of reading image information.

[0082] When the tension measurement value is normally input under the control of the control unit 260 or when the image can be read, a tension information start signal and an image information start signal are generated respectively and transmitted to the mode switching judgment unit 220.

[0083] When the mode switching determination unit 220 receives a tension information start signal from the control unit 260, it generates a tension state change control mode change signal that changes the control mode from image-based flexible tube control to tension-based flexible tube control. When the mode switching determination unit 220 receives an image information start signal from the control unit 260, it continues control in the image-based flexible tube control mode. Therefore, no special change of control mode is required.

[0084] If no change in state occurs even under the control of the control unit 260, and control cannot be performed based on tension and image information, feedback is sent to the user.

[0085] The mode switching unit 230 receives the first switching condition control mode change signal, the second switching condition control mode change signal, the third switching forced condition control mode change signal, and the tension state change control mode change signal from the mode switching judgment unit 220. Based on each control mode change signal from the mode switching judgment unit 220, it switches the control mode from flexible tube control based on tension information to flexible tube control based on image information, or vice versa.

[0086] The description of the learning unit 240 and the compensation control unit 250 is superseded by the foregoing description.

[0087] like Figure 7 As shown, firstly, flexible tube control based on tension information is performed (S21). When performing flexible tube control based on tension information, if it is determined that the preset tension modeling is not met (first switching condition (S22), then the control mode is changed to flexible tube control based on image information and control is performed (S23, S24).

[0088] On the other hand, during the transition from flexible tube control based on tension information to flexible tube control based on image information, when the first switching condition is released (second switching condition), or when the first switching condition is not released and the image of the surgical instrument cannot be read, thus making it impossible to continue maintaining flexible tube control based on image information, the following controls are performed respectively.

[0089] First, when the first switching condition is released (second switching condition) and the measured tension value is greater than the set threshold (at this time, for example, the threshold is "0") (second switching condition), the control mode is changed from flexible tube control based on image information to flexible tube control based on tension information for control (S31, S32, S33, S34).

[0090] On the other hand, if the first switching condition is not released and the image of the surgical instrument cannot be read (S51), control is performed in the following three ways.

[0091] First, as the first method, it is determined whether the control mode change mandatory condition (third switching mandatory condition) is met. If the third switching mandatory condition is met, the control mode is changed from flexible tube control based on image information to flexible tube control based on tension information and then controlled (S41, S54a).

[0092] As a second method, the line is repeatedly pulled or released to control it and thus release the first switching condition (S52a). When the tension measurement value of the line is normally input as the line is controlled (fourth switching condition), the control mode is changed back from flexible tube control based on image information to flexible tube control based on tension information (S53a, S54a). If the fourth switching condition is not met due to the failure to input a normal tension measurement value, the method is switched to the third method described later or feedback is ultimately given to the user.

[0093] As a third method, control is performed by controlling the position of the surgical instrument or the surgical instrument image capturing unit 211 to read the image of the surgical instrument (S52b). If it is determined that the image can be read according to the controlled position (fourth switching condition), the flexible tube control based on the image information is maintained unchanged (S53b, S54b). If the fourth switching condition is not met because the image cannot be read, the method is switched to the second method described above or feedback is finally given to the user.

[0094] (Structure and function of the hysteresis compensation control device for flexible outer tube: Third embodiment)

[0095] Generally, hysteresis varies depending on the degree of friction between the thread and the sheath, and the shape of the sheath varies depending on the shape of the outer sheath. Therefore, the friction between the thread and the sheath may change, resulting in changes in hysteresis characteristics. Depending on the shape of the outer sheath, even with the same thread driving force, the pose (bending angle) of the flexible surgical instrument will change. Therefore, the hysteresis compensation control device for the flexible outer sheath according to the third embodiment of the present invention can search for a corresponding hysteresis characteristic model and derive compensation values ​​even if the shape of the outer sheath changes and the pose of the flexible surgical instrument changes. The following will refer to... Figure 8 and 9 A hysteresis compensation control device for a flexible outer jacket according to a third embodiment of the present invention is described in detail.

[0096] In this invention, a hysteresis compensation model is first learned and corrected through hysteresis model learning. The learned and corrected hysteresis compensation model is searched and determined according to the model search conditions. Then, the compensation value is derived according to the determined compensation model, and the flexible surgical instruments 21 and 22 are compensated and controlled.

[0097] First, refer to Figure 8 This illustrates the learning lag model. For example... Figure 8 As shown, the flexible surgical instrument pose calculation unit 310 calculates the pose values ​​of the flexible surgical instruments 21 and 22, which are inserted into the outer sheath channel and driven by the traction of a wire. The pose values ​​of the flexible surgical instruments can be calculated using the following two methods.

[0098] In the first method, the flexible surgical instrument pose detection unit 311 detects and calculates the pose value of the flexible surgical instrument using sensors. Various sensors can be configured for this purpose. Examples of sensors include fiber Bragg grating sensors (FBG sensors), magnetic sensors, or cameras. The pose of the flexible surgical instrument can be the bending angle of the flexible surgical instrument or a measurement based on the forward, backward, or rotational movement of the flexible surgical instrument. Tension information can be a tension value or the amount of wire stretching measured by an encoder.

[0099] The second method involves the flexible surgical instrument image acquisition unit 312 acquiring images of the flexible surgical instruments. The acquired images of each flexible surgical instrument are stored as a dataset. The flexible surgical instrument pose measurement unit 313 matches the input image dataset of the flexible surgical instruments with the pose values ​​of the flexible surgical instruments and stores this as a dataset. Therefore, the flexible surgical instrument pose measurement unit 313 can estimate the actual pose values ​​of the matched flexible surgical instruments based on the image dataset transmitted from the flexible surgical instrument image acquisition unit 312. The image information of the flexible surgical instruments is the image dataset observed by the camera 111 within the channel inserted into the outer sheath.

[0100] The model condition setting unit 321 sets the conditions for the hysteresis compensation model to control the shape of the outer sheath to a preset shape or the movement of the flexible surgical instrument to a preset movement. For example, the outer sheath can deform into various shapes, and the movement of the flexible surgical instrument can also change according to the surgical procedure. Therefore, various possible environmental variables (the shape of the outer sheath and the movement of the flexible surgical instrument) are set, and a hysteresis compensation model is generated based on these environmental variables.

[0101] The tension information and image information acquisition unit 322 acquires corresponding tension information and image information according to the various hysteresis compensation model conditions set by the model condition setting unit 321. Furthermore, the tension information and image information acquisition unit 322 stores the acquired tension information and image information as an input dataset (input information), which is used to input the hysteresis compensation model generation unit. Tension information can be measured by the aforementioned tension measurement unit 112, and image information can be acquired by the aforementioned flexible surgical instrument image acquisition unit 312 (or surgical instrument image capturing unit 111).

[0102] The flexible surgical instrument pose export unit 323 exports the pose value of the flexible surgical instrument when the input information is obtained from the tension information and image information acquisition unit 322. As described above, the pose value of the flexible surgical instrument can be exported from a sensor or image dataset. Furthermore, the flexible surgical instrument pose export unit 323 stores the exported pose value of the flexible surgical instrument as an output dataset (output information).

[0103] The hysteresis compensation model generation unit 324 receives tension information and image information of the line, as well as the pose value of the flexible surgical instrument. The tension information and image information are input datasets obtained based on the shape of the outer sheath and the motion control of the flexible surgical instrument as set by the model condition setting unit 321. The pose value of the flexible surgical instrument is the output dataset obtained when the input information is acquired. The hysteresis compensation model generation unit 324 receives the input and output datasets and generates corresponding hysteresis compensation models based on various environmental variables set by the model condition setting unit 321 (based on changes in the shape of the outer sheath and the motion changes of the flexible surgical instrument).

[0104] The compensation model learning unit 325 learns and calibrates the hysteresis compensation model by learning from tension information, image information, and the hysteresis compensation model generated by the hysteresis compensation model generation unit 324.

[0105] As described above, a hysteresis compensation model is learned and corrected through hysteresis model learning. Then, a learned and corrected hysteresis compensation model is searched and determined according to model search conditions. The determined compensation model is used to derive compensation values ​​for compensation control of flexible surgical instruments 21 and 22. In the following text, references... Figure 9 This will be described.

[0106] The model search condition input unit 330 acquires tension information and image information caused by the specific movement of the flexible surgical instrument after the outer cannula reaches the target location with the lesion. The tension information can be measured by the tension measurement unit 112 as described above, and the image information can be acquired by the flexible surgical instrument image acquisition unit 312 (or the surgical instrument image capturing unit 111). The acquired tension information and image information are transmitted to the hysteresis compensation model search unit 340.

[0107] The hysteresis compensation model search unit 340 searches for the hysteresis compensation model learned and corrected by the compensation model learning unit 325 based on the tension information and image information input by the model search condition input unit 330.

[0108] The hysteresis compensation model determination unit 350 determines the learned and corrected hysteresis compensation model searched by the hysteresis compensation model search unit 340, and calculates the compensation value as the tension error value through the determined hysteresis compensation model.

[0109] The compensation control unit 360 performs compensation control on the flexible tube with hysteresis characteristics based on the tension error value of the hysteresis compensation model determined by the hysteresis compensation model determination unit 350.

[0110] The hysteresis change detection unit 370 detects movement of the outer tube or patient movement. Movement of the outer tube or patient constitutes a hysteresis change condition. When a hysteresis change occurs, it may be necessary to re-search the established hysteresis compensation model. Therefore, when the hysteresis change detection unit 370 detects a hysteresis change condition, it transmits a detection signal to the model search condition input unit 330. Upon receiving the hysteresis change detection signal, the model search condition input unit 330 re-acquires tension information and image information to re-search the hysteresis compensation model.

[0111] For example, the movement detection of the outer tube can occur in the following situations: when the image changes without inputting a control signal for controlling the outer tube or flexible surgical instruments, or when a main operation signal for controlling the outer tube is input.

[0112] Additionally, the shape of the sheath inserted into the body may deform when the patient moves. Therefore, it is necessary to detect patient movement to predict hysteresis changes. For example, patient movement can be assessed by examining changes in the patient's breathing or in images taken of the patient.

[0113] In describing this invention, descriptions of matters obvious to those skilled in the art or of the art may be omitted, and the omitted descriptions of components (methods) and functions may be fully referenced without departing from the technical spirit of this invention. Furthermore, the components of this invention described above are merely for illustrative purposes; undescribed components may be added without departing from the technical spirit of this invention.

[0114] The structures and functions of the above parts are described separately for ease of description. As needed, a certain structure or function can be merged into other constituent elements or further subdivided.

[0115] The foregoing description has been based on an embodiment of the present invention, but the invention is not limited thereto, and various modifications and applications are possible. That is, those skilled in the art should readily understand that various modifications can be made without departing from the spirit of the invention. Furthermore, it should be noted that detailed descriptions of known functions and their configurations associated with the invention, or the combinations of various configurations of the invention, have been omitted if they are deemed unnecessary to obscure the spirit of the invention.

Claims

1. A hysteresis compensation control method for a flexible tube, characterized in that, Includes the following steps: Based on image information of flexible surgical instruments that are flexible tubes, compensatory control is performed on flexible surgical instruments with hysteresis characteristics. Based on the image information used to determine the current state of the flexible surgical instrument, it is determined whether the flexible surgical instrument is visible; If it is determined that the flexible surgical instrument is not visible, the control mode is switched from compensation control based on the image information to compensation control based on tension information. as well as Based on the tension information of the flexible surgical instrument, compensation control is performed on the flexible surgical instrument with hysteresis characteristics.

2. The hysteresis compensation control method for flexible tubes according to claim 1, characterized in that, After switching the control mode from compensation control based on the image information to compensation control based on tension information, the following steps are also included: Based on the tension information of the flexible surgical instrument, determine whether it conforms to the predefined tension control modeling; If it is determined that the tension control model does not conform to the predefined model, the control mode will be switched from compensation control based on the tension information to compensation control based on the image information. as well as Based on image information of flexible surgical instruments, compensatory control is performed on flexible surgical instruments with hysteresis characteristics.

3. A hysteresis compensation control device for a flexible tube, characterized in that, include: The flexible surgical instrument pose calculation unit calculates the pose value of the flexible surgical instrument inserted into the channel of the outer sheath and driven by the traction of the wire. The hysteresis compensation model generation unit receives tension information and image information of the line and pose value of the flexible surgical instrument, respectively, and generates hysteresis compensation models based on the shape change of the outer tube and the motion change of the flexible surgical instrument. The tension information and image information are input information obtained based on the shape of the outer tube and the motion control of the flexible surgical instrument, and the pose value of the flexible surgical instrument is the output information obtained when the input information is obtained. as well as The hysteresis model learning unit learns and corrects the hysteresis compensation model by learning the tension information, image information, and the hysteresis compensation model generated by the hysteresis compensation model generation unit.

4. The hysteresis compensation control device for flexible tubes according to claim 3, characterized in that, The flexible surgical instrument pose calculation unit uses attached sensors to calculate the pose value of the flexible surgical instrument, or calculates the pose value of the flexible surgical instrument that matches image information composed of a dataset.

5. The hysteresis compensation control device for flexible tubes according to claim 3, characterized in that, Also includes: The model condition setting unit controls the shape of the outer sleeve to a preset shape or the movement of the flexible surgical instrument to a preset movement in order to set the hysteresis compensation model conditions. The tension information and image information acquisition unit acquires tension information and image information according to the hysteresis compensation model conditions, and stores the acquired tension information and image information as the input dataset of the input information; as well as The flexible surgical instrument pose export unit exports the pose value of the flexible surgical instrument when the input information is acquired, and stores the exported pose value of the flexible surgical instrument as the output dataset of the output information.

6. The hysteresis compensation control device for flexible tubes according to claim 3, characterized in that, Also includes: The model search condition input unit acquires tension information and image information generated by the specific movement of the flexible surgical instrument after the outer sheath reaches the target position; The hysteresis compensation model search unit searches for the learned and corrected hysteresis compensation model based on the tension information and image information input by the model search condition input unit. The hysteresis compensation model determination unit determines the learned and corrected hysteresis compensation model searched by the hysteresis compensation model search unit, and calculates the compensation value as the tension error value based on the determined hysteresis compensation model. as well as The compensation control unit performs compensation control on the flexible tube with hysteresis characteristics based on the tension error value.

7. The hysteresis compensation control device for flexible tubes according to claim 6, characterized in that, Also includes: The hysteresis change detection unit detects hysteresis change conditions caused by the detection of movement of the outer sheath or the patient.

8. The hysteresis compensation control device for flexible tubes according to claim 7, characterized in that, The hysteresis change detection unit detects the hysteresis change condition and transmits a detection signal to the model search condition input unit, thereby enabling the model search condition input unit to reacquire tension information and image information and re-search for the hysteresis compensation model.

Images