Determination of reverse direction exit of robotically controlled endoscope

Abstract

A method for robotically controlling an endoscope includes navigating an elongate shaft in a patient's body, the elongate shaft including a tip at a distal end; articulating the tip in a first direction using a first pull wire coupled to the two-wire pulley; reversing the articulation of the tip to a second direction using a second pull wire coupled to the two-wire pulley based on a non-linear response region of the kinematic model; determining an end point of the nonlinear response area; and articulating the tip based on a linear response region when the articulation of the tip in the second direction reaches the endpoint.

Description

[0001] Related Applications

[0002] This application claims priority to U.S. Provisional Application Serial No. 63 / 435,687, filed Dec. 28, 2022, entitled "KINEMATIC ENDOSCOPE MODEL FOR DUAL WIRE PULLEY", and U.S. Provisional Application Serial No. 63 / 435,697, filed Dec. 28, 2022, entitled "DETERMINATION OF DIRECTION REVERSAL EXITS FOR ROBOTICALLY CONTROLLED ENDOSCOPES", the entire disclosures of which are hereby incorporated by reference in their entireties. BACKGROUND OF THE DISCLOSURE FIELD OF THE DISCLOSURE

[0003] The present disclosure relates to robotic medical systems.

[0004] Related Art

[0005] Certain robotic medical procedures can involve the use of shaft-type instruments, such as endoscopes, that can be inserted into a patient's body through an aperture (e.g., a natural aperture) and advanced to a target anatomical site. Such medical devices can be articulable, such that the end and / or other portions of the shaft can be deflected in one or more dimensions using robotic control. SUMMARY OF THE DISCLOSURE

[0006] Systems, devices, and methods are described herein that facilitate control of instrument articulation associated with certain medical procedures. In particular, systems, devices, and methods according to one or more aspects of the present disclosure can facilitate monitoring of shaft articulation and / or shaft articulation cable tension and tautness. For example, cable tension for the purpose of articulating an instrument shaft can be relieved in certain aspects in response to determined / detected articulation and / or tension conditions.

[0007] In some aspects, the techniques described herein relate to a robotic system that includes: an end effector that includes one or more drive outputs configured to: navigate an elongate shaft within a patient's body, the elongate shaft including a distal end; articulate the distal end in a first direction using a first cable coupled to a dual-line pulley; and reverse the articulation of the distal end to a second direction using a second cable coupled to the dual-line pulley, wherein the distal end articulates based on a non-linear response region of a kinematic model; a processor; and a memory that stores computer-executable instructions that, when executed, cause the processor to: determine endpoints of the non-linear response region; and control the articulation of the distal end based on a linear response region when the articulation of the distal end in the second direction reaches the endpoint.

[0008] In some aspects, the techniques described herein relate to a robotic system, wherein the memory further includes computer-executable instructions that, when executed, cause the processor to estimate the non-linear response region of the kinematic model using a sigmoid equation.

[0009] In some aspects, the techniques described herein relate to a robotic system, wherein the sigmoid equation is a generalized logistic function.

[0010] In some aspects, the techniques described herein relate to a robotic system, wherein the memory further includes computer-executable instructions that, when executed, cause the processor to: receive a percentage value associated with the non-linear response region; and calculate a sigmoid curve that crosses the endpoint of the non-linear response region at the percentage value of the traversal during the traversal of the sigmoid curve, wherein, prior to controlling the articulation of the distal end based on the linear response region, the articulation of the distal end is controlled based on the sigmoid curve.

[0011] In some aspects, the techniques described herein relate to a robotic system, wherein determining the endpoints of the non-linear response region includes: monitoring tension on at least one of the first cable or the second cable; and determining the endpoint based on the tension satisfying a threshold condition.

[0012] In some aspects, the techniques described herein relate to a robotic system, wherein the threshold condition is (i) the tension increases in one direction and (ii) the tension increases by at least a threshold amount.

[0013] In some aspects, the techniques described herein relate to a robotic system, wherein the threshold condition is (i) the tension increases in one direction and (ii) the tension changes its sign.

[0014] In some aspects, the techniques described herein relate to a robotic system, wherein determining the endpoint of the non-linear response region includes: calculating a first pulley rotation of a first joint movement in the non-linear response region for a first time sample; calculating a second joint movement in the linear response region based on the first pulley rotation for the first time sample; calculating a second pulley rotation of a third joint movement in the non-linear response region for a second time sample, the second time sample being later in time than the first time sample; calculating a fourth joint movement in the linear response region based on the second pulley rotation for the second time sample; and determining the endpoint based on a comparison between (i) the first joint movement and the second joint movement and (ii) the third joint movement and the fourth joint movement.

[0015] In some aspects, the techniques described herein relate to a robotic system, wherein (i) the first joint movement is less than the second joint movement, and (ii) the third joint movement is greater than the fourth joint movement.

[0016] In some aspects, the techniques described herein relate to a robotic system, wherein (i) the first joint movement is greater than the second joint movement, and (ii) the third joint movement is less than the fourth joint movement.

[0017] In some aspects, the techniques described herein relate to a robotic system, the robotic system including: an end effector including one or more drive outputs configured to: navigate an elongate shaft within a patient's body, the elongate shaft including a distal end; a processor; and a memory storing computer-executable instructions that, when executed, cause the processor to: monitor the tension on at least one of a first cable or a second cable, the at least one of the first cable or the second cable being coupled to the distal end; and determine a point associated with a kinematic model based on the tension.

[0018] In some aspects, the techniques described herein relate to a robotic system, wherein the kinematic model includes at least one linear response region and at least one non-linear response region.

[0019] In some aspects, the techniques described herein relate to a robotic system, wherein the point is associated with a transition between the at least one non-linear response region and the at least one linear response region.

[0020] In some aspects, the techniques described herein relate to a robotic system, wherein determining the point on the kinematic model includes: determining the point based on the tension satisfying at least one threshold condition.

[0021] In some aspects, the techniques described herein relate to a robotic system, where the at least one threshold condition is (i) the tension increases in one direction and (ii) the tension changes its sign.

[0022] In some aspects, the techniques described herein relate to a robotic system, where the at least one threshold condition is (i) the tension increases in one direction and (ii) the tension increases by at least a threshold amount.

[0023] In some aspects, the techniques described herein relate to a robotic system, where the memory further includes computer-executable instructions that, when executed, cause the processor to: update an end of the at least one non-linear response region based on the point; and determine a post-S-shaped linear response.

[0024] In some aspects, the techniques described herein relate to a robotic system, where the memory further includes computer-executable instructions that, when executed, cause the processor to: control joint movement of the end based on the post-S-shaped linear response when joint movement of the end reaches the point.

[0025] In some aspects, the techniques described herein relate to a method for robotically controlling an endoscope, the method including: navigating an elongate shaft in a patient's body, the elongate shaft including an end at a distal end; causing the end to move in a first direction using a first cable coupled to a dual-line pulley; reversing the joint movement of the end to a second direction using a second cable coupled to the dual-line pulley, where the end moves in a joint movement based on a non-linear response region of a kinematic model; determining an end point of the non-linear response region; and causing the end to move in a joint movement based on a linear response region when the joint movement of the end in the second direction reaches the end point.

[0026] In some aspects, the techniques described herein relate to a method that further includes: receiving a percentage value associated with the non-linear response region; and calculating an S-shaped curve that crosses the end point of the non-linear response region at the percentage value of the traversal during the traversal of the S-shaped curve, where the end moves in a joint movement based on the S-shaped curve before moving in a joint movement based on the linear response region.

[0027] For purposes of summarizing the present disclosure, certain aspects, advantages, and novel features have been described. It will be understood that not necessarily all such advantages may be achieved in accordance with any particular embodiment. Accordingly, the disclosed embodiments may be carried out in a manner that realizes or optimizes one advantage or a group of advantages taught herein without necessarily realizing other advantages that may be taught or presented herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] For illustrative purposes, various embodiments are depicted in the drawings and should in no way be construed as limiting the scope of the invention. Additionally, various features of different disclosed embodiments can be combined to form additional embodiments that are part of the present disclosure. Throughout the drawings, reference numerals may be reused to indicate corresponding relationships between reference elements.

[0029] Figure 1 An embodiment of a robotic medical system according to one or more embodiments is illustrated, the robotic medical system including a shaft-type instrument coupled to a robotic end effector.

[0030] Figure 2 A robotic system configured for diagnostic and / or therapeutic bronchoscopy according to one or more embodiments is illustrated.

[0031] Figure 3 A table-based robotic system according to one or more embodiments is illustrated.

[0032] Figure 4 An example of a medical system component that can be implemented in any of a medical system as described herein according to one or more embodiments is illustrated. Figures 1 to 3 of any of the medical systems described herein

[0033] Figure 5 A shaft-type instrument capable of articulating movement according to one or more embodiments is illustrated.

[0034] Figure 6 An exploded view of an instrument manipulator assembly associated with a robotic end effector according to one or more embodiments is shown.

[0035] Figure 7 An instrument according to one or more embodiments is shown having one or more wire pulley systems for articulating the shaft of the instrument.

[0036] Figure 8 is a diagram showing the relationship between pulley rotation and instrument deflection of a plastic instrument shaft according to one or more embodiments.

[0037] Figure 9 is a diagram showing the relationship between pulley rotation and instrument deflection of an elastic instrument shaft according to one or more embodiments.

[0038] Figure 10 is a diagram showing the relationship between pulley rotation and instrument deflection of a hybrid elastic and plastic instrument shaft according to one or more embodiments.

[0039] Figures 11A to 11B is a kinematic model and tension response according to one or more embodiments.

[0040] Figure 12 is a flowchart of a process for controlling the movement of a robotic joint based on a kinematic model according to one or more embodiments.

[0041] Figures 13A to 13B Illustrates a jump response scenario and a hysteresis response scenario according to one or more embodiments.

[0042] Figure 14 Illustrates a percentage-based reverse exit determination according to one or more embodiments.

[0043] Figure 15 Illustrates a tension-based reverse exit determination according to one or more embodiments.

[0044] Figure 16 is a flowchart of a process for controlling the movement of a robotic joint based on a tension-based reverse exit determination according to one or more embodiments.

[0045] Figures 17A to 17B Illustrates a linear response crossover-based reverse exit determination according to one or more embodiments.

[0046] Figure 18 Illustrates a comparison graph showing the relationship between the inclination of a user command and the actual inclination delivered by the system according to one or more embodiments.

[0047] Figure 19 Illustrates a closed-loop algorithm for eliminating the error between the inclination of a user command and the actual inclination according to one or more embodiments. Detailed Description

[0048] The titles provided herein are for convenience only and do not necessarily affect the scope or meaning of the claimed invention. Although specific preferred embodiments and examples are disclosed below, the subject matter of the present invention extends beyond the specifically disclosed embodiments to other alternative embodiments and / or uses and their modifications and equivalents. Thus, the scope of the claims that may arise herein is not limited by any one of the specific embodiments described below. For example, in any method or process disclosed herein, the acts or operations of the method or process may be performed in any suitable sequence and are not necessarily limited to any particular disclosed sequence. The various operations may then be described as multiple discrete operations in a manner that may aid in understanding some embodiments; however, the described order should not be construed as implying that these operations are order-dependent. Additionally, the structures, systems, and / or devices described herein may be embodied as integrated components or separate components. For purposes of comparing various embodiments, certain aspects and advantages of these embodiments are described. Not all such aspects or advantages may be achieved by any particular embodiment. Thus, for example, various embodiments may be carried out in a manner that achieves or optimizes one advantage or group of advantages taught herein without necessarily achieving other aspects or advantages that may also be taught or suggested herein.

[0049] For convenience of devices, components, systems, features, and / or modules having features that are similar in one or more respects, a particular reference numeral is reused in different figures of the group of figures of the present disclosure. However, for any one of the embodiments disclosed herein, the reuse of a common reference numeral in the figures does not necessarily indicate that such features, devices, components, or modules are the same or similar. Instead, one of ordinary skill in the art may be informed by the context of the degree of similarity that may be implied by the use of the common reference numeral among the referenced subject matter. The use of a particular reference numeral in the context of the description of a particular figure may be understood to refer to the device, component, aspect, feature, module, or system identified in that particular figure and not necessarily to any device, component, aspect, feature, module, or system identified by the same reference numeral in another figure. Additionally, aspects of separate figures identified by a common reference numeral may be interpreted as sharing characteristics or being completely independent of each other. In some contexts, features associated with separate figures identified by a common reference numeral are not relevant and / or similar with respect to at least some aspects.

[0050] The present disclosure provides systems, devices, and methods for monitoring and controlling the articulation of an instrument shaft, such as a medical endoscope. The articulation of an instrument according to the present disclosure may be achieved by tensioning one or more tendons that pass through the shaft of the instrument, which are referred to herein as "wire pulls". With respect to the medical devices described in the present disclosure, the term "instrument" is used in its broad and ordinary sense and may refer to any type of tool, device, component, system, subsystem, equipment, part, etc. In some contexts herein, the term "device" may be used substantially interchangeably with the term "instrument". Additionally, the term "shaft" is used herein in its broad and ordinary sense and may refer to any type of elongated cylinder, tube, endoscope (e.g., an endoscope), prism (e.g., a rectangular, oval, elliptical, or rectangular prism), wire, or the like, regardless of the cross-sectional shape. It should be understood that any reference herein to a "shaft" or "instrument shaft" may be understood to potentially refer to an endoscope.

[0051] Medical Procedure

[0052] Although certain aspects of the present disclosure are described herein in the context of renal, urinary, and / or kidney procedures, such as kidney stone removal / treatment procedures, it should be understood that such context is provided for convenience and clarity, and the concepts of instrument articulation control disclosed herein are applicable to any suitable medical procedure, such as robotic bronchoscopy, laparoscopy, arthroscopy, colonoscopy, laryngoscopy, neuroendoscopy, proctoscopy, anoscopy, gastroscopy, sigmoidoscopy, thoracoscopy, colposcopy, esophagoscopy, or other endoscopy or elongated shaft-based procedures.

[0053] In certain medical procedures, such as ureteroscopy procedures, an elongated medical device that enters the treatment site through an access sheath may be utilized to remove debris (such as kidney stones and stone fragments or other waste or contaminants) from the treatment site. Nephrolithiasis (also known as urolithiasis) is a medical condition that involves the formation of solid masses of material, called "kidney stones" or "urinary stones", in the urinary tract. Urinary stones may form and / or be found in the kidneys, ureters, and bladder (called "bladder stones"). Such urinary stones may form due to the concentration of minerals in the urine and can cause significant abdominal pain once such stones reach a size sufficient to block the flow of urine through the ureter or urethra. Urinary stones may be formed from calcium, magnesium, ammonia, uric acid, cystine, and / or other compounds or combinations thereof.

[0054] Several methods can be used to treat patients suffering from kidney stones, including observation, medical treatment (such as lithotripsy therapy), non-invasive treatment (such as extracorporeal shock wave lithotripsy (ESWL)), minimally invasive or surgical treatment (such as ureteroscopy and percutaneous nephrolithotomy ("PCNL")), etc. In some methods (e.g., ureteroscopy and PCNL), a physician can access the stone, break the stone into smaller fragments or pieces, and use a basket device and / or aspiration to remove relatively small stone fragments / particles from the kidney.

[0055] In some procedures, a surgeon can insert an endoscope (e.g., a ureteroscope) through the urethra into the urinary tract to remove urinary stones from the bladder and ureter. Generally, a ureteroscope includes a camera at its distal end, which is configured to visualize the urinary tract. The ureteroscope may also include or allow placement of a lithotripsy device in the working channel of the ureteroscope, which is configured to capture or break up urinary stones. In some procedures (such as procedures for removing relatively large stones / fragments), a physician can use percutaneous nephrolithotomy ("PCNL") techniques, which involve inserting a nephroscope through the skin (i.e., percutaneously) and intervening tissue to provide access to the treatment site in order to break up and / or remove the stone. Percutaneous access devices (e.g., nephroscopes, sheaths, sheath assemblies, and / or catheters) for providing access to a target anatomical site (and / or directly to an endoscope) may include one or more fluid channels for providing a flush fluid stream to and / or aspirating fluid from the target site (e.g., by passive outflow and / or active suction).

[0056] Robot-assisted ureteroscopy procedures can be implemented in conjunction with various medical procedures (such as kidney stone removal procedures), where robotic tools can enable a physician / urologist to perform endoscopic target access as well as percutaneous access / treatment. Advantageously, aspects of the present disclosure relate to systems, devices, and methods for robotic control of the articulation of a device shaft (e.g., an endoscope shaft) in a manner that reduces the risk of injury or damage to patient anatomy and / or instruments.

[0057] Medical System

[0058] Figure 1An example medical system 100 for performing various medical procedures in accordance with aspects of the present disclosure is illustrated. The medical system 100 can be used, for example, in endoscopic (e.g., ureteroscopic) procedures. As mentioned and described above, a particular ureteroscopic procedure involves the treatment / removal of kidney stones. In some embodiments, the treatment of kidney stones can benefit from the assistance of certain robotics / devices. Compared to a completely manual procedure, a robotic medical solution can provide relatively high precision, superior control, and / or superior hand-eye coordination with respect to a particular instrument. For example, according to some procedures, robotic-assisted ureteroscopic access to the kidney can advantageously enable a urologist to articulate the ureteroscope using a robotically controlled gear / driver coupled to the handle / base portion of the ureteroscope. Although Figure 1 the medical system 100 is presented in the context of a ureteroscopic procedure, it should be understood that the principles disclosed herein can be implemented in any type of endoscopic procedure.

[0059] The medical system 100 includes a robotic system 10 (e.g., a mobile robotic cart) configured to engage and / or control a medical device 19 (e.g., an endoscope / ureteroscope) to perform a direct access procedure on a patient 7, the medical device including a proximal handle / base 31 and a shaft 40 coupled to the handle 31 at its proximal portion. In some cases, the term "medical device" can interchangeably refer to any part of the medical device 19, including the proximal handle / base 31, the shaft 40, the scope, the scope tip, etc. The term "direct access" is used herein in its broad and ordinary sense and can refer to any access by instrumentation through a natural or artificial opening in a patient's body. For example, referring to Figure 1 , the scope / shaft 40 can be directly inserted into the urinary tract of the patient 7 via the urethra 65.

[0060] It should be understood that the direct access device 19 can be any type of shaft-based medical device, including endoscopes (such as ureteroscopes), catheters (such as steerable or non-steerable catheters), nephroscopes, laparoscopes, or other types of medical devices. Embodiments of the present disclosure regarding ureteroscopic procedures for removing kidney stones through a ureteral access sheath (e.g., ureteral access sheath 190) are also applicable to solutions for removing objects through percutaneous access (such as through a percutaneous access sheath). For example, the device can be percutaneously inserted into the kidney through, for example, a percutaneous access sheath to capture and remove kidney stones. The term "percutaneous access" is used herein in its broad and ordinary sense and can refer to any entry of an instrument through a patient's skin and any other body layers necessary to reach a target anatomical location (e.g., the caliceal network of the kidney 70) associated with the procedure, such as through a puncture and / or a small incision.

[0061] The medical system 100 includes a control system 50 that is configured to interact with the robotic system 10, provide information about the procedure, and / or perform a variety of other operations. For example, the control system 50 may include one or more displays 56 that are configured to present certain information to assist the physician 5 and / or other technicians or individuals. The medical system 100 may include a table 15 configured to hold the patient 7. The medical system 100 may also include an electromagnetic (EM) field generator 18 that may be held by one or more of the robotic arms 12 of the robotic system 10, or may be a stand-alone device and / or mounted to the table 15. Although the various robotic arms 12 are shown in various positions and coupled to various tools / devices, it should be understood that such configurations are shown for convenience and illustrative purposes, and such robotic arms may have different configurations over time and / or at different points during the medical procedure. In addition, the robotic arm 12 may be coupled to Figure 1The devices / instruments shown are different devices / instruments, and in some cases or time periods, one or more of these arms may not be utilized or coupled to the medical device. The articulating movement of the shaft 40 can be robotically controlled such as by the operation of an end effector associated with the robotic arm 12a, where such operation can be controlled by the control system 50 and / or the robotic system 10. The term "end effector" is used herein in its broad and ordinary sense and can refer to any type of robotic manipulator device, component, and / or assembly. In a particular implementation where an adapter (such as a sterile adapter) is coupled to the robotic end effector or other robotic manipulator, the term "end effector" can refer to the adapter (e.g., sterile adapter) or any other robotic manipulator device, component, or assembly associated with and / or coupled to the end effector. In some contexts, the combination of the robotic end effector and the adapter can be referred to as an instrument manipulator assembly, where such an assembly may or may not also include a medical device (or instrument handle / base) physically coupled to the adapter and / or end effector. The terms "robotic manipulator" and "robotic manipulator assembly" are used according to their broad and ordinary meanings and can refer to the robotic end effector and / or the sterile adapter or other adapter components (collectively or individually) coupled to the end effector. For example, the terms "robotic manipulator" and "robotic manipulator assembly" can refer to an instrument device manipulator (IDM) that includes one or more drive outputs, whether embodied in the robotic end effector, the sterile adapter, and / or other components. The terms "associated" and "associated with" are used herein according to their broad and ordinary meanings. For example, in a case where a first feature, element, component, device, or member is described as being "associated with" a second feature, element, component, device, or member, such a description should be understood to indicate that the first feature, element, component, device, or member is physically coupled, attached, or connected to the second feature, element, component, device, or member, integrated with the second feature, element, component, device, or member, at least partially embedded within the second feature, element, component, device, or member, or otherwise physically related to the second feature, element, component, device, or member, either directly or indirectly.

[0062] In an example use case, if patient 7 has a kidney stone (or stone fragment) 180 located in kidney 70, physician 5 may perform a procedure to remove stone 180 through the urinary tract (63, 60, 65). In some embodiments, physician 5 may interact with control system 50 and / or robotic system 10 to cause / control robotic system 10 to advance medical instrument shaft 40 (e.g., endoscope) from urethra 65, through bladder 60, up ureter 63, and into renal pelvis 71 and / or calyx network of kidney 70 where stone 180 is located. Physician 5 may further interact with control system 50 and / or robotic system 10 to cause / control basketing device or other instrument to be advanced through working channel of instrument shaft 40 to facilitate capture and removal of kidney stone or stone fragment. Control system 50 may provide information associated with medical instrument 40 and / or other instruments of medical system 100 via display 56, such as real-time endoscopic images captured by the medical instrument, to assist physician 5 in navigating / controlling such instruments.

[0063] The renal anatomy is described herein with reference to specific medical procedures relevant to aspects of the inventive concepts. Figure 1 The kidneys 70, shown in their typical anatomical position in FIG, generally comprise two bean-shaped organs located on the left and right sides of the retroperitoneum. In adult humans, the kidneys are typically approximately 11 cm tall / long. The kidneys receive blood from paired renal arteries 69; blood leaves the kidneys via paired renal veins 67. Each kidney 70 is fluidly coupled to a corresponding ureter 63, which generally comprises a tube that carries secreted urine from the kidney 70 to the bladder 60.

[0064] The kidneys 70 are typically positioned relatively high in the abdominal cavity and at a slightly oblique angle in a retroperitoneal position. The asymmetry within the abdominal cavity, generally caused by the position of the liver, often results in the right kidney (in Figure 1 The left kidney (shown in detail in Figure 1) is slightly lower and smaller than the left kidney and is positioned slightly more medially than the left kidney. Atop each kidney is an adrenal gland (not shown). The upper portion of kidney 70 is partially protected by the 11th and 12th ribs (not shown). Each kidney and its adrenal gland are typically surrounded by two layers of fat: perirenal fat, located between the renal fascia and the renal capsule, and pararenal fat, located above the renal fascia.

[0065] The kidneys 70 help control the various body fluid compartments, fluid osmotic pressure, acid-base balance, various electrolyte concentrations, and the amount of toxins removed. The kidneys 70 provide a filtering function by secreting certain substances and reabsorbing others. Examples of substances secreted into urine are hydrogen, ammonium, potassium, and uric acid. In addition, the kidneys perform various other functions, such as hormone synthesis.

[0066] The recessed area on the concave edge of the kidney 70 is the renal hilum 181, where the renal artery 69 enters the kidney 70 and the renal vein 67 and the ureter 63 leave. The kidney 70 is surrounded by tough fibrous tissue, and the renal capsule 74 itself is surrounded by perirenal fat, renal fascia, and paranephric fat. The anterior (front) surface of these tissues is the peritoneum, and the posterior (back) surface is the transversalis fascia.

[0067] The functional substance or parenchyma of the kidney 70 is divided into two main structures: the outer renal cortex 77 and the inner renal medulla 187. These structures are in the shape of multiple generally conical renal lobes, each conical renal lobe containing renal cortex that surrounds a part of the medulla called the renal pyramid 72. Between the renal pyramids 72 are cortical projections called renal columns 73. The nephron ( Figure 1 (not shown in detail in the figure) (the urine-producing functional structure of the kidney) spans the cortex 77 and the medulla 187. The initial filtering part of the nephron is the renal corpuscle, which is located in the cortex and is followed by renal tubules that extend from the cortex deep into the medullary pyramids. The part of the renal cortex, the medullary rays, is a collection of renal tubules that drain into a single collecting duct.

[0068] The tip / apex or papilla 79 of each renal pyramid empties urine into the corresponding minor calyx 75; the minor calyces 75 empty into the major calyces 76, and the major calyces 76 empty into the renal pelvis 71, which transitions to the ureter 63. The manifold-like collection of minor and major calyces can be referred to herein as the "calyx network" of the kidney. At the renal hilum 181, the ureter 63 and the renal vein 67 leave the kidney and the renal artery 69 enters the kidney. The renal hilum fat and lymphoid tissue with lymph nodes surround these structures. The renal hilum fat abuts a fat-filled cavity called the renal sinus. The renal sinus collectively contains the renal pelvis 71 and the calyces 75, 76 and separates these structures from the renal medullary tissue. The funnel-shaped / tubular anatomical structure associated with the calyx can be called the infundibulum / infundibula. That is, the infundibulum generally leads to the termination of the calyx, where the papilla is exposed within the calyx.

[0069] Referring further to medical system 100, a medical device shaft 40 (e.g., an endoscope, a direct access device, etc.) can be advanced through the urinary tract into the kidney 70. Specifically, a ureteral access sheath 190 can be positioned within the urinary tract in an area near the kidney 70. The shaft 40 can pass through the ureteral access sheath 190 to access the internal anatomy of the kidney 70, as shown. The distal portion of the endoscope / shaft 40 deployed from the sheath 190 can be articulable to allow a surgeon 5 to use an input of a control device 55 to articulate the robot system 10 to direct the shaft 40 toward a target kidney stone. Once at the site of the kidney stone 180 (e.g., within a target calyx 75 of the kidney 70 through which the stone 180 can be accessed), a basket device can be delivered / guided to the target location using the medical device 19 and / or its shaft 40. Once the stone 180 has been captured in the distal basket portion of the basket device / component, the kidney stone 180 can be extracted from the patient 7 using the ureteral access path utilized.

[0070] The various endoscope / shaft type instruments disclosed herein (such as the shaft 40 of medical system 100) can be configured to navigate within the human anatomy, such as within a natural orifice or lumen of the human anatomy. The terms "endoscope" and "endoscopy" are used herein in their broad and ordinary sense and can refer to any type of elongated (e.g., shaft-type) medical device having image generation, viewing, and / or capture capabilities and configured to be introduced into any type of organ, cavity, lumen, chamber, or space of the body. An endoscope can include, for example, a ureteroscope (e.g., for accessing the urinary tract), a laparoscope, a nephroscope (e.g., for accessing the kidney), a bronchoscope (e.g., for accessing the airways, such as the bronchi), a colonoscope (e.g., for accessing the colon), an arthroscope (e.g., for accessing a joint), a cystoscope (e.g., for accessing the bladder), a colonoscope (e.g., for accessing the colon and / or rectum), a ureteroscope, etc. In some cases, an endoscope / endoscopy can include at least partially rigid and / or flexible tubing and can be sized to pass within an outer sheath, catheter, introducer, or other lumen-type device, or can be used without such a device.

[0071] Figure 2Illustrated is a cart-based robotic system 101 arranged for diagnostic and / or therapeutic bronchoscopy. During bronchoscopy, the arm 12 of the robotic system 10 may be configured to drive a medical device shaft 40, such as a steerable endoscope (which may be a procedure-specific bronchoscope for bronchoscopy), through a natural aperture entry point (e.g., the mouth of patient 7 positioned on table 15 in this example) to deliver diagnostic, treatment tools, and / or therapy. Depending on the medical procedure being performed, the robotic arm 12 may include more or fewer arms. As shown, the robotic system 10 (e.g., the cart) may be positioned close to the upper torso of the patient to provide access to the entry point. Similarly, the robotic arm 12 may be actuated to position the bronchoscope / shaft 40 relative to the entry point. When performing a GI procedure using a gastroscope (a dedicated endoscope for gastrointestinal (GI) procedures), the arrangement in Figure 2 may also be utilized.

[0072] Once the robotic system 10 is properly positioned, the robotic arm 12 may insert the steerable / articulating endoscope 40 into the patient robotically, manually, or in combination thereof. The endoscope 40 may be advanced within an outer sheath 190, where each of the endoscope 40 and the sheath 190 may be coupled to and / or associated with one of a set of instrument feeders and / or instrument handles 11, and each instrument feeder / handle 11 is coupled to the distal end of a respective robotic arm 12. This linear arrangement of the feeders / handles 11 may form a "virtual track" 104 that may be repositioned in space by manipulating one or more robotic arms 12 to different angles and / or positions. One or more of the instrument feeders / handles 11 may be configured to effect robotic articulation of the shaft 40 and may be configured for this purpose in accordance with one or more of the embodiments disclosed herein.

[0073] After insertion, the endoscope 40 may be directed down into the patient's trachea and lungs using precise articulation commands from the robotic system 10 until the target surgical site is reached. For example, the endoscope 40 may be guided to deliver a biopsy needle to a target, such as a lesion or nodule within the patient's lung. The needle may be deployed down the working channel that extends the length of the endoscope to obtain a tissue sample to be analyzed by a pathologist. Depending on the pathology results, additional tools may be deployed down the working channel of the endoscope for additional biopsies. For example, when a nodule is identified as malignant, the endoscope 40 may deliver tools through the endoscope to excise the potential cancerous tissue. In some cases, the diagnostic and treatment procedures may be delivered in separate procedures. In these cases, the endoscope 40 may also be used to deliver fiducial markers to "mark" the location of the target nodule.

[0074] In the robotic system 101, the patient guide 102 is attached to the patient 7 via a port (not shown; e.g., a surgical tube). The curvature of the patient guide 102 enables the robotic system 10 to manipulate the instrument 40 from a position that is not directly axially aligned with the patient access port, thereby allowing for more flexible placement of the robotic system 10 within the room. Additionally, the curvature of the patient guide 102 may allow the robotic arm 12 of the robotic system 10 to be substantially horizontally aligned with the patient guide 102, which may facilitate manual movement of the robotic arm 12 (if desired). The control system 50 and / or the robotic cart 10 may include control circuitry configured to implement endoscopic joint motion control as described herein.

[0075] For reference, Figure 2 Details of certain respiratory anatomical structures in which the endoscope 40 can be advanced and / or the endoscope can be articulated are shown. Generally speaking, the respiratory system includes certain passageways, blood vessels, organs, and muscles that assist the body in gas exchange between air and blood and between blood and body cells. The respiratory system includes the upper respiratory tract, which includes the nose / nasal cavity, pharynx (i.e., throat), and larynx (i.e., voice box). The respiratory system also includes the lower respiratory tract, which is shown in detail and includes the trachea 6, the lungs 4, and the various segments of the bronchial tree 30, which include alveoli and alveolar ducts, and the alveoli and alveolar ducts include clusters of small air sacs responsible for gas exchange between the lungs and the pulmonary blood vessels. The bronchial tree 30 is an example network of lumens in which a robotically controlled instrument can be navigated and articulated according to the inventive solutions presented herein. However, although aspects of the present disclosure are presented in the context of a network of lumens of the bronchial network of the airway (e.g., lumens, branches) of the patient's lungs, embodiments of the present disclosure can be implemented in other types of lumen networks, such as renal networks, cardiovascular networks (e.g., arteries and veins), gastrointestinal tracts, urinary tracts, etc. The organs of the lower respiratory tract are located inside the chest cavity and are surrounded by the sternum (i.e., breastbone) and the rib cage in front and the vertebrae (i.e., spinal column) in the back, which together protect the lungs and other organs in the chest.

[0076] The trachea 6 is located directly below the larynx 5 and provides the main airway to the lungs 4. The left lung 4 l and the right lung 4 rResponsible for supplying oxygen to the capillaries and exhaling carbon dioxide. The bronchi 7 branch from the trachea 6 into each lung 4 and form a network of complex passages that supply air to the lungs 4. The diaphragm is the main respiratory muscle that contracts and relaxes to allow air to enter the lungs. The trachea 6 is the tube that brings air into and out of the lungs 4. Each lung 4 has tubes 7 called bronchi that are associated with it and are connected to the trachea. The trachea and bronchi form the bronchial tree 30. The bronchial tree 30 includes the main bronchi 81, which branch into smaller secondary bronchi 88 and tertiary bronchi 85, and terminate in even smaller tubes called bronchioles 87. Each bronchiole tube is coupled to a cluster of alveoli. During the inhalation phase of the respiratory cycle, air enters through the mouth and nose and travels down the larynx into the trachea 6, enters the lungs 4 through the right and left main bronchi 81, enters the smaller bronchial airways 88, 85, enters the smaller bronchiole tubes 87, and enters the alveoli, where oxygen and carbon dioxide exchange occurs.

[0077] Lung cancer and other cancers generally involve abnormal cell growth (e.g., in regions of the lung or other anatomical structures), which can have the potential to invade or spread to other parts of the body. For example, cancer can form in the tissues of the lung, such as in cells lining the various air passages. When not treated in an effective and / or timely manner, lung cancer can spread / metastasize to lymph nodes or other organs within the body, which can severely impact the patient's recovery prospects. In Figure 2 the patient 7 is shown as having a mass of tissue 89 that has formed in a region of the lung 4, called a lung nodule. Such lung nodules can be benign or cancerous. Determining whether a lung nodule is cancerous can involve using one or more anatomical imaging modalities and / or minimally invasive lung biopsies in combination with certain thoracoscopic, bronchoscopic, and / or robotic procedures. For example, robotically controlled instruments can be implemented to perform diagnostic biopsy procedures from within the bronchial network.

[0078] In the illustrated example, the medical device 19 includes an endoscope 40. The endoscope 40 is slidably positioned within the working channel of the sheath 190. The endoscope 40 may have a lumen (i.e., a "working channel") through which instruments such as biopsy needles and / or injection needles, cytology brushes, and / or tissue sampling forceps may be passed to the target tissue site of the nodule 89. The terms "lumen" and "channel" are used herein in their broad and ordinary sense and may refer to a physical structure that forms a cavity, void, conduit, or other passageway, such as an at least partially rigid, elongated tubular structure, or may refer to a cavity, void, passageway, or other channel itself that occupies the space within an elongated structure (e.g., a tubular structure). Thus, with respect to an elongated tubular structure (such as a shaft, tube, etc.), the term "lumen" or "channel" may refer to the elongated tubular structure and / or to the channel or space within the elongated tubular structure. The telescoping arrangement of the sheath 190 and the endoscope 40 may allow for a relatively thinner design of the endoscope 40 and may improve the bending radius of the endoscope 40 while providing structural support via the sheath 190. As shown, to reach the nodule 89, the endoscope shaft 40 may be navigated or guided through the lumen or branches of the lumen network 7. An operator (such as a surgeon) may use various advancement and articulation movement commands to navigate the instrument 40 to the nodule 89.

[0079] Figure 3 A table-based robotic system 103 is illustrated in accordance with one or more embodiments of the present disclosure. The system 103 combines robotic components 105 with a table / platform 147, thereby allowing for a reduced amount of capital equipment within an operating room compared to some cart-based robotic systems, which in some cases may allow for greater access to the patient 7. Very similar to cart-based systems, the instrument device manipulator assemblies associated with one or more robotic arms 212a - 212c of the system 103 generally may include instruments and / or instrument feeders that are designed to manipulate an elongated medical device / shaft along a virtual track or path, such as the endoscope 40 and the like.

[0080] As shown, the robot-enabled table system 103 can include a column 144 coupled to one or more carriers 141 (e.g., an annular movable structure), from which robotic arms 212a - 212c can extend. The carrier 141 can translate along a vertical column interface extending along at least a portion of the length of the column 144 to provide different vantage points from which the robotic arms 212a - 212c can be positioned to reach the patient 7. In some embodiments, the carrier 141 can rotate about the column 144 using a mechanical motor positioned within the column 144 to allow the robotic arms 212a - 212c to access multiple sides of the table / platform 147. The rotation and / or translation of the carrier 141 can allow the system 103 to align medical devices such as the endoscope 40 and the sheath 190 to different entry points on the patient 7. By providing vertical adjustment, the robotic arms 212a - 212c can be advantageously configured to be stored compactly beneath the table / platform 147 of the table system 103 and then raised during the procedure.

[0081] The robotic arms 212a - 212c can be mounted on the carrier 141 via one or more arm supports 145, which can include a series of joints that can rotate individually and / or extend telescopically to provide additional configurability to the robotic arms 212a to 212c. The column 144 structurally supports the table / platform 147 and provides a path for the vertical translation of the carrier 141. The column 144 can also transmit power and control signals to the carrier 141 and / or the robotic arms 212a - 212c mounted thereon. The system 103 can include certain control circuits configured to control the drive and / or joint movement of the instrument shaft 40 using an end effector of one of the robotic arms 212a - 212c. Although not shown for visual clarity in Figure 3 the control tower / system, it should be understood that the system 103 can have a control tower / system as in any of the embodiments disclosed herein.

[0082] Figure 4 is shown Figures 1 to 3Example implementation of a control system for any of the above. The associated control system 50 may be configured to provide various functions to assist in performing a medical procedure. In some implementations, the control system 50 may be coupled to the robotic system 10 and operate in cooperation with the robotic system to perform a medical procedure on the patient 7. For example, the control system 50 may communicate with the robotic system 10 via a wireless connection or a wired connection (e.g., to control the robotic system 10). Additionally, in some implementations, the control system 50 may communicate with the robotic system 10 to receive position data related to the position of the distal end of the endoscope 40. Such position data related to the position of the endoscope 40 may be derived using one or more electromagnetic sensors associated with the corresponding components, endoscope image processing functionality, and / or at least partially based on robotic system data (e.g., arm position data, known parameters / dimensions of various system components, etc.).

[0083] Figure 4 Also shown is Figures 1 to 3 Example implementation of a robotic system for any of the above. The robotic system 10 may be configured to at least partially facilitate the performance of a medical procedure. The robotic system 10 may be arranged in a variety of ways, depending on the particular procedure. The robotic system 10 may include one or more robotic arms 12, which are configured to engage and / or control, for example, an endoscope 40 to perform one or more aspects of the procedure. As shown, each robotic arm 12 may include a plurality of arm segments 23 coupled to joints 24, which may provide multiple degrees of mobility / freedom. When the robotic system 10 is properly positioned, the endoscope 40 may be inserted into the patient 7 robotically, manually by the physician 5, or a combination thereof using the robotic arms 12. Referring Figure 1 to, the endoscope driver / feeder instrument coupler 11 (i.e., instrument device manipulator (IDM)) may be attached to the distal end actuator 22 of one of the arms 12b to facilitate robotic control / advancement of the endoscope 40. The other arm 12a of the arms may have an instrument base / handle 31 associated therewith, where the endoscope 40 is physically coupled to the handle 31 at the proximal end of the endoscope 40. The endoscope 40 may include one or more working channels 44 through which additional tools, such as a lithotripter, a basket device, forceps, etc., may be introduced into the treatment site.

[0084] Referring Figures 1 to 4In any system within the system, the robotic system 10 can be coupled to any component of the medical system, such as the control system 50, the table 15, the EM field generator 18, the endoscope 40, and / or any type of percutaneous access device (e.g., needle, catheter, nephroscope, etc.). In some embodiments, the robotic system 10 is communicatively coupled to the control system 50. For example, the robotic system 10 can be configured to receive control signals from the control system 50 to perform certain operations, such as positioning one or more of the robotic arms 12 in the robotic arm 12 in a specific manner, manipulating (e.g., advancing, articulating) the endoscope 40, etc. In response, the robotic system 10 can use certain control circuits 211 to control the actuators 217 and / or other components of the robotic system 10 to perform the operations. For example, the control circuit 211 can control the articulation of the shaft / endoscope 40 by actuating the drive output of the end effector 22 coupled to the instrument handle 31. In some embodiments, the robotic system 10 and / or the control system 50 are configured to receive images and / or image data from the endoscope 40, and these images and / or image data represent the internal anatomy of the patient 7 and / or parts of the access sheath or other device components.

[0085] The robotic system 10 generally includes an elongate support structure 14 (also referred to as a "column"), a robotic system base 25, and a console 13 at the top of the column 14. The column 14 can include one or more arm supports 17 (also referred to as "carriages") for supporting the deployment of one or more robotic arms 12 (three are shown in Figure 1 and Figure 2 . The arm support 17 can include individually configurable arm mounts that rotate along a vertical axis to adjust the base of the robotic arm 12 to be positioned as desired relative to the patient.

[0086] The arm support 17 can be configured to translate vertically along the column 14. In some embodiments, the arm support 17 is connected to the column 14 by slots 20 that are positioned on opposite sides of the column 14 to guide the vertical translation of the arm support 17. The slots 20 include a vertical translation interface to position and hold the arm support 17 relative to the robotic system base 25 at various vertical heights. The vertical translation of the arm support 17 allows the robotic system 10 to adjust the reach of the robotic arms 12 to accommodate a variety of table heights, patient body shapes, and physician preferences. Similarly, the individually configurable arm mounts on the arm support 17 can allow the robotic arm bases 21 of the robotic arms 12 to be angled in a variety of configurations.

[0087] The robotic arm 12 generally may include a robotic arm base 21 and an end effector 22 separated by a series of link arm segments 23, which are connected by a series of joints 24, and each joint 24 includes one or more independent actuators 217. Each actuator may include a separately controllable motor. Each separately controllable joint 24 may provide or represent a separate degree of freedom available to the robotic arm. In some embodiments, each arm in the arm 12 has seven joints and thus provides seven degrees of freedom, including "redundant" degrees of freedom. The redundant degrees of freedom allow the robotic arm 12 to position its corresponding end effector 22 at a particular position, orientation, and trajectory in space using different link mechanisms and joint angles. This allows the system to position and guide a medical device from a desired point in space while allowing the physician 5 to move the arm joints to a clinically advantageous position away from the patient to create greater access while avoiding arm collisions.

[0088] The robotic system base 25 balances the weight of the column 14, the arm support 17, and the arm 12 on the floor. Thus, the robotic system base 25 may house certain relatively heavy components, such as electronics, motors, power supplies, and components that selectively enable the robotic system to move and / or be fixed. For example, the robotic system base 25 includes wheeled casters 28 that may allow the robotic system to be easily moved around an operating room prior to a procedure. After reaching the appropriate position, the casters 28 may be locked in place using wheel locks to hold the robotic system 10 in place during the procedure.

[0089] The console 13 positioned at the upper end of the column 14 may provide both a user interface for receiving user input and a display screen 16 (or a dual-purpose device, such as, for example, a touch screen) for providing preoperative and intraoperative data to the physician / user 5. Potential preoperative data on the console / display (e.g., Figure 1 and Figure 2 the display screen 16) or the display 56 may include preoperative planning, navigation, and mapping data derived from preoperative computed tomography (CT) scans and / or records from preoperative patient interviews. Intraoperative data on the display may include optical information provided from tools, sensors, and coordinate information from sensors, as well as vital patient statistics, such as respiration, heart rate, and / or pulse. The console 13 may be positioned and tilted to allow the physician 5 to access the console from the side of the column 14 opposite the arm support 17. From this position, the physician 5 may view the console 13, the robotic arm 12, and the patient while operating the console 13 from behind the robotic system 10. As shown, the console 13 may also include a handle 27 for assisting in maneuvering and stabilizing the robotic system 10.

[0090] The end effector 22 of each robotic arm in the robotic arm 12 may include an instrument device manipulator (IDM) (e.g., an instrument base / handle) 11 or may be configured to couple the IDM thereto, and the IDM may be attached using a sterile adapter component in some cases. The combination of the end effector 22 and the associated IDM and any intervening mechanism or coupling (e.g., a sterile adapter) may be referred to as a manipulator assembly. In some embodiments, the IDM 11 may be removed and replaced with a different type of IDM. For example, a first type of IDM / instrument may be configured to manipulate an endoscope / shaft, while a second type of IDM / instrument 31 may be associated with (e.g., coupled to its proximal portion) the shaft 40 and configured to articulate the shaft. Another type of IDM / instrument may be configured to hold the electromagnetic field generator 18. The IDM may provide a power and control interface. For example, the interface may include connectors for transferring pneumatic pressure, power, electrical signals, and / or optical signals from the robotic arm 12 to the IDM 11. The IDM 11 may be configured to manipulate a medical device (e.g., a surgical tool / instrument), such as an endoscope 40, using techniques including, for example, direct drive, harmonic drive, gear drive, belt and pulley, magnetic drive, etc. In some embodiments, the device manipulator 11 may be attached to a corresponding robotic arm in the robotic arm 12, where the robotic arm 12 is configured to insert the corresponding coupled medical device into a treatment site or retract it outward from the treatment site.

[0091] As referenced above, Figures 1 to 4 The system may include specific control circuits configured to perform the specific functionality described herein, including the control circuit 211 of the robotic system 10 and the control circuit 251 of the control system 50. That is, the control circuits of the systems 100, 101, 103, 400 may be part of the robotic system 10, the control system 50, or some combination thereof. Thus, any reference herein to a control circuit may refer to that embodied in the robotic system, the control system, or a medical system (such as respectively in Figures 1 to 4The circuitry in any other component of the systems 100, 101, 103, and 400 shown. The term "control circuit" is used herein in its broad and ordinary meaning and can refer to any collection of the following: processors, processing circuitry, processing modules / units, chips, dies (e.g., semiconductor dies including one or more active and / or passive devices and / or connectivity circuitry), microprocessors, microcontrollers, digital signal processors, microcomputers, central processing units, field programmable gate arrays, programmable logic devices, state machines (e.g., hardware state machines), logic circuitry, analog circuitry, digital circuitry, and / or any device that manipulates signals based on circuitry and / or hard-coded operating instructions. The control circuits mentioned herein may also include one or more circuit substrates (e.g., printed circuit boards), conductive traces and vias, and / or mounting pads, connectors, and / or components. The control circuits mentioned herein may also include one or more storage devices, which may be embodied in a single memory device, multiple memory devices, and / or embedded circuitry of a device. Such data storage devices may include read-only memory, random access memory, volatile memory, non-volatile memory, static memory, dynamic memory, flash memory, cache, data storage registers, and / or any device that stores digital information. It should be noted that in embodiments where the control circuit includes hardware and / or software state machines, analog circuitry, digital circuitry, and / or logic circuitry, the data storage device / register storing any associated operating instructions may be embedded within or external to the circuitry including the state machine, the analog circuitry, the digital circuitry, and / or the logic circuitry.

[0092] The control circuits 211, 251 may include a computer-readable medium that stores and / or is configured to store hard-coded and / or operating instructions that correspond to at least some of the steps and / or functions shown in one or more of the figures herein and / or described herein. In some cases, such computer-readable media may be included in an article of manufacture. The control circuits 211, 251 may be maintained / set entirely locally or may be at least partially remotely located (e.g., communicatively coupled indirectly via a local area network and / or a wide area network). Either of the control circuits 211, 251 may be configured to perform any aspect of the various processes disclosed herein, including the processes shown in FIGS. 11 and 13, as described below.

[0093] Regarding the robotic system 10, at least a portion of the control circuit 211 may be integrated with the base 25, column 14, and / or console 13 of the robotic system 10 and / or integrated with another system communicatively coupled to the robotic system 10. Regarding the control system 50, at least a portion of the control circuit 251 may be integrated with the console base 51 and / or display unit 56 of the control system 50. It should be understood that any description herein of a functional control circuit or associated functionality may be understood to be embodied in the robotic system 10, control system 50, or any combination thereof and / or at least partially embodied in one or more other local or remote systems / devices, such as a control circuit associated with the handle / base of a shaft-type instrument (e.g., an endoscope) according to any of the disclosed embodiments.

[0094] The control circuit 211 and / or the control circuit 251 may be communicatively coupled to one or more torque sensors 216 configured to generate signals indicative of torque on one or more actuators of the robotic system 10. The torque sensors 216 may have any suitable or desired configuration. For example, the torque sensors 216 may act as sensing mounting structures or load cells. In some embodiments, the torque sensors 216 are configured as reaction torque sensors that use one or more self-contained strain gauges to measure the strain caused by torque to produce a load cell. Although the torque sensors 216 of the robotic system are described herein in the context of determining the tension on the wire / cable of an endoscope instrument coupled to the robotic system 10, such reference may be understood to represent any type of sensor or sensing mechanism configured to generate a signal indicative of the tension of the wire / cable, such as a strain gauge, etc. The reference herein to a strain gauge may be any type of sensor configured to measure the force / load on a robotic actuator, whether such force is rotational or linear in nature. That is, although a rotary robotic output drive is disclosed in some contexts herein, it should be understood that the inventive concepts disclosed herein apply to other types of actuators, such as linear drives.

[0095] Further reference Figures 1 to 4, the control system 50 may include various I / O components 258 that are configured to assist the physician 5 or others in performing a medical procedure. For example, the input / output (I / O) components 258 may be configured to allow a user to input control / navigation of the endoscope 40 and / or other robotic control devices (e.g., a basket system) within the patient 7. In some embodiments, for example, the physician 5 may use one or more input controls 255 to provide input to the control system 50 and / or the robotic system 10, and in response to such input, a control signal may be sent to the robotic system 10 to manipulate the endoscope 40. The control system 50 may include one or more display devices 56 to provide various information about the procedure. For example, the display 56 may provide information about the endoscope 40. For example, the control system 50 may receive real-time images captured by the endoscope 40 and display these real-time images via the display 56. Additionally or alternatively, the control system 50 may receive signals (e.g., analog signals, digital signals, electrical signals, acoustic / acoustic signals, pneumatic signals, tactile signals, hydraulic signals, etc.) from medical monitors and / or sensors associated with the patient 7, and the display 56 may present information about the health or environment of the patient 7. Such information may include information displayed via medical monitors, including, for example, information related to heart rate (e.g., ECG, HRV, etc.), blood pressure / blood rate, muscle biosignals (e.g., EMG), body temperature, blood oxygen saturation (e.g., SpO2), CO2, brain waves (e.g., EEG), environment, and / or local or core body temperature, etc. Similarly, the robotic system 10 may include various I / O components 218 that are integrated on the console 13 and are configured to assist the physician 5 or others in setting up the robotic system 10.

[0096] Figures 1 to 4 Various components of the system may be communicatively coupled to each other via a network, which may include a wireless network and / or a wired network. Example networks include one or more personal area networks (PAN), local area networks (LAN), wide area networks (WAN), internet area networks (IAN), cellular networks, the internet, personal area networks (PAN), body area networks (BAN), etc. For example, Figures 1 to 4 the various communication interfaces 214, 254 of the system may be configured to communicate with one or more devices / sensors / systems, such as via a wireless network connection and / or a wired network connection. In some embodiments, the various communication interfaces 214, 254 may implement wireless technologies such as Bluetooth, Wi-Fi, near field communication (NFC), etc. Additionally, in some embodiments, the various components of the system may be connected via one or more support cables, tubes, etc. for data communication, fluid exchange, power exchange, etc.

[0097] The control system 50 and / or the robotic system 10 may include certain user controls (e.g., control 55), which may include any type of user input (and / or output) device or device interface, such as one or more buttons, keys, joysticks, hand-held controllers (e.g., video game type controllers), computer mice, touch pads, trackballs, control pads, and / or sensors that capture gestures and finger postures (e.g., motion sensors or cameras), touch screens, and / or interfaces / connectors therefor. Such user controls are communicatively and / or physically coupled to corresponding control circuits. In some embodiments, a user may engage the user control 55 to command robotic axis joint movement as described herein. Additionally, the control system 50 and / or the robotic system 10 may include one or more power interfaces 219, 259 configured to supply power.

[0098] Figure 4 Further shown are details of an example articulable scope assembly / instrument 19 that can be in line of sight with any of the embodiments of the present disclosure. In some embodiments, the scope assembly 19 includes a handle or base 31 coupled to an endoscope / shaft 40. For example, the endoscope (i.e., “scope” or “shaft”) may include an elongate shaft that includes one or more lights 49 and one or more cameras or other imaging devices such as camera 48. In some implementations, the imaging device may be a separate tool external to the scope assembly 19, or may be a tool releasably attached to or slidable within the scope 40. The scope 40 may also include one or more working channels 44 that extend the length of the scope 40. The scope assembly 19 may be powered and / or controlled via a power interface 39 and a control interface 38, each or both of which may interface with a robotic arm / component of the robotic system 10. The scope assembly 19 may also include one or more sensors 32, such as pressure sensors and / or other force sensing sensors, configured to generate signals indicative of forces at and / or experienced by one or more components of the scope assembly 19.

[0099] The scope assembly 19 includes certain mechanisms for articulating / deflecting the shaft 40 relative to its axis. For example, the shaft 40 may already be associated with its proximal portion, with one or more drive inputs 34 associated with and / or integrated with one or more pulleys / spools 33 configured to tension / untension a pull wire 45 of the endoscope shaft 40 to effect articulation of the shaft 40. The terms “untension” and “de-tension” are used herein in their broad and ordinary sense and may refer to a reduction in the tension of a wire, cable, line, or the like, and such terms may be used interchangeably.

[0100] Figure 5 An endoscope 40 that is coupled to a robotic end effector and capable of robotic joint motion in accordance with one or more embodiments is illustrated. Figure 6 An exploded view of an instrument device manipulator assembly 150 associated with a robotic arm 12 in accordance with one or more embodiments is shown. Robotic endoscope control can provide relatively greater precision, control, and / or coordination as compared to strictly manual procedures. The following description may be understood in the context of Figure 5 and Figure 6 and other embodiments presented herein.

[0101] The scope / shaft (e.g., endoscope / ureteroscope) 40 may include a tubular and flexible medical shaft / instrument that is configured to be inserted into a patient's anatomy to capture an image of the anatomy and perform a particular task using one or more of its working channels. In some embodiments, the scope 40 may house wires and / or optical fibers to transmit signals to / from an optical assembly at the distal end 42 of the scope 40, which may include one or more imaging devices 48, such as an optical camera. The scope 40 may also include one or more light sources 49, such as an LED or fiber optic light source / lens.

[0102] The scope 40 may be capable of joint motion relative to the distal portion 42 of the scope 40 such that the scope 40 can be steered within the human anatomy. In some embodiments, the scope 40 is configured to joint motion in, for example, six degrees of freedom, including XYZ coordinate movement, as well as pitch, yaw, and roll. A position sensor (e.g., an electromagnetic sensor) (where implemented) of the scope 40 may similarly have a similar degree of freedom with respect to the position information generated / provided by the position sensor.

[0103] For a robotic implementation, the robotic arm / rail 12 of the robotic system can be configured to / capable of being configured to manipulate the endoscope 40. For example, an instrument device manipulator (e.g., an endoscope handle) 31 can be coupled to the end effector 22 of the robotic arm / rail 12 and can use an elongate movement member to manipulate the endoscope 40. The elongate movement member can include one or more wire ropes (e.g., wire ropes or push wires), cables, tendons, fibers, and / or flexible shafts. For example, the robotic end effector can be configured to actuate a plurality of wire ropes (not shown) coupled to the endoscope 40 to deflect the distal end 42 of the endoscope 40. The wire ropes can comprise any suitable or desired material, such as metallic and non-metallic materials, such as stainless steel, Kevlar, tungsten, carbon fiber, etc. In some embodiments, the endoscope 40 is configured to exhibit non-linear behavior in response to forces applied by the elongate movement member. The non-linear behavior can be based on the stiffness and compression coefficient of the endoscope, as well as the variability of the slack or stiffness between different elongate movement members. The robotic arm 12 can include one or more hinges 382 and / or joints that are configured to allow the distal portion 384 of the robotic arm 12 to extend in various directions and / or at various angles.

[0104] The endoscope 40 can further be configured to house optical fibers to carry light from a proximal light source (such as a light-emitting diode) to the distal end 42 of the endoscope. In some embodiments, the endoscope 40 is configured to be controlled by a robotic system that is similar in one or more respects to the systems 100, 101, 103, and 400 respectively shown in Figures 1 to 4 .

[0105] In some embodiments, the shaft (e.g., endoscope) 40 includes a sensor that is configured to generate sensor position data and / or send the sensor position data to another device or produce a detectable distortion or feature in an electromagnetic field. The sensor position data can indicate the position and / or orientation of the medical device 40 (e.g., its distal end 42), and / or can be used to determine / infer the position / orientation of the medical device. For example, the sensor (sometimes referred to as a “position sensor”) can include other forms / embodiments of electromagnetic (EM) sensors having coils or antennas of conductive material.

[0106] The instrument base / handle 31 can be configured to attach, mount, or otherwise connect or couple to the robotic end effector 22. For example, a robotic arm can include an instrument drive mechanism / component 150 that includes the end effector 22 and / or the sterile adapter 8, and an instrument base / handle 31 attached to the end effector 22 and / or the adapter 8. The instrument drive mechanism can include drive outputs 302, 309 that are configured to engage and actuate corresponding drive inputs 602 on the instrument base / handle 31 to manipulate the medical device 19. For example, one or more drive outputs 302 of the robotic end effector 22 can be configured to control articulation motion, as described in detail herein. The drive output 302 of the end effector 22 can be coupled to one or more drive couplers of an adapter (e.g., a sterile adapter), and the one or more drive couplers are configured to transfer drive torque from the drive output 302 of the end effector 22 to the drive output 309 of the adapter 8. References herein to the robotic end effector and / or its drive outputs or other features can be understood to refer to an adapter (e.g., a sterile adapter) coupled to the end effector and / or the drive outputs of the adapter. For example, a description of an instrument docking on the end effector should be understood to refer to docking the instrument on the adapter when the adapter is coupled to the end effector.

[0107] In some configurations, the elongate shaft 40 of the medical device 19 is arranged to form a service loop 43 between the instrument handle 31 and the instrument feeder 11 and / or between associated robotic arms. The service loop 43 can include a length of the shaft 40 between the instrument base / handle 31 and the feeder device 11. The service loop 43 can provide slack in the shaft 40, which can be used to allow for faster insertion and / or retraction of the shaft 40. For example, during insertion, the slack in the service loop 49 can be taken up (shortened or contracted the service loop 49). During retraction, the service loop 49 can be generated (increased or expanded in length).

[0108] The endoscope 40 can be capable of deflecting in one or two directions within a first / principal plane P p The endoscope 40 can also be capable of deflecting in one or two directions within a second / secondary plane P s which can be orthogonal to the principal plane P p For example, it may be desirable for at least the distal segment 42 of the endoscope 40 to be capable of deflecting in more than one plane to reach a desired region. Although the principal P p and secondary P s deflection planes are shown in a particular configuration, it should be understood that the secondary plane P s shown can be the principal plane P p and vice versa.

[0109] In some embodiments, one or more cables, tendons, wires, or wire segments may extend along the length of the shaft 40. Manipulation / tensioning of one or more wires causes actuation or deflection of the distal segment 42 of the endoscope 40. Manipulation / tensioning of one or more wires may be controlled via one or more instrument drivers / pulleys located within or attached to the instrument base / handle 31.

[0110] The instrument base / handle 31 may generally include an attachment interface having one or more mechanical drive inputs 602 (e.g., receivers, pulleys, spools, female inputs, etc.) designed to reciprocally mate with one or more torque couplers on an attachment surface of an instrument driver. The instrument handle 31 may include a plurality of drive inputs 602, each associated with a respective wire articulation pulley. Multiple wires may be coupled to the plurality of drive inputs 602 (and corresponding pulleys) and extend along the flexible shaft 40. The plurality of drive inputs 602 may be configured to control multiple wires or apply tension to multiple wires in response to rotation of the drive output 302 of the coupled robotic system.

[0111] For the endoscope 40 to navigate through an anatomical structure, the articulating segment of the endoscope 40 may be capable of deflecting in the primary plane P p The distal segment of the articulating segment may also be capable of deflecting in two directions in the secondary plane P s Thus, the distal portion 42 of the articulating segment of the endoscope 40 may be capable of deflecting in two planes and in four directions (e.g., left / right and up / down). The bending radius of the endoscope 40 may be greater in the primary plane P s e.g., up to 270° or greater in either direction) compared to in the secondary plane P p e.g., 180° or less in either direction).

[0112] In the instrument device manipulator assembly 150 (see Figure 6)In embodiments including adapter component 8, adapter 8 may be capable of being mounted to end effector 22 and configured to provide a drive interface between end effector 22 and instrument handle 31. In some embodiments, adapter 8 and / or instrument handle 31 may be capable of being removed or detached from robotic arm 12 and may be free of any electromechanical components, such as motors. This split design may be driven by the need to sterilize medical devices used in medical procedures; and the inability to adequately sterilize expensive capital equipment due to the complex mechanical components and sensitive electronics of the expensive capital equipment. Thus, instrument handle 31 and / or adapter 8 may be designed to be detached, removed, and interchanged from end effector 22 (and thus from the system) for separate sterilization or disposal. In contrast, end effector 22 may not need to be altered or sterilized in some cases and may be covered (e.g., using drape 301) for protection. Drape 301 may be coupled to adapter 8 in a manner that allows mechanical torque to be translated from end effector 22 to adapter 8. Adapter 8 generally may be configured to maintain a seal around its actuating components such that adapter 8 itself provides a sterile barrier. In the case where arm 12 is covered in plastic, a physician and / or other technician may interact with arm 12 and / or other components of the robotic cart (e.g., the screen) during the procedure. The covering may also prevent contamination of the device with biological hazards and / or minimize cleanup after the procedure.

[0113] In some embodiments, the adapter 8 may include connectors for transferring pneumatic pressure, power, electrical signals, and / or optical signals from the robotic arm 12 and / or the end effector 22 to the instrument handle 31. The robotic arm 12 may advance / insert the coupled instrument handle 31 into or retract it from the treatment site. In some embodiments, the instrument handle 31 may be removable and replaced with a different type of instrument. The end effector 22 of the robotic arm 12 may include various components / elements configured to connect to, and / or be aligned with, the adapter 8, the instrument handle 31, and / or the shaft 40. For example, the end effector 22 may include a drive output 302 (e.g., a drive spline, gear, or rotatable disk with engagement features) for controlling the articulation of a medical device, a reader 304 (e.g., a radio frequency identification (RFID) reader for reading a serial number from a medical device) for reading data from the medical device 31, one or more fasteners 306 for attaching the instrument handle 31 and / or the adapter 8 to the end effector 22, and a marker 308 for assisting in instrument alignment and / or defining the front surface of the device manipulator assembly 150. In some embodiments, a portion (e.g., a plate) 315 of the adapter 8 may be configured to rotate / spin independently of the adapter 8 and / or one or more other components of the end effector 22 when coupled to the end effector 22. The adapter 8 may be configured to release from the end effector 22 via a release protrusion 303 and / or a similar mechanism.

[0114] The instrument handle 31 may include a plurality of drive inputs 602 on the surface 336 of the housing 80 of the instrument handle 31. In the illustrated embodiment, the instrument handle 31 includes two drive inputs 602, but in other embodiments may include other numbers of drive inputs. The drive inputs may be at fixed positions spaced apart along the mating surface 336 of the instrument handle 31, which facilitates coupling the drive inputs 602 to corresponding drive outputs 302 of the end effector 22, which may be at fixed positions spaced apart along a corresponding mating surface designed for modular use and attachment to a variety of other instruments. The handle 31 may include a latch clip or other latch features / members for physically coupling to the adapter 8 and / or corresponding structures of the end effector 22.

[0115] The mechanical components within the instrument handle 31 may allow a drive input 602 for articulating the drive shaft 40. Each drive input within the drive input 602 may be configured to engage a corresponding drive output 302 on the end effector 22. For example, each drive input may include a receiver configured to mate with a drive output configured as a spline. The drive input and the drive output may be configured to engage to transfer motion therebetween. Thus, the drive output may be rotated to cause a corresponding rotation of the drive input, thereby controlling the various functions of the instrument handle 31.

[0116] As used herein, references to “instrument device manipulator assemblies,” “instrument manipulator assemblies,” “manipulators,” “manipulator assemblies,” and other variations thereof may refer to Figure 6 any subset of the components of the assembly 150 shown in, including a robotic arm, an end effector of the robotic arm, an adapter configured to couple to the robotic end effector, an instrument base / handle configured to couple to the end effector and / or the adapter, and / or other actuator components, structures, and / or mechanisms associated with the instrument base / handle. Additionally, it should be understood that references to “actuators” as used herein may refer to Figure 6 any component of the assembly 150 shown in that directly or indirectly affects or causes movement of an instrument / component that engages, couples, or is otherwise actuated by a component of the assembly 150. For example, according to the embodiments disclosed herein, an “actuator” may include any collection or subset of the following devices or components: feed rollers, shaft drive wheels / rollers, feed roller channels, instrument feeder drive inputs, adapter drive outputs, adapter drive inputs, pulleys, belts, gears, pins, end effector drive outputs, and / or structures and / or control circuits configured to cause their actuation. For example, an actuator may be any component, device, or structure configured such that its movement causes a corresponding movement in another component, device, or structure, whether the other component, device, or structure is integrated with or separate from the actuator.

[0117] Double-line Pulley

[0118] Figure 7 An example instrument is shown in accordance with one or more embodiments, the instrument having one or more dual-line pulley systems for articulating the shaft 40 of the instrument. As shown, the instrument 700 may include a shaft 40 and a handle 31. The handle 31 may include one or more dual-line pulleys, such as a first dual-line pulley 701 and a second dual-line pulley 702. Each dual-line pulley 701, 702 may utilize a received drive output (e.g., Figure 6The drive output unit 309) performs robotic control / rotation. In other words, the drive output unit can provide torque to rotate each of the double-line pulleys 701, 702 by a certain amount of rotation. The double-line pulleys 701, 702 can rotate in the clockwise or counterclockwise direction.

[0119] As shown, the first double-line pulley 701 can be configured to have a common axis of rotation with the drive output unit. In some embodiments, the first double-line pulley 701 can rotate on Figure 5 the first / main plane P p As shown, the second double-line pulley 702 can be configured to have an axis of rotation perpendicular to both the axis of rotation of the drive output unit and the axis formed along the shaft 40 and the handle 31. In some embodiments, the second double-line pulley 702 can rotate on the second / secondary plane P s The handle 31 can employ any known mechanism to convert the torque / force received from the axis of rotation of the drive output unit to the axis of rotation of the second double-line pulley 702 and transmit the torque / force.

[0120] The first set of wire ropes 91 can be attached to the first double-line pulley 701. In some embodiments, the first set of wire ropes 91 can include a first wire rope 91a and a second wire rope 91b, which can be referred to as agonist wire rope and antagonist wire rope, respectively. The first set of wire ropes 91 can be coupled to the first double-line pulley 701 on opposite sides of the first double-line pulley 701 such that the tension on the first wire rope 91a can be increased via the rotation of the first double-line pulley 701 without increasing the tension on the second wire rope 91b, and vice versa. For example, as shown, the first wire rope 91a can be coupled to the "left" side of the first double-line pulley 701, and the second wire rope 91b can be coupled to the "right" side of the first double-line pulley 701. Continuing with the illustrated example, a counterclockwise rotation of the first double-line pulley 701 can pull the first wire rope 91a (e.g., increase the tension thereon), while potentially releasing the tension on the second wire rope 91b, and vice versa.

[0121] Similarly, a second set of pull wires 92 may be attached to the second double pulley 702. In some embodiments, the second set of pull wires 92 may include a third pull wire 92a and a fourth pull wire 92b, which may be referred to as an agonist pull wire and an antagonist pull wire, respectively. The second set of pull wires 92 may be coupled to the second double pulley 701 on opposite sides of the second double pulley 702 such that the tension on the third pull wire 92a may be increased via rotation of the second double pulley 702 without increasing the tension on the fourth pull wire 92b, and vice versa. For example, as shown, the third pull wire 92a may be coupled to the "back" side of the second double pulley 702, and the fourth pull wire 92b may be coupled to the "front" side of the second double pulley 702. Continuing with the illustrated example, an illustrated counterclockwise rotation of the second double pulley 702 may pull the third pull wire 92a (e.g., increase the tension thereon), while potentially releasing the tension on the fourth pull wire 92b, and vice versa.

[0122] The end of the shaft 40 may be coupled to the other ends of the pull wires 91a, 91b, 92a, 92b. More specifically, the first pull wire 91a and the second pull wire 91b of the first set of pull wires 91 may be coupled to opposite ends of the end in the primary plane P P . Similarly, the third pull wire 92a and the fourth pull wire 92b of the second set of pull wires 92 may be coupled to opposite ends of the end in the secondary plane P S . An enlarged view of the shaft 40 and the handle 31 illustrates the corresponding attachment of the pull wires 91a, 91b, 92a, 92b.

[0123] Continuing with the example of the instrument 700, the end may be articulated based on the rotation of the pulleys of the double pulleys 701, 702. For example, to articulate the end of the shaft 40 to the left, the first double pulley 701 may be rotated counterclockwise. The rotational force of the first double pulley 701 pulls the attached first pull wire 91a, thereby transferring the rotational force as tension on the first pull wire 91a. The first pull wire 91a applies tension on the other end of the first pull wire 91a attached to the end. At the same time, the rotational force releases the attached second pull wire 91b, thereby enabling the end to tilt more freely towards the side to which the first pull wire 91a is attached. In combination with the increased tension on the first pull wire 91a, the end may tilt to the left as the first double pulley 701 rotates counterclockwise. Conversely, a clockwise rotation of the double pulley 701 may tilt the end to the right. Thus, the rotation of the first double pulley 701 may change the tilt of the end in P P . Similarly, counterclockwise and clockwise rotations of the second double pulley 702 may be translated into tension on the second set of pull wires 92 and may change the tilt of the end in P S towards the back of the page and outside of the page, respectively.

[0124] Articulating the end in the respective planes P P and PS The operations of the dual-line pulleys 701, 702 corresponding to the respective changes in the inclination can be considered independent of each other. Thus, the combined operation of the dual-line pulleys 701, 702 can achieve articulation of the distal end in any direction.

[0125] The use of the dual-line pulleys 701, 702 provides many advantages over single-line pulleys (not shown). For example, a single-line pulley with its only cable attached to the distal end can cause the distal end to articulate in only one direction. If the distal end is to articulate in the opposite direction, another cable attached to a different single-line pulley and the distal end is required. Thus, a two-way implementation based on single-line pulleys requires the overhead of additional single-line pulleys and corresponding drive outputs. The overhead can be costly not only in terms of the additional pulleys and drive output components required, but also in terms of the use of the limited physical space in the handle 31. Additionally, in a single-line pulley implementation, the single-line pulleys must be synchronized when tension is applied (e.g., the pulling of one cable must be accompanied by the release of another cable), or the single-line pulleys may apply an undesirably elevated tension, which may be unsafe for the instrument. In contrast, the exemplary instrument 700 based on the dual-line pulleys 701, 702 can reduce the total number of drive outputs and greatly simplify the synchronization of the cable sets 91, 92.

[0126] Double-line Pulley Joint Motion Response

[0127] Figures 8 to 10 are diagrams 800, 900, 1000 showing the relationship between pulley rotation and instrument deflection / articulation. Each of the diagrams 800, 900, 1000 is plotted on a plane having an X-axis representing pulley rotation (e.g., amount of rotation) and a Y-axis representing deflection / articulation (e.g., inclination). The centerline on the X-axis can indicate zero rotation of the dual-line pulley that is neither rotating clockwise nor counterclockwise. Moving along the X-axis to the right of the centerline can indicate increasing clockwise rotation, while moving to the left can indicate increasing counterclockwise rotation. The centerline on the Y-axis can indicate zero inclination of the distal end that is neither toward the left (e.g., negative inclination) nor toward the right (e.g., positive inclination). Moving along the Y-axis toward the top of the centerline can indicate increasing right inclination, while moving toward the bottom can indicate increasing left inclination. With this understanding, each relationship in the relationship will be described in a counterclockwise traversal of the relationship. It should be understood that each starting coordinate and counterclockwise traversal are chosen for ease of the following description and can be considered arbitrary. Additionally, some aspects of the diagrams 800, 900, 1000 may be exaggerated for ease of description.

[0128] Figure 8FIG. 800 is a diagram showing the relationship between pulley rotation and the inclination of the end of a plastic (e.g., malleable, flexible, pliable, soft, bendable, etc.) instrument shaft according to one or more embodiments. The plastic instrument shaft can be a shaft that exhibits a tendency to remain inclined once positioned in an inclined position. Examples of plastic instrument shafts can be shafts that are soft and do not rely on themselves to return to a neutral position (e.g., zero inclination).

[0129] This relationship will be described in a counterclockwise manner, starting from the first configuration 802 to the second configuration 804, the third configuration 806, the fourth configuration 808, the fifth configuration 810, the sixth configuration 812, and returning to the first configuration 802.

[0130] The first configuration 802 is drawn with clockwise (e.g., positive) pulley rotation and zero inclination. At the first configuration 802, based on the clockwise pulley rotation, the left drawstring (e.g., Figure 7 the first drawstring 91a) has zero tension or non-substantial tension, while the right drawstring (e.g., Figure 7 the second drawstring 91b) is taut. Here, the traversal is about to start towards the second configuration 804. In other words, the end will tilt towards the right direction.

[0131] The second configuration 804 is drawn with increasing clockwise pulley rotation and a right inclination. Between the first configuration 802 and the second configuration 804, the increase in tension on the right drawstring causes a linear increase in the right inclination. Although the right drawstring has increased tension compared to the first configuration 802, the left drawstring remains at zero tension or non-substantial tension.

[0132] The third configuration 806 is drawn with zero pulley rotation and a right inclination. Between the second configuration 804 and the third configuration 806, counterclockwise pulley rotation turns the pulley rotation to zero and correspondingly reduces the tension on the right drawstring. However, the counterclockwise pulley rotation does not change the right inclination of the joint movement at the previous second configuration 804. As shown, the flat (e.g., parallel to the X-axis) response between the second configuration 804 and the third configuration 806 indicates a constant inclination of the end. A lack of inclination response during the traversal between the second configuration 804 and the third configuration 806 can be observed when both the left drawstring and the right drawstring are slack (e.g., have no meaningful tension) and thus cannot adjust the inclination. Since the plastic instrument shaft is soft and does not rely on itself to return to its neutral position, no change in inclination is observed during the traversal here.

[0133] When the traversal reaches the third configuration 806, the left drawstring becomes taut based on the counterclockwise pulley rotation. Here, the traversal is about to start towards the fourth configuration 808. In other words, the end is about to tilt left from its right inclination.

[0134] The fourth configuration 808 is plotted with counterclockwise (e.g., negative) pulley rotation and zero inclination. At the fourth configuration 808, based on the counterclockwise pulley rotation, the left cable is taut while the right cable has zero or non-substantial tension.

[0135] The fifth configuration 810 is plotted with increasing counterclockwise pulley rotation and left inclination. Between the third configuration 806 and the fifth configuration 810, an increase in the tension on the left cable causes a linear increase in the left inclination. The right cable remains at zero or non-substantial tension.

[0136] The sixth configuration 812 is plotted with zero pulley rotation and left inclination. Between the fifth configuration 810 and the sixth configuration 812, clockwise pulley rotation turns the pulley rotation to zero and correspondingly reduces the tension on the left cable. However, the clockwise pulley rotation does not change the left inclination of the joint movement at the previous fifth configuration 810. As shown, the flat (e.g., parallel to the X-axis) response between the fifth configuration 810 and the sixth configuration 812 indicates a constant inclination of the end. A lack of inclination response during the traversal between the fifth configuration 810 and the sixth configuration 812 can be observed when both the left and right cables are slack (e.g., have no meaningful tension) and thus cannot adjust the inclination. Since the plastic instrument shaft is flexible and does not return to its neutral position by itself, no change in inclination is observed during the traversal here.

[0137] When the traversal reaches the sixth configuration 812, the right cable becomes taut based on the clockwise pulley rotation. Here, the traversal will return to the first configuration 802. In other words, the end is about to start tilting to the right from its left inclination.

[0138] As shown, the plastic instrument shaft can exhibit at least two traversal zones / regions between the second configuration 804 and the third configuration 806 and between the fifth configuration 810 and the sixth configuration 812, and these at least two traversal zones / regions exhibit an unchanged inclination even when there are changes in pulley rotation. Since the zones do not produce a change in the Y-axis in response to a change in the X-axis, the zones are flat (or nearly flat) in the illustration 800. These flat zones can be considered "dead zones" where the plastic instrument shaft can remain unresponsive to some amount of pulley rotation.

[0139] Figure 9 Illustration 900 shows the relationship between pulley rotation and the instrument inclination of the end of an elastic (e.g., stiff, resilient, rigid, etc.) instrument shaft according to one or more embodiments. The elastic instrument shaft can be a shaft that exhibits a tendency to return to its pre-inclination position when tilted. Examples of elastic instrument shafts can be shafts that are stiff and return to the neutral position (e.g., zero inclination) by themselves.

[0140] This relationship will be described in a counterclockwise manner, starting from the first configuration 902 to the second configuration 904, the third configuration 906, the fourth configuration 908, the fifth configuration 910, the sixth configuration 912, the seventh configuration 914, and returning to the first configuration 902.

[0141] The first configuration 902 is drawn with clockwise (e.g., positive) pulley rotation and zero tilt. At the first configuration 902, based on the clockwise pulley rotation, the left cable (e.g., Figure 7 the first cable 91a) has zero or non-substantial tension, while the right cable (e.g., Figure 7 the second cable 91b) is taut. Here, the traversal is about to start towards the second configuration 904. In other words, the end will tilt towards the right direction.

[0142] The second configuration 904 is drawn with increasing clockwise pulley rotation and a right tilt. Between the first configuration 902 and the second configuration 904, the increase in tension on the right cable causes a linear increase in the right tilt. Although the right cable has increased tension compared to the first configuration 902, the left cable maintains zero or non-substantial tension.

[0143] The third configuration 906 is drawn after applying some counterclockwise pulley rotation during the right tilt. Here, since the shaft is stiff and exhibits a tendency to return to the neutral position, the right cable is taut and resists the tendency of the end. The right tilt continues to be proportional to the total clockwise pulley rotation (e.g., the pulley rotation is to the right of the X-axis centerline) and the tension on the right cable. The left cable has zero or non-substantial tension.

[0144] The fourth configuration 908 and the fifth configuration 910 are drawn within the range of pulley rotation that provides a neutral position. Between the fourth configuration 906 and the fifth configuration 908, any clockwise or counterclockwise pulley rotation is overcome by the elastic tendency of the shaft, causing the shaft to remain in the neutral position. In some embodiments, pulley rotation within this range does not provide the threshold tension level required to tilt the end. In some embodiments, the left and right cables may provide zero or some non-substantial tension, such that the elastic tendency completely controls the return of the end to the neutral position. In any case, when counterclockwise pulley rotation is provided, the end will only tilt to the left, i.e., to the left of the fifth configuration 910, and when clockwise pulley rotation is provided, the end will only tilt to the right, i.e., to the right of the first configuration 902. Referring again to the counterclockwise traversal of the illustration 900, at the fifth configuration 910, the counterclockwise pulley rotation has not caused a left tilt.

[0145] The sixth configuration 912 is depicted with an increasing counterclockwise pulley rotation and a left tilt, and this sixth configuration is a mirror image of the second configuration 904. Between the fifth configuration 910 and the sixth configuration 912, an increase in the tension on the left drawstring causes a linear increase in the left tilt. The right drawstring maintains zero tension or non-substantial tension.

[0146] From the sixth configuration 912 to the seventh configuration 914, a clockwise pulley rotation is applied. Here, since the shaft is stiff and exhibits a tendency to return to the neutral position, the left drawstring is taut and resists the tendency at the end. The left tilt continues to be proportional to the total counterclockwise pulley rotation (e.g., the pulley rotation is to the left of the X-axis centerline) and the tension on the left drawstring. Meanwhile, the right drawstring has zero tension or non-substantial tension.

[0147] At the seventh configuration 914, the end reaches the neutral position again due to its elastic tendency. From this seventh configuration until the first configuration 902, any clockwise or counterclockwise pulley rotation is overcome by the elastic tendency of the shaft, causing the shaft to remain at the neutral position. Here, the elastic instrument shaft behaves in a manner similar to its behavior between the fourth configuration 908 and the fifth configuration 910.

[0148] As shown, the elastic instrument shaft can exhibit at least two traversal zones / regions between the fourth configuration 908 and the fifth configuration 910 and between the seventh configuration 914 and the first configuration 902. Even when there are changes in the pulley rotation, these at least two traversal zones / regions exhibit an unchanged tilt. Since the zones do not produce a change in the Y-axis in response to a change in the X-axis, the zones are flat (or nearly flat) in the illustration 900. These flat zones can be considered "dead zones" where the elastic instrument shaft can remain unresponsive to some amount of pulley rotation.

[0149] Figure 10 Illustration 1000 shows the relationship between pulley rotation and instrument deflection of a hybrid (both plastic and elastic) instrument shaft according to one or more embodiments. The hybrid instrument shaft can be ideally plastic and elastic. In other words, unlike a plastic instrument shaft, the end of the hybrid instrument shaft can have a tendency to return to its neutral position by itself, but less than that of an elastic instrument shaft. Thus, the hybrid instrument shaft can combine or consider the response curves exhibited by various different materials to provide a more desirable or optimal response curve for a specific application. Additionally, the hybrid instrument shaft can better simulate the joint movement behavior of a real instrument shaft.

[0150] Compared with the previous relationship between the plastic instrument shaft and the elastic instrument shaft, the relationship of the hybrid instrument shaft has a linear region 1002 and a non-linear region 1004. The linear region 1002 may include a clockwise linear region 1002a and a counterclockwise linear region 1002b. Within the linear region 1002, the rotation of the pulley can cause a proportional inclination associated with the slope of its corresponding line.

[0151] The non-linear region 1004 may include a first non-linear region 1004a and a second non-linear region 1004b. As shown, the non-linear region 1004 connects the clockwise linear region 1002a and the counterclockwise linear region 1002b. In other words, when the rotation direction of the dual-line pulley of the hybrid instrument shaft is reversed from the clockwise direction to the counterclockwise direction, the end of the hybrid instrument shaft can traverse the first non-linear region 1004a. Similarly, when the rotation direction of the dual-line pulley of the hybrid instrument shaft is reversed from the counterclockwise direction to the clockwise direction, the end of the hybrid instrument shaft can traverse the second non-linear region 1004b.

[0152] The curves of the first non-linear region 1004a and the second non-linear region 1004b illustrate little dead zone. Therefore, the end of the hybrid instrument shaft remains responsive to any change in the pulley rotation. In addition, since the hybrid instrument shaft is neither too wobbly nor too stiff, it can reduce the operator's frustration and, in some cases, improve the durability of the instrument.

[0153] Kinematics Model

[0154] The hybrid instrument shaft and its kinematic response to pulley rotation can be represented based on a kinematic model. The kinematic model can estimate the relationship between the pulley rotation in a plane and the corresponding endoscopic joint motion (e.g., deflection / inclination). Based on this relationship, the kinematic model can achieve the determination of the resulting joint motion provided by a given pulley rotation. Conversely, the kinematic model can achieve the determination of the predicted pulley rotation for a desired joint motion. In the case of involving a kinematic model, the determination of joint motion, pulley rotation, joint motion response, or any of its regions can be synonymously described as estimation, calculation, operation, or identification.

[0155] In some embodiments, the kinematic model can be a mathematical model representing the joint motion response according to an equation. Such a mathematical kinematic model can advantageously achieve the calculation of the resulting joint motion or the predicted pulley rotation. The equation can depend on the following example parameters and variables to represent the joint motion response:

[0156]

[0157] Table 1: Definitions of parameters and variables used in the kinematic model

[0158] According to the formula for representing joint movement response, there may be additional or fewer parameters and variables compared to those shown in Table 1.

[0159] Figure 11A It is a kinematic model 1100 of a hybrid instrument axis showing the relationship between pulley rotation and instrument deflection according to one or more embodiments. The kinematic model 1100 is plotted on a plane with an X-axis representing the commanded pulley rotation (denoted as j cmd ) and a Y-axis representing the deflection (denoted as φ).

[0160] The kinematic model 1100 can be formulated using a combination of linear and non-linear piecewise continuous functions. Specifically, the kinematic model can include eight joint motion response regions: four linear regions (e.g., the first linear region 1101, the second linear region 1103, the third linear region 1104, and the fourth linear region 1106) and four non-linear regions (e.g., the first non-linear region 1102, the second non-linear region 1105, the third non-linear region 1107, and the fourth non-linear region 1108). The kinematic model 1100 that plots the linear and non-linear regions can be defined (e.g., determined) at least in part based on pulley rotation (as shown on the X-axis) and associated joint motion (as shown on the Y-axis). When any of the wire pulls is under tension, the joint motion response can be linear, and thus, the linear function can model the endoscopic response during joint motion in the first linear region 1101 or the third linear region 1104 and during reset joint motion in the second linear region 1103 or the fourth linear region 1106. The joint motion in the first linear region 1101 or the third linear region 1104 is when the endoscope deflects and continues to deflect in a certain direction from the neutral position. The reset joint motion in the second linear region 1103 or the fourth linear region 1106 is when the endoscope returns from its previous deflection to the neutral position. During the transition of tension from one wire pull to another (e.g., agonist wire to antagonist wire), the response is non-linear. The tension transition can occur during the direction reversal. The non-linear function can model the endoscopic response during the reversal in the first non-linear region 1102, the second non-linear region 1105, the third non-linear region 1107, or the fourth non-linear region 1108 (e.g., when changing from joint motion to reset joint motion or vice versa). The joint motion reversal in the first non-linear region 1102 or the second non-linear region 1105 can occur at any time during the joint motion in the first linear region 1101 or the third linear region 1104, and the reset joint motion reversal in the third non-linear region 1107 or the fourth non-linear region 1108 can occur at any time during the reset joint motion in the second linear region 1103 or the fourth linear region 1106. In some embodiments, an S-shaped function can be used to model the joint motion response during the reversal in the first non-linear region 1102, the second non-linear region 1105, the third non-linear region 1107, or the fourth non-linear region 1108. More specifically, a generalized logistic function can be used to model the joint motion response.

[0161] The kinematic model 1100 includes one or more “dead zones” where pulley rotation is not likely to cause endoscope joint movement. Dead zones may occur due to various instrument characteristics such as friction in the endoscope mechanism, the endoscope's anatomy, material properties, pulley characteristics, component wear, etc. An example dead zone is the central dead zone 1109 where, before the endoscope in neutral position deflects, pulley rotation must meet a positive threshold level (denoted as +j dz ) or a negative threshold level (denoted as -j dz ). That is, pulley rotation within the central dead zone 1109 does not deflect the endoscope in neutral position.

[0162] Figure 11B is the tension response 1150 of the hybrid instrument axis showing the relationship between pulley rotation and the applied tension according to one or more embodiments. The tension response 1150 is plotted on a plane having an X-axis representing pulley rotation (denoted as j cmd ) and a Y-axis representing the net tension on a pair of pull wires.

[0163] Similar to the kinematic model, the tension response shows a combination of linear and non-linear regions. Specifically, the tension response shows six regions: two linear tension regions 1151, 1152 and four non-linear tension regions 1153a, 1153b, 1154a, 1154b. When either pull wire is in a tensioned state, the tension response is linear. For example, positive net tension (e.g., the first pull wire causes tension) in the first linear tension region 1151 and endoscope joint movement in the first direction. Positive net tension can cause Figure 11A joint movement in the first linear region 1101 and reset joint movement in the fourth linear region 1106 of the kinematic model 1100 in

[0164] During the reversals associated with the non-linear tension regions 1153a, 1153b, 1154a, 1154b, the tension is non-linear. For example, the first reversal associated with the first non-linear tension region 1153a can occur at the moment when the dual-line pulley starts rotating counterclockwise from the maximum clockwise pulley rotation. The moment of the first reversal associated with the first non-linear tension region 1153a can correspond to the top of the joint movement in the first linear region 1101 of the kinematic model 1100. As the pulley continues to rotate counterclockwise, the first cable loses tension, resulting in a reduced absolute net tension, as illustrated during the reversal associated with the first non-linear tension region 1153a. Eventually, the second cable starts providing a negative net tension on the second linear tension region 1152 for the reset joint movement in the second linear region 1103 and the joint movement in the third linear region 1104 of the kinematic model 1100. Conversely, the second reversal associated with the second non-linear tension region 1153b can occur at the moment when the dual-line pulley starts rotating clockwise from the maximum counterclockwise pulley rotation. The moment of the second reversal associated with the first non-linear tension region 1153b can correspond to the bottom of the joint movement in the third linear region 1104 of the kinematic model 1100. As the pulley continues to rotate clockwise, the second cable loses tension, resulting in a reduced absolute net tension, as illustrated during the reversal associated with the second non-linear tension region 1153b. Eventually, the first cable starts providing a positive net tension on the first linear tension region 1151 for the reset joint movement in the fourth linear region 1106 and the joint movement in the first linear region 1101 of the kinematic model 1100.

[0165] In some cases, the reversal can occur before the maximum pulley rotation. For example, the third reversal associated with the third non-linear tension region 1154a can occur before the dual-line pulley reaches the maximum clockwise pulley rotation. Similarly, the fourth reversal associated with the fourth non-linear tension region 1154b can occur before the dual-line pulley reaches the maximum counterclockwise pulley rotation. The tension responses for the third reversal associated with the third non-linear tension region 1154a and the fourth reversal associated with the fourth non-linear tension region 1154b are illustrated.

[0166] The tension response 1150 provides some insights into the operation of the dead zone, such as the central dead zone 1109 of the kinematic model 1100. During the tension transition from one cable to another during the reversal, the range of pulley rotation 1155 can provide a minimum tension (or minimum net tension) on the cable. Additionally, during the range of pulley rotation 1155, there may be little change in the tension (or net tension). Therefore, when within the range of pulley rotation 1155, clockwise or counterclockwise rotation of the dual-line pulley is unlikely to cause endoscope deflection and contributes to the formation of the dead zone.

[0167] Note that some aspects of the kinematic model 1100 and the tension response 1150 may be exaggerated for ease of description. For simplicity, the relationships in the kinematic model 1100 and the tension response 1150 are limited to a single double pulley setup. However, it should be understood that the modeling of multiple double pulley setups can easily extend the relationships in the kinematic model 1100 and the tension response 1150 with additional dimensions for each additional double pulley.

[0168] The kinematic model 1100 and the tension response 1150 described above can be mathematically modeled using the parameters and variables in Table 1. First, the joint motion in the first linear region 1101 or the third linear region 1104 can be modeled as:

[0169] Φ = k 弯曲 (j cmd - j dz0 )(Eq.1).

[0170] Second, the reset joint motion in the second linear region 1103 or the fourth linear region 1106 can be modeled as:

[0171] Φ = j cmd k 松弛 + constant (Eq.2).

[0172] The joint motion equation Eq.1 and the reset joint motion equation Eq.2 are essentially linear, and thus they are easily invertible.

[0173] Finally, the inversion of the first non - linear region 1102, the second non - linear region 1105, the third non - linear region 1107 or the fourth non - linear region 1108 can be modeled as:

[0174]

[0175] where,

[0176] The inversion equation Eq.3 is essentially non - linear. Although the inversion of the first non - linear region 1102, the second non - linear region 1105, the third non - linear region 1107 or the fourth non - linear region 1108 can be modeled using a variety of non - linear functions, the inversion equation Eq.3 is selected as S - shaped using a generalized logistic function with favorable properties. Generally speaking, the inversion of non - linear equations is not very intuitive. On the contrary, the inversion equation Eq.3 is invertible, and furthermore, the inverse inversion equation has a unique solution. As will be described in more detail, compared with other non - linear equations that are rarely invertible or rarely have closed - form solutions, the direct invertibility makes the inversion equation Eq.3 more preferable.

[0177] Based on equations Eq.1, Eq.2, and Eq.3, all linear and non-linear regions in the kinematic model 1100 can be described mathematically. However, the kinematic model 1100 may need to be fitted for each individual endoscope (e.g., an individual endoscope may need to be calibrated to the kinematic model 1100). The parameters can include, for example, k 弯曲 , j dz0 , Q, B, and nu, which can depend on manufacturing tolerances and can vary for each endoscope.

[0178] These variations can be due to many factors, including part and component tolerances specific to each endoscope, and cause the endoscope to respond differently to joint motion commands (e.g., wire pull commands). An endoscope is a flexible, soft, and compliant mechanism driven by cables (e.g., pull wires), and the mechanical characteristics of the endoscope can be essential for understanding endoscope motion, modeling endoscope behavior, developing control algorithms, making mechanical design decisions, and / or testing the durability of the endoscope. Accurately and responsively controlling the endoscope can be challenging without characterizing the effects of these differences on the endoscope's response, especially when attempting to do so robotically.

[0179] Calibration can help characterize the differences for each endoscope. Methods for characterizing and calibrating an endoscope are described below. The method can characterize the input-to-output behavior of each endoscope by using the measured end position of the endoscope as a basis for controlling the joint motion of the endoscope.

[0180] The method can involve a setup that can consist of a fixture for mounting and holding the endoscope, a mechanism / sensor for rotating and measuring the positions of the respective pulley shafts of the endoscope, a mechanism / sensor for measuring wire displacement and tension, and / or a mechanism / sensor for measuring the joint motion (e.g., end position / orientation) of the endoscope. In some embodiments, the end of the endoscope can be controlled by one or more pulleys attached to one or more pulley shafts. For example, the end can be controlled by four pulleys attached to two or four pulley shafts.

[0181] Using this setup, some or all of the following steps can be performed to characterize the endoscope response:

[0182] 1. The endoscope can be mounted in the setup and coupled to various input mechanisms.

[0183] 2. Some sensors can be used to measure the starting / reference positions of one or more pulley shafts, and other sensors (e.g., EM sensors, image sensors, and / or any other sensors) can be used to record the position / orientation of the end of the endoscope in two-dimensional / three-dimensional space.

[0184] 3. Starting from the starting / reference position, the distal end can be articulated by rotating the pulley shaft, while continuously measuring / sampling the pulley rotation, the wire tension, and the endoscopic articulation. In some embodiments, for single-wire movement characteristics, a single pulley shaft can be rotated to articulate the distal end, while using a distal position / orientation measurement sensor to measure the distal position / orientation. In some embodiments, for dual-wire movement characteristics, two pulley shafts can be rotated simultaneously by a fixed amount or a predetermined ratio between the two pulley shafts. This step can be repeated until the endoscope reaches all predetermined articulation targets and all desired input combinations.

[0185] 4. After the above input-output (e.g., articulation) data collection process, visualizations can be generated from the collected data. Articulation responses (e.g., V-curves, I-curves, etc.) can be generated by plotting the pulley rotation and the endoscopic distal-end articulation on a plane (such as on the X-axis and Y-axis, or vice versa). Force / tension responses (e.g., V-curves, I-curves, etc.) can be generated by plotting the wire tension and the endoscopic distal-end articulation on a plane (such as on the X-axis and Y-axis, or vice versa). Figure 11A FIG. 11B respectively illustrates example articulation and tension responses.

[0186] The plotted responses can enable measurement of some endoscopic-specific mechanical characteristics including: a central dead zone, articulation and re-articulation slope, direction reversal transition region, and direction reversal dead zone. These features are described with respect to Figures 11A to 11B Based on the measurements and equations, the endoscope can be calibrated. That is, for example, the k 弯曲 、j dz0 、Q, B, and nu for adjusting / fitting the kinematic model to the endoscope can be determined.

[0187] Through calibration, endoscope-specific parameters can be determined for each endoscope. In some embodiments, the calibration parameters can be encoded on a scannable medium and attached to the endoscope. For example, the parameters can be programmed in an RFID tag inserted into the endoscope, or printed on a QR code printing material attached to the endoscope. A reader (e.g., Figure 6 reader 304) can scan the parameters and correspondingly fit the kinematic model 1100 to the endoscope. Once calibrated, the endoscope can be robotically controlled, such as determining the predicted pulley rotation for achieving a desired articulation based on the fitted kinematic model.

[0188] In addition to the parameters, Equations Eq.1, Eq.2, and Eq.3 can also depend on variables. The variables can include, for example, k φ 、k j 、φ 偏移 、φ @EoS 、φ@反转 , j @EoS , j@ 反转 , j c and j cmd . Some variables can be determined at moments specific to inversion, such as an inversion variable (e.g., (.)@ 反转 variable). For example, the inversion variable can capture specific joint motions and pulley rotations at the moment of inversion. The inversion variable can indicate the state of the endoscope at that moment, including which linear region the endoscope was traversing before inversion. Some other variables can be calculated, such as an S-shaped end variable (e.g., (.) @EoS variable). For example, the S-shaped end deflection (φ @EoS ) and pulley rotation (j @EoS ) can be calculated based on the following:

[0189] Φ @EoS = k Φ Φ @反转 - Φ 偏移 (Eq.4)

[0190] And

[0191]

[0192] Based on the parameters and variables, the kinematic model 1100 can be fitted for any endoscope and describe its current state using equations Eq.1, Eq.2, and Eq.3. Since all equations are reversible and a given endoscope state provides a unique solution, the predicted pulley rotations for achieving a desired or commanded deflection can be easily calculated by solving the inverse equations. Such a kinematic model can be received, calculated, or otherwise obtained by the robotic cart / system and / or control tower / system of the present disclosure.

[0193] Figure 12 is a flowchart 1200 of a process for controlling the motion of a robotic joint based on a kinematic model according to one or more embodiments. This process can be used to calculate the predicted pulley rotations (or required tensions) for achieving the desired joint motion of an endoscope. This process can be implemented in conjunction with the joint motion of an endoscope by robotic control of a wire tensioning pulley / mechanism associated with the endoscope (e.g., incorporated into the handle of the endoscope). This process can be at least partially implemented by the control circuitry of any of the system components disclosed herein (such as the robotic cart / system and / or control tower / system). For ease of description, it is assumed that the endoscope is initially in a neutral (e.g., straightened) position and the pulley rotation is reset. However, this process can be applied to an endoscope in any position and any pulley rotation as long as its position is accurately identified on its kinematic model.

[0194] At block 1202, a desired joint motion may be received. The desired joint motion may be a commanded joint motion received from an operator. The desired joint motion may be represented as an angle within a range of articulable motion defined around some reference in various ways. For example, some possible ranges of articulable motion definitions may include [-90°, 90°], [0°, 180°], [-π / 2, π / 2], [0, π], etc.

[0195] At block 1204, it may be checked whether the desired joint motion causes a direction reversal. In some embodiments, checking for a direction reversal may involve comparing the currently commanded pulley rotation direction of the desired joint motion with the previously commanded pulley rotation direction. For example, assume that the last pulley rotation involved a clockwise rotation. If the commanded pulley rotation direction is also clockwise, then the commanded pulley rotation does not cause a direction reversal. Otherwise, if the commanded pulley rotation is counterclockwise, then the commanded pulley rotation causes a direction reversal.

[0196] In some embodiments, checking for a direction reversal may involve comparing the desired joint motion with the previous actual joint motion. For example, the direction of change of the previous joint motion may be determined by sampling the previous actual joint motion. If the desired joint motion continues in the same direction as the previous direction, then the desired joint motion does not cause a direction reversal. On the other hand, if the desired joint motion does not continue in the same direction as the previous direction, then the desired joint motion causes a direction reversal.

[0197] When no direction reversal is detected, blocks 1206, 1208, 1210, 1212 may be optional, and the process may jump to block 1214. When a direction reversal is detected, the process continues to block 1206.

[0198] At block 1206, parameters and variables of an S-shape may be set. As described, the parameters may include calibration parameters for fitting a kinematic model to the endoscope, such as k 弯曲 , j dz0 , Q, B, and nu. The variables may be observed variables associated with the current endoscope state, such as k φ , k j , φ 偏移 , φ@ 反转 , j@ 反转 and j c . Specifically, the observed reversed joint motion (φ@ 反转 ) and the reversed pulley rotation (j@ 反转 ) may indicate at which joint motion and pulley rotation the direction reversal occurs. In addition, the reversed joint motion and the reversed pulley rotation may help identify the region in which the robotic control of the endoscope's kinematic model lies.

[0199] At block 1208, the end joint motion (φ @EoS ) and the end pulley rotation (j @EoS ) of the S-shape can be calculated. When the joint motion of the command for direction reversal occurs at a point in the joint motion response region corresponding to the first linear region 1101, the third linear region 1104, the third non-linear region 1107, or the fourth non-linear region 1108, the end joint motion and the end pulley rotation of the S-shape can be calculated based on Eq. 4 and Eq. 5, respectively. However, if the direction reversal is towards one of the linear responses in the linear response of the joint motion in the first linear region 1101 or the third linear region 1104 (i.e., when the joint motion of the command for direction reversal occurs at a point in the joint motion response region corresponding to the first non-linear region 1102, the second linear region 1103, the second non-linear region 1105, or the fourth linear region 1106), the end joint motion of the S-shape can be calculated based on Eq. 4, and since the end pulley rotation of the S-shape is on one of the linear responses in the linear response, the calculation of the end pulley rotation of the S-shape can be simplified by using the inverse form of Eq. 1 (instead of using Eq. 5) at the end joint motion of the S-shape. The calculated variables of the end joint motion and the end pulley rotation of the S-shape together with the parameters and observed variables from block 1206 can define the S-shape for direction reversal.

[0200] At block 1210, the post-S-shape linear response can be determined. When the joint motion of the command for direction reversal occurs at a point in the joint motion response region corresponding to the first linear region 1101, the third linear region 1104, the third non-linear region 1107, or the fourth non-linear region 1108, the post-S-shape linear response can be determined by connecting the calculated end joint motion and end pulley rotation of the S-shape to the end of the central dead zone 1109 (e.g., -j dz or +j dz ). However, if the direction reversal is towards one of the linear responses in the linear response of the joint motion in the first linear region 1101 or the third linear region 1104 (i.e., when the joint motion of the command for direction reversal occurs at a point in the joint motion response region corresponding to the first non-linear region 1102, the second linear region 1103, the second non-linear region 1105, or the fourth linear region 1106), the determination of the post-S-shape linear response can be simplified by considering the post-S-shape linear response to be equivalent to the linear response in the first linear region 1101 or the third linear region 1104 (since the post-S-shape linear response is consistent with the joint motion in the linear region).

[0201] At block 1212, a target region for a desired joint movement can be identified. Specifically, it is determined whether the desired joint movement is on a post-S-shaped linear response (e.g., the first linear region 1101, the second linear region 1103, the third linear region 1104, or the fourth linear region 1106) or on a direction-reversed S-shape that is located on a non-linear region (e.g., the first non-linear region 1102, the second non-linear region 1105, the third non-linear region 1107, or the fourth non-linear region 1108) somewhere between the reversed joint movement (φ@ 反转 ) and the end joint movement (φ @EoS ) of the S-shape.

[0202] At block 1214, a predicted pulley rotation for the desired joint movement is calculated. The predicted pulley rotation is calculated based on the identified target region. If the target region is a post-S-shaped linear response determined at block 1210, then Eq. 1 is used to calculate the predicted pulley rotation when the post-S-shaped linear response coincides with the joint movement (e.g., in the first linear region 1101 or the third linear region 1104), or Eq. 2 is used to calculate the predicted pulley rotation when the post-S-shaped linear response coincides with the de-joint movement (e.g., in the second linear region 1103 or the fourth linear region 1106). Alternatively, if the target region at block 1212 is on a direction-reversed S-shape, then Eq. 3 is used to calculate the predicted pulley rotation.

[0203] At block 1216, the IDM is driven based on the predicted pulley rotation calculated at block 1214 for achieving the desired joint movement. Providing the predicted pulley rotation should result in the desired joint movement.

[0204] Reverse Exit Determination

[0205] Previously, a kinematic model for a hybrid instrument shaft was proposed. The kinematic model includes linear regions and non-linear regions. To account for instrument variations, the kinematic model is fit to each individual endoscope (e.g., the endoscope is calibrated to conform to the kinematic model). The fit kinematic model enables the calculation of a predicted pulley rotation that, when applied to a wire via one or more drive outputs, will achieve the desired joint movement.

[0206] The accuracy of the predicted pulley rotation calculation during and after a direction reversal can depend on the calculated end joint movement (φ @EoS ) of the S-shape and the calculated end pulley rotation (j @EoS) accuracy. This is because the end joint movement in an S shape and the end pulley rotation in an S shape specify the position at the end of the non-linear region and the start of the linear region after the S shape, which coincides with the reverse exit point. Thus, the reverse exit point is the expected position calculated based on calibration parameters specific to a particular endoscope. However, the parameters may deviate from the calibration parameters. For example, over time, components may deteriorate and render the calibration parameters inaccurate.

[0207] When some parameters change, the expected reverse exit point calculated based on the calibration parameters may no longer match the actual reverse exit point of the endoscope. The mismatched reverse exit point may result in inaccuracies in the expected non-linear response and the linear response after the S shape determined based on the expected exit point. That is, the changed parameters may cause a divergence between the expected response of the kinematic model and the actual response of the endoscope.

[0208] When controlling the endoscope, the divergence between the expected response and the actual response may result in jump behavior or hysteresis behavior. Jump behavior may occur when the pulley rotation for the joint movement expected by the expected command causes a sudden and larger change in the joint movement. Hysteresis behavior may occur when the pulley rotation for the joint movement expected by the expected command causes a slower and smaller change in the joint movement. Both of these behaviors can impede the control of the endoscope and, in some cases, the results of the operation.

[0209] Figure 13A Illustrates a jump response scenario 1300 according to one or more embodiments. At Figure 13A the top is the expected kinematic model, which can be used to calculate the predicted pulley rotation for achieving the desired joint movement. The expected kinematic model may include an expected non-linear region (e.g., S-shaped region) 1301 and an expected linear region (e.g., linear region after the S shape) 1302 divided based on the expected transition point 1303 (e.g., the expected end point of the S shape). At Figure 13A the bottom is the actual tension response observed on the endoscope under various pulley rotations. The actual tension response includes a first tension region 1305 and a second tension region 1306 divided based on the actual transition point 1304.

[0210] Suppose the desired joint motion involves changing the pulley rotation through a pulley rotation range 1308 between an expected transition point 1303 and an actual transition point 1304. According to the expected kinematic model, the pulley rotation range 1308 should cause little change in joint motion. For a dead zone where the expected joint motion response is minimal, the end effector can accelerate the pulley rotation within the pulley rotation range 1308 to provide the operator with a constant rate of change in joint motion. However, according to the actual tension response, the pulley rotation range 1308 corresponds to a significant change in tension, as indicated by the steep slope of the tension response. Accelerating the pulley rotation over a significant change in tension may cause a sudden jerk in the endoscopic joint motion, which is a jump response. Therefore, if a constant rate of joint motion is desired during the pulley rotation range 1358, the pulley rotation should be decelerated (e.g., the rotational speed of the pulley rotation should be decreased).

[0211] Figure 13B An example of a hysteresis response scenario 1350 according to one or more embodiments is illustrated. At Figure 13B the top is the expected kinematic model, which can be used to calculate the predicted pulley rotation for achieving the desired joint motion. The expected kinematic model may include an expected non - linear region (e.g., an S - shaped region) 1351 and an expected linear region (e.g., a post - S - shaped linear region) 1352 divided based on an expected transition point 1353 (e.g., an expected end point of an S - shape). At Figure 13B the bottom is the actual tension response observed on the endoscope at various pulley rotations. The actual tension response includes a first tension region 1355 and a second tension region 1356 divided based on an actual transition point 1354.

[0212] Suppose the desired joint motion involves changing the pulley rotation through a pulley rotation range 1358 between an expected transition point 1353 and an actual transition point 1354. According to the expected kinematic model, the pulley rotation range 1358 should cause a significant change in joint motion. For a significant change in the expected joint motion, the end effector can decelerate the pulley rotation within the pulley rotation range 1358 to provide the operator with a constant rate of change in joint motion. However, according to the actual tension response, the pulley rotation range 1358 corresponds to little change in tension, as indicated by the flat slope of the tension response. Decelerating the pulley rotation over little change in tension may cause a deceleration in the endoscopic joint motion, which is a hysteresis response. Therefore, if a joint motion response similar to a constant rate of joint motion is desired during the pulley rotation range 1358, the pulley rotation should be accelerated (e.g., the rotational speed of the pulley rotation should be increased).

[0213] When the expected transition points 1303, 1353 match the actual transition points 1304, 1354, jumping or lagging responses of the endoscope can be avoided. Therefore, accurate determination of the actual transition points 1304, 1354 (e.g., reverse exit points) can ensure accurate estimation of the non-linear region, the linear region, and the transitions between them. Accurate transitions can be the key to ensuring smooth and predictable joint motion responses. Figures 14 to 1 7 describes three different methods for accurately triggering transitions: (i) a percentage-based method, (ii) a tension-based method, and (iii) a linear response crossing-based method.

[0214] Figure 14 Illustrates a percentage-based reverse exit determination 1400 according to one or more embodiments. Figure 14 At the top of Figure 13A the expected kinematic model of the jumping response scenario 1300. Figure 14 At the bottom of Figure 13A the actual tension response of the jumping response scenario 1300. In the jumping response scenario 1300, the expected transition point 1303 arrives late during traversal compared to the actual transition point 1304. It is the late arrival of the expected transition point 1303 that may erroneously inform the robot controller that acceleration of pulley rotation during the pulley rotation range 1308 will be safe. Although the jumping response scenario 1300 is illustrated, it should be understood that the percentage-based reverse exit determination 1400 technique can also be applied to Figure 13B the lagging response scenario 1350.

[0215] The percentage-based reverse exit determination 1400 enables the robot controller to make a smooth transition from the non-linear region to the linear region, which avoids jumping or lagging responses by ensuring that joint motion of the endoscope based on the kinematic model does not cause overly aggressive pulley rotation near the reverse exit. For example, it is described that a response region with a flat slope between the expected transition point 1303 and the point 1404 associated with the actual transition point 1304 associated with the pulley rotation range 1308 can cause aggressive pulley rotation leading to a jumping response. To avoid causing such a jumping response, the percentage-based reverse exit determination 1400 can calculate a new trajectory 1401 to provide a steeper slope than the flat slope for the response region associated with the pulley rotation range 1308. When joint motion of the endoscope is performed near the expected transition point 1303, the steeper slope within the response region associated with the pulley rotation range 1308 can inform the robot controller to be more cautious (e.g., less aggressive) about pulley rotation. That is, the percentage-based reverse exit determination 1400 can help the robot control avoid aggressively exiting from the non-linear region 1301 to the linear region 1302 by calculating and providing the new trajectory 1401 as a temporary kinematic model.

[0216] In some embodiments, the generation of the new trajectory 1401 can involve a two-step process: (i) generating a traditional trajectory (e.g., the non-linear region 1301) and (ii) applying a percentage value to generate the new trajectory 1401. Regarding (i) the generation of the traditional trajectory, the traditional trajectory can be generated by calculating the expected transition points 1303 based on a kinematic model, as previously described. In particular, the kinematic model can calculate the end joint motion of an S-shape and the end pulley rotation of an S-shape (e.g., φ @EoS and j @EoS ), which together provide the expected transition points 1303. A trajectory that connects the current joint motion to the end joint motion of the S-shape and the current pulley rotation to the end pulley rotation of the S-shape can be generated or otherwise calculated. The resulting calculated trajectory can be the non-linear region 1301. As shown, the non-linear region 1301 can be an S-shaped curve.

[0217] Regarding (ii) applying a percentage value to generate the new trajectory 1401, the new trajectory 1401 can be generated or calculated such that the new trajectory has the expected transition points 1303 at a percentage value of the total length of the non-linear region 1301. That is, during the traversal of the new trajectory 1401, the expected transition points 1303 should be located at the percentage value of the traversal (e.g., the new trajectory 1401 passes through the expected transition points 1303). For example, for a percentage value of 80%, the new trajectory 1401 can be calculated such that the expected transition points 1303 are located at 80% of the complete traversal of the new trajectory 1401. Although 80% is used as an example percentage value, any suitable percentage value can be selected. As shown, the new trajectory 1401 can terminate at a new end point 1402 of an S-shape.

[0218] The robotic controller can cause the endoscope to perform joint motion based on the new trajectory 1401. That is, the robotic controller can cause the endoscope to perform joint motion in the response region associated with the pulley rotation range 1308 based on the steeper new trajectory 1401 to exit onto the linear region 1302. The percentage value can be adjusted such that the transition from the non-linear region to the linear region occurs seamlessly. Thus, the percentage-based reverse exit determination 1400 can ensure a smooth and predictable transition during the control of the endoscope using the robotic controller.

[0219] Although the use of percentage values has been described to generate the new trajectory 1401, it should be understood that the percentage values can be calculated dynamically to ensure smooth transitions. For example, instead of a set percentage value such as 80%, a percentage value can be calculated in real time and used that will give the new trajectory 1401 a slope that best matches (e.g., is parallel, equal, or within a threshold level) the slope of the linear region 1302. In these specific implementations, the percentage value can be a reported value rather than the value used to generate the new trajectory 1401. Additionally, it should be understood that a variety of different percentage values can be used. The percentage values can have many variations, including the following examples: (i) percent traversal length, (ii) percentage pulley rotation, or (iii) percentage joint movement. In some cases, multiple of the above percentages can be used simultaneously.

[0220] Figure 15 An example of a tension-based reverse exit determination 1500 according to one or more embodiments is illustrated. Figure 15 At the top of Figure 13A the expected kinematic model of the jump response scenario 1300. Figure 15 At the bottom of Figure 13A the actual tension response of the jump response scenario 1300. Although the jump response scenario 1300 is illustrated, it should be understood that the tension-based reverse exit determination 1500 technique can be applied to Figure 13B the hysteresis response scenario 1350.

[0221] When a direction reversal occurs, the tension on the first cable begins to decrease while the tension on the second cable begins to increase. At the end of the direction reversal on the non-linear region and when the linear region begins, the tension on the second cable takes over and the tension on the first cable can decrease to zero (or some minimum value). Thus, the increasing tension on the second cable can be used as an indication of the transition from the non-linear region to the linear region. The tension-based reverse exit determination 1500 is illustrated based on net tension, but it should be understood that, where applicable, the technique can be based on the individual tensions on each cable.

[0222] As illustrated by the actual tension response, the net tension has a positive value during the non-linear region 1305, as indicated by values above 0 N, and a negative value during the linear region 1306, as indicated by values below 0 N. The location where the net tension crosses zero can coincide with the actual transition point 1304. Thus, tension monitoring can help identify the actual transition point 1304 and the corresponding point 1504. During traversal of the non-linear region 1301, the tension-based reverse exit determination 1500 can indicate to the robot controller to exit onto the linear region 1302 at the corresponding point 1504. More details regarding Figure 16 the tension-based reverse exit determination 1500 are described.

[0223] Figure 16 FIG. 1600 is a flowchart of a process for controlling the movement of a robotic joint based on tension-based reverse exit determination according to one or more embodiments.

[0224] At block 1602, a direction reversal may be detected. Various determination methods described in block 1204 with respect to Figure 12 may be utilized to detect the direction reversal.

[0225] At block 1604, the tension may be monitored. The individual tension on each cable may be monitored separately and / or the net tension of the cables may be monitored jointly. The tension may be monitored by a tension sensor attached to the cable, or in some cases, may be calculated based on the torque applied to the cable via the drive output (e.g., torque divided by the lever arm of the drive output).

[0226] At block 1606, it may be determined whether one or more early reverse exit conditions are met. The determination that the early reverse exit condition is met may involve a first condition and / or a second condition. The first condition is (i) increasing tension and the tension increases at least a threshold in the same sign direction, and the second condition is (ii) increasing tension and the tension increases above the threshold in the opposite sign direction. For example, referring to Figure 15 , at the actual transition point 1304, the tension on the first cable may decrease to zero (or some minimum value), and the tension on the second cable may start to increase from zero (or some minimum value). During continued traversal on the S-shaped post-linear region 1306, the tension on the second cable will increase above the threshold, thus meeting the first condition (i). As another example, when traversing before the actual transition point 1304 (e.g., to the right of point 1304), the net tension of both the first cable and the second cable has a positive value. When traversing after the actual transition point 1304 (e.g., to the left of point 1304), the net tension has a negative value, such that the sign changes. Thus, during traversal, the net tension increases in the negative direction and the net tension increases above the threshold in the opposite direction, thus meeting the second condition (ii). Many variations are possible.

[0227] At block 1608, the current state of the joint movement and pulley rotation may be determined. The current joint movement at the moment of reverse exit is an S-shaped end joint movement that is more accurate than the expected S-shaped end joint movement calculated based on the kinematic model. Similarly, the current pulley rotation at the moment of reverse exit is an S-shaped end pulley rotation that is more accurate than the expected S-shaped end pulley rotation based on the kinematic model. The expected S-shaped end joint movement may be reset using the current joint movement. The expected S-shaped end pulley rotation may be reset using the current pulley rotation.

[0228] At block 1610, a new post-S-shaped linear region can be calculated based on the current state. Compared to the expected post-S-shaped linear region determined based on a kinematic model, the new post-S-shaped linear region more accurately models the transition point and the post-transition response. Thus, the tension-based reverse exit determination 1500 can ensure a smooth and predictable transition during controlling the endoscope shaft with a robotic controller.

[0229] Figures 17A to 17B Illustrates a reverse exit determination based on a linear response crossover according to one or more embodiments. When there is a direction reversal, an S-shape is calculated. Desirably, the calculated S-shape end point should coincide with the start point of the actual post-S-shaped linear region and ensure a smooth and predictable transition. In practice, it is likely that the calculated S-shape end point does not exactly coincide with the start point of the actual post-S-shaped linear response. Thus, if the reverse exit only reoccurs at the end of the calculated S-shape, the transition from the S-shape to the actual post-S-shaped linear region will likely be jerky or laggy. Additionally, in some cases, the S-shape (e.g., Figure 17A the S-shape 1702) may cross the linear response more than once. In these cases, the initial linear response crossover should be the reverse exit because the draw wire has engaged the linear response. However, unless corrected, the robotic controller will continue to articulate the endoscope based on the S-shape and introduce highly undesirable oscillatory behavior. The reverse exit determination based on the linear response crossover can help determine the exit point and eliminate the oscillatory behavior. The reverse exit determination can be particularly advantageous when the direction reversal is a rapid direction reversal.

[0230] Figure 17A Illustrates a first scenario 1700 without a reverse exit determination, and Figure 17B illustrates a second scenario 1750 with a reverse exit determination. The first scenario 1700 illustrates a post-S-shaped linear region 1701 and a corresponding S-shape 1702. The S-shape 1702 can be an S-shape calculated based on various reverse states. As shown, the S-shape 1702 crosses the linear region 1701 twice and transitions onto the linear region 1701 at the end point 1706 of the S-shape.

[0231] The robotic control based on the S-shape 1702 traverses a set of points 1703a - 1703c, which includes a first point 1703a, a second point 1703b, and a third point 1703c. At the third point 1703c, the draw wire is engaged. However, the first scenario 1700 does not expect to transition onto the linear region 1701 until the end point 1706 of the S-shape and maintains the rate at which its pulley rotates through the second point 1703b and the third point 1703c, resulting in an undesirable response (e.g., a jerky or laggy response). In the first scenario 1700, the undesirable response is a jerky response.

[0232] Determination of reverse exit based on linear response crossover can resolve an undesired response. During traversal, for each time sample, joint motion and pulley rotation on the S-shape 1702 can be determined. Each point 1703a, 1703b, 1703c in the set of points 1703a - 3b is associated with the joint motion and pulley rotation. Then, the pulley rotation can be used to calculate the corresponding joint motion on the linear region 1701 for each time sample. The corresponding joint motion on the linear region 1701 is compared with the joint motion on the S-shape 1702 sharing the same pulley rotation. When the sign relationship of the comparison changes, a reverse exit is detected.

[0233] For example, assume that the first point 1703a is traversed at the first time sample "t - 2", the second point 1703b is traversed at the second time sample "t - 1", and the third point 1703c is traversed at the third time sample "t". The set of points 1703a - 1703c is associated with the pulley rotation. Based on the pulley rotation, the corresponding joint motions 1704a - 1704c on the linear region 1701 can be calculated. They are the first corresponding joint motion 1704a at the first time sample, the second corresponding joint motion 1704b at the second time sample, and the third corresponding joint motion at the third time sample.

[0234] At the first time sample, the S-shape joint motion at the first point 1703a is greater than the first corresponding joint motion 1704a. At the second time sample, the S-shape joint motion at the second point 1703b is still greater than the second corresponding joint motion 1704b. However, at the third time sample, the S-shape joint motion at the third point 1703c is less than the third corresponding joint motion 1704c. Between the second time sample and the third time sample, the sign relationship changes. Therefore, a reverse exit is detected between the second time sample and the third time sample. When the endoscope is joint-moved after the third time sample, the robot controller can stop joint-moving the endoscope based on the S-shape 1702 and start joint-moving the endoscope based on the linear region 1701.

[0235] The second scenario 1750 illustrates such a switch of the joint motion scheme from based on the S-shape 1702 to based on the linear region 1701. Traversing from the lower left to the upper right of the second scenario 1750, the robot controller initially joint-moves the endoscope based on the S-shape 1702 and detects a change in the sign relationship at the reverse exit point 1751 during the joint motion. At the reverse exit point 1751, the robot controller that joint-moves the endoscope based on the S-shape 1702 can start joint-moving the endoscope based on the linear region 1701 from the reverse exit point 1751, such that the further joint motion 1752 is consistent with the linear region 1701.

[0236] RegardingFigures 14 to 1 The reverse exit determination method described in 7 can work independently or in any combination. This method can be implemented in combination with the joint movement of the endoscope by robotically controlling the wire tensioning pulley / mechanism associated with the endoscope (e.g., integrated with the handle of the endoscope). This method can be implemented at least in part by the control circuit of any of the system components disclosed herein, such as the robotic cart / system and / or the control tower / system.

[0237] Closed Loop

[0238] Due to the presence of non - linearities in the motion, such as characteristic dead zones, the endoscope may require a large wire displacement when reversing the direction of joint movement. However, inherent limitations such as motor bandwidth and rotational speed may prevent the IDM from delivering the desired instantaneous acceleration / deceleration. Due to these limitations, a trajectory that takes into account the actual motor commands can be generated and used to smooth the control of the IDM.

[0239] Figure 18 Comparison plot 1800 illustrates the relationship between the joint movement 1801 of a user command (desired) and the actual joint movement 1802 delivered by the system according to one or more embodiments. The actual joint movement 1802 delivered by the system can be based on the trajectory. Many robotic controllers are open - loop controllers and assume that the joint movement commanded by the user is instantaneously delivered by the system. However, this assumption may be incorrect and may lead to an error between the joint movement commanded by the user and the actual joint movement. In a robotically controlled endoscope system with an open - loop controller, unless corrected, this error may grow without bound and may ultimately result in uncommanded endoscope movement. Uncommanded endoscope movement may disorient the operator and, even worse, may potentially damage the anatomy. As will be described in more detail with respect to Figure 19 As described in more detail, the closed - loop algorithm can help correct the error by providing feedback of the current joint movement.

[0240] Figure 19 Closed - loop algorithm 1900, according to one or more embodiments, is illustrated that can reduce or eliminate the error between the joint movement commanded by the user and the actual joint movement. Before executing the closed - loop algorithm 1900, some variables can be initialized. For example, the error can be reset to zero. After initialization is complete, the closed - loop algorithm 1900 can be executed for each time sample.

[0241] At box 1902, user input may be received. For example, a robotic system may include components for receiving joint movement commands from a user. Such user input may be received via a controller or other user input device, where manual (or other) engagement with one or more input mechanisms (e.g., buttons, joysticks, sliders, levers, knobs, etc.) may generate joint movement commands received by the robotic system (e.g., robotic control tower / cart).

[0242] In some embodiments, the user input may include joint movement in a certain direction in a plane, such as joint movement in the positive or negative direction on a plane (e.g., Figure 5 the P P ) or a combination of multiple planes. The user input may also include the magnitude of the joint movement of the direction. An example user input could be a slight joint movement to the right.

[0243] At box 1904, a desired joint movement may be determined based on the user input and the previous joint movement calculated at box 1910 for a previous time sample. If the endoscope previously moved slightly to the left in joint, applying a slight joint movement to the right may result in a desired joint movement of neutral joint movement. If the endoscope previously moved slightly to the right, applying a slight joint movement to the right may result in a desired joint movement of a moderate joint movement to the right.

[0244] At box 1906, the pulley rotation required to achieve the desired joint movement may be calculated. As described above, an inverse kinematic model of the desired joint movement may be used to calculate the predicted pulley rotation.

[0245] At box 1908, a trajectory may be generated. As described, the IDM may not be able to deliver the desired instantaneous acceleration / deceleration for the predicted pulley rotation. The trajectory may smooth the control of the IDM by providing the predicted pulley rotation over multiple time samples. The provided pulley rotation for the current time sample is the actual pulley rotation that is fed into the IDM during the current time sample.

[0246] At box 1910, the current joint movement may be calculated. The actual pulley rotation is fed into the IDM to change the endoscope joint movement. A kinematic model may be used to calculate the current joint movement based on the actual pulley rotation. The calculated joint movement may be provided as the previous joint movement for the next time sample to box 1904, thereby providing a feedback mechanism for the desired joint movement for the next time sample. The feedback serves as a closed loop 1912 to ensure that any desired joint movement is instantaneous (or near instantaneous) and reflects the current joint movement rather than the planned joint movement of the trajectory. The closed loop 1912 may limit the error between the user commanded joint movement and the actual joint movement, thereby preventing the error from growing without bound.

[0247] As described, the closed-loop algorithm 1900 can be performed for each time sample. In some embodiments, the closed-loop algorithm 1900 can be performed sporadically. For example, when the error becomes greater than a threshold, the closed-loop algorithm 1900 can be performed. The threshold can be set to an error perceptible to the human eye or an error that may damage the anatomical structure. In some embodiments, the closed-loop algorithm 1900 can be performed according to the requirements of a trigger event. The trigger event can be an error check relative to a threshold, an event known to cause error growth (e.g., direction reversal, model adjustment, safety trigger, etc.), or a combination of events.

[0248] Additional Embodiments

[0249] According to embodiments, certain actions, events, or functions of any of the processes or algorithms described herein can be performed in a different order, can be added, combined, or completely ignored. Thus, in certain embodiments, not all of the described actions or events are necessary for the practice of the process.

[0250] Unless otherwise specifically stated or otherwise understood within the context in which it is used, conditional language used herein, such as "can", "able to", "may", "could", "for example", etc., is meant in its ordinary sense and generally is intended to convey that certain embodiments include while other embodiments do not include certain features, elements, and / or steps. Thus, such conditional language generally is not intended to imply that one or more embodiments necessarily require features, elements, and / or steps, or that one or more embodiments definitely include logic for determining, with or without author input or prompting, whether these features, elements, and / or steps are included in any particular embodiment or will be performed in any particular embodiment. The terms "comprising", "including", "having", etc. are synonymous and are used in their ordinary sense and inclusively in an open-ended manner and do not exclude additional elements, features, acts, operations, etc. Also, the term "or" is used in its inclusive sense (and not in its exclusive sense) such that when used, for example, to connect a series of elements, the term "or" means one, some, or all of the elements of the series. Unless otherwise specifically stated, conjunctive language such as the phrase "at least one of X, Y, and Z" is understood in the context of general use to convey that an item, term, element, etc. can be X, Y, or Z. Thus, such conjunctive language generally is not intended to imply that a particular embodiment requires the presence of at least one of each of X, at least one of Y, and at least one of Z.

[0251] It should be understood that in the above description of the embodiments, for the purpose of simplifying the present disclosure and facilitating the understanding of one or more of the various inventive aspects, various features are sometimes grouped together in a single embodiment, figure, or its description. However, this method of the present disclosure should not be construed as reflecting the intention that any claim requires more features than those expressly recited in that claim. In addition, any component, feature, or step illustrated and / or described in a particular embodiment herein can be applied to any other embodiment or used in conjunction with any other embodiment. Further, for each embodiment, no component, feature, step, or group of components, features, or steps is necessary or indispensable. Accordingly, it is intended that the scope of the invention disclosed herein and claimed below not be limited by the particular embodiments described above, but rather should be determined only by a fair reading of the appended claims.

[0252] It should be understood that for ease of reference, certain ordinal terms (e.g., "first" or "second") may be provided and do not necessarily imply physical characteristics or ordering. Thus, as used herein, ordinal terms (e.g., "first," "second," "third," etc.) used to modify elements such as structures, components, operations, etc. do not necessarily indicate the priority or order of that element relative to any other element, but rather may generally distinguish that element from another element having a similar or identical name (but using an ordinal term). Additionally, as used herein, the indefinite article ("a") may indicate "one or more" rather than "one." Further, an operation performed "based on" a certain condition or event may also be performed based on one or more other conditions or events not expressly recited.

[0253] Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which the example embodiments belong. It will be further understood that terms, such as those defined in a commonly used dictionary, should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and not in an idealized or overly formal sense, unless expressly so defined herein.

[0254] For ease of description, spatial relative terms such as "outer", "inner", "upper", "lower", "below", "above", "vertical", "horizontal", and like terms may be used herein to describe the relationship between one element or component illustrated in the drawings and another. It should be understood that the spatial relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the drawings. For example, if the device shown in the drawings is flipped, a device located "below" or "beneath" another device may be positioned "above" the other device. Thus, the illustrative term "below" can include both a lower and an upper orientation. The device may also be oriented in another direction, and thus the spatial relative terms may be interpreted differently depending on the orientation.

[0255] Unless expressly stated otherwise, comparative and / or quantitative terms, such as "less", "more", "greater", etc., are intended to encompass the concept of equality. For example, "less" may not only mean "less" in the strictest mathematical sense, but may also mean "less than or equal to".

Claims

1. A robotic system comprising: An end effector comprising one or more drive outputs configured to: navigating an elongated shaft within a patient's body, the elongated shaft including a tip at a distal end; articulating the distal end in a first direction using a first cable coupled to a dual-cable pulley; as well as Reversing the articulation of the distal end to a second direction using a second cable coupled to the dual-cable pulley, wherein the distal end articulates based on a nonlinear response region of a kinematic model; processor; and a memory storing computer-executable instructions that, when executed, cause the processor to: determining endpoints of the nonlinear response region; as well as When the joint motion of the tip in the second direction reaches the end point, the joint motion of the tip is controlled based on a linear response region.

2. The robot system according to claim 1, wherein: The memory also includes computer-executable instructions that, when executed, cause the processor to estimate the nonlinear response region of the kinematic model using a sigmoid equation.

3. The robot system according to claim 2, wherein: The sigmoid equation is a generalized logistic function.

4. The robot system according to any one of claims 1 to 3, wherein: The memory further includes computer-executable instructions that, when executed, cause the processor to: receiving a percentage value associated with the nonlinear response region; as well as An S-shaped curve is calculated that passes through the endpoints of the nonlinear response region at the percentage value of the traversal during the traversal of the S-shaped curve, wherein the joint motion of the tip is controlled based on the S-shaped curve before the joint motion of the tip is controlled based on the linear response region.

5. The robot system according to any one of claims 1 to 3, wherein: Determining the endpoint of the nonlinear response region includes: monitoring tension in at least one of the first pull wire or the second pull wire; and The endpoint is determined based on the tension satisfying a threshold condition.

6. The robot system according to claim 5, wherein: The threshold conditions are (i) the tension increases in one direction and (ii) the tension increases by at least a threshold amount.

7. The robot system according to claim 5, wherein: The threshold conditions are (i) the tension increases in one direction and (ii) the tension changes its sign.

8. The robot system according to any one of claims 1 to 3, wherein: Determining the endpoint of the nonlinear response region includes: calculating a first pulley rotation for a first joint motion in the nonlinear response region for a first time sample; calculating a second joint motion in the linear response region based on the first pulley rotation for the first time sample; calculating a second pulley rotation for a third joint motion in the nonlinear response region for a second time sample that is later in time than the first time sample; calculating a fourth joint motion in the linear response region based on the second pulley rotation for the second time sample; and The endpoint is determined based on a comparison between (i) the first joint motion and the second joint motion and (ii) the third joint motion and the fourth joint motion.

9. The robot system according to claim 8, wherein: (i) the first joint motion is smaller than the second joint motion, and (ii) the third joint motion is larger than the fourth joint motion.

10. The robot system according to claim 8, wherein: (i) the first joint motion is greater than the second joint motion, and (ii) the third joint motion is less than the fourth joint motion.

11. A robotic system comprising: an end effector comprising one or more drive outputs configured to navigate an elongated shaft within a patient's body, the elongated shaft including a tip at a distal end; processor; and a memory storing computer-executable instructions that, when executed, cause the processor to: monitoring tension on at least one of a first pull wire or a second pull wire, the at least one of the first pull wire or the second pull wire being coupled to the distal end; and Points associated with a kinematic model are determined based on the tension.

12. The robot system according to claim 11, wherein: The kinematic model includes at least one linear response region and at least one nonlinear response region.

13. The robot system according to claim 12, wherein: The point is associated with a transition between the at least one non-linear response region and the at least one linear response region.

14. The robot system according to claim 13, wherein: Determining the point on the kinematic model includes: The point is determined based on the tension satisfying at least one threshold condition.

15. The robotic system according to claim 14, wherein: The at least one threshold condition is (i) the tension increases in one direction and (ii) the tension changes its sign.

16. The robotic system according to claim 14, wherein: The at least one threshold condition is that (i) the tension increases in one direction and (ii) the tension increases by at least a threshold amount.

17. The robotic system according to claim 14, wherein: The memory further includes computer-executable instructions that, when executed, cause the processor to: updating an end portion of the at least one nonlinear response region based on the point; and Determine the S-shaped and then linear response.

18. The robotic system according to claim 17, wherein: The memory further includes computer-executable instructions that, when executed, cause the processor to: When the joint motion of the end portion reaches the point, the joint motion of the end portion is controlled based on the S-shaped post-linear response.

19. A method for robotically controlling an endoscope, the method comprising: navigating an elongated shaft within a patient's body, the elongated shaft including a tip at a distal end; articulating the distal end in a first direction using a first cable coupled to a dual-cable pulley; Reversing the articulation of the distal end to a second direction using a second cable coupled to the dual-cable pulley, wherein the distal end articulates based on a nonlinear response region of a kinematic model; determining endpoints of the nonlinear response region; and When the articulation of the tip in the second direction reaches the end point, the tip is articulated based on a linear response region.

20. The method according to claim 19, further comprising: receiving a percentage value associated with the nonlinear response region; as well as calculating a sigmoid curve that, during a traversal of the sigmoid curve, passes through the endpoints of the nonlinear response region at the percentage value of the traversal, Before the end portion is subjected to joint motion based on the linear response region, the end portion is subjected to joint motion based on the S-shaped curve.

Images