Robotic control for continuum robot

Abstract

One or more devices, systems, methods, and storage mediums for performing robotic control, correction, adjustment, and / or smoothing for a continuum robot are provided herein. Examples of such control, correction, adjustment, and / or smoothing include, but are not limited to, correction of one or more sections or portions of a continuum robot as the continuum robot is moved, state transition(s) or change(s), and path smoothing for a continuum robot. Examples of applications include imaging, evaluating, and diagnosing biological objects, such as, but not limited to, for Gastro-intestinal, cardio, bronchial, and / or ophthalmic applications, and being obtained via one or more optical instruments, such as, but not limited to, optical probes, catheters, endoscopes, and bronchoscopes. Techniques provided herein also improve processing and imaging efficiency while achieving images that are more precise, and also achieve devices, systems, methods, and storage mediums that reduce mental and physical burden and improve ease of use.

Description

ROBOTIC CONTROL FOR CONTINUUM ROBOTCROSS-REFERENCE TO RELATED APPLICATION(S)

[0001] This application relates, and claims priority, to U.S. Prov. Patent Application Serial No. 63 / 585,128, filed September 25, 2023, the disclosure of which is incorporated by reference herein in its entirety.FIELD OF THE DISCLOSURE

[0002] The present disclosure generally relates to imaging and, more particularly, to a continuum robot apparatus, method, and storage medium to implement robotic control for all sections of a catheter or imaging device / apparatus or system to match a state or states when each section reaches or approaches a same or similar, or approximately a same or similar, state or states of a first section of the catheter or imaging device, apparatus, or system. One or more endoscopic, medical, camera, catheter, or imaging devices, systems, and methods and / or storage mediums for use with same, are discussed herein. One or more devices, methods, or storage mediums may be used for medical applications and, more particularly, to steerable, flexible medical devices that may be used for or with guide tools and devices in medical procedures, including, but not limited to, endoscopes, cameras, and catheters.BACKGROUND

[0003] Endoscopy, bronchoscopy, catheterization, and other medical procedures facilitate the ability to look inside a body. During such a procedure, a flexible medical tool may beinserted into a patient’s body, and an instrument may be passed through the tool to examine or treat an area inside the body. For example, a bronchoscope is an endoscopic instrument to view inside the airways of a patient. Catheters and other medical tools may be inserted through a tool channel in the bronchoscope to provide a pathway to a target area in the patient for diagnosis, planning, medical procedure(s), treatment, etc.

[0004] Robotic bronchoscopes, robotic endoscopes, or other robotic imaging devices may be equipped with a tool channel or a camera and biopsy tools, and such devices (or users of such devices) may insert / retract the camera and biopsy tools to exchange such components. The robotic bronchoscopes, endoscopes, or other imaging devices may be used in association with a display system and a control system.

[0005] An imaging device, such as a camera, may be placed in the bronchoscope, the endoscope, or other imaging device / system to capture images inside the patient and to help control and move the bronchoscope, the endoscope, or the other type of imaging device, and a display or monitor may be used to view the captured images. An endoscopic camera that may be used for control may be positioned at a distal part of a catheter or probe (e.g., at a tip section).

[0006] The display system may display, on the monitor, an image or images captured by the camera, and the display system may have a display coordinate used for displaying the captured image or images. In addition, the control system may control a moving direction of the tool channel or the camera. For example, the tool channel or the camera may be bent according to a control by the control system. The control system may have an operational controller (such as, but not limited to, a joystick, a gamepad, a controller, an input device, etc.), and physicians may rotate or otherwise move the camera, probe, catheter, etc. to control same. However, such control methods or systems are limited in effectiveness. Indeed, while information obtained from an endoscopic camera at a distal end or tip section may help decidewhich way to move the distal end or tip section, such information does not provide details on how the other bending sections or portions of the bronchoscope, endoscope, or other type of imaging device may move to best assist the navigation.

[0007] However, while a camera may provide information for how to control a most distal part of a catheter or a tip of the catheter, the information is limited in that the information does not provide details about how the other bending sections of the catheter or probe should move to best assist the navigation.

[0008] As such, there is a need for devices, systems, methods, and / or storage mediums that provide the feature(s) or details on how the other bending sections or portions of such imaging devices, imaging systems, etc. (e.g., endoscopic devices, bronchoscopes, other types of imaging devices / systems, etc.) may move to best assist navigation and / or state or state(s) for same, to keep track of a path of a tip of the imaging devices, imaging systems, etc., and to address any discontinuous path or shortcomings of directing such devices or systems along a path that may occur.

[0009] Accordingly, it would be desirable to provide at least one imaging, optical, or control device, system, method, and storage medium for controlling one or more endoscopic or imaging devices or systems, for example, by implementing automatic (e.g., robotic) or manual control of each portion or section of the at least one imaging, optical, or control device, system, method, and storage medium to keep track of and to match the state or state(s) of a first portion or section in a case where each portion or section reaches or approaches a same or similar, or approximately same or similar, state or state(s) and to address any discontinuous path that may occur due to movement or a change in a state or state(s). It would also be desirable to provide at least one imaging, optical, or control device, system, method, and storage medium for controlling one or more endoscopic or imaging devices or systems whileusing or including additional information (z.e., in addition to historical tip pose information for each stage position).SUMMARY

[0010] Accordingly, it is a broad object of the present disclosure to provide imaging (e.g., computed tomography (CT), Magnetic Resonance Imaging (MRI), etc.) apparatuses, systems, methods, and storage mediums for using a navigation and / or control method or methods (manual or automatic) in one or more apparatuses or systems (e.g., an imaging apparatus or system, an endoscopic imaging device or system, etc.). The navigation and / or control maybe employed so that an apparatus or system having multiple portions or sections (e.g., multiple bending portions or sections) operates to: (i) keep track of a path of a portion (e.g., a tip) or of each of the multiple portions or sections of an apparatus or system; (ii) have a state or states of each of the multiple portions or sections match a state or states of a first portion or section of the multiple portions or sections in a case where each portion or section reaches or approaches a same, similar, or approximately similar state (e.g., a position or other state(s) in a target, object, or specimen; a position or other state(s) in a patient; a target position or state(s) in an image or frame; a set or predetermined position or state(s) in an image or frame; a set or predetermined position or state(s) in an image or frame where the first portion or section reaches or approaches the set or predetermined position or state(s) at one point in time and one or more of other portions or sections of the multiple portions or sections reach the set or predetermined position or state(s) at one or more other points in time; any other state (which may include, but is not limited to, an orientation, a position, a pose, a navigation, a path (whether continuous or discontinuous), a state transition, any other desired motion(s) or combination of motion(s) (e.g., one or more features of the present disclosure may assist navigation, orientation, or any other types of motions discussed herein or as desired by a user), etc.), a combination of any state(s) and / or motion(s) discussed herein or desired by a user, etc.) of another portion or section of the one or more devices, systems, methods, and / or storage mediums of the present disclosure, etc.); (iii) utilize additional data (such as, but notlimited to, target pose or state information, final pose or state information, interpolated pose or state information, measured pose or state information, converting pose or state information between different states (e.g., drive wire position(s) or state(s); coordinates (three- dimensional (3D) position(s), orientation(s), and / or state(s)); plane and / or angle information for pose(s) or state(s); state position or state information (e.g., target, interpolated, measured pose(s) and / or state(s)); force sensor(s) information; draw or current draw information of one or more actuator motors; section dimension information (e.g., size, shape, length, etc.) for one or more sections of the catheter or probe (e.g., a tip section of the catheter or probe, a middle section of the catheter or probe, a distal section of the catheter or probe, a proximal section of the catheter or probe, any combination thereof, etc.) from an entire device or system (e.g., using forwards or inverse kinematics of the device or system, using other internal sensor(s) or information of the device or system, etc.) and / or external source(s) (e.g., one or more external sensors (e.g., an electromagnetic (EM) sensor, a shape sensor, any other sensor discussed herein or known to those skilled in the art, etc.) in a robotic control algorithm; (iv) utilize differences between a previous or expected robotic control state and a new control state in future calculation(s) of robotic control state(s); and / or (v) address any discontinuous path that may occur (e.g., due to a change (e.g., of a state or states) or other movement or state change / transition of any portion of the apparatus or system), for example, by smoothing out any difference in the discontinuous path over one or more multiple stage positions or states or over one or more other path-like information positions or states, by considering target or object movement(s) (e.g., movement of a patient or a portion of a patient while a probe or catheter is disposed in the patient, etc.).

[0011] In one or more embodiments, pose or state information may be stored in a lookup table or tables, and the pose or state information for one or more sections of the catheter or probe may be updated in the lookup table based on new information (e.g., environmental change(s) for the catheter or probe, movement of a target or sample, movement of a patient, user control, relaxation state changes, etc.). The new information or the updated informationmay be used to control the one or more sections of the catheter or probe more efficiently during navigation (forwards and / or backwards). For example, in a case where a previously stored pose or state may have shifted or changed due to a movement or relaxation of the target, object, or sample (e.g., a patient may move), the previously stored pose or state may not be ideal or may work less efficiently as compared with an updated pose or state modified or updated in view of the new information (e.g., the movement, in this example). As such, one or more embodiments of the present disclosure may update or modify the pose or state information such that robotic control of the catheter or probe may work efficiently in view of the new information, movement, relaxation, and / or environmental change(s). In addition to having the update or change affect the previously stored history or known histoiy at that point in space (e.g., similar to dragging that point (e.g., of the target, object, or sample (e.g., a patient, a portion of a patient, a vessel, a spline, a lung, etc.); of the catheter or probe; etc.) and recalculating the path), in one or more embodiments, the update or change may also affect a number of other points (e.g., all points in a lookup table or tables, all points forward beyond the initially changed point, one or more future points or points beyond the initially changed point as desired, etc.). For example, in one or more embodiments, the transform (or difference, change, update, etc.) between the previous pose or state and the new or updated post or state may be propagated to all points going forward or may be propagated to one or more of forward points (e.g., for a predetermined or set range, for a predetermined or set distance, etc.). Doing so in one or more embodiments may operate to shift all or part of the future path based on how the pose or state of the catheter or probe was adjusted, using that location as a pivot point. Such update(s) may be obtained from one or more internal sources (e.g., one or more processors, one or more sensors, combination(s) thereof, etc.) or may be obtained from one or more external sources (e.g., one or more other processors, one or more external sensors, combination(s) thereof, etc.). For example, a difference between a real-time target, sample, or object (e.g., an airway) and the previous target, sample, or object (e.g., a previous airway) may be detected using machine vision (of the endoscope image) or using multiple medical images. Body, target, object, or sample divergence may also be estimatedfrom other sensors, like one measuring breathing or the motion of the body (or another predetermined or set motion or change to track).

[0012] One or more robotic control methods of the present disclosure may be employed in one or more embodiments. For example, one or more of the following techniques or methods maybe used: Hold the Line, Close the Gap, and / or Stay the Course. In one or more embodiments, a Tip position or state may be converted to a coordinate (e.g., X, Y, Z coordinate) during navigation (e.g., while a stage moves forward, while the navigation is mapped to Z stage position, etc.). In a case where a pose or state is being determined for other section(s) of the catheter or probe, one or more embodiments use a pose or state which operates to keep an end effector of the section(s) as close as possible to the Tip position or state coordinate at that point along the insertion path. The pose or state coordinates of preceding section(s) of the catheter or probe may be known or determined in one or more embodiments. Body, target, object, or sample divergence may be applied with a coordinate shift. In cases where a length of each section or sections of the catheter or probe are not considered, applying a Tip pose or state to a middle / proximal section of the catheter or probe may result in a situation where a middle / proximal end effector maybe in a drastically different location than the Tip end effector at that point along the path (e.g., a middle / proximal section maybe longer or maybe twice as long as the Tip section, for example). As such, by considering the lengths of each section or sections of the catheter or probe, appropriate adjustment(s) may be made so that the middle / proximal end effector, and / or an effector of another section or portion of the catheter or probe, may be in an appropriate / desired / intended location or may be in the same location as the Tip end effector was at that point along the path. Using such techniques, one or more embodiments of the present disclosure may apply one or more Hold the Line features to a catheter or probe to achieve efficient and accurate control of the catheter or probe.

[0013] In one or more embodiments, one or more other or additional robotic control methods or techniques may be employed. For example, one or more gaps may be closed by adjusting a pose or state of a section or sections of a catheter or probe to reduce, minimize, or eliminate a relative difference between the pose or state of the section or sections versus a pose or state of the subsequent / next / future / following section or sections. In one or more embodiments, a trajectory of the subsequent / next / future / following section or sections may be known or determined. Closing such a difference or gap operates to eliminate any motion outside of a desired direction or trajectory. For example, if the Tip section faced left (or to a first direction), then moved forwards slightly, and then faced right (or a towards a second direction), then a middle section may also be moved left in a case where the middle section reaches the point that the Tip section was facing left (or to the first direction). However, since the Tip section wants to be moving right (or towards the second direction) after that point, then the motion of the Tip section to the left (or first) direction will be moving counter-actively to the desired direction, which may be the right (or the second) direction. Instead, using one or more features of the present disclosure, any counter-active motion(s) may be avoided or eliminated.

[0014] In one or more embodiments, one or more further robotic control methods or techniques may be employed. For example, a catheter or probe may be controlled to stay the desired course. For example, a pose or state of a section or sections may be adjusted to minimize any deviation of a pose or state of one or more next sections out of the predetermined, targeted, desired trajectory and maximum motion along the trajectory. In one or more embodiments, the coordinates and the trajectory of subsequent / following / next / future sections may be known, set, or determined, and information for one or more prior sections may be known, set, or determined. By considering section lengths, one or more advantageous results may be achieved. As aforementioned, in one or more embodiments, a middle / proximal section end effector may device from a location of a Tip section at that point. Such a deviation may cause the Tip section to move in directionsthat are not in the direction it is facing. This effect may be compounded in a case where a proximal / middle section is included in the movement or control. As such, by using one or more features of the present disclosure, any counter-active or undesired motion(s) may be avoided or eliminated.

[0015] In one or more embodiments, a continuum robot for performing robotic control may include: one or more processors that operate to: instruct or command a first bending section or portion of a catheter or a probe of the continuum robot such that the first bending section or portion achieves, or is disposed at, a pose, position, or state at a position along a path, the catheter or probe of the continuum robot having a plurality of bending sections or portions and a base; instruct or command each of the other bending sections or portions of the plurality of bending sections or portions of the catheter or probe to match, substantially match, or approximately match the pose, position, or state of the first bending section or portion at the position along the path in a case where each section or portion reaches or approaches a same, similar, or approximately similar state or states at the position along the path; and instruct or command the plurality of bending sections or portions such that the first bending section or portion or a Tip or distal bending section or portion is located in a predetermined pose, position, or state at or near a distal end of the path. A first bending section or portion or the Tip or distal bending section or portion may include a camera, an endoscopic camera, a sensor, or other imaging device or system to obtain one or more images of or in a target, sample, or object; and the one or more processors may further operate to command the camera, sensor, or other imaging device or system to obtain the one or more images of or in the target, sample, or object at the predetermined pose, position, or state, and the one or more processors operate to receive the one or more images and / or display the one or more images on a display.

[0016] In one or more embodiments, the one or more processors may further operate to:(i) track a respective path of one or more of sections or paths of the catheter or probe, or tracka respective path of each section or portion of the catheter or probe; (ii) instruct or command each section or portion to adjust the respective path of each section or path to match, substantially match, or approximately match the path of the first section or portion of the catheter or probe and / or to address any discontinuous path due a change in a state or states and / or due to a change or a transition of a state or movement of the catheter or probe and / or the continuum robot or of the target, sample, or object; and (iii) utilize additional data to control or command the plurality of sections or portions by including the additional data in the robotic control and / or by updating historical information or the pose, position, or state information for the sections or portions of the catheter or probe with the additional data and / or with differences between a previous or expected robotic control state or states and a new control state or states in future calculation(s) of robotic control state(s) such that an efficiency of the navigation or movement of, or of the robotic control of, the sections or portions of the continuum robot is increased or improved. The additional data may include one or more of the following: target pose, position, or state information; final pose, position, or state information; interpolated pose, position, or state information; measured pose, position, or state information; information for converting pose, position, or state information between different states; target pose, position, or state information; measured pose, position, or state information; force sensor(s) information; draw or current draw information of one or more actuator motors of the catheter or probe; section or portion size and / or shape information; section or portion length information; kinematic information of the catheter or probe and / or of the continuum robot; information for internal and / or external sensor(s) of the catheter or probe and / or the continuum robot; electromagnetic sensor information; shape sensor information; sensor information; target, sample, or object divergence or movement information; information for environmental change(s) for the catheter or probe; relaxation state change(s); and / or changes due to one or more user controls of or for the continuum robot; and the different states may include one or more of the following: drive wire position(s) or state(s) for one or more drive wires of the catheter or probe; coordinates for the sections or portions; three-dimensional (3D) position(s), orientation(s), and / or state(s) for the sectionsor portions; plane and / or angle information for pose(s) or state(s) of the sections or portions; and / or position(s) or state(s) of the base or a translational stage or motor(s) of the continuum robot. In one or more embodiments, the additional data and / or the pose, position, or state information may be stored in a lookup table or tables or is used to update or change a number of points or values for the sections or portions of the catheter or probe; and, in a case where the additional data and / or the pose, position, or state information changes or updates the number of points, the additional data and / or the pose, position, or state information may operate to one or more of the following: affect or update one or more points or values in the lookup table or tables, affect or update all points or values forward or beyond an initially changed point or value; affect or update one or more points or values different than an initially changed point or value; affect or update one or more points or values for a predetermined or set range or distance after or from an initially changed point or value; and / or affect or update one or more points or values different than an initially changed point or value by using the initially changed point or value as a pivot point or value.

[0017] The one or more processors may further operate to utilize the additional data to control or command the plurality of sections or portions of the catheter or probe by including the additional data and / or by updating the historical information or the pose, position, or state information by using one or more of the following modes: a Hold the Line mode, a Close the Gap mode, and / or a Stay the Course mode. In a case where the one or more processors are using the Hold the Line mode, the one or more processors may further operate to: (i) instruct or command the catheter or probe to move forward or to move along the path; (ii) calculate coordinates for a Tip or distal end effector of the Tip or distal bending section or portion or for an end effector of the first section or portion; (iii) add the calculated coordinate information to the path, or to a three-dimensional (3D) path, for the Tip or distal end, section, or portion and / or the catheter or probe; (iv) calculate coordinates for a Middle / proximal end effector of a Middle / proximal, or other section or portion subsequent to or following the Tip or distal or first bending section or portion, of the catheter or probe; (v) identify or calculate a distancefrom a closest point along the path or the 3D path for the Middle / proximal end effector and / or the Tip or distal or first portion end effector; (vi) convert the identified or calculated distance to a change in a pose, position, or state of the Tip or distal end effector, of the Tip or distal bending section or portion, of the Middle / proximal end effector, of the Middle / proximal section or portion, and / or of the catheter or probe; and (vii) update the pose, position, or state of the Middle / proximal section or portion of the catheter or probe to match the pose, position, or state of the Tip or distal bending section or portion, or of the first section or portion, of the catheter or probe at that point along the path or the 3D path. In one or more embodiments, one or more of the following may occur: the one or more processors further operate to evaluate or determine whether the identified or calculated distance is greater than zero and / or is within a threshold, and / or whether a change in the pose, position, or state occurred for the Tip or distal bending section or portion or for the first section or portion; in a case where the catheter or probe is instructed or commanded to move forward, the catheter or probe moves forward during one or more of the following: while a stage or linear stage of the catheter or robot moves forward; while the navigation is mapped to Z stage position; while converting a position, pose, or state of the Tip or distal bending section or portion or the first section or portion of the catheter or probe to a coordinate; and / or during navigation; the coordinate(s) use an X, Y, Z coordinate system; in a case where a pose, position, or state is being determined for other section(s) or portion(s) of the catheter or probe, the one or more processors further operate to use a pose, position, or state which operates to keep an end effector of the section(s) or portion(s) as close as possible to the Tip or distal pose, position, or state coordinate at that point along the path or the 3D path; the pose, position, or state coordinates of preceding section(s) or portion(s) of the catheter or probe are known or determined; the one or more processors further operate to apply divergence or difference of the target, object, or sample with a coordinate shift; and / or the one or more processors further operate to consider a length of each section(s) or portion(s) of the catheter or probe to update or change a respective pose, position, or state for the section(s) or portion(s) so that each of the section(s) or portion(s) are located in a predetermined or target location or are located in the same or similar location asthe Tip or distal bending section or portion or the first bending section or portion at an evaluated point along the path or the 3D path.

[0018] In a case where the one or more processors are using the Close the Gap mode, the one or more processors may further operate to: (i) identify, determine, calculate, or obtain a pose, position, or state of a Middle / proximal section or portion of the catheter or probe; (ii) identify, determine, calculate, or obtain a pose, position, or state of a Tip or distal bending section or portion or of the first bending section or portion of the catheter or probe; (iii) determine, identify, calculate or obtain a difference between the poses, positions, or states of the Tip or distal bending section or portion and of the Middle / proximal, or other subsequent or following, bending section or portion; (iv) interpolate, over a set or predetermined length of the path or a three-dimensional (3D) path, the pose, position, or state difference between the Tip or distal bending section or portion or the first section or portion and the Middle / proximal, or other subsequent or following, bending section or portion; and (v) update the pose, position, or state of the Middle / proximal, or other subsequent or following, bending section or portion of the catheter or probe using a corresponding interpolated pose, position, or state difference. In one or more embodiments, one or more of the following may occur: the one or more processors further operate to consider a motion of the catheter or probe when updating the pose, position, or state of the Middle / proximal, or other subsequent or following, bending section or portion of the catheter or probe; information for navigation of the catheter or probe is obtained and used to update the pose, position, or state of the Middle / proximal, or other subsequent or following, bending section or portion of the catheter or probe; the one or more processors further operate to evaluate or determine whether the determined, identified, calculated, or obtained difference is greater than zero and / or is within a threshold, and / or whether a change in pose, position, or state occurred for the Tip or distal bending section or portion, the first bending section or portion, and / or the Middle / proximal, or other subsequent or following, bending section or portion of the catheter or probe; the one or more processors further operate to close one or more gaps between different sections or portions by adjustinga pose, position, or state of the section(s) or portion(s) of the catheter or probe to reduce, minimize, or eliminate a relative difference between the pose, position, or state of the Tip or distal, or the first, section or portion as compared with the pose, position, or state of the Middle / proximal bending section or portion or any other subsequent or following section(s) or portion(s) of the catheter or probe; a respective trajectory of each of the plurality of sections or portions are known or determined; and / or the one or more processors instruct or control movement or navigation of each of the plurality of sections or portions such that any counteractive motion or motions are avoided or eliminated and such that any gap between respective trajectories of each of the plurality of sections or portions is eliminated or avoided.

[0019] In a case where the one or more processors are using the Stay the Course mode, the one or more processors further operate to: (i) instruct or command the catheter or probe to move forward or to move along the path; (ii) calculate a vector, a normal path, or a trajectory for a Tip or distal end effector of the Tip or distal bending section or portion or for an end effector of the first section or portion; (iii) calculate a deviation of the Tip or distal end effector of the Tip or distal bending section or portion or for the end effector of the first section or portion from the vector, the normal path, or the trajectory due to a shape and / or motion of the catheter or probe; (iv) calculate a change to the pose, position, or state of the Middle / proximal section or portion, or other section or portion subsequent to or following the Tip or distal bend section or portion or the first bending section or portion, of the catheter or probe to counteract or remove the calculated deviation; and (v) update the pose, position, or state of the Middle / proximal, or other subsequent or following, section or portion of the catheter or probe to match, substantially match, or approximately match the pose, position, or state of the Tip or distal bending section or portion, or of the first bending section or portion, of the catheter or probe at that point along the path, to eliminate or remove the calculated deviation. In one or more embodiments, one or more of the following may occur: the motion of the catheter or the probe is one or more of the following: a motion from movement of a stage or translational stage of the continuum robot; a motion from one or more changes due to anenvironment or due to the target, object, or sample in which the catheter or probe is located; a body divergence of a patient, target, object, or sample divergence; and / or a motion from another source external to the continuum robot; the one or more processors further operate to evaluate or determine whether a path, vector, or trajectory deviation exists due to a shape and / or motion of the catheter or probe; a motion of the catheter or probe includes one or more of the following: a motion of the base or a stage; a motion due to translational movement of the catheter or probe and / or the continuum robot; a motion or movement of the target, object, or sample; and / or a motion from a force or influence originating outside of the continuum robot; the one or more processors further operate to use a normal path, vector, or trajectory deviation as a trigger for, and in, the calculation of the calculated deviation of the Tip or distal end effector of the Tip or distal bending section or portion or for the end effector of the first section or portion from the vector, the normal path, or the trajectory due to a shape and / or motion of the catheter or probe; the one or more processors operate to control the catheter or probe to stay or maintain a desired course along the path; the one or more processors operate to control and adjust a pose, position, or state of a section(s) or portion(s) of the catheter or probe to minimize any deviation of a pose, position, or state of one or more Middle / proximal, subsequent, or following section(s) or portion(s) based on a predetermined, targeted trajectory and a maximum motion along the predetermined, targeted trajectory; coordinates for, and the vector, normal path, or trajectory of subsequent, following, or future section(s) or portion(S) are known, set, or determined; information for one or more prior section(s) or portion(s) of the catheter or probe are known, set, or determined; and / or the one or more processors instruct or control movement or navigation of each of the plurality of sections or portions such that any counter-active motion or motions are avoided or eliminated and such that a predetermined or targeted course is maintained between respective trajectories of each of the plurality of sections or portions.

[0020] In a case where a difference exists, or the differences exist, between a previous or expected pose, position, or state and a new pose, position, or state for any of the sections orportions of the catheter or probe or in a case where a difference exists between real-time information for the target, sample, or object and previous information for the target, sample, or object, the difference or differences may be detected using machine vision of the one or more images and / or using machine vision of the catheter or probe image(s) or the difference or differences are detected using one or more sensors for detecting the difference or divergence.

[0021] One or more embodiments of a method for performing robotic control for a continuum robot may include: instructing or commanding a first bending section or portion of a catheter or a probe of the continuum robot such that the first bending section or portion achieves, or is disposed at, a pose, position, or state at a position along a path, the catheter or probe of the continuum robot having a plurality of bending sections or portions and a base; instructing or commanding each of the other bending sections or portions of the plurality of bending sections or portions of the catheter or probe to match, substantially match, or approximately match the pose, position, or state of the first bending section or portion at the position along the path in a case where each section or portion reaches or approaches a same, similar, or approximately similar state or states at the position along the path; and instructing or commanding the plurality of bending sections or portions such that the first bending section or portion or a Tip or distal bending section or portion is located in a predetermined pose, position, or state at or near a distal end of the path. The method may further include utilizing additional data to control or command the plurality of sections or portions of the catheter or probe by including the additional data and / or by updating historical information or pose, position, or state information by using one or more of the following modes: a Hold the Line mode, a Close the Gap mode, and / or a Stay the Course mode. The method may further include using the Hold the Line mode and performing the following: (i) instructing or commanding the catheter or probe to move forward or to move along the path; (ii) calculating coordinates for a Tip or distal end effector of the Tip or distal bending section or portion or for an end effector of the first section or portion; (iii) adding the calculated coordinate information to thepath, or to a three-dimensional (3D) path, for the Tip or distal end, section, or portion and / or the catheter or probe; (iv) calculating coordinates for a Middle / proximal end effector of a Middle / proximal, or other section or portion subsequent to or following the Tip or distal or first bending section or portion, of the catheter or probe; (v) identifying or calculating a distance from a closest point along the path or the 3D path for the Middle / proximal end effector and / or the Tip or distal or first portion end effector; (vi) converting the identified or calculated distance to a change in a pose, position, or state of the Tip or distal end effector, of the Tip or distal bending section or portion, of the Middle / proximal end effector, of the Middle / proximal section or portion, and / or of the catheter or probe; and (vii) updating the pose, position, or state of the Middle / proximal section or portion of the catheter or probe to match the pose, position, or state of the Tip or distal bending section or portion, or of the first section or portion, of the catheter or probe at that point along the path or the 3D path.

[0022] The method(s) may further include using the Close the Gap mode and performing the following: (i) identifying, determining, calculating, or obtaining a pose, position, or state of a Middle / proximal section or portion of the catheter or probe; (ii) identifying, determining, calculating, or obtaining a pose, position, or state of a Tip or distal bending section or portion or of the first bending section or portion of the catheter or probe; (iii) determining, identifying, calculating, or obtaining a difference between the poses, positions, or states of the Tip or distal bending section or portion and of the Middle / proximal, or other subsequent or following, bending section or portion; (iv) interpolating, over a set or predetermined length of the path or a three-dimensional (3D) path, the pose, position, or state difference between the Tip or distal bending section or portion or the first section or portion and the Middle / proximal, or other subsequent or following, bending section or portion; and (v) updating the pose, position, or state of the Middle / proximal, or other subsequent or following, bending section or portion of the catheter or probe using a corresponding interpolated pose, position, or state difference.

[0023] The method(s) may further include using the Stay the Course mode and performing the following: (i) instructing or commanding the catheter or probe to move forward or to move along the path; (ii) calculating a vector, a normal path, or a trajectory for a Tip or distal end effector of the Tip or distal bending section or portion or for an end effector of the first section or portion; (iii) calculating a deviation of the Tip or distal end effector of the Tip or distal bending section or portion or for the end effector of the first section or portion from the vector, the normal path, or the trajectory due to a shape and / or motion of the catheter or probe; (iv) calculating a change to the pose, position, or state of the Middle / proximal section or portion, or other section or portion subsequent to or following the Tip or distal bend section or portion or the first bending section or portion, of the catheter or probe to counteract or remove the calculated deviation; and (v) updating the pose, position, or state of the Middle / proximal, or other subsequent or following, section or portion of the catheter or probe to match, substantially match, or approximately match the pose, position, or state of the Tip or distal bending section or portion, or of the first bending section or portion, of the catheter or probe at that point along the path, to eliminate or remove the calculated deviation.

[0024] The method(s) may further include any of the features discussed herein that may be used in the one or more apparatuses of the present disclosure.

[0025] In one or more embodiments, a non-transitory computer- readable storage medium may store at least one program for causing a computer to execute a method for performing correction, adjustment, and / or smoothing for a continuum robot, where the method may include: instructing or commanding a first bending section or portion of a catheter or a probe of the continuum robot such that the first bending section or portion achieves, or is disposed at, a pose, position, or state at a position along a path, the catheter or probe of the continuum robot having a plurality of bending sections or portions and a base; instructing or commanding each of the other bending sections or portions of the plurality of bending sections or portions of the catheter or probe to match, substantially match, or approximately match the pose,position, or state of the first bending section or portion at the position along the path in a case where each section or portion reaches or approaches a same, similar, or approximately similar state or states at the position along the path; and instructing or commanding the plurality of bending sections or portions such that the first bending section or portion or a Tip or distal bending section or portion is located in a predetermined pose, position, or state at or near a distal end of the path. The storage medium(s) may further include any of the features discussed herein that may be used in the one or more apparatuses or methods of the present disclosure, including, but not limited to, one or more of the following techniques or methods maybe used: Hold the Line, Close the Gap, and / or Stay the Course.

[0026] In one or more embodiments, an orientation, pose, or state may include one or more degrees of freedom. For example, in at least one orientation embodiment, two (2) degrees of freedom may be used, which may include an angle for a magnitude of bending and a plane for a direction of bending. In one or more embodiments, matching state(s) may involve matching, duplicating, mimicking, or otherwise copying other characteristics, such as, but not limited to, vectors for each section or portion of the one or more sections or portions of a probe or catheter, for different portions or sections of the catheter or probe. For example, a transition or change from a base angle / plane to a target angle / plane may be set or predetermined using transition values (e.g., while not limited hereto, a base orientation or state may have a stage at o mm, an angle at o degrees, and a plane at o degrees whereas a target orientation or state may have a stage at 20mm, an angle at 90 degrees, and a plane at 180 degrees. The intermediate values for the stage, angle, and plane may be set depending on how many transition orientations or states may be used).

[0027] In one or more embodiments, a continuum robot or steerable catheter may include one or more of the following: (i) a distal bending section or portion, wherein the distal bending section or portion is commanded or instructed automatically or based on an input of a user of the continuum robot or steerable catheter; (ii) a plurality of bending sections or portionsincluding a distal or most distal bending portion or section and the rest of the plurality of the bending sections or portions; and / or (iii) the one or more processors further operate to instruct or command the forward motion, or the motion in the set or predetermined direction, of the motorized linear stage and / or of the continuum robot or steerable catheter automatically and / or based on an input of a user of the continuum robot. A continuum robot or steerable catheter may further include: a base and an actuator that operates to bend the plurality of the bending sections or portions independently; and a motorized linear stage and / or a sensor that operates to move the continuum robot or steerable catheter forward and backward, and / or in the predetermined or set direction or directions, wherein the one or more processors operate to control the actuator and the motorized linear stage and / or the sensor. The plurality of bending sections or portions may each include driving wires that operate to bend a respective section or portion of the plurality of sections or portions, wherein the driving wires are connected to an actuator so that the actuator operates to bend one or more of the plurality of bending sections or portions using the driving wires. One or more embodiments may include a user interface of or disposed on a base, or disposed remotely from a base, the user interface operating to receive an input from a user of the continuum robot or steerable catheter to move one or more of the plurality of bending sections or portions and / or a motorized linear stage and / or a sensor, wherein the one or more processors further operate to receive the input from the user interface, and the one or more processors and / or the user interface operate to use a base coordinate system. One or more displays may be provided to display a path (e.g., a control path) of the continuum robot or steerable catheter. In one or more embodiments, one or more of the following may occur: (i) the continuum robot may further include an operational controller or j oystick that operates to issue or input one or more commands or instructions as an input to one or more processors, the input including an instruction or command to move one or more of a plurality of bending sections or portions and / or a motorized linear stage and / or a sensor; (ii) the continuum robot may further include a display to display one or more images taken by the continuum robot; and / or (iii) the continuum robot may further include an operational controller or joystick that operates toissue or input one or more commands or instructions to one or more processors, the input including an instruction or command to move one or more of a plurality of bending sections or portions and / or a motorized linear stage and / or a sensor, and the operational controller or joystick operates to be controlled by a user of the continuum robot. In one or more embodiments, the continuum robot or the steerable catheter may include a plurality of bending sections or portions and may include an endoscope camera, wherein one or more processors operate or further operate to receive one or more endoscopic images from the endoscope camera, and wherein the continuum robot further comprises a display that operates to display the one or more endoscopic images.

[0028] Any discussion of a state, pose, position, orientation, navigation, path, or other state type discussed herein is discussed merely as a non-limiting, non-exhaustive embodiment example, and any state or states discussed herein may be used interchangeably / alternatively or additionally with the specifically mentioned type of state. Driving and / or control technique(s) may be employed to adjust, change, or control any state, pose, position, orientation, navigation, path, or other state type that maybe used in one or more embodiments for a continuum robot or steerable catheter.

[0029] Physicians or other users of the apparatus or system may have reduced or saved labor and / or mental burden using the apparatus or system due to the navigation, control, and / or orientation (or pose, or position, etc.) feature(s) of the present disclosure. Additionally, one or more features of the present disclosure may achieve a minimized or reduced interaction with anatomy (e.g., of a patient), object, or target (e.g., tissue) during use, which may reduce the physical and / or mental burden on a patient or target. In one or more embodiments of the present disclosure, a labor of a user to control and / or navigate (e.g., rotate, translate, etc.) the imaging apparatus or system or a portion thereof (e.g., a catheter, a probe, a camera, one or more sections or portions of a catheter, probe, camera, etc.) is saved or reduced via use of the navigation and / or control technique(s) of the present disclosure.

[0030] In one or more embodiments, an imaging device or system, or a portion of the imaging device or system (e.g., a catheter, a probe, etc.), the continuum robot, and / or the steerable catheter may include multiple sections or portions, and the multiple sections or portions may be multiple bending sections or portions. In one or more embodiments, the imaging device or system may include manual and / or automatic navigation and / or control features. For example, a user of the imaging device or system (or steerable catheter, continuum robot, etc.) may control each section or portion, and / or the imaging device or system (or steerable catheter, continuum robot, etc.) may operate to automatically control (e.g., robotically control) each section or portion, such as, but not limited to, via one or more navigation, movement, and / or control techniques of the present disclosure.

[0031] Navigation, control, and / or orientation feature(s) may include, but are not limited to, implementing mapping of a pose (angle value(s), plane value(s), etc.) of a first portion or section (e.g., a tip portion or section, a distal portion or section, a predetermined or set portion or section, a user selected or defined portion or section, etc.) to a stage position / state (or a position / state of another structure being used to map path or path-like information), controlling angular position(s) of one or more of the multiple portions or sections, controlling rotational orientation or position(s) of one or more of the multiple portions or sections, controlling (manually or automatically (e.g., robotically)) one or more other portions or sections of the imaging device or system (e.g., continuum robot, steerable catheter, etc.) to match the navigation / orientation / position / pose of the first portion or section in a case where the one or more other portions or sections reach (e.g., subsequently reach, reach at a different time, etc.) the same position (e.g., in a target, in an object, in a sample, in a patient, in a frame or image, etc.) during navigation in or along a first direction of a path of the imaging device or system, controlling each of the sections or portions of the imaging device or system to retrace and match prior respective position(s) of the sections or portions in a case where the imaging device or system is moving or navigated in a second direction (e.g., in an opposite directionalong the path, in a return direction along the path, in a retraction direction along the path, etc.) along the path, etc. For example, an imaging device or system (or portion thereof, such as, but not limited to, a probe, a catheter, a camera, etc.) may enter a target along a path where a first section or portion of the imaging device or system (or portion of the device or system) is used to set the navigation or control path and position(s), and each subsequent section or portion of the imaging device or system (or portion of the device or system) is controlled to follow the first section or portion such that each subsequent section or portion matches the orientation and position of the first section or portion at each location along the path. During retraction, each section or portion of the imaging device or system is controlled to match (or be similar to, to be approximate to, be substantially matching, etc.) the prior orientation, position, state, etc. (for each section or portion) for each of the locations along the path. In other words, each section or portion of the device or system may follow a leader (or more than one leader), may use one or more FTL technique(s), or may use any other technique(s) (e.g., Hold the Line, Stay the Course, Close the Gap, etc.) discussed herein. As such, an imaging device or system (or catheter, probe, camera, etc. of the device or system) may enter and exit a target, an object, a specimen, a patient (e.g., a lung of a patient, an esophagus of a patient, a spline, another portion of a patient, another organ of a patient, a vessel of a patient, etc.), etc. along the same, similar, approximately same or similar, etc. path and using the same orientation, position, state, etc. for entrance and exit to achieve an optimal navigation, orientation, and / or control path. The navigation, control, orientation, and / or state feature(s) are not limited thereto, and one or more devices or systems of the present disclosure may include any other desired navigation, control, orientation, and / or state specifications or details as desired for a given application or use. In one or more embodiments and while not limited thereto, the first portion or section may be a distal or tip portion or section of the imaging and / or continuum robot device or system. In one or more embodiments, the first portion or section may be any predetermined or set portion or section of the imaging device or system, and the first portion or section may be predetermined or set manually by a user ofthe imaging and / or continuum robot device or system or may be set automatically by the imaging device or system (or a combination of manual and automatic control).

[0032] In one or more embodiments of the present disclosure (and while not limited to only this definition), a “change of orientation” or a “change of state” (or a transition of state) maybe defined in terms of direction and magnitude. For example, each interpolated step may have a same direction, and each interpolated step may have a larger magnitude as each step approaches a final orientation. Due to kinematics of one or more embodiments, any motion along a single direction may be the accumulation of a small motion in that direction. The small motion may have a unique or predetermined set of wire position or state changes to achieve the orientation change. Large or larger motion(s) in that direction may use a plurality of the small motions to achieve the large or larger motion(s). Dividing a large change into a series of multiple changes of the small or predetermined / set change may be used as one way to perform interpolation. Interpolation may be used in one or more embodiments to produce a desired or target motion, and at least one way to produce the desired or target motion may be to interpolate the change of wire positions or states.

[0033] In one or more embodiments of the present disclosure, an apparatus or system may include one or more processors that operate to: instruct or command a distal bending section or portion of a catheter or a probe of the continuum robot such that the distal bending section or portion achieves, or is disposed at, a bending pose or position, the catheter or probe of the continuum robot having a plurality of bending sections or portions and a base; store or obtain the bending pose or position of the distal bending section or portion and store or obtain a position or state of a motorized linear stage (or other structure used to map path or path-like information) that operates to move the catheter or probe of the continuum robot in a case where the one or more processors instruct or command forward motion, or a motion in a set or predetermined direction or directions, of the motorized linear stage (or other predetermined or set structure for mapping path or path-like information); generate a goal ortarget bending pose or position for each corresponding section or portion of the catheter or probe from, or based on, the previous bending section or portion; generate interpolated poses or positions for each of the sections or portions of the catheter or probe between the respective goal or target bending pose or position and a respective current bending pose or position of each of the sections or portions of the catheter or probe, wherein the interpolated poses or positions are generated such that an orientation vector of the interpolated poses or positions are on a plane that an orientation vector of the respective goal or target bending pose or position and an orientation vector of a respective current bending pose or position create or define; and instruct or command each of the sections or portions of the catheter or probe to move to or be disposed at the respective interpolated poses or positions during the forward motion, or the motion in the set or predetermined direction, of the previous section(s) or portion(s) of the catheter or probe.

[0034] In one or more embodiments, an apparatus / device or system may have one or more of the following exist or occur: (i) the distal bending section or portion maybe the most distal bending section or portion, and the most distal bending section or portion may be commanded or instructed automatically or based on an input of a user of the continuum robot in a case where the motorized linear stage (or other structure used for mapping path or pathlike information) is stable or stationary; (ii) the plurality of bending sections or portions may include the distal or most distal bending portion or section and the rest of the plurality of the bending sections or portions; (iii) the one or more processors may further operate to instruct or command the forward motion, or the motion in the set or predetermined direction, of the motorized linear stage (or other structure used for mapping path or path-like information) automatically or based on an input of a user of the continuum robot; and / or (iv) the plane may be created or defined based on a base coordinate system or based on a system substantially close to the base coordinate system.

[0035] In one or more embodiments, an apparatus or system (e.g., of or including a continuum robot) may further include: an actuator that operates to bend the plurality of the bending sections or portions independently and the base; and the motorized linear stage (or other structure used for mapping path or path-like information) that operates to move the continuum robot forward and backward, and / or in the predetermined or set direction or directions, wherein the one or more processors operate to control the actuator and the motorized linear stage (or other structure used for mapping path or path-like information). One or more embodiments may include a user interface of or disposed on the base, or disposed remotely from the base, the user interface operating to receive an input from a user of the continuum robot to move one or more of the plurality of bending sections or portions and / or the motorized linear stage (or other structure used for mapping path or path-like information), wherein the one or more processors further operate to receive the input from the user interface, and the one or more processors and / or the user interface operate to use a base coordinate system.

[0036] In one or more embodiments, the plurality of bending sections or portions may each include driving wires that operate to bend a respective section or portion of the plurality of sections or portions, wherein the driving wires are connected to the actuator so that the actuator operates to bend the plurality of bending sections or portions using the driving wires.

[0037] In one or more embodiments, the navigation, movement, and / or control may occur such that any intermediate orientations of one or more of the plurality of bending sections or portions is guided towards respective desired, predetermined, or set orientations (e.g., such that the steerable catheter, continuum robot, or other imaging device or system may reach the one or more targets).

[0038] In one or more embodiments, the catheter or probe of the continuum robot may be a steerable catheter or probe including the plurality of bending sections or portions andincluding an endoscope camera, wherein the one or more processors further operate to receive one or more endoscopic images from the endoscope camera, and wherein the continuum robot further comprises a display that operates to display the one or more endoscopic images.

[0039] One or more embodiments may include one or more of the following features: (i) the continuum robot may further include an operational controller or joystick that operates to issue or input one or more commands or instructions as an input to the one or more processors, the input including an instruction or command to move one or more of the plurality of bending sections or portions and / or the motorized linear stage (or other structure used for mapping path or path-like information); (ii) the continuum robot may further include a display to display one or more images taken by the continuum robot; and / or (iii) the continuum robot may further include an operational controller or joystick that operates to issue or input one or more commands or instructions to the one or more processors, the input including an instruction or command to move one or more of the plurality of bending sections or portions and / or the motorized linear stage (or other structure used for mapping path or path-like information), and the operational controller or joystick operates to be controlled by a user of the continuum robot.

[0040] In one or more embodiments of the present disclosure, an apparatus or system may include one or more processors that operate to: receive or obtain an image or images showing pose or position information of a tip section of a catheter or probe having a plurality of sections including at least the tip section; track a history of the pose or position information of the tip section of the catheter or probe during a period of time; and use the history of the pose or position information of the tip section to determine how to align or transition, move, or adjust (e.g., robotically, manually, automatically, etc.) each section of the plurality of sections of the catheter or probe.

[0041] In accordance with one or more embodiments of the present disclosure, apparatuses and systems, and methods and storage mediums for performing correction(s) and / or adjustment(s) to a direction or view, and / or for performing navigation, movement, and / or control, may operate to characterize biological objects, such as, but not limited to, blood, mucus, tissue, etc.

[0042] One or more embodiments of the present disclosure may be used in clinical application(s), such as, but not limited to, intervascular imaging, intravascular imaging, bronchoscopy, atherosclerotic plaque assessment, cardiac stent evaluation, intracoronary imaging using blood clearing, balloon sinuplasty, sinus stenting, arthroscopy, ophthalmology, ear research, veterinary use and research, etc.

[0043] In accordance with at least another aspect of the present disclosure, one or more technique(s) discussed herein may be employed as or along with features to reduce the cost of at least one of manufacture and maintenance of the one or more apparatuses, devices, systems, and storage mediums by reducing or minimizing a number of optical and / or processing components and by virtue of the efficient techniques to cut down cost (e.g., physical labor, mental burden, fiscal cost, time and complexity, etc.) of use / manufacture of such apparatuses, devices, systems, and storage mediums.

[0044] The following paragraphs describe certain explanatory embodiments. Other embodiments may include alternatives, equivalents, and modifications. Additionally, the explanatory embodiments may include several novel features, and a particular feature may not be essential to some embodiments of the devices, systems, and methods that are described herein.

[0045] According to other aspects of the present disclosure, one or more additional devices, one or more systems, one or more methods, and one or more storage mediums usingimaging adjustment or correction and / or other technique(s) are discussed herein. Further features of the present disclosure will in part be understandable and will in part be apparent from the following description and with reference to the attached drawings.BRIEF DESCRIPTION OF THE DRAWINGS

[0046] For the purposes of illustrating various aspects of the disclosure, wherein like numerals indicate like elements, there are shown in the drawings simplified forms that may be employed, it being understood, however, that the disclosure is not limited by or to the precise arrangements and instrumentalities shown. To assist those of ordinary skill in the relevant art in making and using the subject matter hereof, reference is made to the appended drawings and figures, wherein:

[0047] FIG. 1 illustrates at least one embodiment of an imaging, continuum robot, or endoscopic apparatus or system in accordance with one or more aspects of the present disclosure;

[0048] FIG. 2 is a schematic diagram showing at least one embodiment of an imaging, steerable catheter, or continuum robot apparatus or system in accordance with one or more aspects of the present disclosure;

[0049] FIGS. 3A-3B illustrate at least one embodiment example of a continuum robot and / or medical device that may be used with one or more technique(s), including robotic control technique(s), in accordance with one or more aspects of the present disclosure;

[0050] FIGS. 3C-3D illustrate one or more principles of catheter or continuum robot tip manipulation by actuating one or more bending segments of a continuum robot or steerable catheter 104 of FIGS. 3A-3B in accordance with one or more aspects of the present disclosure;

[0051] FIG. 4 is a schematic diagram showing at least one embodiment of an imaging, continuum robot, steerable catheter, or endoscopic apparatus or system in accordance with one or more aspects of the present disclosure;

[0052] FIG. 5 is a schematic diagram showing at least one embodiment of a console or computer that may be used with one or more robotic control technique(s) in accordance with one or more aspects of the present disclosure;

[0053] FIG. 6 is a flowchart of at least one embodiment of a method for planning an operation of at least one embodiment of a continuum robot or steerable catheter apparatus or system in accordance with one or more aspects of the present disclosure;

[0054] FIGS. 7A-7B are flowcharts of at least one embodiment of a method for performing navigation, movement, and / or control for a continuum robot or steerable catheter in accordance with one or more aspects of the present disclosure;

[0055] FIGS. 8A-8B are flowcharts of at least one embodiment of a method for performing navigation, movement, and / or control for a continuum robot or steerable catheter in accordance with one or more aspects of the present disclosure;

[0056] FIGS. 9A-9B are flowcharts of at least one embodiment of a method for performing navigation, movement, and / or control for a continuum robot or steerable catheter in accordance with one or more aspects of the present disclosure;

[0057] FIG. 10 is a flowchart of at least one embodiment of a method for performing adjustment, correction, or smoothing for a continuum robot in accordance with one or more aspects of the present disclosure;

[0058] FIG. 11 illustrates a diagram of a continuum robot that may be used with one or more technique(s) or method(s) in accordance with one or more aspects of the present disclosure;

[0059] FIG. 12 illustrates a block diagram of at least one embodiment of a continuum robot in accordance with one or more aspects of the present disclosure;

[0060] FIG. 13 illustrates a block diagram of at least one embodiment of a controller in accordance with one or more aspects of the present disclosure;

[0061] FIG. 14 shows a schematic diagram of an embodiment of a computer that may be used with one or more embodiments of an apparatus or system, or one or more methods, discussed herein in accordance with one or more aspects of the present disclosure; and

[0062] FIG. 15 shows a schematic diagram of another embodiment of a computer that may be used with one or more embodiments of an imaging apparatus or system, or methods, discussed herein in accordance with one or more aspects of the present disclosure.DETAILED DESCRIPTION OF THE PRESENT DISCLOSURE

[0063] One or more devices, systems, methods and storage mediums for viewing, imaging, and / or characterizing tissue, or an object or sample, using one or more imaging techniques or modalities (such as, but not limited to, computed tomography (CT), Magnetic Resonance Imaging (MRI), any other techniques or modalities used in imaging (e.g., Optical Coherence Tomography (OCT), Near infrared fluorescence (NIRF), Near infrared auto-fluorescence (NIRAF), Spectrally Encoded Endoscopes (SEE)), etc.) are disclosed herein. Several embodiments of the present disclosure, which may be carried out by the one or more embodiments of an apparatus, system, method, and / or computer-readable storage medium of the present disclosure, are described diagrammatically and visually in FIGS. 1 through 15.

[0064] One or more embodiments of the present disclosure avoid the aforementioned issues by providing a simple and fast method or methods that provide robotic control technique(s) as discussed herein and / or adjustment, correction, and / or smoothing technique(s) as discussed herein. In one or more embodiments, the robotic control techniques may be used with a co-registration (e.g., computed tomography (CT) co-registration, conebeam CT (CBCT) co-registration, etc.) to enhance a successful targeting rate for a predetermined sample, target, or object (e.g., a lung, a portion of a lung, a vessel, a nodule, etc.) by minimizing human error. CBCT maybe used to locate a target, sample, or object (e.g., the lesion(s) or nodule(s) of a lung or airways) along with an imaging device (e.g., a steerable catheter, a continuum robot, etc.) and to co-register the target, sample, or object (e.g., the lesions or nodules) with the device shown in an image to achieve proper guidance.

[0065] Accordingly, it is a broad object of the present disclosure to provide imaging (e.g., computed tomography (CT), Magnetic Resonance Imaging (MRI), etc.) apparatuses, systems, methods, and storage mediums for using a navigation and / or control method or methods (manual or automatic) in one or more apparatuses or systems (e.g., an imaging apparatus or system, an endoscopic imaging device or system, etc.). It is also a broad object of the present disclosure to provide imaging (e.g., computed tomography (CT), Magnetic Resonance Imaging (MRI), etc.) apparatuses, systems, methods, and storage mediums for using a navigation and / or control method or methods for achieving navigation, movement, and / or control through a target, sample, or object (e.g., lung airway(s) during bronchoscopy, a vessel, a patient, a portion of a patient, etc.) in one or more apparatuses or systems (e.g., an imaging apparatus or system, an endoscopic imaging device or system, etc.).

[0066] Accordingly, it is a broad object of the present disclosure to provide imaging (e.g., computed tomography (CT), Magnetic Resonance Imaging (MRI), etc.) apparatuses, systems, methods, and storage mediums for using a navigation and / or control method or methods(manual or automatic) in one or more apparatuses or systems (e.g., an imaging apparatus or system, an endoscopic imaging device or system, etc.). The navigation and / or control maybe employed so that an apparatus or system having multiple portions or sections (e.g., multiple bending portions or sections) operates to: (i) keep track of a path of a portion (e.g., a tip) or of each of the multiple portions or sections of an apparatus or system; (ii) have a state or states of each of the multiple portions or sections match a state or states of a first portion or section of the multiple portions or sections in a case where each portion or section reaches or approaches a same, similar, or approximately similar state (e.g., a position or other state(s) in a target, object, or specimen; a position or other state(s) in a patient; a target position or state(s) in an image or frame; a set or predetermined position or state(s) in an image or frame; a set or predetermined position or state(s) in an image or frame where the first portion or section reaches or approaches the set or predetermined position or state(s) at one point in time and one or more of other portions or sections of the multiple portions or sections reach the set or predetermined position or state(s) at one or more other points in time; any other state (which may include, but is not limited to, an orientation, a position, a pose, a navigation, a path (whether continuous or discontinuous), a state transition, any other desired motion(s) or combination of motion(s) (e.g., one or more features of the present disclosure may assist navigation, orientation, or any other types of motions discussed herein or as desired by a user), etc.), a combination of any state(s) and / or motion(s) discussed herein or desired by a user, etc.) of another portion or section of the one or more devices, systems, methods, and / or storage mediums of the present disclosure, etc.); (iii) utilize additional data (such as, but not limited to, target pose or state information, final pose or state information, interpolated pose or state information, measured pose or state information, converting pose or state information between different states (e.g., drive wire position(s) or state(s); coordinates (three- dimensional (3D) position(s), orientation(s), and / or state(s)); plane and / or angle information for pose(s) or state(s); state position or state information (e.g., target, interpolated, measured pose(s) and / or state(s)); force sensor(s) information; drawer current draw information of one or more actuator motors; section dimension information (e.g., size, shape, length, etc.) for oneor more sections of the catheter or probe (e.g., a tip section of the catheter or probe, a middle section of the catheter or probe, a distal section of the catheter or probe, a proximal section of the catheter or probe, any combination thereof, etc.) from an entire device or system (e.g., using forwards or inverse kinematics of the device or system, using other internal sensor(s) or information of the device or system, etc.) and / or external source(s) (e.g., one or more external sensors (e.g., an electromagnetic (EM) sensor, a shape sensor, any other sensor discussed herein or known to those skilled in the art, etc.) in a robotic control algorithm; (iv) utilize differences between a previous or expected robotic control state and a new control state in future calculation(s) of robotic control state(s); and / or (v) address any discontinuous path that may occur (e.g., due to a change (e.g., of a state or states) or other movement or state change / transition of any portion of the apparatus or system), for example, by smoothing out any difference in the discontinuous path over one or more multiple stage positions or states or over one or more other path-like information positions or states, by considering target or object movement(s) (e.g., movement of a patient or a portion of a patient while a probe or catheter is disposed in the patient, etc.).

[0067] In one or more embodiments, pose or state information may be stored in a lookup table or tables, and the pose or state information for one or more sections of the catheter or probe may be updated in the lookup table based on new information (e.g., environmental change(s) for the catheter or probe, movement of a target or sample, movement of a patient, user control, relaxation state changes, etc.). The new information or the updated information may be used to control the one or more sections of the catheter or probe more efficiently during navigation (forwards and / or backwards). For example, in a case where a previously stored pose or state may have shifted or changed due to a movement or relaxation of the target, object, or sample (e.g., a patient may move), the previously stored pose or state may not be ideal or may work less efficiently as compared with an updated pose or state modified or updated in view of the new information (e.g., the movement, in this example). As such, one or more embodiments of the present disclosure may update or modify the pose or state informationsuch that robotic control of the catheter or probe may work efficiently in view of the new information, movement, relaxation, and / or environmental change(s). In addition to having the update or change affect the previously stored history or known history at that point in space (e.g., similar to dragging that point (e.g., of the target, object, or sample (e.g., a patient, a portion of a patient, a vessel, a spline, a lung, etc.); of the catheter or probe; etc.) and recalculating the path), in one or more embodiments, the update or change may also affect a number of other points (e.g., all points in a lookup table or tables, all points forward beyond the initially changed point, one or more future points or points beyond the initially changed point as desired, etc.). For example, in one or more embodiments, the transform (or difference, change, update, etc.) between the previous pose or state and the new or updated post or state may be propagated to all points going forward or may be propagated to one or more of forward points (e.g., for a predetermined or set range, for a predetermined or set distance, etc.). Doing so in one or more embodiments may operate to shift all or part of the future path based on how the pose or state of the catheter or probe was adjusted, using that location as a pivot point. Such update(s) may be obtained from one or more internal sources (e.g., one or more processors, one or more sensors, combination(s) thereof, etc.) or may be obtained from one or more external sources (e.g., one or more other processors, one or more external sensors, combination(s) thereof, etc.). For example, a difference between a real-time target, sample, or object (e.g., an airway) and the previous target, sample, or object (e.g., a previous airway) may be detected using machine vision (of the endoscope image) or using multiple medical images. Body, target, object, or sample divergence may also be estimated from other sensors, like one measuring breathing or the motion of the body (or another predetermined or set motion or change to track).

[0068] One or more robotic control methods of the present disclosure may be employed in one or more embodiments. For example, one or more of the following techniques or methods maybe used: Hold the Line, Close the Gap, and / or Stay the Course. In one or more embodiments, a Tip position or state may be converted to a coordinate (e.g., X, Y, Zcoordinate) during navigation (e.g., while a stage moves forward, while the navigation is mapped to Z stage position, etc.). In a case where a pose or state is being determined for other section(s) of the catheter or probe, one or more embodiments use a pose or state which operates to keep an end effector of the section(s) as close as possible to the Tip position or state coordinate at that point along the insertion path. The pose or state coordinates of preceding section(s) of the catheter or probe may be known or determined in one or more embodiments. Body, target, object, or sample divergence may be applied with a coordinate shift. In cases where a length of each section or sections of the catheter or probe are not considered, applying a Tip pose or state to a middle / proximal section of the catheter or probe may result in a situation where a middle / proximal end effector maybe in a drastically different location than the Tip end effector at that point along the path (e.g., a middle / proximal section maybe longer or maybe twice as long as the Tip section, for example). As such, by considering the lengths of each section or sections of the catheter or probe, appropriate adjustment(s) may be made so that the middle / proximal end effector, and / or an effector of another section or portion of the catheter or probe, may be in an appropriate / desired / intended location or may be in the same location as the Tip end effector was at that point along the path. Using such techniques, one or more embodiments of the present disclosure may apply one or more Hold the Line features to a catheter or probe to achieve efficient and accurate control of the catheter or probe.

[0069] In one or more embodiments, one or more other or additional robotic control methods or techniques may be employed. For example, one or more gaps may be closed by adjusting a pose or state of a section or sections of a catheter or probe to reduce, minimize, or eliminate a relative difference between the pose or state of the section or sections versus a pose or state of the subsequent / next / future / following section or sections. In one or more embodiments, a trajectory of the subsequent / next / future / following section or sections may be known or determined. Closing such a difference or gap operates to eliminate any motion outside of a desired direction or trajectory. For example, if the Tip section faced left (or to afirst direction), then moved forwards slightly, and then faced right (or a towards a second direction), then a middle section may also be moved left in a case where the middle section reaches the point that the Tip section was facing left (or to the first direction). However, since the Tip section wants to be moving right (or towards the second direction) after that point, then the motion of the Tip section to the left (or first) direction will be moving counter-actively to the desired direction, which may be the right (or the second) direction. Instead, using one or more features of the present disclosure, any counter-active motion(s) may be avoided or eliminated.

[0070] In one or more embodiments, one or more further robotic control methods or techniques may be employed. For example, a catheter or probe may be controlled to stay the desired course. For example, a pose or state of a section or sections may be adjusted to minimize any deviation of a pose or state of one or more next sections out of the predetermined, targeted, desired trajectory and maximum motion along the trajectory. In one or more embodiments, the coordinates and the trajectory of subsequent / following / next / future sections may be known, set, or determined, and information for one or more prior sections may be known, set, or determined. By considering section lengths, one or more advantageous results may be achieved. As aforementioned, in one or more embodiments, a middle / proximal section end effector may device from a location of a Tip section at that point. Such a deviation may cause the Tip section to move in directions that are not in the direction it is facing. This effect may be compounded in a case where a proximal / middle section is included in the movement or control. As such, by using one or more features of the present disclosure, any counter-active or undesired motion(s) may be avoided or eliminated.

[0071] In one or more embodiments, an orientation, pose, or state may include one or more degrees of freedom. For example, in at least one orientation embodiment, two (2) degrees of freedom may be used, which may include an angle for a magnitude of bending anda plane for a direction of bending. In one or more embodiments, matching state(s) may involve matching, duplicating, mimicking, or otherwise copying other characteristics, such as, but not limited to, vectors for each section or portion of the one or more sections or portions of a probe or catheter, for different portions or sections of the catheter or probe. For example, a transition or change from a base angle / plane to a target angle / plane may be set or predetermined using transition values (e.g., while not limited hereto, a base orientation or state may have a stage at o mm, an angle at o degrees, and a plane at o degrees whereas a target orientation or state may have a stage at 20mm, an angle at 90 degrees, and a plane at 180 degrees. The intermediate values for the stage, angle, and plane may be set depending on how many transition orientations or states may be used).

[0072] In one or more embodiments, a continuum robot or steerable catheter may include one or more of the following: (i) a distal bending section or portion, wherein the distal bending section or portion is commanded or instructed automatically or based on an input of a user of the continuum robot or steerable catheter; (ii) a plurality of bending sections or portions including a distal or most distal bending portion or section and the rest of the plurality of the bending sections or portions; and / or (iii) the one or more processors further operate to instruct or command the forward motion, or the motion in the set or predetermined direction, of the motorized linear stage and / or of the continuum robot or steerable catheter automatically and / or based on an input of a user of the continuum robot. A continuum robot or steerable catheter may further include: a base and an actuator that operates to bend the plurality of the bending sections or portions independently; and a motorized linear stage and / or a sensor that operates to move the continuum robot or steerable catheter forward and backward, and / or in the predetermined or set direction or directions, wherein the one or more processors operate to control the actuator and the motorized linear stage and / or the sensor. The plurality of bending sections or portions may each include driving wires that operate to bend a respective section or portion of the plurality of sections or portions, wherein the driving wires are connected to an actuator so that the actuator operates to bend one or more of theplurality of bending sections or portions using the driving wires. One or more embodiments may include a user interface of or disposed on a base, or disposed remotely from a base, the user interface operating to receive an input from a user of the continuum robot or steerable catheter to move one or more of the plurality of bending sections or portions and / or a motorized linear stage and / or a sensor, wherein the one or more processors further operate to receive the input from the user interface, and the one or more processors and / or the user interface operate to use a base coordinate system. One or more displays may be provided to display a path (e.g., a control path) of the continuum robot or steerable catheter. In one or more embodiments, one or more of the following may occur: (i) the continuum robot may further include an operational controller or joystick that operates to issue or input one or more commands or instructions as an input to one or more processors, the input including an instruction or command to move one or more of a plurality of bending sections or portions and / or a motorized linear stage and / or a sensor; (ii) the continuum robot may further include a display to display one or more images taken by the continuum robot; and / or (iii) the continuum robot may further include an operational controller or joystick that operates to issue or input one or more commands or instructions to one or more processors, the input including an instruction or command to move one or more of a plurality of bending sections or portions and / or a motorized linear stage and / or a sensor, and the operational controller or joystick operates to be controlled by a user of the continuum robot. In one or more embodiments, the continuum robot or the steerable catheter may include a plurality of bending sections or portions and may include an endoscope camera, wherein one or more processors operate or further operate to receive one or more endoscopic images from the endoscope camera, and wherein the continuum robot further comprises a display that operates to display the one or more endoscopic images.

[0073] Any discussion of a state, pose, position, orientation, navigation, path, or other state type discussed herein is discussed merely as a non-limiting, non-exhaustive embodiment example, and any state or states discussed herein may be used interchangeably / alternativelyor additionally with the specifically mentioned type of state. Driving and / or control technique(s) may be employed to adjust, change, or control any state, pose, position, orientation, navigation, path, or other state type that maybe used in one or more embodiments for a continuum robot or steerable catheter.

[0074] Physicians or other users of the apparatus or system may have reduced or saved labor and / or mental burden using the apparatus or system due to the navigation, control, and / or orientation (or pose, or position, etc.) feature(s) of the present disclosure. Additionally, one or more features of the present disclosure may achieve a minimized or reduced interaction with anatomy (e.g., of a patient), object, or target (e.g., tissue) during use, which may reduce the physical and / or mental burden on a patient or target. In one or more embodiments of the present disclosure, a labor of a user to control and / or navigate (e.g., rotate, translate, etc.) the imaging apparatus or system or a portion thereof (e.g., a catheter, a probe, a camera, one or more sections or portions of a catheter, probe, camera, etc.) is saved or reduced via use of the navigation and / or control technique(s) of the present disclosure.

[0075] In one or more embodiments, an imaging device or system, or a portion of the imaging device or system (e.g., a catheter, a probe, etc.), the continuum robot, and / or the steerable catheter may include multiple sections or portions, and the multiple sections or portions may be multiple bending sections or portions. In one or more embodiments, the imaging device or system may include manual and / or automatic navigation and / or control features. For example, a user of the imaging device or system (or steerable catheter, continuum robot, etc.) may control each section or portion, and / or the imaging device or system (or steerable catheter, continuum robot, etc.) may operate to automatically control (e.g., robotically control) each section or portion, such as, but not limited to, via one or more navigation, movement, and / or control techniques of the present disclosure.

[0076] Navigation, control, and / or orientation feature(s) may include, but are not limited to, implementing mapping of a pose (angle value(s), plane value(s), etc.) of a first portion or section (e.g., a tip portion or section, a distal portion or section, a predetermined or set portion or section, a user selected or defined portion or section, etc.) to a stage position / state (or a position / state of another structure being used to map path or path-like information), controlling angular position(s) of one or more of the multiple portions or sections, controlling rotational orientation or position(s) of one or more of the multiple portions or sections, controlling (manually or automatically (e.g., robotically)) one or more other portions or sections of the imaging device or system (e.g., continuum robot, steerable catheter, etc.) to match or substantially or approximately match (or be close to or similar to) the navigation / orientation / position / pose of the first portion or section in a case where the one or more other portions or sections reach (e.g., subsequently reach, reach at a different time, etc.) the same or similar, or approximately the same or similar, position or state (e.g., in a target, in an object, in a sample, in a patient, in a frame or image, etc.) during navigation in or along a first direction of a path of the imaging device or system, controlling each of the sections or portions of the imaging device or system to retrace and match (or substantially or approximately match or be close / similar to) prior respective position(s) of the sections or portions in a case where the imaging device or system is moving or navigated in a second direction (e.g., in an opposite direction along the path, in a return direction along the path, in a retraction direction along the path, etc.) along the path, etc. For example, an imaging device or system (or portion thereof, such as, but not limited to, a probe, a catheter, a camera, etc.) may enter a target along a path where a first section or portion of the imaging device or system (or portion of the device or system) is used to set the navigation, control, or state path and state(s) / position(s), and each subsequent section or portion of the imaging device or system (or portion of the device or system) is controlled to follow the first section or portion such that each subsequent section or portion matches (or is similar to, approximate to, substantially matching, etc.) the orientation, position, state, etc. of the first section or portion at each location along the path. During retraction, each section or portion of the imaging device orsystem is controlled to match (or be similar to, be approximate to, be substantially matching, etc.) the prior orientation, position, state, etc. (for each section or portion) for each of the locations along the path. In other words, each section or portion of the device or system may follow a leader (or more than one leader) or may use one or more FTL technique(s) discussed herein. Additionally or alternatively, as discussed herein, one or more embodiments may use one or more Hold the Line, Close the Gap, Stay the Course, and / or any other control feature(s) of the present disclosure. As such, an imaging or continuum robot device or system (or catheter, probe, camera, etc. of the device or system) may enter and exit a target, an object, a specimen, a patient (e.g., a lung of a patient, an esophagus of a patient, a spline, another portion of a patient, another organ of a patient, a vessel of a patient, etc.), etc. along the same, similar, approximately same or similar, etc. path and using the same orientation, pose, state, etc. for entrance and exit to achieve an optimal navigation, orientation, control, and / or state path. The navigation, control, orientation, and / or state feature(s) are not limited thereto, and one or more devices or systems of the present disclosure may include any other desired navigation, control, orientation, and / or state specifications or details as desired for a given application or use. In one or more embodiments and while not limited thereto, the first portion or section may be a distal or tip portion or section of the imaging or continuum robot device or system. In one or more embodiments, the first portion or section may be any predetermined or set portion or section of the imaging or continuum robot device or system, and the first portion or section may be predetermined or set manually by a user of the imaging or continuum robot device or system or may be set automatically by the imaging device or system (or by a combination of manual and automatic control).

[0077] In one or more embodiments of the present disclosure (and while not limited to only this definition), a “change of orientation” or a “change of state” (or a transition of state) maybe defined in terms of direction and magnitude. For example, each interpolated step may have a same direction, and each interpolated step may have a larger magnitude as each step approaches a final orientation. Due to kinematics of one or more embodiments, any motionalong a single direction may be the accumulation of a small motion in that direction. The small motion may have a unique or predetermined set of wire position or state changes to achieve the orientation change. Large or larger motion(s) in that direction may use a plurality of the small motions to achieve the large or larger motion(s). Dividing a large change into a series of multiple changes of the small or predetermined / set change may be used as one way to perform interpolation. Interpolation may be used in one or more embodiments to produce a desired or target motion, and at least one way to produce the desired or target motion may be to interpolate the change of wire positions or states.

[0078] In one or more embodiments of the present disclosure, an apparatus or system may include one or more processors that operate to: instruct or command a distal bending section or portion of a catheter or a probe of the continuum robot such that the distal bending section or portion achieves, or is disposed at, a bending pose or position, the catheter or probe of the continuum robot having a plurality of bending sections or portions and a base; store or obtain the bending pose or position of the distal bending section or portion and store or obtain a position or state of a motorized linear stage (or other structure used to map path or path-like information) that operates to move the catheter or probe of the continuum robot in a case where the one or more processors instruct or command forward motion, or a motion in a set or predetermined direction or directions, of the motorized linear stage (or other predetermined or set structure for mapping path or path-like information); generate a goal or target bending pose or position for each corresponding section or portion of the catheter or probe from, or based on, the previous bending section or portion; generate interpolated poses or positions for each of the sections or portions of the catheter or probe between the respective goal or target bending pose or position and a respective current bending pose or position of each of the sections or portions of the catheter or probe, wherein the interpolated poses or positions are generated such that an orientation vector of the interpolated poses or positions are on a plane that an orientation vector of the respective goal or target bending pose or position and an orientation vector of a respective current bending pose or position create ordefine; and instruct or command each of the sections or portions of the catheter or probe to move to or be disposed at the respective interpolated poses or positions during the forward motion, or the motion in the set or predetermined direction, of the previous section(s) or portion(s) of the catheter or probe.

[0079] In one or more embodiments, an apparatus / device or system may have one or more of the following exist or occur: (i) the distal bending section or portion maybe the most distal bending section or portion, and the most distal bending section or portion may be commanded or instructed automatically or based on an input of a user of the continuum robot in a case where the motorized linear stage (or other structure used for mapping path or pathlike information) is stable or stationary; (ii) the plurality of bending sections or portions may include the distal or most distal bending portion or section and the rest of the plurality of the bending sections or portions; (iii) the one or more processors may further operate to instruct or command the forward motion, or the motion in the set or predetermined direction, of the motorized linear stage (or other structure used for mapping path or path-like information) automatically or based on an input of a user of the continuum robot; and / or (iv) the plane may be created or defined based on a base coordinate system or based on a system substantially close to the base coordinate system.

[0080] In one or more embodiments, an apparatus or system (e.g., of or including a continuum robot) may further include: an actuator that operates to bend the plurality of the bending sections or portions independently and the base; and the motorized linear stage (or other structure used for mapping path or path-like information) that operates to move the continuum robot forward and backward, and / or in the predetermined or set direction or directions, wherein the one or more processors operate to control the actuator and the motorized linear stage (or other structure used for mapping path or path-like information). One or more embodiments may include a user interface of or disposed on the base, or disposed remotely from the base, the user interface operating to receive an input from a user of thecontinuum robot to move one or more of the plurality of bending sections or portions and / or the motorized linear stage (or other structure used for mapping path or path-like information), wherein the one or more processors further operate to receive the input from the user interface, and the one or more processors and / or the user interface operate to use a base coordinate system.

[0081] In one or more embodiments, the plurality of bending sections or portions may each include driving wires that operate to bend a respective section or portion of the plurality of sections or portions, wherein the driving wires are connected to the actuator so that the actuator operates to bend the plurality of bending sections or portions using the driving wires.

[0082] In one or more embodiments, the navigation, movement, and / or control may occur such that any intermediate orientations of one or more of the plurality of bending sections or portions is guided towards respective desired, predetermined, or set orientations (e.g., such that the steerable catheter, continuum robot, or other imaging device or system may reach the one or more targets).

[0083] FIG. 1 illustrates a simplified representation of a medical environment, such as an operating room, where a robotic catheter system 1000 may be used. FIG. 2 illustrates a functional block diagram that may be used in at least one embodiment of the robotic catheter system 1000. FIGS. 3A-3D represent at least one embodiment of the catheter 104 (see FIGS. 3A-3B) and bending for the catheter 104 (as shown in FIGS. 3C-3D). FIG. 4 illustrates a logical block diagram that may be used for the robotic catheter system 1000. In at least this embodiment example, the system 1000 may include a computer cart (see e.g., the controller too, 102 in FIG. 1) operatively connected to a steerable catheter or continuum robot 104 via a robotic platform 108. The robotic platform 108 includes one or more than one robotic arm 132 and a rail 110 (see e.g., FIGS. 1-2) and / or linear translation stage 122 (see e.g., FIG. 2).

[0084] As shown in FIGS. 1-4 of the present disclosure, one or more embodiments of a system 1000 for performing robotic control (e.g., for a continuum robot, a steerable catheter, etc.) may include one or more of the following: a display controller too, a display 101-1, a display 101-2, a controller 102, an actuator 103, a continuum device (also referred to herein as a “steerable catheter” or “an imaging device”) 104, an operating portion 105, a tracking sensor 106 (e.g., an electromagnetic (EM) tracking sensor), a catheter tip position / orientation / pose / state detector 107, and a rail 110 (which may be attached to or combined with a linear translation stage 122) (for example, as shown in at least FIGS. 1-2). The system 1000 may include one or more processors, such as, but not limited to, a display controller 100, a controller 102, a CPU 120, a controller 50, a CPU 51, a console or computer 1200 or 1200’, a CPU 1201, any other processor or processors discussed herein, etc., that operate to execute a software program, to control the one or more adjustment, control, and / or smoothing technique(s) discussed herein, and to control display of a navigation screen on one or more displays 101-1, 101-2, etc. The one or more processors (e.g., the display controller 100, the controller 102, the CPU 120, the controller 50, the CPU 51, the console or computer 1200 or 1200’, the CPU 1201, any other processor or processors discussed herein, etc.) may generate a three dimensional (3D) model of a structure (for example, a branching structure like airway of lungs of a patient, an object to be imaged, tissue to be imaged, etc.) based on images, such as, but not limited to, CT images, MRI images, etc. Alternatively, the 3D model may be received by the one or more processors (e.g., the display controller 100, the controller 102, the CPU 120, the controller 50, the CPU 51, the console or computer 1200 or 1200’, the CPU 1201, any other processor or processors discussed herein, etc.) from another device. A two-dimensional (2D) model may be used instead of 3D model in one or more embodiments. The 2D or 3D model maybe generated before a navigation starts. Alternatively, the 2D or 3D model may be generated in real-time (in parallel with the navigation). In the one or more embodiments discussed herein, examples of generating a model of branching structure are explained. However, the models may not be limited to a model of branching structure. For example, a model of a route direct to a target may be used instead of the branching structure.Alternatively, a model of a broad space maybe used, and the model maybe a model of a place or a space where an observation or a work is performed by using a continuum robot 104 explained below.

[0085] In FIG. 1, a user U (e.g., a physician, a technician, etc.) may control the robotic catheter system 1000 via a user interface unit (operation unit) to perform an intraluminal procedure on a patient P positioned on an operating table B. The user interface may include at least one of a main or first display 101-1 (a first user interface unit), a second display 101-2 (a second user interface unit), and a handheld controller 105 (a third user interface unit). The main or first display 101-1 may include, for example, a large display screen attached to the system 1000 and / or the controllers 101, 102 of the system 1000 or mounted on a wall of the operating room and maybe, for example, designed as part of the robotic catheter system 1000 or maybe part of the operating room equipment. Optionally, there maybe a secondaiy display 101-2 that is a compact (portable) display device configured to be removably attached to the robotic platform 108. Examples of the second or secondaiy display 101-2 may include, but are not limited to, a portable tablet computer, a mobile communication device (a cellphone), a tablet, a laptop, etc.

[0086] The steerable catheter 104 may be actuated via an actuator unit 103. The actuator unit 103 may be removably attached to the robotic platform 108 or any component thereof (e.g., the robotic arm 132, the rail 110, and / or the linear translation stage 122). The handheld controller 105 may include a gamepad-like controller with a joystick having shift levers and / or push buttons, and the controller 105 may be a one-handed controller or a two-handed controller. In one embodiment, the actuator unit 103 may be enclosed in a housing having a shape of a catheter handle. One or more access ports 126 may be provided in or around the catheter handle. The access port 126 may be used for inserting and / or withdrawing end effector tools and / or fluids when performing an interventional procedure of the patient P.

[0087] In one or more embodiments, the system 1000 includes at least a system controller 102, a display controller too, and the main display 101-1. The main display 101-1 may include a conventional display device such as a liquid crystal display (LCD), an OLED display, a QLED display, any other display discussed herein, any other display known to those skilled in the art, etc. The main display 101-1 may provide or display a graphic interface unit (GUI) configured to display one or more views. These views may include a live view image 134, an intraoperative image 135, a preoperative image 136, and other procedural information 138. Other views that maybe displayed include a model view, a navigational information view, and / or a composite view. The live image view 134 may be an image from a camera at the tip of the catheter 104. The live image view 134 may also include, for example, information about the perception and navigation of the catheter 104. The preoperative image 136 may include pre-acquired 3D or 2D medical images of the patient P acquired by conventional imaging modalities such as computer tomography (CT), magnetic resonance imaging (MRI), ultrasound imaging, or any other desired imaging modality. The intraoperative image 135 may include images used for image guided procedure such images may be acquired by fluoroscopy or CT imaging modalities (or another desired imaging modality). The intraoperative image 135 may be augmented, combined, or correlated with information obtained from a sensor, camera image, or catheter data.

[0088] In the various embodiments where a catheter tip tracking sensor 106 is used, the sensor may be located at the distal end of the catheter 104. The catheter tip tracking sensor 106 maybe, for example, an electromagnetic (EM) sensor. If an EM sensor is used, a catheter tip position detector 107 maybe included in the robotic catheter system 1000; the catheter tip position detector 107 may include an EM field generator operatively connected to the system controller 102. One or more other embodiments of the catheter / continuum robot 104 may not include or use the EM tracking sensor 106. Suitable electromagnetic sensors for use with a steerable catheter may be used with any feature of the present disclosure, including thesensors discussed, for example, in U.S. Pat. No. 6,201,387 and in International Pat. Pub. WO2020 / 194212 Al, which are incorporated by reference herein in their entireties.

[0089] While not limited to such a configuration, the display controller 100 may acquire position / orientation / navigation / pose / state (or other state) information of the continuum robot 104 from a controller 102. Alternatively, the display controller 100 may acquire the position / orientation / navigation / pose / state (or other state) information directly from a tip position / orientation / navigation / pose / state (or other state) detector 107. The continuum robot 104 maybe a catheter device (e.g., a steerable catheter or probe device). The continuum robot 104 maybe attachable / detachable to the actuator 103, and the continuum robot 104 may be disposable.

[0090] Similar to FIG. 1, FIG. 2 illustrates the robotic catheter system 1000 including the system controller 102 operatively connected to the display controller too, which is connected to the first display 101-1 and to the second display 101-2. The system controller 102 is also connected to the actuator 103 via the robotic platform 108 or any component thereof (e.g., the robotic arm 132, the rail 110, and / or the linear translation stage 122). The actuator unit 103 may include a plurality of motors 144 that operate to control a plurality of drive wires 160 (while not limited to any particular number of drive wires 160, FIG. 2 shows that six (6) drive wires 160 are being used in the subject embodiment example). The drive wires 160 travel through the steerable catheter or continuum robot 104. One or more access ports 126 maybe located on the catheter 104 (and may include an insertion / extraction detector 109). The catheter 104 may include a proximal section 148 located between the actuator 103 and the proximal bending section 152, where the drive wires 160 operate to actuate the proximal bending section 152. Three of the six drive wires 160 continue through the distal bending section 156 where the drive wires 160 operate to actuate the distal bending section 156 and allow for a range of movement. FIG. 2 is shown with two bendable sections 152, 156 (although one or more bendable sections may be used in one or more embodiments). Otherembodiments as described herein may have three bendable sections (see e.g., FIGS. 3A-3D). In some embodiments, a single bending section maybe provided, or alternatively, four or more bendable sections may be present in one or more embodiments of the catheter 104.

[0091] FIGS. 3A-3B show at least one embodiment of a continuum robot 104 that may be used in the system 1000 or any other system discussed herein. FIG. 3A shows at least one embodiment of a steerable catheter 104. The steerable catheter 104 may include a nonsteerable proximal section 148, a steerable distal section 156, and a catheter tip 320. The proximal section 148 and distal bendable section 156 (including portions 152, 154, and 156 in FIG. 3A) are joined to each other by a plurality of drive wires 160 arranged along the wall of the catheter 104. The proximal section 148 is configured with through-holes (or thru-holes) or grooves or conduits to pass drive wires 160 from the distal section 152, 154, 156 to the actuator unit 103. The distal section 152, 154, 156 is comprised of a plurality of bending segments including at least a distal segment 156, a middle segment 154, and a proximal segment 152. Each bending segment is bent by actuation of at least some of the plurality of drive wires 160 (driving members). The posture of the catheter 104 may be supported by supporting wires (support members) also arranged along the wall of the catheter 104 (as discussed in U.S. Pat. Pub. US2021 / 0308423, which is incorporated by reference herein in its entirety). The proximal ends of drive wires 160 are connected to individual actuators or motors 144 of the actuator unit 103, while the distal ends of the drive wires 160 are selectively anchored to anchor members in the different bending segments of the distal bendable section 152, 154, 156.

[0092] Each bending segment is formed by a plurality of ring-shaped components (rings) with through-holes (or thru-holes), grooves, or conduits along the wall of the rings. The ringshaped components are defined as wire-guiding members 162 or anchor members 164 depending on a respective function(s) within the catheter 104. The anchor members 164 are ring-shaped components onto which the distal end of one or more drive wires 160 are attachedin one or more embodiments. The wire-guiding members 162 are ring-shaped components through which some drive wires 160 slide through (without being attached thereto).

[0093] As shown in FIG. 3B, detail “A” obtained from the identified portion of FIG. 3A illustrates at least one embodiment of a ring-shaped component (a wire-guiding member 162 or an anchor member 164). Each ring-shaped component 162, 164 may include a central opening which may form a tool channel 168 and may include a plurality of conduits 166 (grooves, sub-channels, or through-holes (or thru-holes)) arranged lengthwise (and which may be equidistant from the central opening) along the annular wall of each ring-shaped component 162, 164. Inside the ring-shaped component(s) 162, 164, an inner cover, such as is described in U.S. Pat. Pub. US2021 / 0369085 and US2022 / 0126060, which are incorporated by reference herein in their entireties, maybe included to provide a smooth inner channel and to provide protection. The non-steerable proximal section 148 may be a flexible tubular shaft and may be made of extruded polymer material. The tubular shaft of the proximal section 148 also may have a central opening or tool channel 168 and plural conduits 166 along the wall of the shaft surrounding the tool channel 168. An outer sheath may cover the tubular shaft and the steerable section 152, 154, 156. In this manner, at least one tool channel 168 formed inside the steerable catheter 104 provides passage for an imaging device and / or end effector tools from the insertion port 126 to the distal end of the steerable catheter 104.

[0094] The actuator unit 103 may include, in one or more embodiments, one or more servo motors or piezoelectric actuators. The actuator unit 103 may operate to bend one or more of the bending segments of the catheter 104 by applying a pushing and / or pulling force to the drive wires 160.

[0095] As shown in FIG. 3A, each of the three bendable segments of the steerable catheter104 has a plurality of drive wires 160. If each bendable segment is actuated by three drivewires 160, the steerable catheter 104 has nine driving wires arranged along the wall of the catheter 104. Each bendable segment of the catheter 104 is bent by the actuator unit 103 by pushing or pulling at least one of these nine drive wires 160. Force is applied to each individual drive wire in order to manipulate / steer the catheter 104 to a desired pose. The actuator unit 103 assembled with steerable catheter 104 may be mounted on the robotic platform 108 or any component thereof (e.g., the robotic arm 132, the rail 110, and / or the linear translation stage 122). The robotic platform 108, the rail 110, and / or the linear translation stage 122 may include a slider and a linear motor. In other words, the robotic platform 108 or any component thereof (e.g., the robotic arm 132, the rail 110, and / or the linear translation stage 122) is motorized, and may be controlled by the system controller 102 to insert and remove the steerable catheter 104 to / from the target, sample, or object (e.g., the patient, the patient’s bodily lumen, one or more airways, a lung, a target or object, a specimen, etc.).

[0096] An imaging device 180 that may be inserted through the tool channel 168 includes an endoscope camera (videoscope) along with illumination optics (e.g., optical fibers or LEDs) (or any other camera or imaging device, tool, etc. discussed herein or known to those skilled in the art). The illumination optics provide light to irradiate the lumen and / or a lesion target which is a region of interest within the target, sample, or object (e.g., in a patient). End effector tools may refer to endoscopic surgical tools including clamps, graspers, scissors, staplers, ablation or biopsy needles, and other similar tools, which serve to manipulate body parts (organs or tumorous tissue) during imaging, examination, or surgery. The imaging device 180 may be what is commonly known as a chip-on-tip camera and may be color (e.g., take one or more color images) or black-and-white (e.g., take one or more black-and-white images). In one or more embodiments, a camera may support color and black-and-white images.

[0097] In some embodiments, a tracking sensor 106 (e.g., an EM tracking sensor) is attached to the catheter tip 320. In this embodiment, the steerable catheter 104 and thetracking sensor 106 may be tracked by the tip position detector 107. Specifically, the tip position detector 107 detects a position of the tracking sensor 106, and outputs the detected positional information to the system controller 102. The system controller 102, receives the positional information from the tip position detector 107, and continuously records and displays the position of the steerable catheter 104 with respect to the coordinate system of the target, sample, or object (e.g., a patient, a lung, an airway(s), a vessel, etc.). The system controller 102 operates to control the actuator unit 103 and the robotic platform 108 or any component thereof (e.g., the robotic arm 132, the rail 110, and / or the linear translation stage 122) in accordance with the manipulation commands input by the user U via one or more of the input and / or display devices (e.g., the handheld controller 105, a GUI at the main display 101-1, touchscreen buttons at the secondary display 101-2, etc.).

[0098] FIG. 3C and FIG. 3D show exemplary catheter tip manipulations by actuating one or more bending segments of the steerable catheter 104. As illustrated in FIG. 3C, manipulating only the most distal segment 156 of the steerable section may change the position and orientation of the catheter tip 320. On the other hand, manipulating one or more bending segments (152 or 154) other than the most distal segment may affect only the position of catheter tip 320, but may not affect the orientation of the catheter tip 320. In FIG. 3C, actuation of distal segment 156 changes the catheter tip from a position Pi having orientation 01, to a position P2 having orientation O2, to position P3 having orientation O3, to position P4 having orientation O4, etc. In FIG. 3D, actuation of the middle segment 152 and / or the middle segment 154 may change the position of the catheter tip 320 from a position Pi having orientation 01 to a position P2 and position P3 having the same orientation 01. Here, it should be appreciated by those skilled in the art that exemplary catheter tip manipulations shown in FIG. 3C and FIG. 3D may be performed during catheter navigation (e.g., while inserting the catheter 104 through tortuous anatomies, one or more targets, samples, objects, a patient, etc.). In the present disclosure, the one or more catheter tip manipulations shown in FIG. 3C and FIG. 3D may apply namely to the targeting mode applied after the catheter tip 320 hasbeen navigated to a predetermined distance (a targeting distance) from the target, sample, or object.

[0099] The actuator 103 may proceed or retreat along a rail 110 (e.g., to translate the actuator 103, the continuum robot / catheter 104, etc.), and the actuator 103 and continuum robot 104 may proceed or retreat in and out of the patient’s body or other target, object, or specimen (e.g., tissue). As shown in FIG. 3B, the catheter device 104 may include a plurality of driving backbones and may include a plurality of passive sliding backbones. In one or more embodiments, the catheter device 104 may include at least nine (9) driving backbones and at least six (6) passive sliding backbones. The catheter device 104 may include an atraumatic tip at the end of the distal section of the catheter device 104.

[0100] FIG. 4 illustrates that a system 1000 may include the system controller 102 which may operate to execute software programs and control the display controller too to display a navigation screen (e.g., a live view image 134) on the main display 101-1 and / or the secondary display 101-2. The display controller 100 may include a graphics processing unit (GPU) or a video display controller (VDC) (or any other suitable hardware discussed herein or known to those skilled in the art.

[0101] FIG. 5 illustrates components of the system controller 102 and / or the display controller 100. The system controller 102 and the display controller 100 may be configured separately. Alternatively, the system controller 102 and the display controller too may be configured as one device. In either case, the system controller 102 and the display controller too may include substantially the same components in one or more embodiments. For example, as shown in FIG. 5, the system controller 102 and the display controller too may include a central processing unit (CPU 120) (which may be comprised of one or more processors (microprocessors)), a random access memory (RAM 130) module, an input / output(I / O 140) interface, a read only memory (ROM 110), and data storage memory (e.g., a hard disk drive (HDD 150) or solid state drive (SSD)).

[0102] The ROM no and / or HDD 150 store the operating system (OS) software, and software programs necessary for executing the functions of the robotic catheter system 1000 as a whole. The RAM 130 is used as a workspace memory. The CPU 120 executes the software programs developed in the RAM 130. The I / O 140 inputs, for example, positional information to the display controller 102, and outputs information for displaying the navigation screen to the one or more displays (main display 101-1 and / or secondary display 101-2). In the embodiments descried below, the navigation screen is a graphical user interface (GUI) generated by a software program but, it may also be generated by firmware, or a combination of software and firmware.

[0103] The system controller 102 may control the steerable catheter 104 based on any known kinematic algorithms applicable to continuum or snake-like catheter robots. For example, the system controller controls the steerable catheter 104 based on an algorithm known as follow the leader (FTL) algorithm. By applying the FTL algorithm, the most distal segment 156 is actively controlled with forward kinematic values, while the middle segment 154 and the other middle or proximal segment 152 (following sections) of the steerable catheter 104 move at a first position in the same way as the distal section moved at the first position or a second position near the first position.

[0104] The display controller too may acquire position information of the steerable catheter 104 from system controller 102. Alternatively, the display controller too may acquire the position information directly from the tip position detector 107. The steerable catheter 104 maybe a single-use or limited-use catheter device. In other words, the steerable catheter 104 maybe attachable to, and detachable from, the actuator unit 103 to be disposable.

[0105] During a procedure, the display controller too may generate and output a live-view image or a navigation screen to the main display 101-1 and / or the secondary display 101-2 based on the 3D model of a target, sample, or object (e.g., a lung, an airway, a vessel, a patient’s anatomy (a branching structure), etc.) and the position information of at least a portion of the catheter (e.g., position of the catheter tip 320) by executing pre-programmed software routines. The navigation screen may indicate a current position of at least the catheter tip 320 on the 3D model. By observing the navigation screen, a user may recognize the current position of the steerable catheter 104 in the branching structure. Upon completing navigation to a desired target, one or more end effector tools may be inserted through the access port 126 at the proximal end of the catheter 104, and such tools may be guided through the tool channel 168 of the catheter body to perform an intraluminal procedure from the distal end of the catheter 104.

[0106] The tool may be a medical tool such as an endoscope camera, forceps, a needle, or other biopsy or ablation tools. In one embodiment, the tool may be described as an operation tool or working tool. The working tool is inserted or removed through the working tool access port 126. In the embodiments below, at least one embodiment of using a steerable catheter 104 to guide a tool to a target is explained. The tool may include an endoscope camera or an end effector tool, which may be guided through a steerable catheter under the same principles. In a procedure there is usually a planning procedure, a registration procedure, a targeting procedure, and an operation procedure.

[0107] In one or more embodiments, the one or more processors, such as, but not limited to, the display controller too, may generate and output a navigation screen to the one or more displays 101-1, 101-2 based on the 2D / 3D model and the position / orientation / navigation / pose / state (or other state) information by executing the software. The navigation screen may indicate a current position / orientation / navigation / pose / state (or other state) of the continuum robot 104 on the2D / 3D model. By using the navigation screen, a user may recognize the current position / orientation / navigation / pose / state (or other state) of the continuum robot 104 in the branching structure. Any feature of the present disclosure may be used with any navigation / pose / state feature(s) or other feature(s) discussed in U.S. Prov. Pat. App. No. 63 / 504,972, filed on May 30, 2023, the disclosure of which is incorporated by reference herein in its entirety, and discussed in International Pat. App. No. PCT / US2024 / 031766, filed on May 30, 2024, the disclosure of which is incorporated by reference herein in its entirety.

[0108] In one or more embodiments, the one or more processors, such as, but not limited to, the display controller too and / or the controller 102, may include, as shown in FIG. 5, at least one storage Read Only Memory (ROM) 110, at least one central processing unit (CPU) 120, at least one Random Access Memory (RAM) 130, at least one input and output (I / O) interface 140 and at least one Hard Disc Drive (HDD) 150 (see e.g., also data storage 150 of FIG. 4). A Solid State Drive (SSD) may be used instead of HDD 150 as the data storage 150. In one or more additional embodiments, the one or more processors, and / or the display controller too and / or the controller 102, may include structure as shown in FIGS. 12-13 and 14-15 as further discussed below.

[0109] The ROMno and / or HDD 150 operate to store the software in one or more embodiments. The RAM 130 may be used as a work memory. The CPU 120 may execute the software program developed in the RAM 130. The I / O 140 operates to input the positional (or other state) information to the display controller 100 (and / or any other processor discussed herein) and to output information for displaying the navigation screen to the one or more displays 101-1, 101-2. In the embodiments below, the navigation screen may be generated by the software program. In one or more other embodiments, the navigation screen may be generated by a firmware.[ono] One or more devices or systems, such as the system 1000, may include a tip position / orientation / navigation / pose / state (or other state) detector 107 that operates to detect a position / orientation / navigation / pose / state (or other state) of the EM tracking sensor 106 and to output the detected positional (and / or other state) information to the controller too or 102 (e.g., as shown in FIGS. 1-2), or to any other processor(s) discussed herein.

[0111] The controller 102 may operate to receive the positional (or other state) information of the tip of the continuum robot 104 from the tip position / orientation / navigation / pose / state (or any other state discussed herein) detector 107. The controller too and / or the controller 102 operates to control the actuator 103 in accordance with the manipulation by a user (e.g., manually), and / or automatically (e.g., by a method or methods run by one or more processors using software, by the one or more processors, using automatic manipulation in combination with one or more manual manipulations or adjustments, etc.) via one or more operation / operating portions or operational controllers 105 (e.g., such as, but not limited to a joystick as shown in FIGS. 1-2; see also, diagram of FIG. 4). The one or more displays 101-1, 101-2 and / or operation portion or operational controllers 105 may be used as a user interface 3000 (also referred to as a receiving device) (e.g., as shown diagrammatically in FIG. 4). In an embodiment shown in FIGS. 1-2 or the embodiment shown in FIG. 4, the system(s) 1000 may include, as an operation unit, the display 101-1 (e.g., such as, but not limited to, a large screen user interface with a touch panel, first user interface unit, etc.), the display 101-2 (e.g., such as, but not limited to, a compact user interface with a touch panel, a second user interface unit, etc.) and the operating portion 105 (e.g., such as, but not limited to, a joystick shaped user interface unit having shift lever / button, a third user interface unit, a gamepad, or other input device, etc.).

[0112] The controller 100 and / or the controller 102 (and / or any other processor discussed herein) may control the continuum robot 104 based on an algorithm known as follow the leader (FTL) algorithm. The FTL algorithm may be used in addition to the robotic controlfeatures of the present disclosure. For example, by applying the FTL algorithm, the middle section and the proximal section (following sections) of the continuum robot 104 may move at a first position (or other state) in the same or similar way as the distal section moved at the first position (or other state) or a second position (or state) near the first position (or state) (e.g., during insertion of the continuum robot / catheter 104, by using the navigation, movement, and / or control feature(s) of the present disclosure, etc.). Similarly, the middle section and the distal section of the continuum robot 104 may move at a first position or state in the same / similar / approximately similar way as the proximal section moved at the first position or state or a second position or state near the first position (e.g., during removal of the continuum robot / catheter 104). Additionally or alternatively, the continuum robot / catheter 104 may be removed by automatically and / or manually moving along the same or similar, or approximately same or similar, path that the continuum robot / catheter 104 used to enter a target (e.g., a body of a patient, an object, a specimen (e.g., tissue), etc.) using the FTL algorithm, including, but not limited to, using FTL with the one or more control, adjustment, correction, state, and / or smoothing technique(s) discussed herein.

[0113] Additionally or alternatively, any feature of the present disclosure may be used with features, including, but not limited to, autonomous navigation feature(s), artificial intelligence feature(s), etc., as discussed in U.S. Prov. Pat. App. No. 63 / 513,803, filed on July 14, 2023, the disclosure of which is incorporated by reference herein in its entirety, and as discussed in International Pat. App. No. PCT / US2024 / 037935, filed on July 12, 2024, the disclosure of which is incorporated by reference herein in its entirety.

[0114] Any of the one or more processors, such as, but not limited to, the controller 102 and the display controller 100, may be configured separately. As aforementioned, the controller 102 may similarly include a CPU 120, a RAM 130, an I / O 140, a ROM 110, and a HDD 150 as shown diagrammatically in FIG. 5. Alternatively, any of the one or more processors, such as, but not limited to, the controller 102 and the display controller too, maybe configured as one device (for example, the structural attributes of the controller too and the controller 102 may be combined into one controller or processor, such as, but not limited to, the one or more other processors discussed herein (e.g., computer, console, or processor 1200, 1200’, etc.).

[0115] The system 1000 may include a tool channel 126 for a camera, biopsy tools, or other types of medical tools (as shown in FIGS. 1-2). For example, the tool may be a medical tool, such as an endoscope, a forceps, a needle, or other biopsy tools, etc. In one or more embodiments, the tool may be described as an operation tool or working tool. The working tool may be inserted or removed through a working tool insertion slot 126 (as shown in FIGS. 1-2). Any of the features of the present disclosure may be used in combination with any of the features, including, but not limited to, the tool insertion slot, as discussed in U.S. Prov. Pat. App. No. 63 / 378,017, filed September 30, 2022, the disclosure of which is incorporated by reference herein in its entirety, and as discussed in U.S. Pat. App. No. 18 / 477,081, filed on September 28, 2023, the disclosure of which is incorporated by reference herein in its entirety, and / or any of the features as discussed in U.S. Prov. Pat. App. No. 63 / 377,983, filed September 30, 2022, the disclosure of which is incorporated by reference herein in its entirety, and as discussed in U.S. Pat. App. No. 18 / 477,081, filed on September 28, 2023, the disclosure of which is incorporated by reference herein in its entirety.

[0116] One or more of the features discussed herein may be used for planning procedures, including using one or more models for robotic control and / or artificial intelligence applications. As an example of one or more embodiments, FIG. 6 is a flowchart showing steps of at least one planning procedure of an operation of the continuum robot / catheter device 104. One or more of the processors discussed herein may execute the steps shown in FIG. 6, and these steps may be performed by executing a software program read from a storage medium, including, but not limited to, the ROMno or HDD 150, by CPU 120 or by any other processor discussed herein. One or more methods of planning using the continuum robot / catheter6odevice 104 may include one or more of the following steps: (i) In step s6oi, one or more images such, as CT or MRI images, maybe acquired; (ii) In step S602, a three dimensional model of a branching structure (for example, an airway model of lungs or a model of an object, specimen or other portion of a body) may be generated based on the acquired one or more images; (iii) In step S603, a target on the branching structure may be determined (e.g., based on a user instruction, based on preset or stored information, etc.); (iv) In step S604, a route of the continuum robot / catheter device 104 to reach the target (e.g., on the branching structure) may be determined (e.g., based on a user instruction, based on preset or stored information, based on a combination of user instruction and stored or preset information, etc.); and / or (v) In step S605, the generated model (e.g., the generated two-dimensional or three-dimensional model) and the decided route on the model may be stored (e.g., in the RAM 130 or HDD or data storage 150, in any other storage medium discussed herein, in any other storage medium known to those skilled in the art, etc.). In this way, a model (e.g., a 2D or 3D model) of a branching structure may be generated, and a target and a route on the model may be determined and stored before the operation of the continuum robot 104 is started.

[0117] In one or more of the embodiments below, embodiments of using a catheter device / continuum robot 104 are explained, such as, but not limited to features for performing navigation, movement, and / or robotic control technique(s).

[0118] In one or more embodiments, the navigation and / or control may be employed so that an apparatus or system having multiple portions or sections (e.g., multiple bending portions or sections) operates to: (i) keep track of a path of a portion (e.g., a tip) or of each of the multiple portions or sections of an apparatus or system; (ii) have a state or states of each of the multiple portions or sections match a state or states of a first portion or section of the multiple portions or sections in a case where each portion or section reaches or approaches a same, similar, or approximately similar state (e.g., a position or other state(s) in a target, object, or specimen; a position or other state(s) in a patient; a target position or state(s) in animage or frame; a set or predetermined position or state(s) in an image or frame; a set or predetermined position or state(s) in an image or frame where the first portion or section reaches or approaches the set or predetermined position or state(s) at one point in time and one or more of other portions or sections of the multiple portions or sections reach the set or predetermined position or state(s) at one or more other points in time; any other state (which may include, but is not limited to, an orientation, a position, a pose, a navigation, a path (whether continuous or discontinuous), a state transition, any other desired motion(s) or combination of motion(s) (e.g., one or more features of the present disclosure may assist navigation, orientation, or any other types of motions discussed herein or as desired by a user), etc.), a combination of any state(s) and / or motion(s) discussed herein or desired by a user, etc.) of another portion or section of the one or more devices, systems, methods, and / or storage mediums of the present disclosure, etc.); (iii) utilize additional data (such as, but not limited to, target pose or state information, final pose or state information, interpolated pose or state information, measured pose or state information, converting pose or state information between different states (e.g., drive wire position(s) or state(s); coordinates (three- dimensional (3D) position(s), orientation(s), and / or state(s)); plane and / or angle information for pose(s) or state(s); state position or state information (e.g., target, interpolated, measured pose(s) and / or state(s)); force sensor(s) information; drawer current draw information of one or more actuator motors; section dimension information (e.g., size, shape, length, etc.) for one or more sections of the catheter or probe (e.g., a tip section of the catheter or probe, a middle section of the catheter or probe, a distal section of the catheter or probe, a proximal section of the catheter or probe, any combination thereof, etc.) from an entire device or system (e.g., using forwards or inverse kinematics of the device or system, using other internal sensor(s) or information of the device or system, etc.) and / or external source(s) (e.g., one or more external sensors (e.g., an electromagnetic (EM) sensor, a shape sensor, any other sensor discussed herein or known to those skilled in the art, etc.) in a robotic control algorithm; (iv) utilize differences between a previous or expected robotic control state and a new control state in future calculation(s) of robotic control state(s); and / or (v) address any discontinuous path thatmay occur (e.g., due to a change (e.g., of a state or states) or other movement or state change / transition of any portion of the apparatus or system), for example, by smoothing out any difference in the discontinuous path over one or more multiple stage positions or states or over one or more other path-like information positions or states, by considering target or object movement(s) (e.g., movement of a patient or a portion of a patient while a probe or catheter is disposed in the patient, etc.).

[0119] In one or more embodiments, pose or state information may be stored in a lookup table or tables, and the pose or state information for one or more sections of the catheter or probe may be updated in the lookup table based on new information (e.g., environmental change(s) for the catheter or probe, movement of a target or sample, movement of a patient, user control, relaxation state changes, etc.). The new information or the updated information may be used to control the one or more sections of the catheter or probe more efficiently during navigation (forwards and / or backwards). For example, in a case where a previously stored pose or state may have shifted or changed due to a movement or relaxation of the target, object, or sample (e.g., a patient may move), the previously stored pose or state may not be ideal or may work less efficiently as compared with an updated pose or state modified or updated in view of the new information (e.g., the movement, in this example). As such, one or more embodiments of the present disclosure may update or modify the pose or state information such that robotic control of the catheter or probe may work efficiently in view of the new information, movement, relaxation, and / or environmental change(s). In addition to having the update or change affect the previously stored history or known histoiy at that point in space (e.g., similar to dragging that point (e.g., of the target, object, or sample (e.g., a patient, a portion of a patient, a vessel, a spline, a lung, etc.); of the catheter or probe; etc.) and recalculating the path), in one or more embodiments, the update or change may also affect a number of other points (e.g., all points in a lookup table or tables, all points forward beyond the initially changed point, one or more future points or points beyond the initially changed point as desired, etc.). For example, in one or more embodiments, the transform (ordifference, change, update, etc.) between the previous pose or state and the new or updated post or state may be propagated to all points going forward or may be propagated to one or more of forward points (e.g., for a predetermined or set range, for a predetermined or set distance, etc.). Doing so in one or more embodiments may operate to shift all or part of the future path based on how the pose or state of the catheter or probe was adjusted, using that location as a pivot point. Such update(s) may be obtained from one or more internal sources (e.g., one or more processors, one or more sensors, combination(s) thereof, etc.) or may be obtained from one or more external sources (e.g., one or more other processors, one or more external sensors, combination(s) thereof, etc.). For example, a difference between a real-time target, sample, or object (e.g., an airway) and the previous target, sample, or object (e.g., a previous airway) may be detected using machine vision (of the endoscope image) or using multiple medical images. Body, target, object, or sample divergence may also be estimated from other sensors, like one measuring breathing or the motion of the body (or another predetermined or set motion or change to track). In one or more embodiments, an amount of transform, update, and / or change maybe different for each point, and / or maybe a function of, for example, a distance from a current point.

[0120] One or more robotic control methods of the present disclosure may be employed in one or more embodiments. For example, one or more of the following techniques or methods maybe used to update historical information of a catheter or probe (or portion(s) or section(s) of the catheter or probe): Hold the Line, Close the Gap, and / or Stay the Course.

[0121] One or more methods of controlling or using a continuum robot / catheter device (e.g., robot or catheter device 104) may use one or more Hold the Line techniques, and may include one or more of the following steps (as shown in FIGS. 7A-7B): (i) In step S700, a catheter or robot device may move forward (e.g., while a stage of the catheter or robot moves forward, while the navigation is mapped to Z stage position (e.g., a position, pose, or state of a Tip section or portion of the catheter or probe may be converted to a coordinate (e.g., X, Y,Z coordinate) during navigation), etc.); (ii) In step S701, coordinates for a Tip end effector of the Tip section or portion may be calculated; (iii) In step S702, add the calculated coordinate information to a 3D path for the Tip end / section / portion and / or catheter or probe; (iv) In step S703, coordinates for a Middle / proximal end effector of a Middle / proximal (or other section or portion subsequent to or following the Tip section or portion) section or portion of the catheter or probe maybe calculated; (v) In step S704, a distance from a closest point along the 3D path may be identified for the Middle / proximal end effector and / or the Tip end effector; (vi) In step S706, the calculated distance may be converted to a change in a pose, position, or state of the Tip end effector, the Middle / proximal end effector, and / or the catheter or probe; and (vii) In step S707, the pose, position, or state of the Middle / proximal section or portion of the catheter or probe may be updated (e.g., to match the pose, position or state of the Tip section or portion of the catheter or probe at that point along the path) and the process may then return to step S703 (and repeat steps S703 through S704 and / or S705 as needed). In one or more embodiments (e.g., as shown in FIG. 7B), one or more methods may include a step S705 in which it is evaluated or determined whether the distance of step S704 is greater than zero and / or is within a threshold, and / or whether a change in pose, position, or state occurred for the Tip section or portion. If “YES” the process proceeds to steps S706-S707 (and may repeat steps S703-S705 as needed). If “NO”, the process may end.

[0122] While not limited thereto, one or more embodiments may employ at least one Hold the Line technique(s) or method(s) shown in FIGS. 7A-7B as aforementioned. In one or more embodiments, a Tip position or state may be converted to a coordinate (e.g., X, Y, Z coordinate) during navigation (e.g., while a stage moves forward, while the navigation is mapped to Z stage position, etc.) (see e.g., step S700 in FIGS. 7A-7B). In a case where a pose, position, or state is being determined for other section(s) or portion(s) of the catheter or probe, one or more embodiments use a pose, position, or state which operates to keep an end effector of the section(s) or portion(s) as close as possible to the Tip pose, position, or state coordinate at that point along the insertion path. The pose, position, or state coordinates of precedingsection(s) or portion(s) of the catheter or probe may be known or determined in one or more embodiments. Body, target, object, or sample divergence may be applied with a coordinate shift. In cases where a length of each section or sections, or each portion or portions, of the catheter or probe are not considered, applying a Tip pose, position, or state to a middle / proximal section or portion of the catheter or probe may result in a situation where a middle / proximal end effector may be in a drastically different location than the Tip end effector at that point along the path (e.g., a middle / proximal section maybe longer or maybe twice as long as the Tip section, for example). Additionally, by considering the lengths of each section or sections of the catheter or probe, appropriate adjustment(s) may be made so that the middle / proximal end effector, and / or an effector of another section or portion of the catheter or probe, maybe in an appropriate / desired / intended location or maybe in the same location as the Tip end effector was at that point along the path. Using such techniques, one or more embodiments of the present disclosure may apply one or more Hold the Line features to a catheter or probe to achieve efficient and accurate control of the catheter or probe.

[0123] One or more methods of controlling or using a continuum robot / catheter device (e.g., robot or catheter device 104) may use one or more Close the Gap techniques, and may include one or more of the following steps (as shown in FIGS. 8A-8B): (i) In step s8oo, a pose, position, or state of a Middle / proximal section or portion of a catheter or probe may be identified, determined, calculated, or otherwise obtained; (ii) In step s8oi, a pose, position, or state of a Tip section or portion of a catheter or probe may be identified, determined, calculated, or otherwise obtained; (iii) In step S802, a difference between the poses, positions, or states of the Tip section or portion and of the Middle / proximal (or other subsequent or following) section or portion maybe determined, identified, calculated, or otherwise obtained; (iv) In step S804, the pose, position, or state difference between the tip section or portion and the Middle / proximal (or other subsequent or following) section or portion may be interpolated over a set or predetermined length; and (v) In step S805, the pose, position, or state of the Middle / proximal (or other subsequent or following) section or portion of thecatheter or probe maybe updated using the corresponding interpolated pose, position, or state difference. In one or more embodiments, step S805 may consider motion of the catheter or probe when updating the pose, position, or state of the Middle / proximal (or other subsequent) portion or section of the catheter or probe. For example, in step S806, information for navigation (e.g., while a stage moves forward or backward, while the navigation is mapped to Z stage position (of an X, Y, Z coordinate), etc.) of the catheter or probe may be obtained and provided to step S805 for updating the pose, position, or state of the Middle / proximal (or other subsequent) portion or section of the catheter or probe. In one or more embodiments (e.g., as shown in FIG. 8B), one or more methods may include a step S803 in which it is evaluated or determined whether the difference of step S802 is greater than zero and / or is within a threshold, and / or whether a change in pose, position, or state occurred for the Tip section or portion and / or the Middle / proximal (or other subsequent) portion or section of the catheter or probe. If “YES”, the process proceeds to steps S804-S806 (and repeats steps S800, S802, and S803 as needed). If “NO”, then the process may end.

[0124] For example, one or more gaps may be closed by adjusting a pose, position, or state of a section or sections of a catheter or probe to reduce, minimize, or eliminate a relative difference between the pose, position, or state of the section or sections versus a pose, position, or state of the subsequent / next / future / following section or sections. In one or more embodiments, a trajectory of the subsequent / next / future / following section or sections may be known or determined. Closing such a difference or gap operates to eliminate any motion outside of a desired direction or trajectory. For example, if the Tip section or portion of the catheter or probe faced left (or to a first direction), then moved forwards slightly, and then faced right (or a towards a second direction), then a Middle / proximal (or other subsequent or following) section or portion of the catheter or probe may also be moved left in a case where the Middle / proximal (or other subsequent or following) section or portion reaches the point that the Tip section or portion was facing left (or to the first direction). However, since the Tip section or portion wants to be moving right (or towards the second direction) after that point,then the motion of the Tip section or portion to the left (or first) direction will be moving counter-actively to the desired direction, which may be the right (or the second) direction. Instead, using one or more features of the present disclosure, any counter-active motion(s) may be avoided or eliminated.

[0125] One or more methods of controlling or using a continuum robot / catheter device (e.g., robot or catheter device 104) may use one or more Stay the Course techniques, and may include one or more of the following steps (as shown in FIGS. 9A-9B): (i) In step S900, a catheter or robot device may move forward (e.g., while a stage of the catheter or robot moves forward, while the navigation is mapped to Z stage position (e.g., a position, pose, or state of a Tip section or portion of the catheter or probe may be converted to a coordinate (e.g., X, Y, Z coordinate) during navigation), etc.; in one or more embodiments, step S900 may be performed similarly or substantially similar to, or the same as, step S700 described above); (ii) In step S901, a vector (e.g., a normal vector or a normal path; a predetermined, targeted, desired trajectory or path; etc.) may be calculated for a Tip end effector of the Tip section or portion of the catheter or probe; (iii) In step S903, a deviation of the Tip end effector from the normal path or vector due to catheter or probe shape and / or motion (e.g., motion from movement of a stage or translational stage, motion from changes due to an environment or the target or sample in which the catheter or probe is located, body divergence, motion from another source, etc.); (iv) In step S904, a change to a pose, position, or state of a Middle / proximal (or other section or portion subsequent to or following the Tip section or portion) section or portion of the catheter or probe may be calculated to counteract or remove the calculated deviation (e.g., from step S903); and (v) In step S905, the pose, position, or state of the Middle / proximal section or portion of the catheter or probe may be updated (e.g., to match the pose, position or state of the Tip section or portion of the catheter or probe at that point along the path, to eliminate or remove the calculated deviation, etc.), and / or a proximal section(s) and / or the stage may be updated or adjusted. In one or more embodiments (e.g., as shown in FIG. 9B), one or more methods may include a step S902 inwhich it is evaluated or determined whether a path deviation due to a catheter or probe shape and / or motion (e.g., due to stage motion, due to translational motion, due to movement or motion of the target, object, or sample, body divergence, due to motion of an outside force or influence on the catheter or probe, etc.) exists. If “YES”, then the process may proceed to steps S903-S905 (and may repeat steps S903-S905 as needed). If “NO”, then the process may end. In one or more embodiments, the existence of the path deviation of step S902 may be used as a trigger for, and used in, the calculation of the step S903.

[0126] As aforementioned, a catheter or probe may be controlled to stay the desired course. For example, a pose, position, or state of a section or sections, or of a portion or portions, of the catheter or probe may be adjusted to minimize any deviation of a pose, position, or state of one or more next (e.g., subsequent, following, proximal, future, Middle / proximal, etc.) sections out of the predetermined, targeted, desired trajectory and maximum motion along the trajectory. In one or more embodiments, the coordinates and the trajectory of subsequent / following / next / future sections may be known, set, or determined, and information for one or more prior sections may be known, set, or determined. By considering section lengths in one or more embodiments, one or more advantageous results may be achieved. As aforementioned, in one or more embodiments, a Middle / proximal section end effector may deviate from a location of a Tip section at that point (e.g., along the path, vector, trajectory; along the predetermined, targeted, desired trajectory; along the normal vector or path; etc.). Such a deviation may cause the Tip section or portion of the catheter or probe to move in directions that are not in the direction it is facing. This effect may be compounded in a case where a proximal / Middle (or other subsequent, next, following, future, etc.) section or portion is included in the movement or control. As such, by using one or more features of the present disclosure, any counter-active or undesired motion(s) may be avoided or eliminated.

[0127] In one or more embodiments, the system controller 102 (or any other controller, processor, computer, etc. discussed herein) may operate to perform a robotic control mode and / or an autonomous navigation mode. During the robotic control mode and / or the autonomous navigation mode, the user does not need to control the bending and translational insertion position of the steerable catheter 104. The autonomous navigation mode may include or comprise: (1) a perception step, (2) a planning step, and (3) a control step. In the perception step, the system controller 102 may receive an endoscope view (or imaging data) and may analyze the endoscope view (or imaging data) to find addressable airways from the current position / orientation of the steerable catheter 104. At an end of this analysis, the system controller 102 identifies or perceives these addressable airways as paths in the endoscope view (or imaging data).

[0128] The planning step is a step to determine a target path, which is the destination for the steerable catheter 104. While there are a couple of different approaches to select one of the paths as the target path, the present disclosure uniquely includes means to reflect user instructions concurrently for the decision of a target path among the identified or perceived paths. Once the system 1000 determines the target paths while considering concurrent user instructions, the target path is sent to the next step, z.e., the control step.

[0129] The control step is a step to control the steerable catheter 104 and the linear translation stage 122 (or any other portion of the robotic platform 108) to navigate the steerable catheter 104 to the target path, pose, state, etc. This step may also be performed as an automatic step. The system controller 102 operates to use information relating to the real time endoscope view (e.g., the view 134), the target path, and an internal design & status information on the robotic catheter system 1000.

[0130] Through these three steps, the robotic catheter system 1000 may navigate the steerable catheter 104 automatically or autonomously, which achieves reflecting the user’s intention efficiently.

[0131] As shown in FIG. 1, the real-time endoscope view 134 may be displayed in a main display 101-1 (as a user input / output device) in the system 1000. The user may see the airways in the real-time endoscope view 134 through the main display 101-1. This real-time endoscope view 134 may also be sent to the system controller 102. In the perception step, the system controller 102 may process the real-time endoscope view 134 and may identify path candidates by using image processing algorithms. Among these path candidates, the system controller 102 may select the paths with the designed computation processes, and then may display the paths with a circle, octagon, or other geometric shape with the real-time endoscope view 134, for example, as discussed in U.S. Prov. Pat. App. No. 63 / 513,803, filed on July 14, 2023, the disclosure of which is incorporated by reference herein in its entirety, and as discussed in International Pat. App. No. PCT / US2024 / 037935, filed on July 12, 2024, the disclosure of which is incorporated by reference herein in its entirety.

[0132] In planning step, the system controller 102 may provide a cursor so that the user may indicate the target path by moving the cursor with the joystick 105. When the cursor is disposed or is located within the area of the path, the system controller 102 operates to recognize the path with the cursor as the target path.

[0133] In a further embodiment example, the system controller 102 may pause the motion of the actuator unit 103 and the linear translation stage 122 while the user is moving the cursor so that the user may select the target path with a minimal change of the real-time endoscope view 134 and paths since the system 1000 would not move in such a scenario.

[0134] In one or more of the embodiments below, embodiments of using a catheter device / continuum robot 104 are explained. Any feature of the present disclosure may be used with autonomous navigation, movement detection, and / or control technique(s), including, but not limited to, the features discussed in U.S. Prov. Pat. App. No. 63 / 513,803, filed on July 14, 2023, the disclosure of which is incorporated by reference herein in its entirety, and as discussed in International Pat. App. No. PCT / US2024 / 037935, filed on July 12, 2024, the disclosure of which is incorporated by reference herein in its entirety.

[0135] The system controller 102 may control the steerable catheter 104 based on any known kinematic algorithms applicable to continuum or snake-like catheter robots. For example, the system controller controls the steerable catheter 104 based on an algorithm known as follow the leader (FTL) algorithm. By applying the FTL algorithm, the most distal segment 156 is actively controlled with forward kinematic values, while the middle segment 154 and the other middle or proximal segment 152 (following sections) of the steerable catheter 104 move at a first position in the same way as the distal section moved at the first position or a second position near the first position. In one or more additional or alternative embodiments, any other algorithm may be applied to control a continuum robot or catheter / probe, such as, but not limited to, Hold the Line, Close the Gap, Stay the Course, any combination thereof, etc.

[0136] In one or more embodiments, where smoothing is performed, interpolated orientations may be combined with values previously mapped to a corresponding stage position / state (or position / state of another structure used for mapping path or path-like information) (e.g., by simply adding the wire positions or states).

[0137] Due to kinematics of a robot, device, or system embodiment of the present disclosure, applying a same “change in position” or a “change in state” to two separate orientations / states may maintain a difference (e.g., a set difference, a predetermineddifference, etc.) between the two separate orientations / states. Since an orientation / state difference may be defined as the difference between wire position / state in one or more embodiments (other embodiments are not limited thereto), changing both sets of wire positions or states by the same amount would not affect the orientation or state difference between the two separate orientations or states.

[0138] Orientations mapped to two subsequent stage positions / states (or positions / states of another structure used for mapping path or path-like information) may have a specific orientation difference between the orientations. In a case where smoothing is applied, the later (or second) stage position / state (or position / state of the another structure) has a same change in orientation that the earlier (or first) stage position / state (or position / state of the another structure) received such that the pose / state difference did not change. The smoothing process may include an additional step of a “small motion”, which operates to cause the pose / state difference to change by an amount of that small motion. Since the “small motion” operates to produce the same orientation / state change regardless of prior orientation / state, the small motion step operates to direct that orientation / state in a table towards a proper (e.g., set, desired, predetermined, selected, etc.) direction, while also maintaining a semblance or configuration of the prior path / state before the smoothing process was applied. Therefore, in one or more embodiments, it maybe most efficient and effective to combine and compare wire positions or states to or with prior orientations or states while using a smoothing process to maintain the pre-existing orientation changes.

[0139] In one or more embodiments, a catheter or probe may transition, move, or adjust using a shortest possible volume. In a case where a following section or portion of the probe or catheter is being transitioned, moved, or adjusted, using the shortest possible volume my reduce or minimize an amount of disruption to positions or states of one or more (or all) of the distal / foll owing sections or portions of the catheter or probe. In one or more embodiments, a process or algorithm may perform the transitioning, moving, or adjustingprocess more efficiently than computing a transformation stackup of each section or portion of the catheter or probe. Preferably, each interpolated step aims towards the final orientation in a desired direction such that any prior orientation which the interpolated step is combined with will also aim towards the desired direction to achieve the final orientation.

[0140] In one or more embodiments of the present disclosure, an apparatus or system may include one or more processors that operate to: receive or obtain an image or images showing pose or position (or other state) information of a tip section of a catheter or probe having a plurality of sections including at least the tip section; track a history of the pose or position (or other state) information of the tip section of the catheter or probe during a period of time; and use the history of the pose or position (or other state) information of the tip section to determine how to align or transition, move, or adjust (e.g., robotically, manually, automatically, etc.) each section of the plurality of sections of the catheter or probe.

[0141] In one or more embodiments, one or more additional image or images may be received or obtained to show the catheter or probe after each section of the plurality of sections of the catheter or probe has been aligned or adjusted (e.g., robotically, manually, automatically, etc.) based on the history of the pose or position (or other state) information of the tip section. In one or more embodiments, the apparatus or system may include a display to display the image or images showing the aligned or adjusted sections of the catheter or probe. In one or more embodiments, the pose or position (or other state) information may include, but is not limited to, a target pose or position (or other state) or a final pose or position (or other state) that the tip section is set to reach, an interpolated pose or position (or other state) of the tip section (e.g., an interpolation of the tip section between two positions or poses (or other states) (e.g., between pose or position (or other state) A to pose or position (or other state) B) where the apparatus or system sends pose (or other state) change information in steps based on a desired, set, or predetermined speed; between poses or positions where each pose or position (or other state) of the catheter or probe takes or is disposed is tracked duringthe transition; etc.), and a measured pose or position (or other state) (e.g., using tracked poses or positions (or other states), using encoder positions (or other states) of each wire motor, etc.) where the one or more processors may further operate to calculate or derive a current position or position (or state) that a section (e.g., the tip section, one of the other sections of the plurality of sections of the probe or catheter, etc.) of the probe or catheter is taking. In addition to using one or more types of poses or positions (or other states), each pose or position (or state) may be converted (e.g., via the one or more processors) between the following formats: Drive Wire Positions (or state(s)); and / or Coordinates (three-dimensional (3D) Position and Orientation (or other state(s))).

[0142] In one or more embodiments, an apparatus or system may include a camera deployed at a tip of a catheter or probe and may be bent with the catheter or probe, and / or the camera may be detachably attached to, or removably inserted into, the steerable catheter or probe. In one or more embodiments, an apparatus or system may include a display controller, or the one or more processors may display the image or images for display on a display.

[0143] One or more features discussed herein may be used for performing control, correction, adjustment, and / or smoothing (e.g., direct FTL smoothing, path smoothing, continuum robot smoothing, etc.). FIG. 10 is a flowchart showing steps of at least one procedure for performing correction, adjustment, and / or smoothing of a continuum robot / catheter device (e.g., such as continuum robot / catheter device 104). One or more of the processors discussed herein may execute the steps shown in FIG. 10, and these steps may be performed by executing a software program read from a storage medium, including, but not limited to, the ROMno or HDD 150, by CPU 120 or by any other processor discussed herein. One or more methods of performing correction, adjustment, and / or smoothing (e.g., direct FTL smoothing) for a catheter or probe of a continuum robot device or system may include one or more of the following steps: (i) in step S1300, instructing a distal bending section or portion of a catheter or a probe of a continuum robot such that the distal bending section orportion achieves, or is disposed at, a bending pose or position; (ii) in step S1301, storing or obtaining the bending pose or position of the distal bending section or portion and storing or obtaining a position of a motorized linear stage that operates to move the catheter or probe of the continuum robot in a case where a forward motion, or a motion in a set or predetermined direction or directions, of the motorized linear stage is instructed or commanded; (iii) in step S1302, generating a goal or target bending pose or position (or other state) for each corresponding section or portion of the catheter or probe from, or based on, the previous bending section or portion or based on a previous pose or state of a Distal bending section or portion; (iv) in step S1303, generating interpolated poses or positions for each of the sections or portions of the catheter or probe between the respective goal or target bending pose or position and a respective current bending pose or position of each of the sections or portions of the catheter or probe, wherein the interpolated poses or positions are generated such that an orientation vector of the interpolated poses or positions are on a plane that an orientation vector of the respective goal or target bending pose or position and an orientation vector of a respective current bending pose or position create or define; and / or (v) in step S1304, instructing or commanding each of the sections or portions of the catheter or probe to move to or be disposed at the respective interpolated poses or positions during the forward motion, or the motion in the set or predetermined direction, of the previous section(s) or portion(s) of the catheter or probe (see FIG. 10).

[0144] In one or more embodiments, a non-transitory computer- readable storage medium may store at least one program for causing a computer to execute a method for performing control, correction, adjustment, and / or smoothing (e.g., direct FTL smoothing, path smoothing, continuum robot smoothing, etc.), such as, but not limited to, the method step(s) of FIG. 10 as discussed above.

[0145] One or more of the aforementioned features may be used with a continuum robot and related features as disclosed in U.S. Provisional Pat. App. No. 63 / 150,859, filed onFebruary 18, 2021, the disclosure of which is incorporated by reference herein in its entirety, as discussed in International Pat. App. No. PCT / US2022 / 016660, filed February 16, 2022, the disclosure of which is incorporated by reference herein in its entirety, and as disclosed in U.S. Pat. App. No. 17 / 673,606, filed on February 16, 2022, the disclosure of which is incorporated by reference herein in its entirety. For example, FIGS. 11 to 13 illustrate features of at least one embodiment of a continuum robot apparatus 10 configuration to implement robotic control and / or automatic correction of a direction to which a tool channel or a camera moves or is bent in a case where a displayed image is rotated. The continuum robot apparatus 10 enables to keep a correspondence between a direction on a monitor (top, bottom, right or left of the monitor) and a direction the tool channel or the camera moves on the monitor according to a particular directional command (up, down, turn right or turn left) even if the displayed image is rotated.

[0146] As shown in FIGS. 11 and 12, the continuum robot apparatus 10 may include one or more of a continuum robot 11, an image capture unit 20, an input unit 30, a guide unit 40, a controller 50, and a display 60. The image capture unit 20 maybe a camera or other image capturing device. The continuum robot 11 may include one or more flexible portions 12 connected together and configured so that the one or more flexible portions 12 may be curved or rotated about in different directions. The continuum robot 11 may include a drive unit 13, a movement drive unit 14, and a linear drive or guide 15. The movement drive unit 14 operates to cause the drive unit 13 to move along the linear drive or guide 15.

[0147] The input unit 30 has an input element 32 and is configured to allow a user to positionally adjust the flexible portions 12 of the continuum robot 11. The input unit 30 may be configured as a mouse, a keyboard, joystick, lever, or another shape to facilitate user interaction. The user may provide an operation input through the input element 32, and the continuum robot apparatus 10 may receive information of the input element 32 and one or more input / output devices, which may include, but are not limited to, a receiver, a transmitter,a speaker, a display, an imaging sensor, a user input device, which may include a keyboard, a keypad, a mouse, a position tracked stylus, a position tracked probe, a foot switch, a microphone, etc. The guide unit 40 is a device that includes one or more buttons, knobs, switches, etc. 42, 44, that a user may use to adjust various parameters of the continuum robot 10, such as the speed (e.g., rotational speed, translational speed, etc.), angle or plane, or other parameters.

[0148] FIG. 13 illustrates at least one embodiment of a controller 50 according to one or more features of the present disclosure. The controller 50 may be configured to control the elements of the continuum robot apparatus 10 and has one or more of a CPU 51, a memory 52, a storage 53, an input and output (I / O) interface 54, and communication interface 55. The continuum robot apparatus 10 may be interconnected with medical instruments or a variety of other devices, and may be controlled independently, externally, or remotely by the controller 50. In one or more embodiments of the present disclosure, one or more features of the continuum robot apparatus 10 and one or more features of the continuum robot or catheter or probe system 1000 maybe used in combination or alternatively to each other.

[0149] The memory 52 may be used as a work memory or may include any memory discussed in the present disclosure. The storage 53 stores software or computer instructions, and may be any type of storage, data storage 150, or other memory or storage discussed in the present disclosure. The CPU 51, which may include one or more processors, circuitry, or a combination thereof, executes the software developed in the memory 52 (or any other memory discussed herein). The I / O interface 54 operates to input information from the continuum robot apparatus 10 to the controller 50 and to output information for displaying to the display 60 (or any other display discussed herein, such as, but not limited to, display 1209 discussed below).

[0150] The communication interface 55 may be configured as a circuit or other device for communicating with components included in the apparatus 10, and with various external apparatuses connected to the apparatus via a network. For example, the communication interface 55 may store information to be output in a transfer packet and may output the transfer packet to an external apparatus via the network by communication technology such as Transmission Control Protocol / Internet Protocol (TCP / IP). The apparatus may include a plurality of communication circuits according to a desired communication form.

[0151] The controller 50 may be communicatively interconnected or interfaced with one or more external devices including, for example, one or more data storages (e.g., the data storage 150, the SSD or storage drive 1207 discussed below, or any other storage discussed herein), one or more external user input / output devices, or the like. The controller 50 may interface with other elements including, for example, one or more of an external storage, a display, a keyboard, a mouse, a sensor, a microphone, a speaker, a projector, a scanner, a display, an illumination device, etc.

[0152] The display 60 may be a display device configured, for example, as a monitor, an LCD (liquid panel display), an LED display, an OLED (organic LED) display, a plasma display, an organic electro luminescence panel, or any other display discussed herein. Based on the control of the apparatus, a screen may be displayed on the display 60 showing one or more images, such as, but not limited to, one or more images being captured, captured images, captured moving images recorded on the storage unit, etc.

[0153] The components may be connected together by a bus 56 so that the components may communicate with each other. The bus 56 transmits and receives data between these pieces of hardware connected together, or the bus 56 transmits a command from the CPU 51 to the other pieces of hardware. The components may be implemented by one or more physical devices that may be coupled to the CPU 51 through a communication channel. Forexample, the controller 50 may be implemented using circuitry in the form of ASIC (application specific integrated circuits) or other similar circuits as discussed herein. Alternatively, the controller 50 may be implemented as a combination of hardware and software, where the software is loaded into a processor from a memory or over a network connection. Functionality of the controller 50 maybe stored on a storage medium, which may include, but is not limited to, RAM (random-access memory), magnetic or optical drive, diskette, cloud storage, etc.

[0154] The units described throughout the present disclosure are exemplary and / or preferable modules for implementing processes described in the present disclosure. However, one or more embodiments of the present disclosure are not limited thereto. The term “unit”, as used herein, may generally refer to firmware, software, hardware, or other component, such as circuitry or the like, or any combination thereof, that is used to effectuate a purpose. The modules maybe hardware units (such as circuitry, firmware, a field programmable gate array, a digital signal processor, an application specific integrated circuit or the like) and / or software modules (such as a computer readable program, instructions stored in a memory or storage medium, etc.). The modules for implementing the various steps are not described exhaustively above. However, where there is a step of performing a certain process, there may be a corresponding functional module or unit (implemented by hardware and / or software) for implementing the same process. Technical solutions by all combinations of steps described and units corresponding to these steps are included in the present disclosure.

[0155] One or more control, adjustment, correction, and / or smoothing features of the present disclosure may be used with one or more image correction or adjustment features in one or more embodiments. One or more adjustments, corrections, or smoothing functions for a catheter or probe device and / or a continuum robot may adjust a path of one or more sections or portions of the catheter or probe device and / or the continuum robot (e.g., the continuum robot 104, the continuum robot device 10, etc.), and one or more embodiments may make a8ocorresponding adjustment or correction to an image view. For example, in one or more embodiments the medical tool may be a bronchoscope.

[0156] While one or more features of the present disclosure have been described with reference to exemplary embodiments, it is to be understood that the present disclosure is not limited to the disclosed exemplary embodiments. The scope of the claims is to be accorded the broadest interpretation so as to encompass all modifications, equivalent structures, and functions.

[0157] A computer, such as the console or computer 1200, 1200’, may perform any of the steps, processes, and / or techniques discussed herein for any apparatus and / or system being manufactured or used, any of the embodiments shown in FIGS. 1-15, any other apparatus or system discussed herein, etc.

[0158] There are many ways to control a continuum robot, correct or adjust an image or a path (or one or more sections or portions of) a continuum robot (or other probe or catheter device or system), or perform any other measurement or process discussed herein, to perform continuum robot method(s) or algorithm(s), and / or to control at least one continuum robot device / apparatus, system and / or storage medium, digital as well as analog. In at least one embodiment, a computer, such as the console or computer 1200, 1200’, may be dedicated to control and / or use continuum robot devices, systems, methods, and / or storage mediums for use therewith described herein.

[0159] The one or more detectors, sensors, cameras, or other components of the apparatus or system embodiments (e.g. of the system 1000 of FIG. 1 or any other system discussed herein) may transmit the digital or analog signals to a processor or a computer such as, but not limited to, an image processor or display controller too, a controller 102, a CPU 120, a controller 50, a CPU 51, a processor or computer 1200, 1200’ (see e.g., at least FIGS. 1-5, 11-13, and 14-15), a combination thereof, etc. The image processor may be a dedicated image processor or a general purpose processor that is configured to process images. In at least one embodiment, the computer 1200, 1200’ may be used in place of, or in addition to, the image processor or display controller 100 and / or the controller 102 (or any other processor or controller discussed herein, such as, but not limited to, the controller 50, the CPU 51, etc.). In an alternative embodiment, the image processor may include an ADC and receive analog signals from the one or more detectors or sensors of the system 1000 (or any other system discussed herein). The image processor may include one or more of a CPU, DSP, FPGA, ASIC, or some other processing circuitry. The image processor may include memory for storing image, data, and instructions. The image processor may generate one or more images based on the information provided by the one or more detectors, sensors, or cameras. A computer or processor discussed herein, such as, but not limited to, a processor of the devices, apparatuses or systems of FIGS. 1-5 and 11-13, the computer 1200, the computer 1200’, the image processor, etc. may also include one or more components further discussed herein below (see e.g., FIGS. 14-15).

[0160] Electrical analog signals obtained from the output of the system 1000 or the components thereof, and / or from the devices, apparatuses, or systems of FIGS. 1-5 and 11-13, maybe converted to digital signals to be analyzed with a computer, such as, but not limited to, the computers or controllers 100, 102 of FIG. 1, the computer 1200, 1200’, etc.

[0161] As aforementioned, there are many ways to control a continuum robot, correct or adjust an image, correct, adjust, or smooth a path (or section or portion) of a continuum robot, or perform any other measurement or process discussed herein, to perform continuum robot method(s) or algorithm(s), and / or to control at least one continuum robot device / apparatus, system and / or storage medium, digital as well as analog. By way of a further example, in at least one embodiment, a computer, such as the computer or controllers 100, 102 of FIG. 1, theconsole or computer 1200, 1200’, etc., may be dedicated to the control and the monitoring of the continuum robot devices, systems, methods and / or storage mediums described herein.

[0162] The electric signals used for imaging may be sent to one or more processors, such as, but not limited to, the processors or controllers too, 102 of FIGS. 1-5, a computer 1200 (see e.g., FIG. 14), a computer 1200’ (see e.g., FIG. 15), etc. as discussed further below, via cable(s) or wire(s), such as, but not limited to, the cable(s) or wire(s) 113 (see FIG. 14). Additionally or alternatively, the computers or processors discussed herein are interchangeable, and may operate to perform any of the feature(s) and method(s) discussed herein.

[0163] Various components of a computer system 1200 (see e.g., the console or computer 1200 as may be used as one embodiment example of the computer, processor, or controllers too, 102 shown in FIG. 1) are provided in FIG. 14. A computer system 1200 may include a central processing unit (“CPU”) 1201, a ROM 1202, a RAM 1203, a communication interface 1205, a hard disk (and / or other storage device) 1204, a screen (or monitor interface) 1209, a keyboard (or input interface; may also include a mouse or other input device in addition to the keyboard) 1210 and a BUS (or “Bus”) or other connection lines (e.g., connection line 1213) between one or more of the aforementioned components e.g., as shown in FIG. 14). In addition, the computer system 1200 may comprise one or more of the aforementioned components. For example, a computer system 1200 may include a CPU 1201, a RAM 1203, an input / output (I / O) interface (such as the communication interface 1205) and a bus (which may include one or more lines 1213 as a communication system between components of the computer system 1200; in one or more embodiments, the computer system 1200 and at least the CPU 1201 thereof may communicate with the one or more aforementioned components of a continuum robot device or system using same, such as, but not limited to, the system 1000, the devices / systems of FIGS. 1-5, and / or the systems / apparatuses of FIGS. 11-13, discussed herein above, via one or more lines 1213), and one or more other computer systems 1200 may include one or more combinations of the other aforementioned components (e.g., the one ormore lines 1213 of the computer 1200 may connect to other components via line 113). The CPU 1201 is configured to read and perform computer-executable instructions stored in a storage medium. The computer-executable instructions may include those for the performance of the methods and / or calculations described herein. The computer system 1200 may include one or more additional processors in addition to CPU 1201, and such processors, including the CPU 1201, may be used for controlling and / or manufacturing a device, system, or storage medium for use with same or for use with any continuum robot technique(s), and / or use with image correction or adjustment technique(s) discussed herein. The system 1200 may further include one or more processors connected via a network connection (e.g., via network 1206). The CPU 1201 and any additional processor being used by the system 1200 may be located in the same telecom network or in different telecom networks (e.g., performing, manufacturing, controlling, calculation, and / or using technique(s) may be controlled remotely).

[0164] The I / O or communication interface 1205 provides communication interfaces to input and output devices, which may include the one or more of the aforementioned components of any of the systems discussed herein (e.g., the controller too, the controller 102, the displays 101-1, 101-2, the actuator 103, the continuum device 104, the operating portion or controller 105, the tracking sensor 106, the position detector 107, the rail 108, etc.), a microphone, a communication cable and a network (either wired or wireless), a keyboard 1210, a mouse (see e.g., the mouse 1211 as shown in FIG. 15), a touch screen or screen 1209, a light pen and so on. The communication interface of the computer 1200 may connect to other components discussed herein via line 113 (as diagrammatically shown in FIG. 14). The Monitor interface or screen 1209 provides communication interfaces thereto.

[0165] Any methods and / or data of the present disclosure, such as, but not limited to, the methods for using and / or controlling a continuum robot or catheter device, system, or storage medium for use with same and / or method(s) for imaging, performing tissue or samplecharacterization or analysis, performing diagnosis, planning and / or examination, for performing control, adjustment, correction, or smoothing technique(s) (e.g., to a path of, to a pose or position of, or to one or more sections or portions of, a continuum robot, a catheter or a probe), and / or for performing image correction or adjustment technique(s), as discussed herein, maybe stored on a computer-readable storage medium. A computer-readable and / or writable storage medium used commonly, such as, but not limited to, one or more of a hard disk (e.g., the hard disk 1204, a magnetic disk, etc.), a flash memory, a CD, an optical disc (e.g., a compact disc (“CD”) a digital versatile disc (“DVD”), a Blu-ray™ disc, etc.), a magneto-optical disk, a random-access memory (“RAM”) (such as the RAM 1203), a DRAM, a read only memory (“ROM”), a storage of distributed computing systems, a memory card, or the like (e.g., other semiconductor memory, such as, but not limited to, a non-volatile memory card, a solid state drive (SSD) (see SSD 1207 in FIG. 15), SRAM, etc.), an optional combination thereof, a server / database, etc. may be used to cause a processor, such as, the processor or CPU 1201 of the aforementioned computer system 1200 to perform the steps of the methods disclosed herein. The computer-readable storage medium may be a non-transitory computer-readable medium, and / or the computer- readable medium may comprise all computer-readable media, with the sole exception being a transitory, propagating signal in one or more embodiments. The computer-readable storage medium may include media that store information for predetermined, limited, or short period(s) of time and / or only in the presence of power, such as, but not limited to Random Access Memory (RAM), register memory, processor cache(s), etc. Embodiment(s) of the present disclosure may also be realized by a computer of a system or apparatus that reads out and executes computer executable instructions (e.g., one or more programs) recorded on a storage medium (which may also be referred to more fully as a “non- transitory computer-readable storage medium”) to perform the functions of one or more of the above-described embodiment(s) and / or that includes one or more circuits (e.g., application specific integrated circuit (ASIC)) for performing the functions of one or more of the above-described embodiment(s), and by a method performed by the computer of the system or apparatus by, for example, reading out and executing the computer executableinstructions from the storage medium to perform the functions of one or more of the above - described embodiment(s) and / or controlling the one or more circuits to perform the functions of one or more of the above-described embodiment(s).

[0166] In accordance with at least one aspect of the present disclosure, the methods, devices, systems, and computer-readable storage mediums related to the processors, such as, but not limited to, the processor of the aforementioned computer 1200, the processor of computer 1200’, the controller too, the controller 102, etc., as described above may be achieved utilizing suitable hardware, such as that illustrated in the figures. Functionality of one or more aspects of the present disclosure may be achieved utilizing suitable hardware, such as that illustrated in FIG. 14. Such hardware may be implemented utilizing any of the known technologies, such as standard digital circuitry, any of the known processors that are operable to execute software and / or firmware programs, one or more programmable digital devices or systems, such as programmable read only memories (PROMs), programmable array logic devices (PALs), etc. The CPU 1201 (as shown in FIG. 14 or FIG. 15, and / or which may be included in the computer, processor, controller and / or CPU 120 of FIGS. 1-5), CPU 51, and / or the CPU 120 may also include and / or be made of one or more microprocessors, nanoprocessors, one or more graphics processing units (“GPUs”; also called a visual processing unit (“VPU”)), one or more Field Programmable Gate Arrays (“FPGAs”), or other types of processing components (e.g., application specific integrated circuit(s) (ASIC)). Still further, the various aspects of the present disclosure may be implemented by way of software and / or firmware program(s) that maybe stored on suitable storage medium (e.g., computer- readable storage medium, hard drive, etc.) or media (such as floppy disk(s), memory chip(s), etc.) for transportability and / or distribution. The computer may include a network of separate computers or separate processors to read out and execute the computer executable instructions. The computer executable instructions may be provided to the computer, for example, from a network or the storage medium. The computers or processors (e.g., too, 102,120, 50, 51? 1200, 1200’, etc.) may include the aforementioned CPU structure, or may be connected to such CPU structure for communication therewith.

[0167] As aforementioned, hardware structure of an alternative embodiment of a computer or console 1200’ is shown in FIG. 15. The computer 1200’ includes a central processing unit (CPU) 1201, a graphical processing unit (GPU) 1215, a random access memory (RAM) 1203, a network interface device 1212, an operation interface 1214 such as a universal serial bus (USB) and a memory such as a hard disk drive or a solid-state drive (SSD) 1207. Preferably, the computer or console 1200’ includes a display 1209 (and / or the displays 101-1, 101-2). The computer 1200’ may connect with one or more components of a system (e.g., the systems / apparatuses of FIGS. 1-5, 11-13, etc.) via the operation interface 1214 or the network interface 1212. The operation interface 1214 is connected with an operation unit such as a mouse device 1211, a keyboard 1210 or a touch panel device. The computer 1200’ may include two or more of each component. Alternatively, the CPU 1201 or the GPU 1215 maybe replaced by the field-programmable gate array (FPGA), the application-specific integrated circuit (ASIC) or other processing unit depending on the design of a computer, such as the computer 1200, the computer 1200’, etc.

[0168] At least one computer program is stored in the SSD 1207 (or any other storage device or drive discussed herein), and the CPU 1201 loads the at least one program onto the RAM 1203, and executes the instructions in the at least one program to perform one or more processes described herein, as well as the basic input, output, calculation, memory writing, and memoiy reading processes.

[0169] The computer, such as the computer 1200, 1200’, the computer, processors, and / or controllers of FIGS. 1-5 and / or FIGS. 11-13, etc., communicates with the one or more components of the apparatuses / systems of FIGS. 1-5, of FIGS. 11-13, and / or of any other system(s) discussed herein, to perform imaging, and reconstructs an image from the acquiredintensity data. The monitor or display 1209 displays the reconstructed image, and the monitor or display 1209 may display other information about the imaging condition or about an object to be imaged. The monitor 1209 also provides a graphical user interface for a user to operate a system, for example when performing CT, MRI, or other imaging modalities or other imaging technique(s), including, but not limited to, controlling continuum robot devices / systems, and / or performing correction, adjustment, and / or smoothing technique(s). An operation signal is input from the operation unit (e.g., such as, but not limited to, a mouse device 1211, a keyboard 1210, a touch panel device, etc.) into the operation interface 1214 in the computer 1200’, and corresponding to the operation signal the computer 1200’ instructs the system (e.g., the system 1000, the systems / apparatuses of FIGS. 1-5, the systems / apparatuses of FIGS. 11- 13, any other system / apparatus discussed herein,, etc.) to start or end the imaging, and / or to start or end continuum robot control(s) and / or performance of correction, adjustment, and / or smoothing technique(s). The camera or imaging device as aforementioned may have interfaces to communicate with the computers 1200, 1200’ to send and receive the status information and the control signals.

[0170] The present disclosure and / or one or more components of devices, systems, and storage mediums, and / or methods, thereof also may be used in conjunction with continuum robot devices, systems, methods, and / or storage mediums and / or with endoscope devices, systems, methods, and / or storage mediums. Such continuum robot devices, systems, methods, and / or storage mediums are disclosed in at least: U.S. Provisional Pat. App. No. 63 / 150,859, filed on February 18, 2021, the disclosure of which is incorporated by reference herein in its entirety, as discussed in International Pat. App. No. PCT / US2022 / 016660, filed February 16, 2022, the disclosure of which is incorporated by reference herein in its entirety, and as disclosed in U.S. Pat. App. No. 17 / 673,606, filed on February 16, 2022, the disclosure of which is incorporated by reference herein in its entirety. Such endoscope devices, systems, methods, and / or storage mediums are disclosed in at least: U.S. Pat. App. No. 17 / 565,319, filed on December 29, 2021, the disclosure of which is incorporated by reference herein in itsentirety; U.S. Pat. App. No. 63 / 132,320, filed on December 30, 2020, the disclosure of which is incorporated by reference herein in its entirety; U.S. Pat. App. No. 17 / 564,534, filed on December 29, 2021, the disclosure of which is incorporated by reference herein in its entirety; and U.S. Pat. App. No. 63 / 131,485, filed December 29, 2020, the disclosure of which is incorporated by reference herein in its entirety. Any of the features of the present disclosure may be used in combination with any of the features as discussed in U.S. Prov. Pat. App. No. 63 / 378,017, filed September 30, 2022, the disclosure of which is incorporated by reference herein in its entirety, any of the features as discussed in U.S. Prov. Pat. App. No. 63 / 383,210, filed November 10, 2022, the disclosure of which is incorporated by reference herein in its entirety, any of the features as discussed in U.S. Prov. Pat. App. No. 63 / 377,983, filed September 30, 2022, the disclosure of which is incorporated by reference herein in its entirety, and / or any of the features as discussed in U.S. Pat. App. No. 18 / 477,081, filed on September 28, 2023, the disclosure of which is incorporated by reference herein in its entirety. Any of the features of the present disclosure may be used in combination with any of the features as discussed in U.S. Pat. Pub. No. 2023 / 0131269, published on April 26, 2023, the disclosure of which is incorporated by reference herein in its entirety.

[0171] Although the disclosure herein has been described with reference to particular embodiments, it is to be understood that these embodiments are merely illustrative of the principles and applications of the present disclosure (and are not limited thereto), and the invention is not limited to the disclosed embodiments. It is therefore to be understood that numerous modifications may be made to the illustrative embodiments and that other arrangements may be devised without departing from the spirit and scope of the present disclosure. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications, equivalent structures, and functions.

Claims

CLAIMS1. A continuum robot for performing robotic control, the continuum robot comprising: one or more processors that operate to: instruct or command a first bending section or portion of a catheter or a probe of the continuum robot such that the first bending section or portion achieves, or is disposed at, a pose, position, or state at a position along a path, the catheter or probe of the continuum robot having a plurality of bending sections or portions and a base; instruct or command each of the other bending sections or portions of the plurality of bending sections or portions of the catheter or probe to match, substantially match, or approximately match the pose, position, or state of the first bending section or portion at the position along the path in a case where each section or portion reaches or approaches a same, similar, or approximately similar state or states at the position along the path; and instruct or command the plurality of bending sections or portions such that the first bending section or portion or a Tip or distal bending section or portion is located in a predetermined pose, position, or state at or near a distal end of the path.

2. The continuum robot of claim 1, wherein:(i) the first bending section or portion or the Tip or distal bending section or portion includes a camera, an endoscopic camera, a sensor, or other imaging device or system to obtain one or more images of or in a target, sample, or object; and(ii) the one or more processors further operate to command the camera, sensor, or other imaging device or system to obtain the one or more images of or in the target, sample, or object at the predetermined pose, position, or state, and the one or more processors operate to receive the one or more images and / or display the one or more images on a display.

3. The continuum robot of claim 2, wherein the one or more processors further operate to:(i) track a respective path of one or more of sections or paths of the catheter or probe, or track a respective path of each section or portion of the catheter or probe;(ii) instruct or command each section or portion to adjust the respective path of each section or path to match, substantially match, or approximately match the path of the first section or portion of the catheter or probe and / or to address any discontinuous path due a change in a state or states and / or due to a change or a transition of a state or movement of the catheter or probe and / or the continuum robot or of the target, sample, or object; and(iii) utilize additional data to control or command the plurality of sections or portions by including the additional data in the robotic control and / or by updating historical information or the pose, position, or state information for the sections or portions of the catheter or probe with the additional data and / or with differences between a previous or expected robotic control state or states and a new control state or states in future calculation(s) of robotic control state(s) such that an efficiency of the navigation or movement of, or of the robotic control of, the sections or portions of the continuum robot is increased or improved.

4. The continuum robot of claim 3, wherein:(i) the additional data includes one or more of the following: target pose, position, or state information; final pose, position, or state information; interpolated pose, position, or state information; measured pose, position, or state information; information for converting pose, position, or state information between different states; target pose, position, or state information; measured pose, position, or state information; force sensor(s) information; draw or current draw information of one or more actuator motors of the catheter or probe; section or portion size and / or shape information; section or portion length information; kinematic information of the catheter or probe and / or of the continuum robot; information for internal and / or external sensor(s) of the catheter or probe and / or the continuum robot; electromagnetic sensor information; shape sensor information; sensor information; target, sample, or object divergence or movement information; information for environmental change(s) for the catheter or probe; relaxation state change(s); and / or changes due to one or more user controls of or for the continuum robot; and(ii) the different states include one or more of the following: drive wire position(s) or state(s) for one or more drive wires of the catheter or probe; coordinates for the sections or portions; three-dimensional (3D) position(s), orientation(s), and / or state(s) for the sections or portions; plane and / or angle information for pose(s) or state(s) of the sections or portions; and / or position(s) or state(s) of the base or a translational stage or motor(s) of the continuum robot.

5. The continuum robot of claim 4, wherein: the additional data and / or the pose, position, or state information is stored in a lookup table or tables or is used to update or change a number of points or values for the sections or portions of the catheter or probe; andin a case where the additional data and / or the pose, position, or state information changes or updates the number of points, the additional data and / or the pose, position, or state information operates to one or more of the following: affect or update one or more points or values in the lookup table or tables, affect or update all points or values forward or beyond an initially changed point or value; affect or update one or more points or values different than an initially changed point or value; affect or update one or more points or values for a predetermined or set range or distance after or from an initially changed point or value; and / or affect or update one or more points or values different than an initially changed point or value by using the initially changed point or value as a pivot point or value.

6. The continuum robot of claim 4, wherein the one or more processors further operate to utilize the additional data to control or command the plurality of sections or portions of the catheter or probe by including the additional data and / or by updating the historical information or the pose, position, or state information by using one or more of the following modes: a Hold the Line mode, a Close the Gap mode, and / or a Stay the Course mode.

7. The continuum robot of claim 6, wherein, in a case where the one or more processors are using the Hold the Line mode, the one or more processors further operate to:(i) instruct or command the catheter or probe to move forward or to move along the path;(ii) calculate coordinates for a Tip or distal end effector of the Tip or distal bending section or portion or for an end effector of the first section or portion;(iii) add the calculated coordinate information to the path, or to a three-dimensional (3D) path, for the Tip or distal end, section, or portion and / or the catheter or probe;(iv) calculate coordinates for a Middle / proximal end effector of a Middle / proximal, or other section or portion subsequent to or following the Tip or distal or first bending section or portion, of the catheter or probe;(v) identify or calculate a distance from a closest point along the path or the 3D path for the Middle / proximal end effector and / or the Tip or distal or first portion end effector;(vi) convert the identified or calculated distance to a change in a pose, position, or state of the Tip or distal end effector, of the Tip or distal bending section or portion, of the Middle / proximal end effector, of the Middle / proximal section or portion, and / or of the catheter or probe; and(vii) update the pose, position, or state of the Middle / proximal section or portion of the catheter or probe to match the pose, position, or state of the Tip or distal bending section or portion, or of the first section or portion, of the catheter or probe at that point along the path or the 3D path.

8. The continuum robot of claim 7, wherein one or more of the following: the one or more processors further operate to evaluate or determine whether the identified or calculated distance is greater than zero and / or is within a threshold, and / or whether a change in the pose, position, or state occurred for the Tip or distal bending section or portion or for the first section or portion; in a case where the catheter or probe is instructed or commanded to move forward, the catheter or probe moves forward during one or more of the following: while a stage or linear stage of the catheter or robot moves forward; while the navigation is mapped to Z stage position; while converting a position, pose, or state of the Tip or distal bending section or portion or the first section or portion of the catheter or probe to a coordinate; and / or during navigation; the coordinate(s) use an X, Y, Z coordinate system; in a case where a pose, position, or state is being determined for other section(s) or portion(s) of the catheter or probe, the one or more processors further operate to use a pose, position, or state which operates to keep an end effector of the section(s) or portion(s) as close as possible to the Tip or distal pose, position, or state coordinate at that point along the path or the 3D path; the pose, position, or state coordinates of preceding section(s) or portion(s) of the catheter or probe are known or determined; the one or more processors further operate to apply divergence or difference of the target, object, or sample with a coordinate shift; and / or the one or more processors further operate to consider a length of each section(s) or portion(s) of the catheter or probe to update or change a respective pose, position, or state for the section(s) or portion(s) so that each of the section(s) or portion(s) are located in a predetermined or target location or are located in the same or similar location as the Tip or distal bending section or portion or the first bending section or portion at an evaluated point along the path or the 3D path.

9. The continuum robot of claim 6, wherein, in a case where the one or more processors are using the Close the Gap mode, the one or more processors further operate to:(i) identify, determine, calculate, or obtain a pose, position, or state of a Middle / proximal section or portion of the catheter or probe;(ii) identify, determine, calculate, or obtain a pose, position, or state of a Tip or distal bending section or portion or of the first bending section or portion of the catheter or probe;(iii) determine, identify, calculate or obtain a difference between the poses, positions, or states of the Tip or distal bending section or portion and of the Middle / proximal, or other subsequent or following, bending section or portion;(iv) interpolate, over a set or predetermined length of the path or a three-dimensional (3D) path, the pose, position, or state difference between the Tip or distal bending section or portion or the first section or portion and the Middle / proximal, or other subsequent or following, bending section or portion; and(v) update the pose, position, or state of the Middle / proximal, or other subsequent or following, bending section or portion of the catheter or probe using a corresponding interpolated pose, position, or state difference.

10. The continuum robot of claim 9, wherein one or more of the following: the one or more processors further operate to consider a motion of the catheter or probe when updating the pose, position, or state of the Middle / proximal, or other subsequent or following, bending section or portion of the catheter or probe; information for navigation of the catheter or probe is obtained and used to update the pose, position, or state of the Middle / proximal, or other subsequent or following, bending section or portion of the catheter or probe; the one or more processors further operate to evaluate or determine whether the determined, identified, calculated, or obtained difference is greater than zero and / or is within a threshold, and / or whether a change in pose, position, or state occurred for the Tip or distal bending section or portion, the first bending section or portion, and / or the Middle / proximal, or other subsequent or following, bending section or portion of the catheter or probe; the one or more processors further operate to close one or more gaps between different sections or portions by adjusting a pose, position, or state of the section(s) or portion(s) of the catheter or probe to reduce, minimize, or eliminate a relative difference between the pose, position, or state of the Tip or distal, or the first, section or portion as compared with the pose,position, or state of the Middle / proximal bending section or portion or any other subsequent or following section(s) or portion(s) of the catheter or probe; a respective trajectory of each of the plurality of sections or portions are known or determined; and / or the one or more processors instruct or control movement or navigation of each of the plurality of sections or portions such that any counter-active motion or motions are avoided or eliminated and such that any gap between respective trajectories of each of the plurality of sections or portions is eliminated or avoided.

11. The continuum robot of claim 6, wherein, in a case where the one or more processors are using the Stay the Course mode, the one or more processors further operate to:(i) instruct or command the catheter or probe to move forward or to move along the path;(ii) calculate a vector, a normal path, or a trajectory for a Tip or distal end effector of the Tip or distal bending section or portion or for an end effector of the first section or portion;(iii) calculate a deviation of the Tip or distal end effector of the Tip or distal bending section or portion or for the end effector of the first section or portion from the vector, the normal path, or the trajectory due to a shape and / or motion of the catheter or probe;(iv) calculate a change to the pose, position, or state of the Middle / proximal section or portion, or other section or portion subsequent to or following the Tip or distal bend section or portion or the first bending section or portion, of the catheter or probe to counteract or remove the calculated deviation; and(v) update the pose, position, or state of the Middle / proximal, or other subsequent or following, section or portion of the catheter or probe to match, substantially match, or approximately match the pose, position, or state of the Tip or distal bending section or portion, or of the first bending section or portion, of the catheter or probe at that point along the path, to eliminate or remove the calculated deviation.

12. The continuum robot of claim 11, wherein one or more of the following: the motion of the catheter or the probe is one or more of the following: a motion from movement of a stage or translational stage of the continuum robot; a motion from one or more changes due to an environment or due to the target, object, or sample in which the catheter orprobe is located; a body divergence of a patient, target, object, or sample divergence; and / or a motion from another source external to the continuum robot; the one or more processors further operate to evaluate or determine whether a path, vector, or trajectory deviation exists due to a shape and / or motion of the catheter or probe; a motion of the catheter or probe includes one or more of the following: a motion of the base or a stage; a motion due to translational movement of the catheter or probe and / or the continuum robot; a motion or movement of the target, object, or sample; and / or a motion from a force or influence originating outside of the continuum robot; the one or more processors further operate to use a normal path, vector, or trajectory deviation as a trigger for, and in, the calculation of the calculated deviation of the Tip or distal end effector of the Tip or distal bending section or portion or for the end effector of the first section or portion from the vector, the normal path, or the trajectory due to a shape and / or motion of the catheter or probe; the one or more processors operate to control the catheter or probe to stay or maintain a desired course along the path; the one or more processors operate to control and adjust a pose, position, or state of a section(s) or portion(s) of the catheter or probe to minimize any deviation of a pose, position, or state of one or more Middle / proximal, subsequent, or following section(s) or portion(s) based on a predetermined, targeted trajectory and a maximum motion along the predetermined, targeted trajectory; coordinates for, and the vector, normal path, or trajectory of subsequent, following, or future section(s) or portion(s) are known, set, or determined; information for one or more prior section(s) or portion(s) of the catheter or probe are known, set, or determined; and / or the one or more processors instruct or control movement or navigation of each of the plurality of sections or portions such that any counter-active motion or motions are avoided or eliminated and such that a predetermined or targeted course is maintained between respective trajectories of each of the plurality of sections or portions.

13. The continuum robot of claim 3, wherein, in a case where a difference exists, or the differences exist, between a previous or expected pose, position, or state and a new pose, position, or state for any of the sections or portions of the catheter or probe or in a case where a difference exists between real-time information for the target, sample, or object and previous information for the target, sample, or object, the difference or differences are detected usingmachine vision of the one or more images and / or using machine vision of the catheter or probe image(s) or the difference or differences are detected using one or more sensors for detecting the difference or divergence.14- A method for performing robotic control for a continuum robot, the method comprising: instructing or commanding a first bending section or portion of a catheter or a probe of the continuum robot such that the first bending section or portion achieves, or is disposed at, a pose, position, or state at a position along a path, the catheter or probe of the continuum robot having a plurality of bending sections or portions and a base; instructing or commanding each of the other bending sections or portions of the plurality of bending sections or portions of the catheter or probe to match, substantially match, or approximately match the pose, position, or state of the first bending section or portion at the position along the path in a case where each section or portion reaches or approaches a same, similar, or approximately similar state or states at the position along the path; and instructing or commanding the plurality of bending sections or portions such that the first bending section or portion or a Tip or distal bending section or portion is located in a predetermined pose, position, or state at or near a distal end of the path.

15. The method of claim 14, further comprising utilizing additional data to control or command the plurality of sections or portions of the catheter or probe by including the additional data and / or by updating historical information or pose, position, or state information by using one or more of the following modes: a Hold the Line mode, a Close the Gap mode, and / or a Stay the Course mode.

16. The method of claim 15, further comprising using the Hold the Line mode and performing the following:(i) instructing or commanding the catheter or probe to move forward or to move along the path;(ii) calculating coordinates for a Tip or distal end effector of the Tip or distal bending section or portion or for an end effector of the first section or portion;(iii) adding the calculated coordinate information to the path, or to a three- dimensional (3D) path, for the Tip or distal end, section, or portion and / or the catheter or probe;(iv) calculating coordinates for a Middle / proximal end effector of a Middle / proximal, or other section or portion subsequent to or following the Tip or distal or first bending section or portion, of the catheter or probe;(v) identifying or calculating a distance from a closest point along the path or the 3D path for the Middle / proximal end effector and / or the Tip or distal or first portion end effector;(vi) converting the identified or calculated distance to a change in a pose, position, or state of the Tip or distal end effector, of the Tip or distal bending section or portion, of the Middle / proximal end effector, of the Middle / proximal section or portion, and / or of the catheter or probe; and(vii) updating the pose, position, or state of the Middle / proximal section or portion of the catheter or probe to match the pose, position, or state of the Tip or distal bending section or portion, or of the first section or portion, of the catheter or probe at that point along the path or the 3D path.

17. The method of claim 16, wherein one or more of the following: the method further includes evaluating or determining whether the identified or calculated distance is greater than zero and / or is within a threshold, and / or whether a change in the pose, position, or state occurred for the Tip or distal bending section or portion or for the first section or portion; in a case where the catheter or probe is instructed or commanded to move forward, the catheter or probe moves forward during one or more of the following: while a stage or linear stage of the catheter or robot moves forward; while the navigation is mapped to Z stage position; while converting a position, pose, or state of the Tip or distal bending section or portion or the first section or portion of the catheter or probe to a coordinate; and / or during navigation; the coordinate(s) use an X, Y, Z coordinate system; in a case where a pose, position, or state is being determined for other section(s) or portion(s) of the catheter or probe, the method further includes using a pose, position, or state which operates to keep an end effector of the section(s) or portion(s) as close as possible to the Tip or distal pose, position, or state coordinate at that point along the path or the 3D path; the pose, position, or state coordinates of preceding section(s) or portion(s) of the catheter or probe are known or determined; the method further includes applying divergence or difference of the target, object, or sample with a coordinate shift; and / orthe method further includes considering a length of each section(s) or portion(s) of the catheter or probe to update or change a respective pose, position, or state for the section(s) or portion(s) so that each of the section(s) or portion(s) are located in a predetermined or target location or are located in the same or similar location as the Tip or distal bending section or portion or the first bending section or portion at an evaluated point along the path or the 3D path.

18. The method of claim 15, further comprising using the Close the Gap mode and performing the following:(i) identifying, determining, calculating, or obtaining a pose, position, or state of a Middle / proximal section or portion of the catheter or probe;(ii) identifying, determining, calculating, or obtaining a pose, position, or state of a Tip or distal bending section or portion or of the first bending section or portion of the catheter or probe;(iii) determining, identifying, calculating, or obtaining a difference between the poses, positions, or states of the Tip or distal bending section or portion and of the Middle / proximal, or other subsequent or following, bending section or portion;(iv) interpolating, over a set or predetermined length of the path or a three- dimensional (3D) path, the pose, position, or state difference between the Tip or distal bending section or portion or the first section or portion and the Middle / proximal, or other subsequent or following, bending section or portion; and(v) updating the pose, position, or state of the Middle / proximal, or other subsequent or following, bending section or portion of the catheter or probe using a corresponding interpolated pose, position, or state difference.

19. The method of claim 18, wherein one or more of the following: the method further includes considering a motion of the catheter or probe when updating the pose, position, or state of the Middle / proximal, or other subsequent or following, bending section or portion of the catheter or probe; information for navigation of the catheter or probe is obtained and used to update the pose, position, or state of the Middle / proximal, or other subsequent or following, bending section or portion of the catheter or probe;the method further includes evaluating or determining whether the determined, identified, calculated, or obtained difference is greater than zero and / or is within a threshold, and / or whether a change in pose, position, or state occurred for the Tip or distal bending section or portion, the first bending section or portion, and / or the Middle / proximal, or other subsequent or following, bending section or portion of the catheter or probe; the method further includes closing one or more gaps between different sections or portions by adjusting a pose, position, or state of the section(s) or portion(s) of the catheter or probe to reduce, minimize, or eliminate a relative difference between the pose, position, or state of the Tip or distal, or the first, section or portion as compared with the pose, position, or state of the Middle / proximal bending section or portion or any other subsequent or following section(s) or portion(s) of the catheter or probe; a respective trajectory of each of the plurality of sections or portions are known or determined; and / or the method further includes instructing or controlling movement or navigation of each of the plurality of sections or portions such that any counter-active motion or motions are avoided or eliminated and such that any gap between respective trajectories of each of the plurality of sections or portions is eliminated or avoided.

20. The method of claim 15, further comprising using the Stay the Course mode and performing the following:(i) instructing or commanding the catheter or probe to move forward or to move along the path;(ii) calculating a vector, a normal path, or a trajectory for a Tip or distal end effector of the Tip or distal bending section or portion or for an end effector of the first section or portion;(iii) calculating a deviation of the Tip or distal end effector of the Tip or distal bending section or portion or for the end effector of the first section or portion from the vector, the normal path, or the trajectory due to a shape and / or motion of the catheter or probe;(iv) calculating a change to the pose, position, or state of the Middle / proximal section or portion, or other section or portion subsequent to or following the Tip or distal bend section or portion or the first bending section or portion, of the catheter or probe to counteract or remove the calculated deviation; and(v) updating the pose, position, or state of the Middle / proximal, or other subsequent or following, section or portion of the catheter or probe to match, substantially match, or approximately match the pose, position, or state of the Tip or distal bending section or portion,or of the first bending section or portion, of the catheter or probe at that point along the path, to eliminate or remove the calculated deviation.

21. The method of claim 20, wherein one or more of the following: the motion of the catheter or the probe is one or more of the following: a motion from movement of a stage or translational stage of the continuum robot; a motion from one or more changes due to an environment or due to the target, object, or sample in which the catheter or probe is located; a body divergence of a patient, target, object, or sample divergence; and / or a motion from another source external to the continuum robot; the method further includes evaluating or determining whether a path, vector, or trajectory deviation exists due to a shape and / or motion of the catheter or probe; a motion of the catheter or probe includes one or more of the following: a motion of the base or a stage; a motion due to translational movement of the catheter or probe and / or the continuum robot; a motion or movement of the target, object, or sample; and / or a motion from a force or influence originating outside of the continuum robot; the method further includes using a normal path, vector, or trajectory deviation as a trigger for, and in, the calculation of the calculated deviation of the Tip or distal end effector of the Tip or distal bending section or portion or for the end effector of the first section or portion from the vector, the normal path, or the trajectory due to a shape and / or motion of the catheter or probe; the method further includes controlling the catheter or probe to stay or maintain a desired course along the path; the method further includes controlling and adjusting a pose, position, or state of a section(s) or portion(s) of the catheter or probe to minimize any deviation of a pose, position, or state of one or more Middle / proximal, subsequent, or following section(s) or portion(s) based on a predetermined, targeted trajectory and a maximum motion along the predetermined, targeted trajectory; coordinates for, and the vector, normal path, or trajectory of subsequent, following, or future section(s) or portion(s) are known, set, or determined; information for one or more prior section(s) or portion(s) of the catheter or probe are known, set, or determined; and / or the method further includes instructing or controlling movement or navigation of each of the plurality of sections or portions such that any counter-active motion or motions areavoided or eliminated and such that a predetermined or targeted course is maintained between respective trajectories of each of the plurality of sections or portions.

22. A non-transitory computer-readable storage medium storing at least one program for causing a computer to execute a method for performing correction, adjustment, and / or smoothing for a continuum robot, the method comprising: instructing or commanding a first bending section or portion of a catheter or a probe of the continuum robot such that the first bending section or portion achieves, or is disposed at, a pose, position, or state at a position along a path, the catheter or probe of the continuum robot having a plurality of bending sections or portions and a base; instructing or commanding each of the other bending sections or portions of the plurality of bending sections or portions of the catheter or probe to match, substantially match, or approximately match the pose, position, or state of the first bending section or portion at the position along the path in a case where each section or portion reaches or approaches a same, similar, or approximately similar state or states at the position along the path; and instructing or commanding the plurality of bending sections or portions such that the first bending section or portion or a Tip or distal bending section or portion is located in a predetermined pose, position, or state at or near a distal end of the path.

23. The storage medium of claim 22, wherein the method further comprises: utilizing additional data to control or command the plurality of sections or portions of the catheter or probe by including the additional data and / or by updating historical information or pose, position, or state information by using one or more of the following modes: a Hold the Line mode, a Close the Gap mode, and / or a Stay the Course mode.

24. The storage medium of claim 23, wherein the method further comprises: using the Hold the Line mode and performing the following:(i) instructing or commanding the catheter or probe to move forward or to move along the path;(ii) calculating coordinates for a Tip or distal end effector of the Tip or distal bending section or portion or for an end effector of the first section or portion;(iii) adding the calculated coordinate information to the path, or to a three- dimensional (3D) path, for the Tip or distal end, section, or portion and / or the catheter or probe;(iv) calculating coordinates for a Middle / proximal end effector of a Middle / proximal, or other section or portion subsequent to or following the Tip or distal or first bending section or portion, of the catheter or probe;(v) identifying or calculating a distance from a closest point along the path or the 3D path for the Middle / proximal end effector and / or the Tip or distal or first portion end effector;(vi) converting the identified or calculated distance to a change in a pose, position, or state of the Tip or distal end effector, of the Tip or distal bending section or portion, of the Middle / proximal end effector, of the Middle / proximal section or portion, and / or of the catheter or probe; and(vii) updating the pose, position, or state of the Middle / proximal section or portion of the catheter or probe to match the pose, position, or state of the Tip or distal bending section or portion, or of the first section or portion, of the catheter or probe at that point along the path or the 3D path.

25. The storage medium of claim 24, wherein one or more of the following: the method further includes evaluating or determining whether the identified or calculated distance is greater than zero and / or is within a threshold, and / or whether a change in the pose, position, or state occurred for the Tip or distal bending section or portion or for the first section or portion; in a case where the catheter or probe is instructed or commanded to move forward, the catheter or probe moves forward during one or more of the following: while a stage or linear stage of the catheter or robot moves forward; while the navigation is mapped to Z stage position; while converting a position, pose, or state of the Tip or distal bending section or portion or the first section or portion of the catheter or probe to a coordinate; and / or during navigation; the coordinate(s) use an X, Y, Z coordinate system; in a case where a pose, position, or state is being determined for other section(s) or portion(s) of the catheter or probe, the method further includes using a pose, position, or state which operates to keep an end effector of the section(s) or portion(s) as close as possible to the Tip or distal pose, position, or state coordinate at that point along the path or the 3D path;the pose, position, or state coordinates of preceding section(s) or portion(s) of the catheter or probe are known or determined; the method further includes applying divergence or difference of the target, object, or sample with a coordinate shift; and / or the method further includes considering a length of each section(s) or portion(s) of the catheter or probe to update or change a respective pose, position, or state for the section(s) or portion(s) so that each of the section(s) or portion(s) are located in a predetermined or target location or are located in the same or similar location as the Tip or distal bending section or portion or the first bending section or portion at an evaluated point along the path or the 3D path.

26. The storage medium of claim 23, wherein the method further comprises using the Close the Gap mode and performing the following:(i) identifying, determining, calculating, or obtaining a pose, position, or state of a Middle / proximal section or portion of the catheter or probe;(ii) identifying, determining, calculating, or obtaining a pose, position, or state of a Tip or distal bending section or portion or of the first bending section or portion of the catheter or probe;(iii) determining, identifying, calculating, or obtaining a difference between the poses, positions, or states of the Tip or distal bending section or portion and of the Middle / proximal, or other subsequent or following, bending section or portion;(iv) interpolating, over a set or predetermined length of the path or a three- dimensional (3D) path, the pose, position, or state difference between the Tip or distal bending section or portion or the first section or portion and the Middle / proximal, or other subsequent or following, bending section or portion; and(v) updating the pose, position, or state of the Middle / proximal, or other subsequent or following, bending section or portion of the catheter or probe using a corresponding interpolated pose, position, or state difference.

27. The storage medium of claim 26, wherein one or more of the following: the method further includes considering a motion of the catheter or probe when updating the pose, position, or state of the Middle / proximal, or other subsequent or following, bending section or portion of the catheter or probe;information for navigation of the catheter or probe is obtained and used to update the pose, position, or state of the Middle / proximal, or other subsequent or following, bending section or portion of the catheter or probe; the method further includes evaluating or determining whether the determined, identified, calculated, or obtained difference is greater than zero and / or is within a threshold, and / or whether a change in pose, position, or state occurred for the Tip or distal bending section or portion, the first bending section or portion, and / or the Middle / proximal, or other subsequent or following, bending section or portion of the catheter or probe; the method further includes closing one or more gaps between different sections or portions by adjusting a pose, position, or state of the section(s) or portion(s) of the catheter or probe to reduce, minimize, or eliminate a relative difference between the pose, position, or state of the Tip or distal, or the first, section or portion as compared with the pose, position, or state of the Middle / proximal bending section or portion or any other subsequent or following section(s) or portion(s) of the catheter or probe; a respective trajectory of each of the plurality of sections or portions are known or determined; and / or the method further includes instructing or controlling movement or navigation of each of the plurality of sections or portions such that any counter-active motion or motions are avoided or eliminated and such that any gap between respective trajectories of each of the plurality of sections or portions is eliminated or avoided.

28. The storage medium of claim 23, further comprising using the Stay the Course mode and performing the following:(i) instructing or commanding the catheter or probe to move forward or to move along the path;(ii) calculating a vector, a normal path, or a trajectory for a Tip or distal end effector of the Tip or distal bending section or portion or for an end effector of the first section or portion;(iii) calculating a deviation of the Tip or distal end effector of the Tip or distal bending section or portion or for the end effector of the first section or portion from the vector, the normal path, or the trajectory due to a shape and / or motion of the catheter or probe;(iv) calculating a change to the pose, position, or state of the Middle / proximal section or portion, or other section or portion subsequent to or following the Tip or distal bend section or portion or the first bending section or portion, of the catheter or probe to counteract or remove the calculated deviation; and(v) updating the pose, position, or state of the Middle / proximal, or other subsequent or following, section or portion of the catheter or probe to match, substantially match, or approximately match the pose, position, or state of the Tip or distal bending section or portion, or of the first bending section or portion, of the catheter or probe at that point along the path, to eliminate or remove the calculated deviation.

29. The storage medium of claim 28, wherein one or more of the following: the motion of the catheter or the probe is one or more of the following: a motion from movement of a stage or translational stage of the continuum robot; a motion from one or more changes due to an environment or due to the target, object, or sample in which the catheter or probe is located; a body divergence of a patient, target, object, or sample divergence; and / or a motion from another source external to the continuum robot; the method further includes evaluating or determining whether a path, vector, or trajectory deviation exists due to a shape and / or motion of the catheter or probe; a motion of the catheter or probe includes one or more of the following: a motion of the base or a stage; a motion due to translational movement of the catheter or probe and / or the continuum robot; a motion or movement of the target, object, or sample; and / or a motion from a force or influence originating outside of the continuum robot; the method further includes using a normal path, vector, or trajectory deviation as a trigger for, and in, the calculation of the calculated deviation of the Tip or distal end effector of the Tip or distal bending section or portion or for the end effector of the first section or portion from the vector, the normal path, or the trajectory due to a shape and / or motion of the catheter or probe; the method further includes controlling the catheter or probe to stay or maintain a desired course along the path; the method further includes controlling and adjusting a pose, position, or state of a section(s) or portion(s) of the catheter or probe to minimize any deviation of a pose, position, or state of one or more Middle / proximal, subsequent, or following section(s) or portion(s) based on a predetermined, targeted trajectory and a maximum motion along the predetermined, targeted trajectory; coordinates for, and the vector, normal path, or trajectory of subsequent, following, or future section(s) or portion(s) are known, set, or determined; information for one or more prior section(s) or portion(s) of the catheter or probe are known, set, or determined; and / or io6the method further includes instructing or controlling movement or navigation of each of the plurality of sections or portions such that any counter-active motion or motions are avoided or eliminated and such that a predetermined or targeted course is maintained between respective trajectories of each of the plurality of sections or portions.

Images