Navigation of elongate robotic drive devices

Abstract

The invention relates to navigation of an elongate robotic drive device. In order to improve workflow and provide further assistance in manipulating an interventional device by a robotic device, a navigation system (10) for assisting robotic anatomical path navigation of an elongate robotic drive device is provided, comprising a data input (12), a data processor (14) and an output interface (16). The data input is configured to receive robot-related data of a designated robot provided for performing at least one intraluminal related task, device-related data related to intrinsic or mechanical properties of at least one elongate device to be driven by the designated robot, and object-related anatomical path image data of a region of interest, in the region of interest, the device is to be moved by a specified robot; and providing the data to a data processor. A data processor is configured to calculate an estimate of a working range of at least one device in an anatomical path based on the robot-related data, the device-related data, and the object-related anatomical path image data, the robot-related data, the device-related data, and the object-related anatomical path image data are inputs for estimation calculations. The output interface is configured to provide an estimated operating range.

Description

Technical Field

[0001] The present invention relates to navigation of an elongated robotic drive device, and in particular to a navigation system for assisting robotic anatomical path navigation of an elongated robotic drive device, a system for robotic anatomical path navigation, and a method for assisting robotic anatomical path navigation of an elongated robotic drive device. Background Art

[0002] Robotic solutions for treating endovascular diseases have been growing in popularity over the past few years. Applications for robotic-assisted procedures span a variety of interventions, including coronary, peripheral, and neurovascular interventions. Various designs have been used for robotic systems, including articulated arms or motorized robotic modules that can translate and rotate vascular devices. Depending on the design of the intervention and the robot, different devices with different working ranges and properties are required. For example, robotic-assisted neurovascular procedures (stroke, aneurysm, etc.) will require a higher working range for the devices to reach their targets in the carotid arteries and brain. In robotic intravascular navigation, different intravascular devices are provided that can be replaced. However, it has been shown that process interruptions for device changes can be cumbersome, requiring the new device to be found, the current device to be unloaded, the new device to be reloaded in a specific manner, and most importantly, replacement significantly increases the risk of complications (such as dissection and clots) and wastes valuable time. Summary of the Invention

[0003] Therefore, there may be a need to improve workflow and further assist in manipulating interventional devices via robotic means.

[0004] The objects of the invention are solved by the subject matter of the independent claims; further embodiments are incorporated into the dependent claims. It should be noted that the aspects of the invention described below also apply to a navigation system for assisting in robotic anatomical path navigation of an elongated robotic drive device, a system for robotic anatomical path navigation, and a method for assisting in robotic anatomical path navigation of an elongated robotic drive device.

[0005] According to the present invention, a navigation system for assisting robotic anatomical path navigation of an elongated robotically driven device is provided. The system includes a data input, a data processor, and an output interface. The data input is configured or enabled to receive robot-related data for a designated robot configured to perform at least one intracavitary task and / or to drive the elongated driving device. Such robot-related data may include data related to the elongated device driven by the robot. The data input is further configured or enabled to receive device-related data related to intrinsic and / or mechanical properties of at least one elongated device to be driven by the designated robot. The data input is further configured or enabled to receive subject-related anatomical path image data of a region of interest, wherein the device is to be moved by the designated robot. The data input is configured to provide the data to the data processor. A non-transitory machine-readable storage medium or memory may be provided and encoded with instructions for execution by the data processor. The data processor is configured to calculate an estimate of the working range of at least one device in the anatomical path based on the robot-related data, the device-related data, and the subject-related anatomical path image data (which are inputs to the estimation calculation). The output interface is configured to provide the estimated working range.

[0006] As an effect, providing information about the travel limits of a device being used for a robot-assisted intervention allows for an optimized use of the device. For example, if the working range is shown to be sufficient for a specific task, the user is provided with confidence that a sudden need to change the device is avoided. Alternatively, if the working range is shown to be insufficient, the user can consider replanning or changing the device to a preferred point in time, for example, before navigating further into the vascular structure. Unwanted flow interruptions are avoided or at least better integrated into the workflow. The provided knowledge about the (estimated) working range allows the user to smooth the workflow. Thus, the knowledge about the working range supports successful navigation.

[0007] In this example, the device's working range changes dynamically as the user navigates. This means that the maximum reach of the robotically controlled device changes based on slack, kinks, device buckling, and energy accumulation along the way. Therefore, a device initially expected to have sufficient length to reach a target may appear to have insufficient length midway through navigation—suggesting that the user replace the device in advance or purge energy accumulation from the system.

[0008] In an example, at least one repetitive cycle of updating the estimated operating range is provided.

[0009] As an advantage, knowledge about i) the specific device, ii) the specific current anatomical situation and iii) the working range of the current robotic device assists the user in manipulating the interventional device(s) via the robotic device and thus improves the workflow.

[0010] In the examples, the term robotic device refers to the robot configuration and state.

[0011] According to an example, the data processor is configured to calculate estimates of operating ranges along different possible paths for the at least one device.The output interface is configured to output the operating ranges along the different possible paths.

[0012] In an option, assessments are calculated for different pathways based on predetermined weighting factors (such as the tortuosity of the pathway, the number of branching channels, narrowed channels, the width of the vessel, the risk importance of the vessel channel, etc., such as relaxation and energy accumulation, which can be another input here. The determined assessment values are indicated to the user.

[0013] According to an example, the data input is configured to provide device-related data, including a plurality of device-related data for a plurality of different devices. The data processor is configured to calculate a plurality of estimates of the operating range based on the plurality of device-related data. The output interface is configured to output the operating ranges along the different possible paths.

[0014] In an option, evaluations are calculated for different devices based on predetermined weighting factors such as reach, maneuverability, etc. The determined evaluation values are indicated to the user.

[0015] According to an example, the working range is provided as graphical information superimposed on anatomical image data of the region of interest. As an option provided additionally or alternatively, the output interface is configured to provide the estimated working range as an image matrix, wherein the maximum working range of each device within each path is uniquely marked in the image.

[0016] According to an example, the data input is configured to provide workflow data.The data processor is configured to further calculate at least one estimate of a working range based on the workflow data.

[0017] According to an example, a neural network-based controller is provided, comprising a convolution filter configured to capture background patterns in image data. Additionally or alternatively, the data processor is configured to calculate the estimate based on training of the neural network.

[0018] According to the present invention, a system for robotic anatomical path navigation is also provided. The system includes a robotic apparatus and a navigation system according to one of the aforementioned examples. The robotic apparatus is configured to control and drive at least one device for insertion into and movement along an anatomical path in a region of interest of a subject. The navigation system is configured to provide an estimate of the working range of the at least one device within the anatomical path in the region of interest of the subject.

[0019] According to an example, an imaging apparatus configured to provide object-related anatomical path image data is provided. As an additionally or alternatively provided option, the object-related anatomical path image data is provided as a 2D X-ray image.

[0020] This allows estimating the working range using live images, which on the one hand simplifies the procedure since no additional data about the current / present anatomy is needed, and which on the other hand provides the most accurate information about the current anatomy in which the device needs to be navigated.

[0021] According to the present invention, there is also provided a method for assisting robotic anatomical path navigation of an elongated robotic drive device. The method comprises the following steps:

[0022] receiving robot-related data for a designated robot provided for performing at least one intracavity-related task;

[0023] receiving equipment-related data relating to intrinsic or mechanical properties of at least one elongated equipment to be driven by the designated robot;

[0024] receiving subject-related anatomical path image data of a region of interest in which the device is to be moved by the designated robot;

[0025] computing an estimate of a working range of the at least one device in the anatomical pathway based on the robot-related data, the device-related data, and the subject-related anatomical pathway image data; and

[0026] Outputs the estimated operating range.

[0027] According to aspects, the travel limits of each device will depend on the relationship between the current device, the robotic system, and the patient anatomy observed in the interventional image. Provided is a method for dynamically calculating and displaying the maximum reach of a robotically controlled intravascular device by combining imaging interpretation, device intelligence, and robot-related data (which may include robotic device data, such as, for example, the configuration of the robot (e.g., geometry, articulation configuration, other design data, intrinsic parameters of the robot, robot state, robot settings, registration and / or calibration data...), and further to drive-related data, such as, for example, drive-related feedback data (e.g., kinematics, encoder data from (one or more) robot motors, motion data - rotation, translation and / or roll and / or positioning data...). The maximum reach of an intravascular device may also be referred to as a working range. Factors determining the maximum reach of a device in an anatomical structure depend on the design of the robotic system, the placement of the intravascular device on the robot, the length of the device, the access site location (e.g., radial artery or femoral artery), and the patient anatomy. Information about the working range can be presented as a graphical overlay, text, audio, or tactile feedback.

[0028] In an example, a procedure may use multiple devices simultaneously, each device having a working range and optimal position to support the distal device (if any) within it.

[0029] The present invention focuses on combining new robotic technology with an interventional imaging platform. The working range estimation system can be used with, for example, a fixed C-arm system and a mobile fluoroscopy system. In an example, the working range estimation system is designed to work with an interventional guided therapy device.

[0030] As an advantage, the usability and interface of the image-guided robotic system is enhanced. It can be available as part of the robot core interface or within a software as a service solution.

[0031] According to one aspect, a working range of a device driven by a robotic device is determined based on a current anatomical situation. The process of calculating the working range is particularly based on image data representing the current situation. The image data is acquired and possible pathways within a given anatomical structure are determined for a given device or selection of possible devices.

[0032] In another example, multiple ways of using image data are provided. In an example, for example, if the device is already in the field of view, only the last image of the patient is used, which captures the anatomy and the device within the vasculature at its last position.

[0033] In another approach, a sequence of images from the past to the present is used. In this case, for example, the last 100 X-ray images from the same patient are used during training / inference, showing how the device has been moved through the vasculature. These images can be from the same anatomical region, or they can be from various anatomical regions and different fields of view stitched together.

[0034] These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Exemplary embodiments of the present invention will be described below with reference to the following drawings:

[0036] Figure 1 An example of a navigation system for assisting in robotic anatomical path navigation of an elongated robotic drive device is schematically shown.

[0037] Figure 2 An example of a system for robotic anatomical path navigation is shown.

[0038] Figure 3 The basic steps of an example of a method for assisting robotic anatomical path navigation of an elongated robotic drive device are shown.

[0039] Figure 4 Another example of a schematic setup for assisting robotic anatomical path navigation is shown.

[0040] Figure 5 Further examples of working schemes are shown.

[0041] Figure 6 An example of a presentation shown on a display is shown.

[0042] Figure 7 Examples of uncertainty indicators are shown. DETAILED DESCRIPTION

[0043] Certain embodiments will now be described in more detail with reference to the accompanying drawings. In the following description, similar reference numerals are used for similar elements even in different drawings. Content defined in the specification, such as detailed configuration and elements, is provided to assist in a comprehensive understanding of the exemplary embodiments. Furthermore, well-known functions or configurations are not described in detail because they would obscure the embodiments with unnecessary detail. In addition, when preceding a list of elements, expressions such as "at least one of..." modify the entire list of elements and do not modify the individual elements of the list.

[0044] Figure 1An example navigation system 10 for assisting in robotic anatomical path navigation of an elongated robotically driven device is schematically illustrated. System 10 includes a data input 12, a data processor 14, and an output interface 16. Data input 12 is configured to receive robot-related data for a designated robot configured to perform at least one endocavity-related task. Data input 12 is also configured to receive device-related data regarding inherent or mechanical properties of at least one elongated device to be driven by the designated robot. Data input 12 is also configured to receive subject-related anatomical path image data of a region of interest in which the device is to be moved by the designated robot. Data input 12 is also configured to provide this data to data processor 14. Data processor 14 includes a memory (not shown in detail) for storing instructions and a processor for executing the instructions. Data processor 14 is configured to calculate an estimate of the working range of at least one device within the anatomical path based on the robot-related data, device-related data, and subject-related anatomical path image data, the robot-related data, the device-related data, and the subject-related anatomical path image data serving as inputs for the estimation calculation. The output interface 16 is configured to provide the estimated operating range.

[0045] Three dashed arrows indicate the input of different types of data: a first arrow 18 indicates the supply or feeding of robot-related data. A second arrow 20 indicates the supply or feeding of device-related data. A third arrow 22 indicates the supply or feeding of object-related anatomical path image data. A fourth dashed arrow 22 indicates the output of data, namely, the provision of a calculated estimate of the working range. Optionally, the estimate is shown on a display 26.

[0046] In one option, the frame 28 indicates that the data input 12, the data processor 14 and the output interface 16 are provided in a common housing. In another option, the data input 12, the data processor 14 and the output interface 16 are provided as individual components.

[0047] In an example not shown in greater detail in the figures, the working range comprises at least one parameter from the group consisting of: travel limit, working length and maximum reach.

[0048] Figure 2An example of a system 100 for robotic anatomical path navigation is shown. It should be noted that, as an option, the system is shown in the context of an operating room with other equipment such as an imaging system. The system 100 can also be provided as a standalone solution. The system 100 includes a robotic apparatus 102 and an example 104 of a navigation system 10 according to one of the aforementioned examples. The robotic apparatus 102 is configured to control and drive (e.g., manipulate) at least one device 106 for insertion into and movement along an anatomical path of an area of interest of an object 108. The navigation system 10 is configured to provide an estimate of the working range of the at least one device 106 within the anatomical path of the area of interest of the object 108.

[0049] The robotic device 102 is shown schematically only. The robotic device 102 may be a stand-alone robot having a stand and one or more manipulators. The robotic device 102 may also be integrated into other movable equipment within an operating room, such as an operating table. The robotic device 102 may be supported on the floor, mounted to a wall structure, or suspended from the ceiling. The robotic device 102 may have one or more "arms" as manipulators. The manipulators may have multiple degrees of freedom of motion, such as three degrees of freedom, or four, five, or six degrees of freedom.

[0050] The robotic system may optionally include a tracking system, such as that used in surgical navigation systems based on IR light or EM tracking or (optical) shape sensing technology. Such a tracking system can track the position of the robotic device relative to the patient and / or relative to equipment in the operating room (such as an X-ray system). In addition to internal encoding methods (e.g., motor encoders, linear encoders, etc.), the tracking system can also provide information about the proximity of the robotic components to the patient access site and the relative position of the robotic components in the device. This is a method of calibrating and / or tracking the relative displacement of the device relative to the patient and the device itself. Alternatively, the tracking system can track the device directly using EM markers or optical tracking fiducials attached to the device(s) or using common computer vision methods with a stereo camera.

[0051] As an option, the object 108 is shown to be arranged on an object support 110. A side controller 112 can be arranged nearby. A hanging monitor device 114 is also shown. In addition, a console 116 for operating the equipment in the operating room is indicated. The console 116 can be arranged in the same room or in a separate room.

[0052] As an option, an imaging device 118 is provided that is configured to provide subject-related anatomical path image data. As an example, the imaging device 118 is an X-ray imaging system having a movably mounted C-arm 120 having an X-ray source 122 and a detector 124 mounted to opposite ends of the C-arm 120.

[0053] A first communication line 126 indicates a data connection between the robotic device 102 and the instance 104 of the navigation system 10. A second communication line 128 indicates a data connection between the imaging device 118 and the instance 104 of the navigation system 10. A third communication line 130 indicates a data connection between the instance 104 of the navigation system 10 and the console 116.

[0054] The data connection may be provided as wire-based communication or wireless communication.

[0055] In an option, the subject-related anatomical path image data is provided as a 2D X-ray image.

[0056] Figure 3 The basic steps of an example of a method 200 for assisting robotic anatomical path navigation of an elongated robotically driven device are shown. The method 200 comprises the following steps: in a first sub-step 202, robot-related data of a designated robot provided for performing at least one endocavity-related task are received. In a second sub-step 204, device-related data relating to intrinsic or mechanical properties of at least one elongated device to be driven by the designated robot are received. In a third sub-step 206, object-related anatomical path image data of a region of interest in which the device is to be moved by the designated robot are received. The three sub-steps may be provided simultaneously or subsequently in any order. In a further step 208, an estimate of a working range is calculated for at least one device in the anatomical path based on the robot-related data, the device-related data and the object-related anatomical path image data. In a next step 210, the estimated working range is provided, for example, to an operator.

[0057] Return Reference Figure 1 and Figure 2 , devices used for endocavity-related tasks (ie, devices 106 used to be inserted into and moved along an anatomical path of a region of interest of a subject) may also be referred to as interventional devices.

[0058] In an example, the elongated robotic drive device 106 is configured to be navigated in an anatomical structure. An anatomical structure refers to any type of structure in an object having different properties for navigating the device. In an example, the anatomical structure refers to a vessel or lumen, such as a vascular structure or an organ.

[0059] The navigation system 10 for robotic anatomical path navigation may also be referred to as a navigation apparatus (for robotic anatomical path navigation) or a navigation device (for robotic anatomical path navigation).

[0060] In a first option, the navigation system 10 is provided as a data processing device. As an example, the navigation system is provided as a kit for upgrading an existing system. As another example, the navigation system is provided as a separate part of a system.

[0061] In a second option, the navigation system 10 further comprises at least one device 106. As an example, the navigation system 10 is provided as a navigation setup with a collection of devices 10, eg several different interventional devices.

[0062] In an example, the instructions, when executed by the data processor 14 , cause the system to receive at least one of the group consisting of: robot-related data, positioning or motion data, device-related data, and image data.

[0063] The term "anatomical path navigation" refers to navigation, i.e., guided movement along a given anatomical lumen suitable for insertion of a device, such as a guidewire, catheter, etc. An anatomical path or lumen can be any hollow anatomical structure, such as a vascular structure, a respiratory passage, or the intestine or gastrointestinal tract. In examples, an anatomical path or lumen is a naturally occurring path.

[0064] The term "data input 12" refers to providing or supplying data for data processing steps. The data input 12 may also be referred to as an image data input 12. The data input 12 may also be referred to as a data supply, an image data supply, an image input unit, an input unit, or simply an input unit. In an example, the data input 12 may be data-connected to an imaging source device. The data input 12 is configured to receive data from a corresponding data source and transmit it to the data processor 14, for example.

[0065] The term "data processor 14" refers to a portion of a processor or processor device that is provided to perform calculation steps using data provided by a data input. The data processor 14 may also be referred to as a data processing device, a processor unit or a processor. In an example, the data processor 14 is data connected to the data input and output interfaces. In an option, the data processor 14 is configured to calculate the range of the device when it is operated and moved by the robot. Therefore, the data processor 14 may be referred to as a "range calculation unit". The range calculation unit receives data collected before, during or after the robot-assisted surgical procedure and calculates the possible working range of each device (e.g., in particular flexible devices) relative to the patient, imaging or robot reference frame.

[0066] The term "output interface 16" refers to an interface for providing processed or calculated data for further use. Output interface 16 may also be referred to as an output portion or output unit. In an example, output interface 16 may be data-connected to a display device or apparatus. In another example, output interface 16 may be data-connected to a display. As an example, signals from a controller may be provided by output interface 16. Output interface 16 is configured to receive data from data processor 14 and transmit it to, for example, a display.

[0067] The term "robot-related data" refers to data derived from the use of a robot. The term "robot-related data" refers to, for example, at least one item from the group consisting of: encoding data, kinematics, end-effector pose, robot tracker data, and user control inputs to the robot used to move the robot. In one example, the robot-related data includes robot movement-related data, such as movement capabilities, options, and limitations. In another example, the robot-related data includes robot state-related data, such as robot state data. In another option, the robot-related data also includes robot calibration data. The robot-related data may also be referred to as robot data.

[0068] The term "device-related data" refers to data about the mechanical characteristics of a device, such as overall length, working length, width, X-ray opacity, stiffness, shape, articulation pattern (for maneuverable devices), how it is mounted to a robot, etc. Device-related data includes data related to the device. As an example, device-related data includes inherent properties of the device and parameters of its interface with the robot. Device-related data can also be referred to as device data. Working length can be the length of a segment that can enter the interior of another device.

[0069] The term "subject-related anatomical path image data" refers to image data from a navigation system. In an example, the anatomical path image data is provided as vascular structure image data. The subject-related anatomical path image data may also be referred to as subject anatomical path image data or subject path image data.

[0070] The term "intraluminal tasks" refers to tasks within a lumen, such as tasks within blood vessels, the respiratory system, or other hollow sections within an anatomical structure. The term "intraluminal tasks" may also be referred to as "intravascular tasks."

[0071] The term "working range" means at least one of the group consisting of: travel limit, working length, and maximum reach.

[0072] The imaging data may be two-dimensional (2D) or three-dimensional (3D).Preferably, 2D image data is provided, such as live fluoroscopic images.

[0073] In the options, anatomical pathway navigation refers to intravascular navigation; and device-related data is data related to an intravascular device.

[0074] In an example, the output interface 16 is configured to provide an estimated working range for assisting an operator.

[0075] In an example, the data processor 14 is configured to calculate an estimate of a working range of at least one device in the anatomical path relative to the target data based on the robot-related data.

[0076] According to the present invention, the "estimation" or better its corresponding model is based on a plurality of movement probabilities of, for example, an elongated device type with kinematics within the constraints of the anatomical environment.

[0077] In one example, a model is obtained by training a neural network using retrospective robotic and imaging data. The neural network can be designed with multiple channels at the output layer, where each channel corresponds to a separate possible estimate of the device limits for a different movement probability.

[0078] The term "travel limit" relates to the distance a device can travel when being moved by a robot, such as along a certain travel path.

[0079] The term "working length" refers to the length of the vessel segment(s) within which the interventional device is moved and along which interventional device operation is possible. Thus, the working length can be a fraction of the travel distance. In an example, the working length is the amount that the device (tip) can reach within the vasculature given the constraints.

[0080] The term "maximum reach" refers to the maximum possible distance a device can reach. In one example, the maximum reach is the actual location within the blood vessel. The maximum reach can also be the distance at which functional operation is still possible. Alternatively, the maximum reach can be longer than the working length.

[0081] In an example not shown in greater detail in the figures, the data processor 14 is configured to calculate estimates of the operating range along different possible paths of the at least one device. Furthermore, the output interface 16 is configured to output the operating range along the different possible paths.

[0082] In an example not shown in greater detail in the figures, the data input 12 is configured to provide device-specific data, including a plurality of device-specific data for a plurality of different devices. The data processor 14 is configured to calculate a plurality of estimates of the operating range based on the plurality of device-specific data. Furthermore, the output interface 16 is configured to output the operating ranges along the different possible paths.

[0083] In an option, estimates of the operating range along different possible paths are provided for a number of different devices.

[0084] As an example, the maximum reach of each device within each vessel or other anatomical pathway is uniquely labeled in the displayed image. An example output of the system is an image matrix in which the maximum reach of each device within each vessel is uniquely labeled in the image. For example, the coordinate corresponding to the maximum reach is labeled with a different intensity compared to other pixels; or the entire trajectory of coordinates in each vessel up to the maximum reach is labeled with a different intensity.

[0085] As an effect, visualization of device constraints allows a user (e.g., an interventionalist) to observe in real time the working range that each robotically controlled device can reach using current settings and imaging feedback, thereby promptly and accordingly adjusting their catheterization strategy and device. As another exemplary effect, visualization of device constraints allows the user to adjust the relative positioning of the robot as needed. The term "relative" refers to the position relative to the object. Optionally, additionally or alternatively, the relative positioning of the object can be adjusted as needed.

[0086] In an option, the processor is configured to compare the operating ranges and provide a ranked proposal of at least two operating ranges.

[0087] In an example, the device-related data includes a plurality of device-related data for a plurality of different devices.

[0088] In an example not shown in greater detail in the accompanying drawings, the working range is provided as graphical information superimposed on the anatomical image data of the region of interest. In an additional or alternatively provided option, the output interface is configured to provide the estimated working range as an image matrix, wherein the maximum working range of each device within each path is uniquely marked in the image.

[0089] As an example, graphical information is provided as an indicator of the travel limit, working length, or maximum reach of the catheter.

[0090] In an example, the travel limit refers to the length of possible pathways along which the device 106 can move (ie, travel).

[0091] In another example, the working length refers to the possible path length that the device 106 can travel while still being able to provide a specified operational task. Such a task can be a forward motion (e.g., imaging or ablation) procedure, a backward motion procedure (such as a pullback), or even a repeated back-and-forth workflow.

[0092] In an example, maximum reach refers to the possible length along the vessel that still allows for appropriate procedural procedures, such as distal imaging reach of an imaging catheter.

[0093] For example, the coordinate corresponding to the maximum reachable range is marked with a different intensity compared to other pixels. In another example, the entire trajectory up to the coordinate of the maximum reachable range in each path segment (eg, blood vessel) is marked with different intensities.

[0094] In an example not shown in greater detail in the accompanying drawings, the robot-related data includes at least one item from the group consisting of: data obtained from the robot such as robot encoder information, robot travel limits, CAD design, forward kinematics, inverse kinematics, velocity, acceleration, end effector pose, user controller input, and robot placement relative to the patient.

[0095] In an example not shown in greater detail in the accompanying drawings, the device-related data includes at least one item from the group consisting of: device-specific insights, device type, device length, device stiffness, data about the device-robot relationship such as the distance between the tip or end of the device and the robot actuation unit, roller or gripper maneuverability information, and manufacturer.

[0096] In another example, the device-related data relates to robot data; for example, rollers and grippers can be robot-driven elements of a robot, as is known in the art, for rolling or gripping, which can contact the elongated device and can directly actuate movement of the elongated device, such as translation or rolling. The robot can be further positioned away from the rollers and grippers to rotate, translate, and / or roll the elongated device.

[0097] Thus, the "roller" and "gripper" are actuators that are related to robot data. As an example, calibration is supplied to provide relevant robot data related to the actuators. The calibration data provides information about the relative position of the devices in the device stack, such as 0 position + length + robot position.

[0098] In the example, the maneuverability information relates to a specific device that can be articulated. As an option, this is provided under the umbrella of a device model that includes shape behavior and controllability; specific articulation for maneuverable devices is added to the standard translation-rotation control.

[0099] In an example not shown in greater detail in the figures, the subject-related vascular structure image data includes at least one item from the group consisting of: data obtained from medical imaging, such as fluoroscopy or ultrasound imaging, and preoperative or intraoperative medical imaging data, such as 3D rotational angiography, CT, CBCT, and MRI images. Furthermore, the data processor 14 is configured to determine anatomical structures in the image data to identify an anatomical path suitable for navigating at least one device within the anatomical path. Additionally or alternatively, the subject-related anatomical path image data includes target data, the target data including at least one item from the group consisting of: a target and a pathway segment.

[0100] As an option, the subject-related anatomical path image data of the region of interest comprises at least one of the group consisting of: a vascular structure, a respiratory tract, or an intestinal passage.

[0101] In an option, the image data is provided as 2D image data, such as live or current fluoroscopy image data.

[0102] The data processor is configured to provide segmentation for identifying anatomical paths.

[0103] In an example not shown in greater detail in the figures, the data input 12 is configured to provide workflow data. Furthermore, the data processor 14 is configured to calculate at least one estimate of the working range also based on the workflow data.

[0104] In an example, the workflow data includes information about the access site. As an example, depending on the entry point (femoral or radial), the distance between the device starting point (device base) and the target anatomy changes, which directly affects the working length.

[0105] In an example, the workflow data includes information related to the target anatomy. As an example, information from the target anatomy can determine the tortuosity along the way, which directly affects the estimation of the working length.

[0106] In an example, the workflow data includes information about the type of procedure being applied. For example, data about the type of procedure (e.g., aneurysm coiling in the brain, stroke embolization, etc.) implicitly informs the system about some complications of device manipulation, navigation pathways, etc. Optionally, when a neural network is applied, the neural network learns from the input data.

[0107] In an example, the workflow data includes information about kinks, buckling, slack, etc. As an example, these components will reduce the effective working length of the device. For example, when the system is aware of a kink or high slack in the system, it can adjust its predictions to output a shorter working length.

[0108] In the options, the above inputs are placed in a descriptor vector, which is then digitized and fed to the neural network during training and inference.

[0109] As examples, access sites include (right / left) femoral and / or radial artery access points. For example, procedure types include mechanical thrombectomy or coiling.

[0110] As an advantage, for example in robotic intravascular interventions, the user is facilitated to correctly select a device that has taken into account its maximum reach inside the anatomical structure. As an option, the robot is placed near the access zone and a plurality of candidate devices, such as catheters, guidewires, guide catheters, microwires, etc., are selected for possible use, i.e. deployment during the intervention. By determining (i.e. estimating) the working range, candidates for devices with a length that is not suitable for a specific task can be deselected, which avoids the situation where the user will have to change the selected device due to lack of working range. Thus, by providing an estimate of the working range, the efficiency of the process is significantly supported and enhanced.

[0111] Estimating the working range also allows for robotic navigation using standard intravascular devices, which may be designed primarily for manual navigation. Such devices may not be long enough for some remote robotic-assisted procedures, but may be suitable for multiple robotic-assisted procedures. Estimating the working range also addresses robots with limited travel range, which directly impacts the working length of the robotically controlled device.

[0112] By facilitating the selection of equipment with the correct length, the need for manual steps such as multiple equipment changes or robot displacements to obtain additional range of motion is avoided or at least minimized, which would interrupt the process and would require significant staff involvement and thus prolong the process.

[0113] This estimation can be implemented in a robot-assisted system for performing the corresponding task. When provided with image data reflecting the current anatomical situation, the user is provided with real-time information about the maximum working range of each device within the anatomy. This estimation can take into account multiple parameters, such as device shape, slack, anatomy, robot type, and device type. As a result, medical workflow is beneficially improved, and a new level of confidence is provided to, for example, interventionalists.

[0114] The auxiliary device is designed to use some or all of the above data and estimate an output that encodes the scope of each device. The output of the device can be numerical, tabular, graphical or other formats.

[0115] Figure 4A schematic arrangement of another example is shown. A range estimator 300 (in the center portion) is connected to a data supply (on the left). As an example of live or current image data, a 2D X-ray image 302 representing an anatomical image of a region of interest of an object is indicated as an input. A first arrow 304 indicates the supply or input of data to the range estimator 300. In addition, a plurality of parameters 306 are provided and supplied (indicated by a second arrow 308) to the range estimator 300 as further inputs. The parameters relate to both the device and the robot used to operate the device. Examples of parameters are device type, device entry point, special equipment, device mounting on the robot, and robot encoder parameters. The range estimator 300 calculates the working range of the device for a given anatomical structure and provides it as an output, indicated by a third arrow 310. A diagram 312 as an output shows the anatomical structure as, for example, an X-ray image, such as an angiographic image, superimposed by a graphical representation 314 of the working range. Figure 4 The working range estimator module is shown using two sets of input data: first, the robot / device state, which includes the inherent parameters of the robot and device and their relative relationships; and second, the imaging feedback, which shows the current configuration of the device inside the vasculature. The output of the "range estimator" module 300 is then superimposed as the maximum working range on the interventional image.

[0116] Figure 5 Another example of a working solution is shown. The range calculation unit 350 is data-connected to a medical imaging system 352, i.e., an imaging device, which supplies image data 354 to the range calculation unit 350. Furthermore, the range calculation unit 350 is data-connected to a robotic system 356, which supplies robotic data 354 to the range calculation unit 350. Furthermore, as an option, a workflow source 360 providing, for example, an event log or audio data of the current scene in the operating room or video of user activity in the operating room is provided and supplied to the robotic data and provided to the range calculation unit 350 as device data 358. The robotic system 356 can also supply data to the device data 362.

[0117] The range calculation unit 350 is data-connected to one or several displays 364. As an option, the range calculation unit 350 is data-connected to an operator's cabin intelligence unit 366, which further utilizes the generated working range data.

[0118] In particular embodiments, the range calculation unit 350 or range estimator 300 or controller or resulting model has been developed based on parameters defined according to the probability of movement of a slender device type (a type defined in the device data) having potential kinematics (included in the robot data) in a determined anatomical environment (e.g., a vascular or respiratory or other endoluminal structure included in the anatomical pathway image data) and may involve parameters related to the target area or location (which may be included in the anatomical pathway image data).

[0119] These parameters can be entered manually or generated based on a dataset.

[0120] In a more specific embodiment, the range calculation unit 350 or range estimator 300 or controller or resulting model has been trained based on a previous data set, which may include robot data, device data, anatomical pathway image data, and may also include determined working range limits (see more exemplary details in subsequent sections).

[0121] In an example not shown in greater detail in the accompanying drawings, the range calculation unit 350 or range estimator 300 or controller or resulting model comprises a neural network-based controller that includes a convolution filter. The filter can be configured to capture background patterns in the image data. As an additional or alternative option, the data processor is configured to calculate the estimate based on training of the neural network.

[0122] In this example, background patterns are learned as weights for convolutional kernels and extracted as feature maps from the input data. For example, using supervised training, the weights are estimated based on minimizing the distance between the estimated working range and the ground-truth working range label. Examples of low-level background features include landmarks on the device, as well as anatomical structures and device boundaries in the image. High-level background patterns include the overall structure of the device relative to the vascular structure.

[0123] In another example, the background pattern is a spatial background pattern. In an example, the spatial background pattern is a pattern that captures the 2D or 3D spatial relationships between different anatomical structures and the device based on the input data. These can be low-level patterns, such as the location of different edges in the image, or high-level patterns, such as the overall position of the device in the vasculature, or the registration between the robot data / state and the device position in the 3D vasculature. As described above, convolution kernels are used in the neural network to capture spatial context.

[0124] In another example, the background pattern is a non-spatial background pattern.

[0125] In the example, the neural network also uses fully connected layers to capture vectorized and numerical patterns and embeddings from robotic data, equipment-related data, or workflow data. The neural network can also use recurrent layers such as RNNs, LSTMs, transformers, etc. to capture temporal dependencies when using time series data.

[0126] In an option, for the purpose of training the network-based controller, various input data are provided as synthetic data obtained in a computer simulation environment, including using different robot setups, synthetic data from device models and target anatomy, as further explained in subsequent sections of this disclosure.

[0127] In the options, a range estimator training phase is provided. The weights used for the neural network's range estimation are learned and stored during the training phase. To create the training data, various data are collected from the robot's manual navigation. All relevant data, such as imaging, robot, equipment, workflow, and data, are stored during the data acquisition step. Whenever the robot reaches a limit during manual navigation, the image coordinates and corresponding intervention data (image, robot, equipment, and workflow data) are stored. The coordinates of the equipment limit are then used as the ground truth labels. Finally, the training data and corresponding labels are used to train the neural network.

[0128] During training, a back-propagation process is used to optimize the neural network weights. As an example, at each iteration, the neural network predictions are compared to the ground truth labels using a distance function. Some relevant distance functions for training neural networks can include (but are not limited to) L-2 distance (Euclidean), L-1 distance, binary cross entropy, and dice loss.

[0129] As an additional option, a range estimator inference phase is provided. During the inference phase, the weights obtained during training are stored and used to calculate the expected maximum working range. In this step, real-time interventional data (images, robots, equipment, workflow, etc.) is fed to the neural network controller in the same format used during training. Finally, the input data is passed through the neural network model forward to generate the output.

[0130] As yet another option, learning in simulation is provided. For example, in a computer simulation environment, simulated interventional sessions are generated based on different setups of an intraluminal (e.g., intravascular) robot, different intraluminal device models, and target anatomical structures. Each device is synthetically advanced through all branches associated with the target process, and device constraints are obtained and stored with each new setup. The synthetically generated [robot, device, imaging] data is then used as input signals to the controller introduced in the main claim. The device constraints will serve as the ground truth labels corresponding to the input data. The set of inputs and ground truth labels generated here will be used to train the neural network controller introduced above.

[0131] In the options, the assignment of graphical elements is provided. As an example, it is provided to assign or change single or multiple visual, audio or textual elements on the graphical display based on the operating range of the device being controlled by the robot. An example of this embodiment is to enhance the travel range on a fluorescent, contrast or road map image, such as Figure 6 shown.

[0132] exist Figure 6 , an example of a presentation shown on a display is provided. An image 400 represents a region of interest of an object 402, which shows a corresponding diagram indicative of a vascular structure 404. A superimposed indicator 406 (eg, highlighted) indicates the calculated working range. Figure 6 The working range is shown as an overlay on the display interface. The overlay can be enhanced onto a 2D or 3D acquired (or simulated) image. As an option, the image can be fluoroscopic, DRR, DSA, roadmap, CBCT, CT, etc.

[0133] In an example not shown in greater detail in the figures, the data processor 14 is configured to determine an operating range uncertainty.The output interface 16 is configured to provide an indication of the operating range uncertainty.

[0134] Figure 7 An example of a probability or uncertainty indicator is shown. In the lower right portion, an object 450 is depicted in a simplified manner. Furthermore, a robotic actuator 452 is shown driving a device 454 partially inserted into the object 452. In an enlarged portion 456, the distal end 458 of the device 454 is shown. A plurality of circles 460 of different grayscale values, colors, or patterns indicate different degrees of certainty, i.e., uncertainty indicators, for the calculated range of (the distal tip of) the device 454.

[0135] As an option, device range uncertainty is estimated using Monte Carlo dropout with Bayesian inference approximating a deep Gaussian process. To calculate uncertainty, a subset of neurons in the neural network controller presented in the main embodiment are turned off during the forward pass to trigger dropout. Next, each batch of incoming data is passed through the model multiple times, for example, ten times. Each time, the dropout mechanism produces a slightly different form of the model, which can subsequently produce a different operating range estimate for the neural network. The results of all these processes are aggregated to calculate the upper and lower bounds of the device's operating range. Finally, these uncertainty bounds are visualized on a display. An example form of this visualization could be using different colors or dashed lines for the uncertainty bounds superimposed on the displayed image.

[0136] In a first example, a system is provided that includes a robotic device and a navigation system.

[0137] In a second example, a system is provided that includes a robotic apparatus and a navigation system and at least one device. The interventional device is configured for insertion into a path of a subject, such as into a vascular structure.

[0138] In an option, a system comprising several devices (ie, at least two or more) is provided. For example, a collection of devices is provided.

[0139] In an example, a robotic apparatus is configured to be registered directly to an object for use in computing an estimate of the working range of a device being controlled and driven or manipulated by the robot.

[0140] In one option, device working ranges are estimated based on registration. The robotic system is configured to register directly to the patient's anatomy to calculate the operating range of the robotically controlled device. This registration loop is closed by finding a relationship between the robotic system and the anatomy visualized in the X-ray system. A method for calculating the registration transform is developed using pre- or intra-operative 3D imaging and 2D / 3D registration techniques.

[0141] In an option, user feedback is provided once a predetermined amount of the operating range has been reached.The user feedback is provided as at least one of the group comprising: visual feedback, audible feedback or tactile feedback.

[0142] For example, when the device is expected to reach the limit calculated in the previous embodiment, the device assigns haptic feedback, such as vibration, to the physical controller.

[0143] In one example, the predetermined amount is provided as half of the operating range, such as approximately 50%, or less than half of the operating range, such as 75%, 80%, 85%, 90%, or 95%, or even more. In another example, the predetermined amount is provided as the full operating range. As an example, visible feedback is provided as an overlay on the anatomical image, or as a separate light signal. In an additional or alternative option, vibration or other tactile feedback is provided to the user when the travel limit is reached or is about to be reached, such as via a handle operated by the user's foot or hand, or as vibration of a floor portion.

[0144] In an option, the system control is changed based on the working range. For example, the robot's controller changes the gain or speed of the system based on the distance of the device tip from the working limit of the device calculated using the methods described in the previous embodiments.

[0145] In an option, system control changes include changes to the imaging system, such as frame rate changes, resolution adaptation, magnification, etc., if the limits of the working range are approached.

[0146] The term "subject" may also be referred to as an individual. A "subject" may also be referred to as a patient, although it is noted that this term does not indicate whether the subject actually has any disease or condition.

[0147] In an example, a computer program is provided that includes instructions that, when executed by a computer, cause the computer to perform the method of the aforementioned example.

[0148] In an example, a computer program or program element for controlling an apparatus according to one of the above examples is provided, which program or program element, when executed by a processing unit, is adapted to perform the method steps of one of the above method examples. In an option, a computer-readable medium having the computer program of the above example stored thereon is provided.

[0149] In a further exemplary embodiment of the present invention, a computer program or a computer program element is provided, which is characterized in that it is adapted to execute the method steps of the method according to one of the preceding embodiments on a suitable system.

[0150] Therefore, the computer program element can be stored on a computer unit or distributed on more than one computer unit, which can also be part of an embodiment of the present invention. The computing unit can be adapted to perform the steps of the method described above or induce the performance of the steps of the method described above. In addition, it can be adapted to operate components of the apparatus described above. The computing unit can be adapted to automatically operate and / or execute user commands. The computer program can be loaded into a working memory of a data processor. The data processor can thus be equipped to perform the method of the present invention.

[0151] Aspects of the present invention can be implemented in a computer program product, which can be a set of computer program instructions that can be run by a computer and stored on a computer-readable storage device. Instructions of the present invention can be any interpretable or executable code mechanism, including but not limited to scripts, interpretable programs, dynamic link libraries (DLLs) or Java classes. Instructions can be provided as complete executable programs, partial executable programs, modifications (e.g., updates) to existing programs, or extensions (e.g., plug-ins) to existing programs. In addition, the parts of the processing of the present invention can be distributed on multiple computers or processors.

[0152] As described above, a processing unit (e.g., a controller) implements the control method. The controller can be implemented in a variety of ways using software and / or hardware to perform the various functions required. A processor is an example of a controller that employs one or more microprocessors that can be programmed using software (e.g., microcode) to perform the required functions. However, a controller can be implemented with or without a processor, and can also be implemented as a combination of dedicated hardware for performing some functions and a processor (e.g., one or more programmed microprocessors and associated circuitry) for performing other functions.

[0153] Examples of controller components that may be employed in various embodiments of the present disclosure include, but are not limited to, conventional microprocessors, application specific integrated circuits (ASICs), and field programmable gate arrays (FPGAs).

[0154] This exemplary embodiment of the invention covers both a computer program that right from the beginning uses the invention and a computer program that by means of an up-date turns an existing program into a program that uses the invention.

[0155] Furthermore, the computer program element can provide all necessary steps for implementing the procedure of an exemplary embodiment of the method as described above.

[0156] According to another exemplary embodiment of the present invention, a computer-readable medium, such as a CD-ROM, is provided, wherein the computer-readable medium has a computer program element stored on the computer-readable medium, the computer program element being described in the preceding section. The computer program can be stored and / or distributed on a suitable medium, such as an optical storage medium or solid-state medium provided with or as part of other hardware, but the computer program can also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems.

[0157] However, the computer program may also be present on a network such as the World Wide Web and can be downloaded from such a network into the working memory of a data processor. According to a further exemplary embodiment of the present invention, a medium for making a computer program element available for downloading is provided, which computer program element is arranged to perform a method according to one of the previously described embodiments of the invention.

[0158] It should be noted that embodiments of the present invention have been described with reference to different subject matters. Specifically, some embodiments are described with reference to method-type claims, while other embodiments are described with reference to apparatus-type claims. However, those skilled in the art will appreciate from the above and following descriptions that, unless otherwise indicated, any combination of features relating to different subject matters, in addition to any combination of features belonging to one type of subject matter, is also considered disclosed by this application. However, all features can be combined to provide synergistic effects that exceed the simple sum of the features.

[0159] Although the present invention has been illustrated and described in detail in the drawings and the foregoing description, such illustration and description are to be considered illustrative or exemplary rather than restrictive. The present invention is not limited to the disclosed embodiments. Other variations of the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention from a study of the drawings, the disclosure, and the appended claims.

[0160] In the claims, the word "comprising" does not exclude other elements or steps, and the word "a" or "an" does not exclude a plurality. A single processor or other unit may perform the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. Any reference signs in the claims should not be construed as limiting the scope.

Claims

1. A controller arranged to provide assistance to actuation of an elongate robotic drive device in robotic anatomical path navigation, the controller comprising: Data input unit (12); Data processor (14); as well as Output interface (16); wherein the data input is enabled to receive: robot-related data of a designated robot, the designated robot being provided for driving the elongated driving device in order to perform at least one endocavity-related task, such robot-related data comprising robot device data and / or data related to the driving; device-related data relating to intrinsic or mechanical properties of at least one elongated device to be driven by the designated robot; and object-related anatomical path image data of a region of interest, in which region of interest the device is to be moved by the designated robot; and the data input is enabled to provide the data to the data processor; wherein the data processor is configured, optionally using computer program instructions, to calculate an estimate of the working range of the at least one elongate device in the anatomical path based on the robot-related data, the device-related data and the subject-related anatomical path image data, the robot-related data, the device-related data and the subject-related anatomical path image data being inputs for the estimation calculation; and Wherein, the data processor is further configured to output the estimated working range via the output interface.

2. A navigation system (10) for assisting robotic anatomical path navigation of an elongated robotic drive device, the system comprising: Data input unit (12); Data processor (14); as well as Output interface (16); wherein the data input is enabled to receive: robot-related data of a designated robot, the designated robot being provided for driving the elongated driving device in order to perform at least one endocavity-related task, such robot-related data comprising robot device data and / or data related to the driving; device-related data relating to intrinsic or mechanical properties of at least one elongated device to be driven by the designated robot; and object-related anatomical path image data of a region of interest, in which region of interest the device is to be moved by the designated robot; and the data input is enabled to provide the data to the data processor; wherein the data processor is configured to calculate an estimate of a working range of the at least one elongate device in an anatomical path based on the robot-related data, the device-related data and the subject-related anatomical path image data, the robot-related data, the device-related data and the subject-related anatomical path image data being inputs for the estimation calculation; and Wherein, the output interface is configured to provide an estimated operating range.

3. The controller according to claim 1, wherein: The working range comprises at least one parameter from the group consisting of: a travel limit, a working length, and a maximum reach.

4. The controller according to claim 1 or 2, wherein: The data processor is configured to calculate estimates of operating ranges along different possible paths for the at least one device; and The output interface is configured to output the working range along the different possible paths.

5. The controller according to claim 1, 2 or 3, wherein: The data input unit is configured to provide the device-related data, wherein the device-related data includes a plurality of device-related data for a plurality of different devices; wherein the data processor is configured to calculate a plurality of estimates of operating ranges based on the plurality of device-related data; and The output interface is configured to output the working range along the different possible paths.

6. The controller according to one of the preceding claims, wherein The working range is provided as graphical information superimposed on anatomical image data of the region of interest; and The output interface is configured to provide the estimated working range as an image matrix, wherein the maximum working range of each device within each path is uniquely marked in the image.

7. The controller according to one of the preceding claims, wherein The robot-related data comprises at least one of the group consisting of: data obtained from the robot such as robot encoder information, robot travel limits, CAD design, forward kinematics, inverse kinematics, velocity, acceleration, end effector pose, user controller input, and robot placement relative to the patient.

8. The controller according to one of the preceding claims, wherein The device-related data includes at least one item from the group consisting of: device-specific insights, device type, device length, device working length, device stiffness, device shape, data about the device-robot relationship such as the distance between the tip or end of the device and the robot actuation unit, forces, torques, speeds, robot type, device manipulation mechanism (roller, gripper, belt, fixed) maneuverability information, and manufacturer.

9. The controller according to one of the preceding claims, wherein The object-related intraluminal structure image data includes at least one of the group consisting of: data obtained from medical imaging such as fluoroscopy or ultrasound imaging, and pre-operative or intra-operative medical imaging data such as 3D rotational angiography, CT, CBCT, and MRI images; and wherein the data processor is configured to determine anatomical structures in the image data to identify an anatomical path suitable for navigating the at least one device within the anatomical path; and The object-related anatomical pathway image data includes target data, the target data including at least one of the group consisting of: a target and a pathway segment.

10. The controller according to one of the preceding claims, wherein The data input is configured to provide workflow data; and Wherein the data processor is configured to further calculate at least one estimate of a working scope based on the workflow data.

11. The controller according to one of the preceding claims, wherein providing a neural network-based controller including a convolution filter configured to capture background patterns in the image data; and Wherein the data processor is configured to calculate the estimate based on training the neural network.

12. The controller according to one of the preceding claims, wherein The calculated estimate includes a determination of uncertainty in the scope of work; and Wherein, the output interface is configured to provide an indication of the uncertainty of the operating range.

13. A navigation system (10) for assisting robotic anatomical path navigation of an elongate robotic drive device, the system comprising a controller according to any one of the preceding claims.

14. The system (100) according to the preceding claim, further comprising: Robotic device (102); as well as Navigation system (10) according to one of the preceding claims; wherein the robotic apparatus is configured for controlling and driving navigation of at least one elongate device along an anatomical path of at least one region of interest of a subject, the robotic apparatus (102) being further arranged to provide robot-related data; Wherein the navigation system is configured to provide an estimate of a working range of the at least one elongated device within the anatomical path of the region of interest of the subject.

15. The system according to claim 13, wherein: providing an imaging device (118) configured to provide image data of an anatomical path associated with the subject; and Wherein, preferably, the object-related anatomical path image data is provided as a 2D X-ray image.

16. A computer program comprising instructions stored and encoded in a non-transitory processor-readable medium, said instructions when said program is executed by the controller according to claim 1.

Images