Control tablet for navigated and robotic workflows

Abstract

Systems and methods for remote control of surgical platforms during surgical procedures are provided. The method includes receiving from a remote location from the customizable hand-held device, a list of a plurality of surgical platforms for selection and in response to a selection of at least one of the surgical platforms, displaying a plurality of tasks associated with the selected surgical platform. The method also includes in response to a selection of at least one task from the plurality of tasks, causing the remote location to select executable instructions for the selected surgical platform to perform the selected task when a tactical control is used to locate a configurable key of the customizable hand-held device for activation and downloading the executable instructions from the remote location for use in connection with the configurable key of the customizable hand-held device.

Description

CONTROL TABLET FOR NAVIGATED AND ROBOTIC WORKFLOWS

[0001] This application claims the benefit of U.S. Provisional Patent Application Serial No. 63 / 722,432, filed 19 November 2024, the entire content of which is incorporated herein by reference.FIELD OF INVENTION

[0002] The present disclosure is generally directed to the field of remote communications and, more specifically, to a programmable remote controller for surgical platforms used in non-sterile and / or sterile environments.BACKGROUND

[0003] Surgical robots may assist a surgeon or other medical provider in carrying out a surgical procedure or may complete one or more surgical procedures autonomously. Imaging may be used by a medical provider for diagnostic and / or therapeutic purposes. Patient anatomy can change over time, particularly following placement of a medical implant in the patient anatomy During surgical procedures, however, intra-operative challenges do exist where surgeons and technicians for surgical platforms (navigation platforms, robotic platforms, etc.) are not on the same page in terms of efficiently utilizing the surgical platforms during surgical procedures. Accordingly, there is a need to provide improvements in remote control of surgical platforms during surgical procedures.BRIEF SUMMARY

[0004] Example aspects of the present disclosure include:

[0005] A customizable hand-held device according to at least one embodiment of the present disclosure comprises: a processor; and a memory storing data for processing by the processor, the data, when processed, causes the processor to: receive from a remote location from the customizable hand-held device, a list of a plurality of surgical platforms for selection; in response to a selection of at least one of the surgical platforms, display a plurality of tasks associated with the selected surgical platform; in response to a selection of at least one task from the plurality of tasks, cause the remote location to select executable instructions for the selected surgical platform to perform the selected task when a tactical control is used to locate a configurable key of the customizable hand-held device for activation; and download the executable instructions from the remote location for use in connection with the configurable key of the customizable hand-held device.

[0006] A method according to at least one embodiment of the present disclosure comprises: receiving from a remote location from a customizable hand-held device, a list of a plurality ofsurgical platforms for selection; in response to a selection of at least one of the surgical platforms, displaying a plurality of tasks associated with the selected surgical platform; in response to a selection of at least one task from the plurality of tasks, causing the remote location to select executable instructions for the selected surgical platform to perform the selected task when a tactical control is used to locate a configurable key of the customizable hand-held device for activation; and downloading the executable instructions from the remote location for use in connection with the configurable key of the customizable hand-held device.

[0007] A system according to at least one embodiment of the present disclosure comprises a storage device and a hand-held device, comprising: a processor and a memory storing data for processing by the processor, the data, when processed, causes the processor to receive from the storage device, a list of a plurality of surgical platforms for selection; in response to a selection of at least one of the surgical platforms, display a plurality of tasks associated with the selected surgical platform; in response to a selection of at least one task from the plurality of tasks, cause the storage device to select executable instructions for the selected surgical platform to perform the selected task when a tactical control is used to locate a configurable key of the hand-held device for activation ; and download the executable instructions from the storage device for use in connection with the configurable key of the hand-held device, wherein the storage device is provided at a location remote from the hand-held device.

[0008] Any aspect in combination with any one or more other aspects.

[0009] Any one or more of the features disclosed herein.

[0010] Any one or more of the features as substantially disclosed herein.

[0011] Any one or more of the features as substantially disclosed herein in combination with any one or more other features as substantially disclosed herein.

[0012] Any one of the aspects / features / embodiments in combination with any one or more other aspects / features / embodiments.

[0013] Use of any one or more of the aspects or features as disclosed herein.

[0014] It is to be appreciated that any feature described herein can be claimed in combination with any other feature(s) as described herein, regardless of whether the features come from the same described embodiment.

[0015] The details of one or more aspects of the disclosure are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the techniques described in this disclosure will be apparent from the description and drawings, and from the claims.

[0016] The phrases “at least one”, “one or more”, and “and / or” are open-ended expressions that are both conjunctive and disjunctive in operation. For example, each of the expressions “at leastone of A, B and C”, “at least one of A, B, or C”, “one or more of A, B, and C”, “one or more of A, B, or C” and “A, B, and / or C” means A alone, B alone, C alone, A and B together, A and C together, B and C together, or A, B and C together. When each one of A, B, and C in the above expressions refers to an element, such as X, Y, and Z, or class of elements, such as Xl-Xn, Yl- Ym, and Zl-Zo, the phrase is intended to refer to a single element selected from X, Y, and Z, a combination of elements selected from the same class (e.g., XI and X2) as well as a combination of elements selected from two or more classes (e g., Y1 and Zo).

[0017] The term “a” or “an” entity refers to one or more of that entity. As such, the terms “a” (or “an”), “one or more” and “at least one” can be used interchangeably herein. It is also to be noted that the terms “comprising”, “including”, and “having” can be used interchangeably.

[0018] The preceding is a simplified summary of the disclosure to provide an understanding of some aspects of the disclosure. This summary is neither an extensive nor exhaustive overview of the disclosure and its various aspects, embodiments, and configurations. It is intended neither to identify key or critical elements of the disclosure nor to delineate the scope of the disclosure but to present selected concepts of the disclosure in a simplified form as an introduction to the more detailed description presented below. As will be appreciated, other aspects, embodiments, and configurations of the disclosure are possible utilizing, alone or in combination, one or more of the features set forth above or described in detail below.

[0019] Numerous additional features and advantages of the present disclosure will become apparent to those skilled in the art upon consideration of the embodiment descriptions provided hereinbelow.BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

[0020] The accompanying drawings are incorporated into and form a part of the specification to illustrate several examples of the present disclosure. These drawings, together with the description, explain the principles of the disclosure. The drawings simply illustrate preferred and alternative examples of how the disclosure can be made and used and are not to be construed as limiting the disclosure to only the illustrated and described examples. Further features and advantages will become apparent from the following, more detailed, description of the various aspects, embodiments, and configurations of the disclosure, as illustrated by the drawings referenced below.

[0021] Fig l is a block diagram of a system according to at least one embodiment of the present disclosure;

[0022] Fig. 2 is a block diagram of a programmable remote controller according to at least one embodiment of the present disclosure;

[0023] Fig. 3 is a block diagram of a user interface for the programmable remote controller according to at least one embodiment of the present disclosure;

[0024] Fig. 4 is a block diagram illustrating the programmable remote controller used in sterile and non-sterile environments according to at least one embodiment of the present disclosure;

[0025] Fig. 5 is a block diagram illustrating the user interface for the programmable remote controller being customized according to at least one embodiment of the present disclosure;

[0026] Fig. 6 is a block diagram illustrating the user interface for the programmable remote controller being customized according to at least one embodiment of the present disclosure;

[0027] Fig. 7 is a block diagram illustrating the user interface for the programmable remote controller being customized according to at least one embodiment of the present disclosure;

[0028] Fig. 8 is a block diagram illustrating the user interface for the programmable remote controller being customized according to at least one embodiment of the present disclosure;

[0029] Fig. 9 is a block diagram illustrating the user interface for the programmable remote controller being customized according to at least one embodiment of the present disclosure; and

[0030] Fig. 10 is a flowchart according to at least one embodiment of the present disclosure.DETAILED DESCRIPTION

[0031] It should be understood that various aspects disclosed herein may be combined in different combinations than the combinations specifically presented in the description and accompanying drawings. It should also be understood that, depending on the example or embodiment, certain acts or events of any of the processes or methods described herein may be performed in a different sequence, and / or may be added, merged, or left out altogether (e.g., all described acts or events may not be necessary to carry out the disclosed techniques according to different embodiments of the present disclosure). In addition, while certain aspects of this disclosure are described as being performed by a single module or unit for purposes of clarity, it should be understood that the techniques of this disclosure may be performed by a combination of units or modules associated with, for example, a computing device and / or a medical device.

[0032] In one or more examples, the described methods, processes, and techniques may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as one or more instructions or code on a computer-readable medium and executed by a hardware-based processing unit. Alternatively, or additionally, functions may be implemented using machine learning models, neural networks, artificial neural networks, or combinations thereof (alone or in combination with instructions). Computer- readable media may include non-transitory computer-readable media, which corresponds to a tangible medium such as data storage media (e g., RAM, ROM, EEPROM, flash memory, or anyother medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer).

[0033] Instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors (e.g., Intel Core i3, i5, i7, or i9 processors; Intel Celeron processors; Intel Xeon processors; Intel Pentium processors; AMD Ryzen processors; AMD Athlon processors; AMD Phenom processors; Apple A10 or 10X Fusion processors; Apple Al l, A12, A12X, A12Z, or A13 Bionic processors; or any other general purpose microprocessors), graphics processing units (e.g., Nvidia GeForce RTX 2000- series processors, Nvidia GeForce RTX 3000-series processors, AMD Radeon RX 5000-series processors, AMD Radeon RX 6000-series processors, or any other graphics processing units), application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Accordingly, the term “processor” as used herein may refer to any of the foregoing structure or any other physical structure suitable for implementation of the described techniques. Also, the techniques could be fully implemented in one or more circuits or logic elements.

[0034] Before any embodiments of the disclosure are explained in detail, it is to be understood that the disclosure is not limited in its application to the details of construction and the arrangement of components set forth in the following description or illustrated in the drawings. The disclosure is capable of other embodiments and of being practiced or of being carried out in various ways. Also, it is to be understood that the phraseology and terminology used herein is for the purpose of description and should not be regarded as limiting. The use of “including,” “comprising,” or “having” and variations thereof herein is meant to encompass the items listed thereafter and equivalents thereof as well as additional items. Further, the present disclosure may use examples to illustrate one or more aspects thereof. Unless explicitly stated otherwise, the use or listing of one or more examples (which may be denoted by “for example,” “by way of example,” “e.g.,” “such as,” or similar language) is not intended to and does not limit the scope of the present disclosure.

[0035] The terms proximal and distal are used in this disclosure with their conventional medical meanings, proximal being closer to the operator or user of the system, and further from the region of surgical interest in or on the patient, and distal being closer to the region of surgical interest in or on the patient, and further from the operator or user of the system.

[0036] During surgical procedures, surgeons need a solution that minimizes reliance on the experience level of a stealth operator (e g., technician, technical operator, sales representative, operating room staff, etc.) to utilize surgical platforms (e.g., navigation platform, robotic platform, Juno platform, StealthStation, Mazor X, etc.). The experience level of the stealthoperator can make the difference between a smooth surgical procedure or a difficult surgical procedure. At times the stealth operator may need to support more than one surgical platform associated with a surgical procedure simultaneously, requiring the stealth operator to leave the operating room for periods of time.

[0037] In general, the stealth operator is required to be in close proximity to the surgical platform and in some cases, in front of the surgical platform to control the display information for the surgical platform. If the stealth operator must leave the operating room, another person is required to operate the surgical platform. In some cases, however, another person is not available to operate the surgical platform during the surgical procedure. Therefore, the stealth operator needs the flexibility to either move around the operating room or exit the operating room without impacting the efficiency of the surgical procedure.

[0038] Surgeons desire a customized solution that allows them to control the surgical platforms as desired. That is, surgeons want to interact with the surgical platforms as needed whenever the surgeon has a free hand to do so. Although surgeons rely on the stealth operator to utilize the surgical platforms during surgical procedures, surgeons desire to continuously interact with the surgical platforms without the assistance of the stealth operator. As an example, the surgeon may want to perform the functions of “Clear Arm,” “Stop,” Move Arm Up / Down Levels” on the robotic arm of a robot of the robotic platform when the surgeon has a free hand to do so. Moreover, surgeons want the ability to customize default options on the Juno platform.Performing operations such as stopping the robot or clearing the arm can be used in extremely time sensitive cases (e.g., the patient is bleeding out).

[0039] Thus, according to at least one embodiment of the present disclosure, a list of a plurality of surgical platforms for selection is received from a remote location from a customizable handheld device. In response to a selection of at least one of the surgical platforms, a plurality of tasks associated with the selected surgical platform is displayed. In response to a selection of at least one task from the plurality of tasks, the remote location is caused to select executable instructions for the selected surgical platform to perform the selected task when tactical controls are used to locate a configurable key of the customizable hand-held device for activation. Afterwards, the executable instructions are downloaded from the remote location for use in connection with the configurable key of the customizable hand-held device.

[0040] Embodiments of the present disclosure provide technical solutions to one or more of the problems of (1) increasing surgeon autonomy during surgical procedures to minimize the reliance on stealth operators to operate the surgical platforms which reduces or eliminates repeated requests to the stealth operators during surgical procedures; (2) allowing for stealth operators to control surgical platforms without being in front of, next to, or in close proximity to the surgicalplatforms during the surgical procedures; (3) providing the ability to the surgeon to control specific steps when operating the surgical platform when the surgeon’s hands are free; (4) providing customizable controls for the surgeon and the stealth operator based on the steps in the surgical workflow which allows for a simple interface that enables pertinent controls without taking up space on the surgical monitor; (5) providing parental control for the stealth operator which maintains the majority of the responsibility on the stealth operator since the stealth operator has a significant stake in being the surgical platform expert; and (6) supporting stealth operator needs while meeting the necessary surgeon needs.

[0041] Turning first to Fig. 1, a block diagram of a system 100 according to at least one embodiment of the present disclosure is shown. The system 100 may be used for remote control of one or more surgical platforms and / or to carry out one or more other aspects of one or more of the methods disclosed herein. The system 100 comprises a computing device 102, one or more imaging devices 112, a robot 114, a navigation system 118, a programmable remote controller 126, a database 130, and / or a cloud or other network 134. Systems according to other embodiments of the present disclosure may comprise more or fewer components than the system 100. For example, the system 100 may not include the imaging device 112, the robot 114, the navigation system 118, one or more components of the computing device 102, the database 130, and / or the cloud 134.

[0042] The computing device 102 comprises a processor 104, a memory 106, a communication interface 108, and a user interface 110. Computing devices according to other embodiments of the present disclosure may comprise more or fewer components than the computing device 102.

[0043] The processor 104 of the computing device 102 may be any processor described herein or any similar processor. The processor 104 may be configured to execute instructions stored in the memory 106, which instructions may cause the processor 104 to carry out one or more computing steps utilizing or based on data received from the imaging device 112, the robot 114, the navigation system 118, the database 130, and / or the cloud 134.

[0044] The memory 106 may be or comprise RAM, DRAM, SDRAM, other solid-state memory, any memory described herein, or any other tangible, non-transitory memory for storing computer-readable data and / or instructions. The memory 106 may store information or data useful for completing, for example, any step of the method 1000 described herein, or of any other methods. The memory 106 may store, for example, instructions and / or machine learning models that support one or more functions of the robot 114. For instance, the memory 106 may store content (e g., instructions and / or machine learning models) that, when executed by the processor 104, enable image processing 120, segmentation 122, registration 124, and / or transformation 128.

[0045] The image processing 120 enables the processor 104 to process image data of an image (received from, for example, the imaging device 112, an imaging device of the navigation system 118, or any imaging device, etc.) for the purpose of, for example, identifying one or more anatomical elements and / or fiducials depicted in the image data. The fiducial may be mounted on the patient on a target anatomical element. The target anatomical element may be, for example, one or more vertebrae though it will be appreciated that the target anatomical element may be any soft tissue and / or hard tissue. The information may comprise, for example, identification of anatomical element(s) and / or the fiducial(s), a boundary between an anatomical element, a boundary of hard tissue and / or soft tissue, etc. The image processing 120 may, for example, identify the anatomical element based on the fiducial(s), and / or a boundary of the anatomical element by determining a difference in or contrast between colors or grayscales of image pixels. For example, a boundary of the anatomical element may be identified as a contrast between lighter pixels and darker pixels. The imaging processing 120 may also use segmentation 122, as described below.

[0046] The segmentation 122 enables the processor 104 to process image data of an image (received from, for example, the imaging device 112, an imaging device of the navigation system 118, or any imaging device) for the purpose of, for example, identifying individual objects and / or anatomical elements in the image data. In some embodiments, the segmentation 122 may be used by the image processing 120. The segmentation 122 may enable the processor 104 to identify a boundary of a fiducial or an anatomical element by using, for example, feature recognition. For example, the segmentation 122 may enable the processor 104 to identify a vertebra in the image data. In other instances, the segmentation 122 may enable the processor 104 to identify a boundary of the fiducial or an anatomical element by determining a difference in or contrast between colors or grayscales of image pixels.

[0047] The fiducial(s) and / or anatomical element(s) identified from the image processing 120 and / or the segmentation 122 may enable the registration 124 to identify a target anatomical element based on the fiducial(s).

[0048] The registration 124 enables the processor 104 to process the identified fiducial(s) and / or anatomical element(s) obtained from the image processing 120 and / or the segmentation 122 to register the anatomical element(s) depicted in the image data based on the identified fiducial(s) to, for example, a preliminary image of the patient. It will be appreciated that though the image processing 120, the segmentation 122, the registration 124, and the transformation are described separately, that the image processing 120 and / or the segmentation 122 may be part of or a step of the registration 124. For example, registering the one or more anatomical elementsmay comprise using the image processing 120 and / or the segmentation 122 to identify one or more fiducial(s) and / or one or more anatomical element(s) depicted in the image data.

[0049] The transformation 128 enables the processor 104 to transform one coordinate system into another coordinate system. In other words, the transformation 128 enables the processor 104 to transform the first coordinate system (e.g., the patient coordinate system) into the second coordinate system (e g., the reference frame coordinate system) based on, for example, the registration of the first coordinate system and the third coordinate system and the registration of the second coordinate system and the third coordinate system.

[0050] Such content, if provided as in instruction, may, in some embodiments, be organized into one or more applications, modules, packages, layers, or engines. Alternatively, or additionally, the memory 106 may store other types of content or data (e.g., machine learning models, artificial neural networks, deep neural networks, etc.) that can be processed by the processor 104 to carry out the various method and features described herein. Thus, although various contents of memory 106 may be described as instructions, it should be appreciated that functionality described herein can be achieved through use of instructions, algorithms, and / or machine learning models. The data, algorithms, and / or instructions may cause the processor 104 to manipulate data stored in the memory 106 and / or received from or via the imaging device 112, the robot 114, the database 130, and / or the cloud 134.

[0051] The computing device 102 may also comprise a communication interface 108. The communication interface 108 may be used for receiving image data or other information from an external source (such as the imaging device 112, the robot 114, the navigation system 118, the database 130, the cloud 134, and / or any other system or component not part of the system 100), and / or for transmitting instructions, images, or other information to an external system or device (e.g., another computing device 102, the imaging device 112, the robot 114, the navigation system 118, the database 130, the cloud 134, and / or any other system or component not part of the system 100). The communication interface 108 may comprise one or more wired interfaces (e.g., a USB port, an Ethernet port, a Firewire port) and / or one or more wireless transceivers or interfaces (configured, for example, to transmit and / or receive information via one or more wireless communication protocols such as 802.1 la / b / g / n, Bluetooth, NFC, ZigBee, and so forth). In some embodiments, the communication interface 108 may be useful for enabling the device 102 to communicate with one or more other processors 104 or computing devices 102, whether to reduce the time needed to accomplish a computing-intensive task or for any other reason.

[0052] The computing device 102 may also comprise one or more user interfaces 110 The user interface 110 may be or comprise a keyboard, mouse, trackball, monitor, television, screen, touchscreen, and / or any other device for receiving information from a user and / or for providinginformation to a user. The user interface 110 may be used, for example, to receive a user selection or other user input regarding any step of any method described herein. Notwithstanding the foregoing, any required input for any step of any method described herein may be generated automatically by the system 100 (e.g., by the processor 104 or another component of the system 100) or received by the system 100 from a source external to the system 100. In some embodiments, the user interface 110 may be useful to allow a surgeon or other user to modify instructions to be executed by the processor 104 according to one or more embodiments of the present disclosure, and / or to modify or adjust a setting of other information displayed on the user interface 110 or corresponding thereto.

[0053] Although the user interface 110 is shown as part of the computing device 102, in some embodiments, the computing device 102 may utilize a user interface 110 that is housed separately from one or more remaining components of the computing device 102. In some embodiments, the user interface 110 may be located proximate one or more other components of the computing device 102, while in other embodiments, the user interface 110 may be located remotely from one or more other components of the computer device 102.

[0054] The imaging device 112 may be operable to image the fiducial(s), anatomical feature(s) (e.g., a bone, veins, tissue, etc.), and / or other aspects of patient anatomy to yield image data (e.g., image data depicting or corresponding to a bone, veins, tissue, etc.). “Image data” as used herein refers to the data generated or captured by an imaging device 112, including in a machine- readable form, a graphical / visual form, and in any other form. In various examples, the image data may comprise data corresponding to the fiducial(s), an anatomical feature of a patient, or to a portion thereof. The image data may be or comprise a preoperative image, an intraoperative image, a postoperative image, or an image taken independently of any surgical procedure. In some embodiments, a first imaging device 112 may be used to obtain first image data (e.g., a first image) at a first time, and a second imaging device 112 may be used to obtain second image data (e.g., a second image) at a second time after the first time. In such embodiments, the first imaging device may use ionizing radiation (e.g., X-ray scans) and the second imaging device may be free of ionizing radiation (e.g., ultrasound scans). In other embodiments, the imaging device 112 may obtain the first image data and the second image data.

[0055] The imaging device 112 may be capable of taking a 2D image or a 3D image to yield the image data. The imaging device 112 may be or comprise, for example, an ultrasound scanner (which may comprise, for example, a physically separate transducer and receiver, or a single ultrasound transceiver), an O-arm, a C-arm, a G-arm, or any other device utilizing X-ray -based imaging (e.g., a fluoroscope, a CT scanner, or other X-ray machine), a magnetic resonance imaging (MRI) scanner, an optical coherence tomography (OCT) scanner, an endoscope, amicroscope, an optical camera, a thermographic camera (e.g., an infrared camera), a radar system (which may comprise, for example, a transmitter, a receiver, a processor, and one or more antennae), or any other imaging device 112 suitable for obtaining images of an anatomical feature of a patient. The imaging device 112 may be contained entirely within a single housing or may comprise a transmitter / emitter and a receiver / detector that are in separate housings or are otherwise physically separated.

[0056] In some embodiments, the imaging device 112 may comprise more than one imaging device 112. For example, a first imaging device may provide first image data and / or a first image, and a second imaging device may provide second image data and / or a second image. In still other embodiments, the same imaging device may be used to provide both the first image data and the second image data, and / or any other image data described herein. The imaging device 112 may be operable to generate a stream of image data. For example, the imaging device 112 may be configured to operate with an open shutter, or with a shutter that continuously alternates between open and shut so as to capture successive images. For purposes of the present disclosure, unless specified otherwise, image data may be considered to be continuous and / or provided as an image data stream if the image data represents two or more frames per second.

[0057] The robot 114 may be any surgical robot or surgical robotic system. The robot 114 may be or comprise, for example, the Mazor X™ Stealth Edition robotic guidance system. The robot 114 may be configured to position the imaging device 112 at one or more precise position(s) and orientation(s), and / or to return the imaging device 112 to the same position(s) and orientation(s) at a later point in time. The robot 114 may additionally or alternatively be configured to manipulate a surgical tool (whether based on guidance from the navigation system 118 or not) to accomplish or to assist with a surgical task. The robot 114 may also be configured to position and / or insert one or more fiducial(s) into the patient and near a target anatomical element. In some embodiments, the robot 114 may be configured to hold and / or manipulate an anatomical element during or in connection with a surgical procedure. The robot 114 may comprise one or more robotic arms 116. In some embodiments, the robotic arm 116 may comprise a first robotic arm and a second robotic arm, though the robot 114 may comprise more than two robotic arms. In some embodiments, one or more of the robotic arms 116 may be used to hold and / or maneuver the imaging device 112. In embodiments where the imaging device 112 comprises two or more physically separate components (e.g., a transmitter and receiver), one robotic arm 116 may hold one such component, and another robotic arm 116 may hold another such component. Each robotic arm 116 may be positionable independently of the other robotic arm. The robotic arms 116 may be controlled in a single, shared coordinate space, or in separate coordinate spaces.

[0058] The robot 114, together with the robotic arm 116, may have, for example, one, two, three, four, five, six, seven, or more degrees of freedom. Further, the robotic arm 116 may be positioned or positionable in any pose, plane, and / or focal point. The pose includes a position and an orientation. As a result, an imaging device 112, surgical tool, or other object held by the robot 114 (or, more specifically, by the robotic arm 116) may be precisely positionable in one or more needed and specific positions and orientations.

[0059] The robotic arm(s) 116 may comprise one or more sensors that enable the processor 104 (or a processor of the robot 114) to determine a precise pose in space of the robotic arm (as well as any object or element held by or secured to the robotic arm).

[0060] In some embodiments, reference markers (e.g., navigation markers) may be placed on the robot 114 (including, e.g., on the robotic arm 116), the imaging device 112, or any other object in the surgical space. The reference markers may be tracked by the navigation system 118, and the results of the tracking may be used by the robot 114 and / or by an operator of the system 100 or any component thereof. In some embodiments, the navigation system 118 can be used to track other components of the system (e.g., imaging device 112) and the system can operate without the use of the robot 114 (e.g., with the surgeon manually manipulating the imaging device 112 and / or one or more surgical tools, based on information and / or instructions generated by the navigation system 118, for example).

[0061] In some aspects of the present disclosure, the navigation system 118 may include one or more of an optical tracking system, an acoustic tracking system, an electromagnetic tracking system, a radar tracking system, an inertial measurement unit (IMU) based tracking system, and a computer vision based tracking system. The navigation system 118 may include a corresponding transmission device 136 capable of transmitting signals associated with the tracking type. In some aspects, the navigation system 118 may be capable of computer vision based tracking of objects present in images captured by the imaging device(s) 112.

[0062] The navigation system 118 may provide navigation for a surgeon and / or a surgical robot during an operation. The navigation system 118 may be any now-known or future-developed navigation system, including, for example, the Medtronic StealthStation™ S8 surgical navigation system or any successor thereof. The navigation system 118 may include one or more cameras or other sensor(s) for tracking one or more reference markers, navigated trackers (e.g., tracking devices 140, etc.) or other objects within the operating room or other room in which some or all of the system 100 is located. The one or more cameras may be optical cameras, infrared cameras, or other cameras. In some implementations, the navigation system 118 may include one or more tracking devices 140 (e.g., electromagnetic sensors, acoustic sensors, etc.).

[0063] In some aspects, the navigation system 118 may include one or more of an optical tracking system, an acoustic tracking system, an electromagnetic tracking system, a radar tracking system, an inertial measurement unit (IMU) based tracking system, and a computer vision based tracking system. The navigation system 118 may include a corresponding transmission device 136 capable of transmitting signals associated with the tracking type. In some aspects, the navigation system 118 may be capable of computer vision based tracking of objects present in images captured by the imaging device(s) 112.

[0064] In various embodiments, the navigation system 118 may be used to track a position and orientation (e.g., a pose) of the imaging device 112, the robot 114 and / or robotic arm 116, and / or one or more surgical tools (or, more particularly, to track a pose of a navigated tracker attached, directly or indirectly, in fixed relation to the one or more of the foregoing). The navigation system 118 may include a display for displaying one or more images from an external source (e.g., the computing device 102, imaging device 112, or other source) or for displaying an image and / or video stream from the one or more cameras or other sensors of the navigation system 118. In some embodiments, the system 100 can operate without the use of the navigation system 118. The navigation system 118 may be configured to provide guidance to a surgeon or other user of the system 100 or a component thereof, to the robot 114, or to any other element of the system 100 regarding, for example, a pose of one or more anatomical elements, whether or not a tool is in the proper trajectory, and / or how to move a tool into the proper trajectory to carry out a surgical task according to a preoperative or other surgical plan.

[0065] The database 130 may store information that correlates one coordinate system to another (e.g., one or more robotic coordinate systems to a patient coordinate system and / or to a navigation coordinate system). The database 130 may additionally or alternatively store, for example, one or more surgical plans (including, for example, pose information about a target and / or image information about a patient’s anatomy at and / or proximate the surgical site, for use by the robot 114, the navigation system 118, and / or a user of the computing device 102 or of the system 100); one or more images useful in connection with a surgery to be completed by or with the assistance of one or more other components of the system 100; one or more surgical platform task; one or more surgeon profiles; and / or any other useful information. The database 130 may be configured to provide any such information to the computing device 102 or to any other device of the system 100 or external to the system 100, whether directly or via the cloud 134. In some embodiments, the database 130 may be or comprise part of a hospital image storage system, such as a picture archiving and communication system (PACS), a health information system (HIS), and / or another system for collecting, storing, managing, and / or transmitting electronic medical records including image data.

[0066] The cloud 134 may be or represent the Internet or any other wide area network. The computing device 102 may be connected to the cloud 134 via the communication interface 108, using a wired connection, a wireless connection, or both. In some embodiments, the computing device 102 may communicate with the database 130 and / or an external device (e.g., the programmable remote controller 126) via the cloud 134 as discussed in greater detail below.

[0067] The system 100 or similar systems may be used, for example, to carry out one or more aspects of any of the method 1000 described herein. The system 100 or similar systems may also be used for other purposes.

[0068] Fig. 2 is a block diagram of a programmable remote controller 200 (similar to the programmable remote controller 126 illustrated in Fig. 1) according to at least one embodiment of the present disclosure. Representations for the programmable remote controller 200 may include, but are not limited to, hand-held electronic devices, personal digital assistant, tablets, Web tablets, laptop computers, extended-functionality mobile or cellular phones, remote control devices, etc., as long as the device provides tactile feel, and swiping capabilities as discussed in greater detail below. The programmable remote controller 200 is adapted to provide wired or wireless control to an external source (such as the computing device 102, the imaging device 112, the robot 114, the navigation system 118, the database 130, the cloud 134, and / or any other system or component not part of the system 100).

[0069] The programmable remote controller 200 generally includes a processor 204, a memory 206, a communication interface 208, and a user interface 210 communicating by a bus 202. The processor 204 of the programmable remote controller 200 may be any processor described herein or any similar processor. The processor 204 may be configured to execute instructions stored in the memory 206, which instructions may cause the processor 204 to carry out one or more computing steps utilizing or based on data received from the computing device 102, the imaging device 112, the robot 114, the navigation system 118, the database 130, and / or the cloud 134.

[0070] The communication interface 208 may be used for receiving image data or other information from an external source (such as the computing device 102, the imaging device 112, the robot 114, the navigation system 118, the database 130, the cloud 134, and / or any other system or component not part of the system 100), and / or for transmitting instructions, images, or other information to an external system or device. The communication interface 208 may comprise one or more wired interfaces (e.g., a USB port, an Ethernet port, a Firewire port) and / or one or more wireless transceivers or interfaces (configured, for example, to transmit and / or receive information via one or more wireless communication protocols such as 802.1 la / b / g / n, Bluetooth, NFC, ZigBee, and so forth). The communication interface 208 may further include an infrared (IF) transmitter and receiver and / or a radio frequency (RF) transmitter and receiver totransmit and receive information to and from the external sources. In some embodiments of the present disclosure, the communication interface 208 may be useful for enabling the programmable remote controller 200 to communicate with one or more other processors 204 or processors 104 of computing devices 102, whether to reduce the time needed to accomplish a computing-intensive task or for any other reason.

[0071] The programmable remote controller 200 may also comprise one or more user interfaces 210. The user interface 210 may be or comprise a keyboard, mouse, trackball, monitor, television, screen, touchscreen, and / or any other device for receiving information from a user and / or for providing information to a user. The user interface 210 may be used, for example, to receive a user selection or other user input regarding any step of any method described herein. Notwithstanding the foregoing, any required input for any step of any method described herein may be generated automatically by the system 100 (e.g., by the processor 104, the processor 204 or another component of the system 100) or received by the system 100 from a source external to the system 100. In some embodiments, the user interface 210 may be useful to allow a surgeon or other user to modify instructions to be executed by the processor 204 according to one or more embodiments of the present disclosure, and / or to modify or adjust a setting of other information displayed on the user interface 210 or corresponding thereto.

[0072] Although the user interface 210 is shown as part of the programmable remote controller 200, in some embodiments of the present disclosure, the programmable remote controller 200 may utilize a user interface 210 that is housed separately from one or more remaining components of the programmable remote controller 200. In some embodiments of the present disclosure, the user interface 210 may be located proximate one or more other components of the programmable remote controller 200, while in other embodiments of the present disclosure, the user interface 210 may be located remotely from one or more other components of the programmable remote controller 200.

[0073] The memory 206 may be or comprise RAM, DRAM, SDRAM, other solid-state memory, any memory described herein, or any other tangible, non-transitory memory for storing computer-readable data and / or instructions. The memory 206 may store information or data useful for completing, for example, any step of the method 1000 described herein, or of any other methods. The memory 206 may store, for example, instructions and / or machine learning models that support one or more functions of the robot 114. For instance, the memory 206 may store content (e.g., instructions and / or machine learning models) that, when executed by the processor 204, enable image processing, segmentation, registration, and / or transformation as discussed above with the computing device 102.

[0074] Moreover, the memory 206 stores programmable remote controller application software in order to control the surgical platforms (e.g., navigation platform, robotic platform, imaging platforms etc.) as well as the features associated with the computing device 102. As illustrated in Fig. 2, the programmable remote controller application software includes remote control software 220, synchronization software 222, IR signaling software 224 and RF module software 228. The remote control software 220 allows the user to control the surgical platforms and the computing device 102. In general, the remote control software 220 includes several segments including, but not limited to a presentation engine, a configuration engine, a hardware communication manager, and a file input / output (VO) manager.

[0075] The presentation engine manages user interface 210 that users interact with to command the operation of the programmable remote controller 200, for example, to cause the transmission of commands to a device such as devices associated with the surgical platforms and the computing device 102. The configuration engine manages the configuration resources and creates display screens using the presentation engine. Resources for the configuration engine are generally stored in a configuration file. The hardware communication manager communicates with devices external to the programmable remote controller 200 utilizing IR, Bluetooth, 8021 l .b RF hardware, etc. The file I / O manager allows data access to be managed and controlled from a single place within the remote control software 220.

[0076] The remote control software 220 may also include the following high level functions: setup wizards; surgical platform and computer device control through IR and / or RF signal transmissions; customization features, reassign keys, setting favorites including rapid assignment of quick keys, configuration for multiple user profiles, configure parental control (“override”) mode, customize the remote control software from the host or remote computer device including the downloading of new software, software extensions; and copying of configuration data from one programmable remote controller to another.

[0077] The setup wizard is used to setup the programmable remote controller 200 to identify one or more users and the preferences of the one or more users. By way of further example, the setup wizard would be used to establish a list of surgical platforms that the programmable remote controller 200 is configured to control as well as how the surgical platforms should be controlled. To this end, a graphical user interface of the programmable remote controller 200 or a graphic user interface of the computing device may start by displaying a small number of device buttons (e.g., soft buttons that would be activated to place the programmable remote controller 200 into a mode to control a device) that would initially be un-setup. Selection of an un-setup soft device button can be used to automatically invoke the setup wizard. It is to be appreciated that the setup wizard will be invoked each time the programmable remote controller 200 is setup to control anew device or when it is desired to add or change settings for an already setup device. The setup wizard can be invoked multiple times to add more users. When invoking a setup wizard, the programmable remote controller 200 may be able to detect if a setup process has already been executed and may display the current settings to the user for confirmation or changing.

[0078] Once the programmable remote controller 200 has been setup with the setup wizard, the surgical platform and computer device control through IR and / or RF signal transmission feature allows a user interface for the programmable remote controller 200 to support the control buttons and features on the interface to support the operations of the various surgical platforms and the control features of computing devices.

[0079] Remote control customization allows a user to modify the graphical user interface, device and / or function control capabilities, or the programmable remote controller 200. Generally, any remote control customization would be assisted by the setup wizard.Modifications achievable via customization include, by way of example, learning any changes to the various surgical platforms, learning any changes for a user’s profile, learning changes for surgical plans, etc. Before modifications are made, an automatic backup of the settings of one or more component parts of the remote control software 220 may be made so that the user can undo changes if they did not achieve the desired results.

[0080] Settings usable in connection with the remote control software 220 (e.g., outdated software) may also be added, deleted, and / or modified based on information stored in external systems. The information may have originated on a website, the computing device 102, etc. For example, a website may provide updates for surgical platforms via the computing device 102. It will be appreciated by those skilled in the art that the programmable remote controller 200 is adapted to communicate directly to a webserver hosting a website via the cloud 134. According to a further embodiment of the present disclosure, the programmable remote controller 200 may be adapted to share all or part of a user’s programmed configuration settings with other compatible programmable remote controllers.

[0081] The synchronization software 222 is used as a communications conduit to and from a remote control companion application that is installed on a host computing device such as computing device 102. The IR signaling software 224 performs the IR signal generating functions to control surgical platforms, supporting a database of IR codes and transmission protocols for use in conjunction with the IR hardware components of the surgical platforms. The RF module software 228 (e.g., a Bluetooth or 802.11 RF module) may optionally perform RF signal generation to control RF devices and / or communicate wirelessly with a server or other applications resident on a host desktop such as computing device 102. It will be appreciated thatthe RF module software 228 may be used either together with or in place of the above-mentioned IR signaling software 224 to effect control of the various surgical platforms.

[0082] Fig. 3 is a block diagram of a user interface 300 for the programmable remote controller 200 according to at least one embodiment of the present disclosure. The user interface 300 for the programmable remote controller 200 generally includes buttons in the form of a plurality of configurable keys (e.g., quick buttons 304-320) and a plurality of preprogrammed standard keys (e.g., track pad 324, parental control button 350, navigation keys 360, horizontal slider 370 and vertical slider 380). According to an embodiment of the present disclosure, the plurality of configurable keys (e.g., quick buttons 304-320) and the plurality of preprogrammed standard keys (e.g., track pad 324, parental control button 350, navigation keys 360, horizontal slider 370 and vertical slider 380) are software keys or virtual keys provided underneath the touchscreen of the interface 300. The quick buttons 304-320 include indented or grooved features that provides tactile feedback to the user to allow users to locate the quick buttons 304-320 and allow the user to select (tap / press) customized options based on a procedure or task (e.g., an administrative panel, layouts, instruments list, save plan, zoom, etc.) without having to look down at the programmable remote controller 200. According to one embodiment of the present disclosure, the quick buttons 304-320 may include the tactile indicators 390a-390e (e.g., raised dots to orient users without the need for the user to see where the user’s fingers are relative to the quick buttons 304-320) as indicated in the drawings. The track pad 324 also includes an indented or grooved feature that provides tactile feedback to the user to allow the user to locate the track pad 324 that acts like a track pad for mouse control. The parental control (e.g., override) button 350 includes an indented or grooved feature that provides tactile feedback to the user to allow the user to locate the parental control button 350 that allows the stealth operator to enable surgeons autonomous control available or customized for a surgical procedure or a particular step of a surgical procedure. The navigation keys 360 also include indented or grooved features that provide tactile feedback to the user to allow the user to locate the navigation keys 360. The navigation keys 360 include arrow keys that allow users to navigate across selection panels similar to a remote control without having to look down at the programmable remote controller 200. The center circular key functions as a selection button. This feature can be adopted to each task based on selectable options from the arrow keys surrounding the center circular key. The feature will light up if the functionality is available during a task. The horizontal slider 370 includes indented or groove features that provide tactile feedback to the user to allow the user to locate the horizontal slider 370 that allows users to control functions like changing a projection or an implant size by sliding the bar of the horizontal slider between the positive and negative ends. Features will light up if the functionality is available during the surgical procedure task.The vertical slider 380 includes indented or grooved features that provide tactile feedback to the user to allow the user to locate the vertical slider 380 and that allows users to control functions such as zoom, move across image slices or image selection by sliding the bar of the vertical slider 380 between the top and bottom arrows. Features will light up if the functionality is available during the surgical procedure or surgical task.

[0083] According to an embodiment of the present disclosure, the intended or grooved features allow for intuitive use of the user interface 300 for the programmable remote controller 200 by providing tactile feedback and requiring minimal visual confirmation.

[0084] Outside United States (OUS) surgeons have minimum stealth operator support in many cases. The OUS surgeons rely on other operating room staff or themselves to thoroughly learn the surgical platforms. Therefore, OUS surgeons need easy-to-use autonomous controls over the surgical platforms during the surgical workflow. Fig. 4 is a block diagram 400 illustrating the programmable remote controller 200 used in sterile and non-sterile environments according to at least one embodiment of the present disclosure. The programable remote controller 200 can be used by a sterile user 404 (e.g., the surgeon, a nurse, surgical support staff, etc.) in a sterile environment (e.g., the operating room or operating theater) as well as a non-sterile user 408 (e.g., a sale representative for the surgical platform or other individuals not in contact with the patient, surgeon, etc.) in a non-sterile environment (e.g., outside of the operating room or operating theater). For the sterile user 404 the programmable remote controller 200 allows small US 412 clients stealth / robot limited controls 416. According to an embodiment of the present disclosure, when in the limited control mode 416, the indented or grooved features remain the same as the comprehensive modes 420 and 428. The difference, however, is when in the limited control mode 416, the underlying software is changed and there may not be software functionality or a soft button underneath that would cause anything to happen. The software button would have no function and do nothing. According to embodiments of the present disclosure, the nonfunctional software button would be programmable and can be modified both through the manufacturer software code or by user settings. Thus, surgeons control and enable autonomous controls. Also, for the sterile user 404, the programmable remote controller 200 allows comprehensive system controls 420 for large OUS 418 clients without a dedicated stealth operator. For the non-sterile user 408, the programmable remote controller 200 allows operator owned 424 comprehensive system controls 428 without being positioned in front of a user interface for the surgical platform. The programmable remote controller 200 allows for the same user interface 432.

[0085] Fig 5 is a block diagram 500 illustrating the user interface 300 for the programmable remote controller 200 being customized according to at least one embodiment of the present disclosure. According to an embodiment of the present disclosure, the programmable remotecontroller 200 is in communication with the computing device 102, the imaging device 112, the robot 114, the navigation system 118, and / or the database 130 over the cloud 134 to generate an image 504 provided on a display of the user interface 110 of the computing device 102, a display of the imaging device 112 or a display of the navigation system 118. The quick buttons 304-320 are arranged to represent one or more icons displayed in the image 504 based on tasks associated with a selected surgical platform. For example, tasks could include a cranial task, an ear, neck and throat (ENT) task or a spine task. Moreover, the quick buttons 304-320 could be arranged, labeled or customized based on a surgeon’s profile or a surgical workflow stored in database 130. As illustrated in Fig. 5, the quick buttons 304-320 are customized based on a selected surgical procedure using quick button 304 labeled “Left Panel”. The tasks associated with the selected surgical platform include various tasks such as “Cranial” represented by quick button 308, “ENT” represented by quick button 312, and “Spine” represented by quick button 316. The image 504 includes the associated icons for the quick buttons 308-316. According to an embodiment of the present disclosure, since there are only three (3) icons displayed, quick button 320 is not assigned. For example, a user can use quick button 308 to select the cranial application. This action results in the surgical platform proceeding to the next display for the cranial application. Thus, the programmable remote controller 200 allows a user to remotely select options already available on the surgical platform per selected workflow.

[0086] Fig. 6 is a block diagram 600 illustrating the user interface 300 for the programmable remote controller 200 being customized according to at least one embodiment of the present disclosure. As illustrated in Fig. 6, quick button 316 (e.g., spine task) is selected. The selection of quick button 316 highlights the icon for spine task illustrated in image 604.

[0087] According to embodiments of the present disclosure, the user interface for the programmable remote controller 200 can be customized for registration procedures. Default options for the registration procedure can be based on an active task. For example, the navigation keys 360 can be used to scroll and select images in the registration procedure or platform or proceed to a navigation procedure or platform. Moreover, the vertical slider 380 can be used to zoom-in or zoom-out on displayed images. Fig. 7 is a block diagram 700 illustrating the user interface 300 for the programmable remote controller 200 being customized according to at least one embodiment of the present disclosure. The user interface 300 for the programmable remote controller 200 can be customized for navigation procedures or platforms. The navigation keys 360 can be used to scroll and select images in layout 704 and the vertical slider 380 can be used to zoom-in or zoom-out on the images in layout 708. Although in the drawing the vertical slider 380 is used to zoom-in or zoom-out on displayed images and the horizontal slider 370 is used to control functions, these elements can be switched such that the vertical slider 380controls features and the horizontal slider 370 is used to zoom-in or zoom-out on displayed images with departing from the spirit and scope of the present disclosure.

[0088] Fig. 8 is a block diagram 800 illustrating the user interface 300 for the programmable remote controller 200 being customized according to at least one embodiment of the present disclosure. The user interface 300 for the programmable remote controller 200 can be customized for navigation procedures or platforms. The quick buttons 304-320 can be arranged to include tasks such as administrative 304, instruments 308, implants 312, projection 316 and layouts 320. The navigation keys 360 can be used to scroll and select images in layout 804 and the vertical slider 380 can be used to zoom-in or zoom-out on the images in layout 808.

[0089] Fig. 9 is a block diagram 900 illustrating the user interface 300 for the programmable remote controller 200 being customized according to at least one embodiment of the present disclosure. As illustrated in Fig. 9, the programmable remote controller 200 is customized for use with the robotic arm 116 of the robot 114. According to an embodiment of the present disclosure, the quick buttons 304-320 are labelled with the tasks administrative, clear arm, stop, move up / down and layouts, respectively. As illustrated in Fig. 9, quick button 308 labeled with the task “clear arm” and quick button 312 labeled with the task “stop” are mapped to the controls (e.g., buttons 904 and 908, respectively) provided on the robotic arm 116. Alternatively, the quick buttons 304-320 can be customized for other options. According to embodiments of the present disclosure, the quick buttons 304-320 may be customized and preset based on planned surgical procedures and stored surgeon preferences.

[0090] According to an embodiment of the present disclosure, a surgeon or stealth operator can remotely control the robotic arm 116 using the programmable remote controller 200 while another operator or personnel in proximity to the robotic arm 116 can still operate the robotic arm 116 using buttons 904 and 908. According to an embodiment of the present disclosure, a remote operator and an operator in proximity to the robotic arm 116 can control or operate the robotic arm 116 at the same time. According to another embodiment of the present disclosure, only one of the remote operator or the operator in proximity to the robotic arm 116 can control or operate the robotic arm 116 at the same time. According to a further embodiment of the present disclosure, limited access can be provided to the remote operator by way of the parental control button 350.

[0091] Fig. 10 depicts a method 1000 for programming a remote control for use with surgical platforms in non-sterile and / or sterile environments. In the following description of method 1000, the operations may be performed in a different order than the order shown, or the operations may be performed in different orders or at different times. Certain operations may also be left out of method 1000, or other operations may be added to the method 1000.

[0092] Generally, method 1000 starts with a START operation at step 1004 and ends with an END operation at step 1036. Method 1000 can be executed as a set of computer-executable instructions executed by a computer system (e.g., computing device 102, programmable remote controller 200, etc.) and encoded or stored on a computer readable medium. Hereinafter, method 1000 shall be explained with reference to the systems, components, modules, applications, software, data structures, user interfaces, etc. described in conjunction with Figs. 1-9.

[0093] The method 1000 (and / or one or more steps thereof) may be carried out or otherwise performed, for example, by at least one processor. The at least one processor may be the same as or similar to the processor(s) 104 of the computing device 102 described above or the processor 204(s) of the programmable remote controller 200 described above. A processor other than any processor described herein may also be used to execute the method 1000. The at least one processor may perform the method 1000 by executing elements stored in a memory such as the memory 106 or memory 206. The elements stored in memory and executed by the processor may cause the processor to execute one or more steps of a function as shown in method 1000. One or more portions of the method 1000 may be performed by the processor executing any of the contents of memory.

[0094] Method 1000 begins with the START operation at step 1004 and proceeds to step 1008, where the processor 204 receives from a remote location from the programmable remote controller 200, a list of a plurality of surgical platforms for selection. According to an embodiment of the present disclosure, the list of the plurality of surgical platforms may include a robotic platform, a navigation platform, Stealth Station Navigation platform, Flex ENT platform, Mazor Stealth XE, Axis Navigation platform, etc. The list of the surgical platforms may be stored in a system (e.g., a computing device 102) and / or one or more components thereof (e.g., a database 130). According to an alternative embodiment of the present disclosure, the list of the plurality of surgical platforms may be stored online and accessed from a website, for example.

[0095] After receiving from a remote location from the programmable remote controller 200, a list of a plurality of surgical platforms for selection at step 1008, method 1000 proceeds to step 1012, where the processor causes the display of the list of the plurality of surgical platforms for selection. According to embodiments of the present disclosure, the list of the plurality of surgical platforms for selection may be displayed on the user interface 110 of the computing device 102 or any display or monitor of the robot 114 or the navigation system 118. After causing the display of the list of the plurality of surgical platforms for selection at step 1012, method 1000 proceeds to step 1016, where the processor 204 receives a selection of at least one of the surgical platforms.

[0096] After receiving a selection of at least one of the surgical platforms at step 1016, method 1000 proceeds to step 1020 where the processor 204, in response to a selection of at least one of the surgical platforms, displays a plurality of tasks associated with the selected at least one of the surgical platforms. According to one embodiment of the present disclosure, tasks for a surgical platform may include a list of various body parts (e.g., cranial, ENT, spine, etc ). According to another embodiment of the present disclosure, tasks for a surgical platform may include a list of items required for surgery (e.g., administration, instruments, implants, projection, layouts, etc.). According to a further embodiment of the present disclosure, task for a surgical platform may include a list of commands for surgical equipment (e g., administration, clear arm, stop, move up / down, layout, etc.). According to alternative embodiments of the present disclosure, tasks for a surgical platform may include a list of procedures to be performed during a surgical procedure or a list of surgeon preferences. According to embodiments of the present disclosure, this information may be stored in a system (e.g., a computing device 102) and / or one or more components thereof (e.g., a database 130).

[0097] After displaying a plurality of tasks associated with the selected at least one of the surgical platforms at step 1020, method 1000 proceeds to step 1024, where the processor 204 receives a selection of at least one task from the plurality of tasks.

[0098] After receiving a selection of at least one task from the plurality of tasks at step 1024, method 1000 proceeds to step 1028 where the processor 204 causes the remote location to select executable instructions for causing the selected at least one of the surgical platforms to perform the selected at least one task when a tactical control is used to locate a configurable key of the programmable remote controller 200 for activation. According to one embodiment of the present disclosure, one or more of the remote software 220, the synchronization software 222, the IR signaling software 224 and the RF module software 228 is used to transfer executable code from the computing device 102 to the programmable remote controller 200. According to another embodiment of the present disclosure, one or more of the remote software 220, the synchronization software 222, the IR signaling software 224 and the RF module software 228 is used to transfer executable code from a website to the programmable remote controller 200. According to a further embodiment of the present disclosure, one or more of the remote software 220, the synchronization software 222, the IR signaling software 224 and the RF module software 228 is used in connection with the computing device 102 and a website to provide the executable code such that the programmable remote controller 200 can communicate with the selected surgical platform.

[0099] After causing the remote location to select executable instructions for causing the selected at least one of the surgical platforms to perform the selected at least one task when atactical control is used to locate a configurable key of the programmable remote controller 200 for activation at step 1028, method 1000 proceeds to step 1032 where the processor 204 downloads the executable instructions from the remote location for use in connection with the configurable key of the customizable hand-held device.

[0100] After downloading the executable instructions from the remote location for use in connection with the configurable key of the customizable hand-held device at step 1032, method 1000 ends at the END operation at step 1036.

[0101] The foregoing is not intended to limit the disclosure to the form or forms disclosed herein. In the foregoing Detailed Description, for example, various features of the disclosure are grouped together in one or more aspects, embodiments, and / or configurations for the purpose of streamlining the disclosure. The features of the aspects, embodiments, and / or configurations of the disclosure may be combined in alternate aspects, embodiments, and / or configurations other than those discussed above. This method of disclosure is not to be interpreted as reflecting an intention that the claims require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed aspect, embodiment, and / or configuration. Thus, the following claims are hereby incorporated into this Detailed Description, with each claim standing on its own as a separate preferred embodiment of the disclosure.

[0102] Moreover, though the foregoing has included description of one or more aspects, embodiments, and / or configurations and certain variations and modifications, other variations, combinations, and modifications are within the scope of the disclosure, e.g., as may be within the skill and knowledge of those in the art, after understanding the present disclosure. It is intended to obtain rights which include alternative aspects, embodiments, and / or configurations to the extent permitted, including alternate, interchangeable and / or equivalent structures, functions, ranges or steps to those claimed, whether or not such alternate, interchangeable and / or equivalent structures, functions, ranges or steps are disclosed herein, and without intending to publicly dedicate any patentable subject matter.

[0103] The techniques of this disclosure may also be described in the following examples:

[0104] Example 1: A customizable hand-held device, comprising: a processor (204); and a memory (206) storing data for processing by the processor, the data, when processed, causes the processor to: receive from a remote location (100) from the customizable hand-held device, a list of a plurality of surgical platforms (112, 114, 118) for selection; in response to a selection of at least one of the surgical platforms, display a plurality of tasks associated with the selected surgical platform; in response to a selection of at least one task from the plurality of tasks, cause the remote location to select executable instructions for the selected surgical platform to performthe selected task when a tactical control (390a-390e) is used to locate a configurable key (304- 380) of the customizable hand-held device for activation; and download the executable instructions from the remote location for use in connection with the configurable key of the customizable hand-held device.

[0105] Example 2: The customizable hand-held device of Example 1, wherein the plurality of surgical platforms includes a robotic platform (116) and a navigation platform (118).

[0106] Example 3: The customizable hand-held device of Example 1 or Example 2, wherein the tactical control provides feedback from activation of the configurable key.

[0107] Example 4: The customizable hand-held device of any of Examples 1-3, wherein the memory stores additional data for processing by the processor, that when processed, cause the processor to download from the remote location, command sets for allowing operations of the plurality of surgical platforms to be controlled via actuations of corresponding preprogrammed controls of the customizable hand-held device.

[0108] Example 5: The customizable hand-held device of Example 4, wherein the corresponding preprogrammed controls include navigation keys (360).

[0109] Example 6: The customizable hand-held device of Example 4 or Example 5, wherein the corresponding preprogrammed controls include vertical and horizontal sliders (370, 380).

[0110] Example 7: The customizable hand-held device of any of Examples 4-6, wherein the corresponding preprogrammed controls include a track pad (324).

[0111] Example 8: The customizable hand-held device of any of Examples 4-7, wherein the corresponding preprogrammed controls include an override key (350).

[0112] Example 9: The customizable hand-held device of any of Examples 1-8, wherein the remote location is a host computing device (102) in communication with the customizable handheld device.

[0113] Example 10: The customizable hand-held device of any of Examples 1-8, wherein the remote location is a server hosting a website.

[0114] Example 11 : The customizable hand-held device of any of Examples 1-10, wherein activation of the configurable key places the customizable hand-held device in a mode to effect control of the selected surgical platform.

[0115] Example 12: A method, comprising: receiving from a remote location (100) from a customizable hand-held device (200), a list of a plurality of surgical platforms (112, 114, 118) for selection; in response to a selection of at least one of the surgical platforms, displaying a plurality of tasks associated with the selected surgical platform; in response to a selection of at least one task from the plurality of tasks, causing the remote location to select executable instructions for the selected surgical platform to perform the selected task when a tactical control (390a-390e) isused to locate a configurable key (304-380) of the customizable hand-held device for activation; and downloading the executable instmctions from the remote location for use in connection with the configurable key of the customizable hand-held device.

[0116] Example 13: The method of Example 12, wherein the plurality of surgical platforms includes a robotic platform (116) and a navigation platform (118).

[0117] Example 14: The method of Example 12 or Example 13, further comprising downloading from the remote location, command sets for allowing operations of the plurality of surgical platforms to be controlled via actuations of corresponding preprogrammed controls of the customizable hand-held device.

[0118] Example 15: The method of Example 14, wherein the corresponding preprogrammed controls include navigation keys (360).

[0119] Example 16: The method of Example 14 or Example 15, wherein the corresponding preprogrammed controls include vertical and horizontal sliders (370, 380).

[0120] Example 17: The method of any of Examples 14-16, wherein the corresponding preprogrammed controls include a track pad (324).

[0121] Example 18: The method of any of Examples 14-17, wherein the corresponding preprogrammed controls include an override key (350).

[0122] Example 19: The method of any of Examples 12-18, wherein the remote location is a host computing device (102) in communication with the customizable hand-held device.

[0123] Example 20: A system, comprising: a storage device (130); and a hand-held device (200), comprising: a processor (204); and a memory (206) storing data for processing by the processor, the data, when processed, causes the processor to: receive from the storage device, a list of a plurality of surgical platforms (112, 114, 118) for selection; in response to a selection of at least one of the surgical platforms, display a plurality of tasks associated with the selected surgical platform; in response to a selection of at least one task from the plurality of tasks, cause the storage device to select executable instructions for the selected surgical platform to perform the selected task when a tactical control (390a-390e) is used to locate a configurable key (304- 370) of the hand-held device for activation; and download the executable instructions from the storage device for use in connection with the configurable key of the hand-held device, wherein the storage device is provided at a location remote from the hand-held device.

Claims

CLAIMSWhat is claimed is:

1. A customizable hand-held device, comprising: a processor (204); and a memory (206) storing data for processing by the processor, the data, when processed, causes the processor to: receive from a remote location (100) from the customizable hand-held device, a list of a plurality of surgical platforms (112, 114, 118) for selection; in response to a selection of at least one of the surgical platforms, display a plurality of tasks associated with the selected surgical platform; in response to a selection of at least one task from the plurality of tasks, cause the remote location to select executable instructions for the selected surgical platform to perform the selected task when a tactical control (390a-390e) is used to locate a configurable key (304-380) of the customizable hand-held device for activation; and download the executable instructions from the remote location for use in connection with the configurable key of the customizable hand-held device.

2. The customizable hand-held device of claim 1, wherein the plurality of surgical platforms includes a robotic platform (116) and a navigation platform (118).

3. The customizable hand-held device of claim 1 or 2, wherein the tactical control provides feedback from activation of the configurable key.

4. The customizable hand-held device of any of claims 1-3, wherein the memory stores additional data for processing by the processor, that when processed, cause the processor to download from the remote location, command sets for allowing operations of the plurality of surgical platforms to be controlled via actuations of corresponding preprogrammed controls of the customizable hand-held device.

5. The customizable hand-held device of claim 4, wherein the corresponding preprogrammed controls include navigation keys (360).

6. The customizable hand-held device of claim 4 or 5, wherein the corresponding preprogrammed controls include vertical and horizontal sliders (370, 380).

7. The customizable hand-held device of any of claims 4-6, wherein the corresponding preprogrammed controls include a track pad (324).

8. The customizable hand-held device of any of claims 4-7, wherein the corresponding preprogrammed controls include an override key (350).

9. A method, comprising: receiving from a remote location (100) from a customizable hand-held device (200), a list of a plurality of surgical platforms (112, 114, 118) for selection; in response to a selection of at least one of the surgical platforms, displaying a plurality of tasks associated with the selected surgical platform; in response to a selection of at least one task from the plurality of tasks, causing the remote location to select executable instructions for the selected surgical platform to perform the selected task when a tactical control (390a-390e) is used to locate a configurable key (304-380) of the customizable hand-held device for activation; and downloading the executable instructions from the remote location for use in connection with the configurable key of the customizable hand-held device.

10. The method of claim 9, wherein the plurality of surgical platforms includes a robotic platform (116) and a navigation platform (118).

11. The method of claim 9 or claim 10, further comprising downloading from the remote location, command sets for allowing operations of the plurality of surgical platforms to be controlled via actuations of corresponding preprogrammed controls of the customizable handheld device.

12. The method of claim 11, wherein the corresponding preprogrammed controls include navigation keys (360).

13. The method of claim 11 or claim 12, wherein the corresponding preprogrammed controls include vertical and horizontal sliders (370, 380).

14. The method of any of claims 11-13, wherein the corresponding preprogrammed controls include a track pad (324).

15. A system, comprising: a storage device (130); and a hand-held device (200), comprising: a processor (204); and a memory (206) storing data for processing by the processor, the data, when processed, causes the processor to: receive from the storage device, a list of a plurality of surgical platforms (112, 114, 118) for selection; in response to a selection of at least one of the surgical platforms, display a plurality of tasks associated with the selected surgical platform; in response to a selection of at least one task from the plurality of tasks, cause the storage device to select executable instructions for the selected surgical platform to perform the selected task when a tactical control (390a-390e) is used to locate a configurable key (304-370) of the hand-held device for activation; and download the executable instructions from the storage device for use in connection with the configurable key of the hand-held device, wherein the storage device is provided at a location remote from the hand-held device.

Images